Oracle advanced settings
In most cases, the advanced settings for the Oracle connector won't be required; however, Matillion Change Data Capture (CDC) provides some additional configuration properties when setting up your CDC pipelines. Take a moment to understand what these advanced settings are before incorporating them into your pipelines.
Contact Matillion for support if you're unsure about a specific setting.
RAC enabled applications
If your Oracle configuration is RAC enabled, set the rac.nodes parameter in Pipeline settings.
Low change frequency sources
The Debezium Oracle connector tracks system change numbers in the connector offsets, so that when the connector is restarted it can begin where it left off. However, in a low change frequency environment, these change numbers may become stale due to older numbers not being retained as part of the local log retention policy. To avoid this, you can set up a heartbeat table and define an interval for inserting a record into that table. This will ensure that changes are regularly applied to avoid this issue. For more information, read Pipeline settings.
You will also need to add supplemental logging for your heartbeat table when you configure your Oracle database for CDC.
Note
The term "heartbeat table" is a standard Oracle table that is built to create a frequent and consistent stream of data to prevent log staleness. For more information, read Low change frequency offset management.
Connect to Oracle
Pipeline advanced settings can be added at the end of the Create Pipeline journey from the Data Loader UI. For more information, read CDC pipeline overview.
defaultRowPrefetch
The default number of rows to prefetch from the server is 10. When there are 1000 or more records in a table, set 'defaultRowPrefetch' to 1000 in the advanced settings. The default value of 10 does not allow retrieving all records before the connection times out.
Pipeline settings
Below is a table listing the configuration properties for the Oracle connector's advanced settings:
Note
For a full list of the configuration properties, read the Debezium documentation.
| Parameter | Value | Notes |
|---|---|---|
rac.nodes |
192.168.1.100,192.168.1.101:1522 | A comma-separated list of Oracle Real Application Clusters (RAC) node host names or addresses. This field is required to enable compatibility with an Oracle RAC deployment. The node will assume the default port as set when connecting to Oracle. If that port needs to be overridden, add the IP address and port number as per the example. For more information, read rac.nodes. |
log.mining.transaction.retention.ms |
2000000 | A positive integer value that specifies the number of milliseconds to retain long running transactions between redo log switches. When set to 0, transactions are retained until a commit or rollback is detected. Any transaction that exceeds this configured value is discarded entirely, and the connector does not emit any messages for the operations that were part of the transaction. For more information, read log.mining.transaction.retention.ms. |
heartbeat.interval.ms |
20000 | Specifies, in milliseconds, how frequently the connector sends messages to a heartbeat topic. Use this property to determine whether the connector continues to receive change events from the source database. It can also be useful to set the property in situations where no change events occur in captured tables for an extended period. For more information, read heartbeat.interval.ms. |
heartbeat.action.query |
INSERT INTO |
Set this property and create a heartbeat table to receive the heartbeat messages to resolve situations in which Debezium fails to synchronize offsets on low-traffic databases that are on the same host as a high-traffic database. For more information, read heartbeat.action.query. |