Detect Changes

Detect Changes is a transformation component that lets you scan two separate (but similar) tables, and insert a new column detailing if data has been inserted, deleted, changed, or is unchanged.

Any rows with key columns that contain NULL values will be ignored. NULL comparison values are considered equal.

Properties

Name = string

A human-readable name for the component.

Master Table = drop-down

Select a master table from the two inputs. This table is the one treated as default in the comparison with the second table.

Match Keys = dual listbox

Select the key columns to join the two tables on. These columns must appear in both tables. NULL values are ignored.

Compare Columns = dual listbox

Select the columns that will be checked for changes. Just like the keys, these columns must appear in both tables; however, the two lists shouldn't overlap.

Output Column Mapping = column editor (optional)

Input Column: Select input columns to map to output names. Defaults are provided automatically but can be changed.
Output Column: Name each output columns that the input columns will map to.

Click the Add all button to automatically populate the Input Column and Output Column with all the columns selected in the Compare Columns property. You can then selectively edit the list if required, but if you click Add all again it will return the list to the default fully populated state, removing any changes you have made.

Indicator Column = string

Enter a name for the new column in the output. By default, this column is named "Indicator". This column contains an indicator that shows the status of each record:

C the record has been changed.
D the record has been deleted.
I the record is identical.
N the record is new.

Switching the master table in the Master Table property will reverse the meaning of new (N) and deleted (D).

Indicators

Indicators are single-letter codes that indicate what the state of a row is regarding Detect Changes. The following indicators are used:

C Changed: the record is present in both tables, with different values, but with the same ID.
D Deleted: the record is present in the master table, but not in the second table.
I Identical: the same record is present in both tables with no changes.
N New: the record isn't present in the master table, but is present in the second table.

Snowflake	Databricks	Amazon Redshift
✅	✅	✅