Skip to content

Detect Changes

The Detect Changes component lets users scan two separate (but similar) tables, and insert a new column detailing if data has been inserted, deleted, changed, or even if the data is unchanged.

Any rows with key columns that contain NULL values will be ignored. NULL comparison values are considered equal.

Properties

Name = string

A human-readable name for the component.

Master Table = drop-down

Select a master table from the two inputs. This table is the one treated as default in the comparison with the second table.

Match Keys = dual listbox

Select the key columns to join the two tables on. These columns must appear in both tables. NULL values are ignored.

Compare Columns = dual listbox

Select the columns that will be checked for changes. Just like the keys, these columns must appear in both tables; however, the two lists should not overlap.

Output Column Mapping = column editor

  • Input Column: Select input columns to map to output names. Sensible defaults are provided automatically; however, these can be changed.
  • Output Column: Name output columns to which selected input columns will map.

Indicator Column = string

Input a name for the new column in the output. By default, this column is named "Indicator". This column contains an indicator that shows the status of each record:

C the record has been changed. D the record has been deleted. I the record is identical. N the record is new.

Switching the master table in the Master Table property will reverse the meaning of new (N) and deleted (D).

Strategy

Detects changed, unchanged, added, or deleted data in a comparison table relative to the designated master table.

Indicators

Indicators are single-letter codes that indicate what the state of a row is with regard to Detect Changes. The table below shows all indicators and their meanings.

Indicator Description
C Changed: the record is present in both tables, with different values, but with the same ID.
D Deleted: the record is present in the master table, but not in the second table.
I Identical: the same record is present in both tables with no changes.
N New: the record is not present in the master table, but is present in the second table.
Snowflake Delta Lake on Databricks Amazon Redshift Google BigQuery Azure Synapse Analytics
✅ ❌ ✅ ✅ ✅