The Detect Changes component lets users scan two separate (but similar) tables, and insert a new column detailing if data has been inserted, deleted, changed, or even if the data is unchanged.
Any rows with key columns that contain NULL values will be ignored. NULL comparison values are considered equal.
Name = string
A human-readable name for the component.
Master Table = drop-down
Select a master table from the two inputs. This table is the one treated as default in the comparison with the second table.
Match Keys = dual listbox
Select the key columns to join the two tables on. These columns must appear in both tables. NULL values are ignored.
Compare Columns = dual listbox
Select the columns that will be checked for changes. Just like the keys, these columns must appear in both tables; however, the two lists should not overlap.
Output Column Mapping = column editor
- Input Column: Select input columns to map to output names. Sensible defaults are provided automatically; however, these can be changed.
- Output Column: Name output columns to which selected input columns will map.
Indicator Column = string
Input a name for the new column in the output. By default, this column is named "Indicator". This column contains an indicator that shows the status of each record:
C the record has been changed. D the record has been deleted. I the record is identical. N the record is new.
Switching the master table in the Master Table property will reverse the meaning of new (N) and deleted (D).
Detects changed, unchanged, added, or deleted data in a comparison table relative to the designated master table.
Indicators are single-letter codes that indicate what the state of a row is with regard to Detect Changes. The table below shows all indicators and their meanings.
|C||Changed: the record is present in both tables, with different values, but with the same ID.|
|D||Deleted: the record is present in the master table, but not in the second table.|
|I||Identical: the same record is present in both tables with no changes.|
|N||New: the record is not present in the master table, but is present in the second table.|
|Snowflake||Delta Lake on Databricks||Amazon Redshift||Google BigQuery||Azure Synapse Analytics|