Incremental load tools
Overview
Incremental loading is the practice of loading only records that are new or changed since the previous load. For large tables, this can represent a significant saving of resources over reloading the entire table every time to ensure it's up-to-date.
Users should schedule their incremental load jobs to run periodically for the job to continually update the created tables. To learn more about scheduling, read Manage Schedules.
The incremental load tools in Matillion ETL allow users to easily set up incremental loads for various data staging components, with wizards to guide users through the configuration process. These tools can be found in the Components panel on Matillion ETL.
Usage
When an Incremental Load component is dragged onto the job canvas, a job wizard opens to begin setup of the Incremental Load Shared Job (read-only).
Each wizard contains several pages of options—including data sources, columns, authentication properties, and staging options that are similar to those found in data staging components. Once configuration is completed, the Incremental Load Shared Job will appear on the canvas, ready for use.
The read-only Shared Job that each wizard uses is also available, unconfigured, from the Shared Jobs panel.
List of incremental load wizards
It is not guaranteed that all of the below wizards are available for all versions of Matillion ETL.