Skip to content

Create or Refresh External Table shared job

This shared job maintains two external tables, used by the Sync Single Table shared job. The first table, named '' contains all the change events from a given CDC pipeline. The second table, named '_VERSION_METADATA' contains the metadata for each version of the source tables.

On first execution, the job will create the external tables over the Snowflake stage. The tables will be created with partition columns to enable efficient querying for new data.

If the external tables already exist, they will be refreshed to contain any new files.

Properties

Target Database = drop-down

The Snowflake database containing the external table with change events and the target table.

Target Schema = drop-down

The Snowflake schema containing the external table with change events and the target table.

Snowflake External Stage = drop-down

The name of the existing external stage that contains the files output by the CDC pipeline.

External Table = drop-down

The external table in Snowflake containing the CDC change events.

Cloud Storage URL = string

The URL of the location where the CDC change event files are stored. This should have the format s3://<bucket>/<prefix>/ or azure://<account>.blob.core.windows.net/<container>/<prefix>/

Snowflake Delta Lake on Databricks Amazon Redshift Google BigQuery Azure Synapse Analytics
✅ ❌ ❌ ❌ ❌