Scheduling improvements, connectors and more!
BatchNew features 🎉Improvements 🔧CDC
Scheduling improvements
Batch pipelines allow you to quickly grab as many sources as you like from your service and load them into a data warehouse... but the real power comes from scheduling these to run more frequently! We've added deeper support for Quartz cron expressions so you can have total control over your schedule frequencies.
New connectors including Workday support
And as usual we're committed to getting your favorite services integrated natively with the Data Productivity Cloud so you can find the following connector components in Designer, today:
Soft deletes for Snowflake CDC
Earlier this quarter, we released direct-to-Snowflake for CDC pipelines which allows you to replicate your data directly to your Snowflake data warehouse. We're now expanding that with a new "Copy Table with Soft Deletes" feature. This allows you to produce a replica in Snowflake of each captured source table, but rows deleted in the source are not deleted in the target table. As well as each column mapped from the corresponding source table, an additional boolean column called MTLN_CDC_DELETED
is included that tells you whether that row is deleted in the source or not.
See the CDC Snowflake Destination documentation for more information.