Scheduling improvements, connectors and more!

BatchNew features 🎉Improvements 🔧CDC

Scheduling improvements

Batch pipelines allow you to quickly grab as many sources as you like from your service and load them into a data warehouse... but the real power comes from scheduling these to run more frequently! We've added deeper support for Quartz cron expressions so you can have total control over your schedule frequencies.

New connectors including Workday support

And as usual we're committed to getting your favorite services integrated natively with the Data Productivity Cloud so you can find the following connector components in Designer, today:

• Workday
• Workday Custom Reports
• Pipedrive
• Chargebee

Soft deletes for Snowflake CDC

Earlier this quarter, we released direct-to-Snowflake for CDC pipelines which allows you to replicate your data directly to your Snowflake data warehouse. We're now expanding that with a new "Copy Table with Soft Deletes" feature. This allows you to produce a replica in Snowflake of each captured source table, but rows deleted in the source are not deleted in the target table. As well as each column mapped from the corresponding source table, an additional boolean column called MTLN_CDC_DELETED is included that tells you whether that row is deleted in the source or not.

See the CDC Snowflake Destination documentation for more information.