November 2024 changelog

Here you'll find the November 2024 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.

19th November

DesignerNew features 🎉

New connectors:

Added a Kafka connector. The Kafka connector is an orchestration component that lets you load data from Kafka topics into either schemaless JSON or JSON schema objects. New to Kafka? Read Apache Kafka's Introduction.
Added the following Flex connectors for developing data pipelines:
- Box
- Fanatics
- Snapchat

Git actions in Designer:

Added a new Compare changes feature that allows you to compare your local changes to the previous commit.
Added commit indicators that show how many commits your local branch is ahead of or behind the remote branch when committing, pushing, or pulling changes.

API:

Added a GET pipeline executions endpoint, which returns a list of summarised pipeline executions.

CDC

DesignerNew features 🎉Improvements 🔧

New connectors:

Added the following Flex connectors for developing data pipelines:
- Attio
- Buildkite

Component improvements:

The Python Pushdown component now includes a Warehouse parameter, allowing you to specify which Snowflake warehouse to run your Python scripts on.
The Detect Changes component is now available on Databricks projects.

DesignerNew features 🎉Improvements 🔧

Copilot is now available for Databricks and Amazon Redshift projects (previously only available for Snowflake projects). In the Data Productivity Cloud, Copilot introduces a new era of AI-driven data productivity tools, letting you prompt generative AI to create data pipelines, improving your efficiency in managing and processing data.
Updated the S3 Unload component for Amazon Redshift projects. Previously, only CSV file types were supported. Now, the component also supports delimited, fixed with, and Parquet file types.

CDC

Data Productivity CloudNew features 🎉

The Data Productivity Cloud is now available in Snowflake Partner Connect, allowing a quick and simple way to create an automatically configured Matillion account directly from a Snowflake instance.