October 2024 changelog

Here you'll find the October 2024 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.

31st October

DesignerNew features 🎉

Connect your own Git repo:

You can now connect your own Bitbucket repository to the Data Productivity Cloud. This feature lets you connect your Bitbucket repository when you create a project in the Data Productivity Cloud. Follow the documentation to get started:
- Prerequisites for connecting a Bitbucket repository to the Data Productivity Cloud
- Troubleshooting guide for connecting a Bitbucket repository to the Data Productivity Cloud

In case you missed our announcements in previous changelogs, we also support GitHub, GitLab, and Azure DevOps. Read Connect your own Git repo for an overview.

Agents:

You can now refresh an agent's credentials, which generates a new client_secret to authenticate with the agent application in your infrastructure (Hybrid SaaS deployments only).

29th October

DesignerNew features 🎉

Components:

JDBC Table Metadata to Grid is now available for Amazon Redshift projects.
Truncate Table is now available for Databricks projects.

22nd October

CDC

Updated the CDC agent version from 2.97.14 to 2.98.2.
Change behaviour of the Snowflake destination - if the Replication Type is Copy Table or Copy Table with Soft Deletes and the source table does not have a primary key then the Replication Type will be set to Change Log for that table.
Library updates and security fixes.

15th October

DesignerImprovements 🔧

Component updates:

Updated Python Pushdown. Added a Snowflake Secret parameter, which is activated when an external access integration is selected. You can select Snowflake secrets and assign them to Snowflake variables, which can then be used in the Python script.

9th October

CDC

Updated the CDC agent version from 2.97.12 to 2.97.14.
Fixed Azure Blob Storage authentication flow.

8th October

DesignerNew features 🎉

New orchestration components:

Azure Document Intelligence. This component uses the Azure AI Document Intelligence API to automate the extraction of text, handwriting, layout elements, and other key data from forms and documents. The output format can be either Markdown or text.
Azure Speech Transcribe. This component uses the Azure Speech service to transcribe speech to text from an audio file in your specified Azure Blob location and Speech resource endpoint. Speaker diarization clarifies which speaker spoke and when, increasing the accuracy of your transcriptions.

New transformation components:

Databricks Vector Search. This component lets you perform a search on an input table to find content that best answers specific questions, using vector embeddings to identify suitable answers within data located in your Databricks account. Only available for Databricks projects.

3rd October

DesignerNew features 🎉

New orchestration components:

Add Partition. This orchestration component lets users define the S3 directory structure for partitioned external table data. Amazon Redshift projects only.
Create External Table. This orchestration component creates a table to reference data stored in an Amazon S3 bucket. The table references external data, meaning the table itself does not hold the data. Amazon Redshift projects only.

2nd October

CDCImprovements 🔧

Updated the CDC agent version from 2.95.33 to 2.97.12.
Improved the process of agent shutdowns so that on clean shutdown the agent status shows as Stopped rather than Unknown.
Library updates and security fixes.