August 2024 changelog

Here you'll find the August 2024 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.

29th August

DesignerNew features 🎉

You can now connect your own GitLab repository to the Data Productivity Cloud. This feature lets you connect your GitLab repository when you create a project in the Data Productivity Cloud. Follow the documentation to get started:
- Prerequisites for connecting a GitLab repository to the Data Productivity Cloud
- Troubleshooting guide for connecting a GitLib repository to the Data Productivity Cloud
Updated Schedules so that you can now specify a time and date to enable pipeline schedules from.
Updated the "Connect your own GitHub" workflow so that you can now view the list of authorized accounts or organizations after a successful authorization with GitHub.

27th August

StreamingNew features 🎉

Added the following destinations to streaming pipelines:
- Amazon S3
- Azure Blob Storage

20th August

DesignerNew features 🎉

Shared pipelines are now available in the Data Productivity Cloud. Any pipeline can be saved as a shared pipeline, helping you to maintain consistency of core functionality over multiple different projects across the organization.
Added the Run Shared Pipeline component, which allows you to call a shared pipeline from within another pipeline.
Added improvements to environments. When you create an environment, you can now set a default access level for that environment. This access level can also be set on a per-user basis.
- As part of this change, all users in existing environments have been set to have Admin access by default, to avoid any disruption. Project admins can change a user's environment access.

16th August

DesignerNew features 🎉

Note

The Salesforce and Salesforce Query items below have been rolled back as of August 30th, 2024. The Salesforce Query component can once again be used for new pipelines.

A new Salesforce connector is now available, replacing the Salesforce Query component.
- Existing pipelines with the Salesforce Query component will continue to run as normal.
Added a Document AI Predict component, which invokes the Snowflake PREDICT function and allows users to call their Document AI models from the context of a Data Productivity Cloud pipeline. This component is available in public preview.

14th August

CDCImprovements 🔧

Updated the CDC agent version from 2.95.22 to 2.95.27.
CDC will now retain snapshot metrics once a pipeline snapshot completes and streaming begins. Note: these will be lost after an agent restart.
Improved resilience of connection between CDC agents and the platform.
Library updates and security fixes.

8th August

StreamingNew features 🎉

Streaming pipelines are now available in the Data Productivity Cloud. Streaming pipelines are a complete end-to-end solution for near-real-time data ingestion, allowing you to capture and synchronize data from source databases and write to a target data warehouse.
- Supported data sources are:
- Supported data targets are:
  - Snowflake
- Streaming pipelines are available on our Hybrid SaaS offering, meaning you'll need to create a streaming agent in your AWS or Azure infrastructure. Read Create an agent in your infrastructure to set up your agent.

DesignerNew features 🎉

Options to run a single component or run a pipeline starting from a specific component have been added to the pipeline canvas.
Options to control the behavior of variables in a branching pipeline have been added.
The Push local changes dialog now shows a list of commits to be pushed.

6th August

DesignerNew features 🎉

Users can now terminate running pipelines from the Pipeline Observability Dashboard.