Skip to content

2024 changelogπŸ”—

Here you'll find the 2024 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.


DecemberπŸ”—

December 12πŸ”—

DesignerNew features πŸŽ‰

  • Added the Retrieve Run endpoint to the dbt Cloud Flex connector.

AgentImprovements πŸ”§

  • The agent template for Azure now deploys with an environment type of "Workload profile - Dedicated" (was previously "Consumption Only"). This allows the agent to better facilitate complex networking, while keeping the ability for the agent to auto-update.

December 9πŸ”—

CDC

  • Updated the CDC agent version from 2.98.7 to 2.101.2.
  • For customers who may have experienced an out-of-memory issue with the agent, we have added a new advanced property matillion.compact-history that can be set to true in any affected pipeline definition. This will reduce the memory footprint of the agent by compacting the schema history of the pipeline.
  • For pipelines that stream data directly to Snowflake, the agent will create a new column MTLN_CDC_LOAD_TIMESTAMP in each target table to store the load timestamp of each row.
  • Library updates and security fixes.

December 6πŸ”—

APINew Endpoints πŸŽ‰

The following endpoints have been added to the Data Productivity Cloud REST API:

Credit consumption

Method Endpoint Description
GET /v1/consumption Get a breakdown of your credit consumption for flat-rated products
GET /v1/consumption/etl-users Get a breakdown of your credit consumption for ETL users

December 5πŸ”—

DesignerNew features πŸŽ‰

  • Added the new Skip on run action to the component action bar. This allows you to temporarily disable a component in a pipeline, without removing it. For more information, read Skip components.

New transformation component:

  • Cortex Parse Document is a transformation component that uses Snowflake Cortex to extract content from a document on an internal or external stage in the form of an object that contains JSON-encoded objects as strings.

December 4πŸ”—

DesignerNew features πŸŽ‰


December 3πŸ”—

DesignerNew features πŸŽ‰

  • Added the option to rename pipeline folders in Designer. For more information, read Pipeline folders.
  • Added a new Logging tab to the detailed view that appears when you click a task in the Task history tab. This shows a detailed log of each step in a task, as described in Logging.

StreamingImprovements πŸ”§


NovemberπŸ”—

November 19πŸ”—

DesignerNew features πŸŽ‰

New connectors:

  • Added a Kafka connector. The Kafka connector is an orchestration component that lets you load data from Kafka topics into either schemaless JSON or JSON schema objects. New to Kafka? Read Apache Kafka's Introduction.
  • Added the following Flex connectors for developing data pipelines:

Git actions in Designer:

  • Added a new Compare changes feature that allows you to compare your local changes to the previous commit.
  • Added commit indicators that show how many commits your local branch is ahead of or behind the remote branch when committing, pushing, or pulling changes.

API:


November 15πŸ”—

APINew Endpoints πŸŽ‰

The following endpoint has been added to the Data Productivity Cloud REST API:

Pipeline execution

Method Endpoint Description
GET /v1/pipeline-executions Returns a list of summarized pipeline executions

November 13πŸ”—

CDC

  • Updated the CDC agent version from 2.98.4 to 2.98.7.
  • Enhanced the resilience of connecting the agent to the Data Productivity Cloud.
  • Library updates and security fixes.

November 12πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

New connectors:

  • Added the following Flex connectors for developing data pipelines:

Component improvements:

  • The Python Pushdown component now includes a Warehouse parameter, allowing you to specify which Snowflake warehouse to run your Python scripts on.
  • The Detect Changes component is now available on Databricks projects.

November 7πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Maia (formerly Copilot) is now available for Databricks and Amazon Redshift projects (previously only available for Snowflake projects). In the Data Productivity Cloud, Maia introduces a new era of AI-driven data productivity tools, letting you prompt generative AI to create data pipelines, improving your efficiency in managing and processing data.
  • Updated the S3 Unload component for Amazon Redshift projects. Previously, only CSV file types were supported. Now, the component also supports delimited, fixed with, and Parquet file types.

November 6πŸ”—

CDC

  • Updated the CDC agent version from 2.98.2 to 2.98.4.
  • Library updates and security fixes.

November 5πŸ”—

Data Productivity CloudNew features πŸŽ‰

  • The Data Productivity Cloud is now available in Snowflake Partner Connect, allowing a quick and simple way to create an automatically configured Matillion account directly from a Snowflake instance.

OctoberπŸ”—

October 31πŸ”—

DesignerNew features πŸŽ‰

Connect your own Git repo:

In case you missed our announcements in previous changelogs, we also support GitHub, GitLab, and Azure DevOps. Read Connect your own Git repo for an overview.

Agents:

  • You can now refresh an agent's credentials, which generates a new client_secret to authenticate with the agent application in your infrastructure (Hybrid SaaS deployments only).

October 29πŸ”—

DesignerNew features πŸŽ‰

Components:


October 22πŸ”—

CDC

  • Updated the CDC agent version from 2.97.14 to 2.98.2.
  • Change behaviour of the Snowflake destination - if the Replication Type is Copy Table or Copy Table with Soft Deletes and the source table does not have a primary key then the Replication Type will be set to Change Log for that table.
  • Library updates and security fixes.

October 15πŸ”—

DesignerImprovements πŸ”§

Component updates:

  • Updated Python Pushdown. Added a Snowflake Secret parameter, which is activated when an external access integration is selected. You can select Snowflake secrets and assign them to Snowflake variables, which can then be used in the Python script.

October 9πŸ”—

CDC

  • Updated the CDC agent version from 2.97.12 to 2.97.14.
  • Fixed Azure Blob Storage authentication flow.

October 8πŸ”—

DesignerNew features πŸŽ‰

New orchestration components:

  • Azure Document Intelligence. This component uses the Azure AI Document Intelligence API to automate the extraction of text, handwriting, layout elements, and other key data from forms and documents. The output format can be either Markdown or text.
  • Azure Speech Transcribe. This component uses the Azure Speech service to transcribe speech to text from an audio file in your specified Azure Blob location and Speech resource endpoint. Speaker diarization clarifies which speaker spoke and when, increasing the accuracy of your transcriptions.

New transformation components:

  • Databricks Vector Search. This component lets you perform a search on an input table to find content that best answers specific questions, using vector embeddings to identify suitable answers within data located in your Databricks account. Only available for Databricks projects.

October 3πŸ”—

DesignerNew features πŸŽ‰

New orchestration components:

  • Add Partition. This orchestration component lets users define the S3 directory structure for partitioned external table data. Amazon Redshift projects only.
  • Create External Table. This orchestration component creates a table to reference data stored in an Amazon S3 bucket. The table references external data, meaning the table itself does not hold the data. Amazon Redshift projects only.

October 2πŸ”—

CDCImprovements πŸ”§

  • Updated the CDC agent version from 2.95.33 to 2.97.12.
  • Improved the process of agent shutdowns so that on clean shutdown the agent status shows as Stopped rather than Unknown.
  • Library updates and security fixes.

SeptemberπŸ”—

September 26πŸ”—

DesignerImprovements πŸ”§

Projects:

  • Snowflake projects now allow the choice of key-pair authentication when configuring environments, so a Snowflake private key can be used.
  • Databricks projects now require a Databricks personal access token for authentication when configuring environments. Username/password has been deprecated by Databricks and can no longer be used.

Git version control:

  • Additional options are available for Git reset, allowing a reset to your selected working state.

New orchestration components:

  • Bash Pushdown. This component enables users running Full SaaS configurations (Matillion-hosted agents) to use SSH connections to run their Bash scripts in their own instances.

September 24πŸ”—

DesignerNew features πŸŽ‰

New orchestration components:

Component updates:

Designer UI:

  • You can now reorder your pipeline tabs in Designer with click-and-drag.

September 12πŸ”—

DesignerNew features πŸŽ‰

New transformation components:

  • Vector Search. This component uses Snowflake vectors to perform a retrieval augmented generation (RAG) search on an input table. This component is available in public preview for Snowflake projects.

Component updates:

Designer:

  • The Git Commit dialog now lists all uncommitted changes.
  • Added a command palette to Designer. In its first phase, the following command is supported:
    • Use cmd + k or ctrl + k to activate a file explorer dialog. Alternatively, click the search icon in the upper-right corner of the UI.

September 10πŸ”—

CDCImprovements πŸ”§

  • Updated the CDC agent version from 2.95.27 to 2.95.33.
  • Library updates and security fixes.

September 5πŸ”—

DesignerNew features πŸŽ‰

New orchestration components:

  • Amazon Textract. This component uses the Textract API to extract text, handwriting, layout elements, and data from scanned documents. You can optionally choose to include footers and page numbers in the extraction process.
  • Amazon Transcribe. This component uses the Transcribe API to extract data from media and audio files and then converts that data into transcriptions.

Component updates:

  • Chunk Text is now out of public preview and generally available for all Snowflake projects.

September 3πŸ”—

DesignerNew features πŸŽ‰

New components:

  • Cortex Embed. This transformation component lets you convert English-language text into a Cortex vector embedding.
  • Remove From Grid. This orchestration component takes in grid variable data and remove data based on the user's definition.

New connector:

  • NetSuite SuiteAnalytics. This new connector is an orchestration component that uses NetSuite SuiteAnalytics Connect to retrieve data and load it into a table.

AugustπŸ”—

August 29πŸ”—

DesignerNew features πŸŽ‰


August 27πŸ”—

StreamingNew features πŸŽ‰


August 20πŸ”—

DesignerNew features πŸŽ‰

  • Shared pipelines are now available in the Data Productivity Cloud. Any pipeline can be saved as a shared pipeline, helping you to maintain consistency of core functionality over multiple different projects across the organization.
  • Added the Run Shared Pipeline component, which allows you to call a shared pipeline from within another pipeline.
  • Added improvements to environments. When you create an environment, you can now set a default access level for that environment. This access level can also be set on a per-user basis.
    • As part of this change, all users in existing environments have been set to have Admin access by default, to avoid any disruption. Project admins can change a user's environment access.

August 16πŸ”—

DesignerNew features πŸŽ‰

  • Added a Document AI Predict component, which invokes the Snowflake PREDICT function and allows users to call their Document AI models from the context of a Data Productivity Cloud pipeline. This component is available in public preview.

August 14πŸ”—

CDCImprovements πŸ”§

  • Updated the CDC agent version from 2.95.22 to 2.95.27.
  • CDC will now retain snapshot metrics once a pipeline snapshot completes and streaming begins. Note: these will be lost after an agent restart.
  • Improved resilience of connection between Streaming agents and the platform.
  • Library updates and security fixes.

August 8πŸ”—

StreamingNew features πŸŽ‰

  • Streaming pipelines are now available in the Data Productivity Cloud. Streaming pipelines are a complete end-to-end solution for near-real-time data ingestion, allowing you to capture and synchronize data from source databases and write to a target data warehouse.

DesignerNew features πŸŽ‰

  • Options to run a single component or run a pipeline starting from a specific component have been added to the pipeline canvas.
  • Options to control the behavior of variables in a branching pipeline have been added.
  • The Push local changes dialog now shows a list of commits to be pushed.

August 6πŸ”—

DesignerNew features πŸŽ‰


JulyπŸ”—

July 30πŸ”—

Data Productivity Cloud APINew features πŸŽ‰

  • The Pipeline execution API now supports overriding scalar or grid variables when making a call. This enhancement allows for greater flexibility and customization during pipeline execution.

CDCImprovements πŸ”§


July 25πŸ”—

DesignerNew features πŸŽ‰

  • Added the Cloud Pub/Sub component, which allows messages to be published to a topic on Google Cloud Platform (GCP). Other applications can subscribe to these topics to receive such messages.

July 23πŸ”—

DesignerNew features πŸŽ‰

  • Google Cloud Storage is now available for Snowflake projects in the following areas:
    • Added the Google Cloud Storage Load component, which lets you load data stored on the Google Cloud Storage service into an existing Snowflake table.
    • Added the Google Cloud Storage Unload component, which writes data from a Snowflake table or view into a specified Google Cloud Storage (GCS) bucket in your specified file format (CSV, JSON, or Parquet).
    • Added Google Cloud Storage as a staging platform to all "Query" components (Facebook Query, Salesforce Query, Shopify Query, and so on).
  • Added Google Cloud Storage as a source and destination to the Data Transfer component.
  • Added Google Cloud Storage as a source and destination to the Excel Query component.
  • Added Google Cloud Storage as a source and destination to the File Iterator component.

Note

To interact with Google Cloud Platform services, you'll need to set up GCP cloud credentials and associate your credentials with your environment.

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.95.1 to 2.95.21.
  • Fixed an issue where the thread cap for table writers could be reached when capturing changes on a large number of tables at once and cause the agent to stop streaming.
  • The Oracle connector will now gracefully handle skipping transactions when the retention.ms property is configured.
  • Library updates and security fixes.

July 18πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Updated the font across the Data Productivity Cloud to a more modern, readable font.
  • Added HTML as a supported data format to the Chunk Text component.
  • Added the ability to use multiple Azure Key Vault vaults to store secrets. When you add a new secret definition, you can choose which of your key vaults the secret is stored in.

July 16πŸ”—

DesignerNew features πŸŽ‰

  • Added a JDBC orchestration component for the Hybrid SaaS offering. The JDBC component allows you to run queries on a JDBC-compatible data source that you have a driver for. You can upload your drivers to the Data Productivity Cloud, and so connect to any sources that Matillion doesn't provide drivers for as standard.
  • Added a Chunk Text orchestration component, which performs pushdown text chunking using a Python user-defined function (UDF) in Snowflake via the computational power of your Snowflake warehouse. You can choose text or Markdown as your data format.
  • Added the following Flex connectors for developing data pipelines:
  • Added a Refresh Table orchestration component, which invalidates cached entries of a specified table or view, including both data and metadata. Databricks projects only.

July 4πŸ”—

DesignerNew features πŸŽ‰


July 2πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§


JuneπŸ”—

June 25πŸ”—

DesignerNew features πŸŽ‰


June 20πŸ”—

CDCImprovements πŸ”§

  • Updated the CDC agent version from 2.93.8 to 2.95.1.
  • Added an option to map date and time source columns to the corresponding Snowflake data types when using Snowflake as a target.
  • Changed the MTLN_CDC_LAST_COMMIT_TIMESTAMP column when using Snowflake as a target with the Change Log transformation type.
  • Library updates and security fixes.

June 18πŸ”—

DesignerNew features πŸŽ‰

  • Added a component action bar that will be displayed when you click on a component in the canvas. The action bar currently supports copying the component and deleting the component.
  • Updated data sampling for transformation components to include a drop-down menu where you can set the number of rows sampled, for example 1, 10, 250, 1000. The default is 25 rows.
  • Added an Append To Grid component, which lets you append or prepend data to a pre-existing grid variable in the Data Productivity Cloud. Running this component can add rows to existing columns, but doesn't add columns to the grid.
  • Added the following Flex connectors for developing data pipelines:

June 10πŸ”—

DesignerNew features πŸŽ‰


June 6πŸ”—

DesignerNew features πŸŽ‰

  • Added support for GPT-4o to the OpenAI Prompt component. When this model is selected, the Image input type is available. Images should be in either Base64 format or a direct, public URL of the image. If you're feeding images in Base64 format, you will need to add data:image/{format_of_your_image};base64, before the encoded image.
  • Added a Volume to Delta Table orchestration component for Databricks, which lets users transfer data from a pre-existing volume in Databricks in to a Delta Lake table without replacing or deleting any existing data.
  • Added a Convert String To Struct transformation component, which takes a string input and converts it to a struct data type.
  • Renamed the Extract Nested Data component to Extract Structured Data for Databricks. This transformation component unpacks arrays of structured data into columns and rows of data in a table. Any existing pipelines will still display an Extract Nested Data component.
  • The Query Result To Grid component is now available for Databricks projects.
  • Added the following Flex connectors for developing data pipelines:
  • A user's account ID is now visible in the user menu (top right of the Data Productivity Cloud UI), to help users easily identify their account.

MayπŸ”—

May 31πŸ”—

Custom ConnectorNew features πŸŽ‰

  • Added a Script paging option for connector pagination. Paging scripts give you complete control over response pagination in Custom Connector.

May 30πŸ”—

DesignerNew features πŸŽ‰

  • Maia (formerly Copilot) for the Data Productivity Cloud is now available in public preview for transformation pipelines in Snowflake projects. Maia lets you prompt generative AI to create data pipelines, improving your efficiency in managing and processing data. Read our docs to get started:
  • The Query Result to Scalar component is now available for Databricks projects. This orchestration component allows you to query a table and return a scalar value. This value can then be mapped to a project or pipeline variable for use in other pipeline components.

May 23πŸ”—

DesignerNew features πŸŽ‰


May 21πŸ”—

DesignerNew features πŸŽ‰


May 16πŸ”—

DesignerNew features πŸŽ‰


May 14πŸ”—

DesignerNew features πŸŽ‰

  • The AI prompt components are now available for Amazon Redshift projects:
  • Added two new connectors:
    • Gmail, which lets you query the Gmail API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
    • SurveyMonkey, which lets you query the SurveyMonkey API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
  • Added the following components for Snowflake projects:
  • The Git menu can now be accessed by clicking the active branch.
  • The Data Transfer component now supports SFTP as a target.

May 10πŸ”—

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.93.1 to 2.93.8.
  • Improved messaging and logging for some scenarios.
  • Library updates and security fixes.

May 2πŸ”—

DesignerGA release πŸŽ‰

  • The Python Pushdown component is now generally available for Snowflake users. This component lets you execute a Python script using the Snowpark service in your Snowflake account.

May 1πŸ”—

DesignerNew features πŸŽ‰

  • Amazon Redshift is now generally available as a cloud data warehouse in the Data Productivity Cloud.

AprilπŸ”—

April 30πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Added a Microsoft Exchange component, which lets you query the Microsoft Exchange API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
  • Added a ServiceNow Query component, which lets you query the ServiceNow API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
  • Added logging to Custom Connector and to Flex and custom connectors used in Designer. You can choose one of five logging options: Error, Warn, Info, Trace, or Debug.
  • Improved component sampling to enable column resizing.

April 24πŸ”—

CDCImprovements πŸ”§

  • Updated Streaming agent version from 2.92.3 to 2.93.1.
  • Removed a number of unnecessary dependency overrides and updated some minor dependencies.

April 18πŸ”—

DesignerNew features πŸŽ‰


April 17πŸ”—

DesignerNew features πŸŽ‰

  • Maia (formerly Copilot) is now available as an individual assignable service in the Edit User menu.
    • Super Admins can enable and disable Maia access for all users in an account.
    • User Admins can enable and disable Maia access for all User Admins and Users in an account (not the Super Admin).

April 11πŸ”—

DesignerImprovements πŸ”§

  • Added an improvement to connecting a GitHub repository to a project. Now, when you connect an existing repository, any active branches will be cloned into Designer and selectable from the Branches tab. Previously only the main branch was cloned. If any of the active branches in your repository only include files that are not Data Pipeline Language (DPL) files, those branches will appear as empty while in Designer until you create a pipeline.

April 4πŸ”—

DesignerNew features πŸŽ‰


April 2πŸ”—

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.92.0 to 2.92.3
  • Library updates and security fixes
  • Improved agent logging around data handling edge cases to help with debugging

MarchπŸ”—

March 28πŸ”—

DesignerNew features πŸŽ‰

  • Added the ability to export pipelines and import them into a new project.
  • Added the following connectors for developing data pipelines:

March 20πŸ”—

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.91.5 to 2.92.0.
  • Added optional STREAMING_AGENT_DEBUG_DISABLE_COMPRESSION Environment Variable to disable compression when uploading to Snowflake Staging area.
  • Library updates and security fixes.

March 19πŸ”—

DesignerNew features πŸŽ‰

  • The following AI components are now generally available to all Data Productivity Cloud users in Designer:
  • AI Note is also now generally available, which lets you invoke generative AI to annotate your data pipelines using notes.
  • The project details page in the Data Productivity Cloud now displays project metadata including the agent type, the cloud data platform, and the type of Git repository (Matillion hosted repository or customer's GitHub repository).

March 14πŸ”—

DesignerNew features πŸŽ‰

APINew Endpoints πŸŽ‰

Projects

Method Endpoint Description
GET /v1/projects List all projects

Environments

Method Endpoint Description
GET /v1/projects/{projectId}/environments Lists all environments in a project

Pipelines

Method Endpoint Description
GET /v1/projects/{projectId}/published-pipelines Lists all published pipelines in a project

Pipeline execution

Method Endpoint Description
POST /v1/projects/{projectId}/pipeline-executions Execute a published pipeline
GET /v1/projects/{projectId}/pipeline-executions/{pipelineExecutionId} Returns the status of a pipeline
PATCH /v1/projects/{projectId}/pipeline-executions/{pipelineExecutionId} Terminate a pipeline
GET /v1/projects/{projectId}/pipeline-executions/{pipelineExecutionId}/steps Represents a single execution of a pipeline triggered from a step of another pipeline

March 13πŸ”—

DesignerNew features πŸŽ‰

  • Added a Text mode option to components and grid variables, to facilitate rapid input and editing of multiple properties.

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.90.16 to 2.91.5.
  • Increased the Streaming agent JVM memory allocation to 60%.
  • Fixed a bug with the Direct to Snowflake flow, in which a change to the scale of a numeric column of a table in the source database would not trigger a schema change in the target Snowflake table.
  • Library updates and security fixes.

March 12πŸ”—

DesignerNew features πŸŽ‰

  • Added support for lineage for transformation pipelines. With lineage, you get a visual representation of each transformation, which can make understanding the evolution of your data faster and easier. Lineage is currently in public preview and includes:
    • Transformation lineage at runtime.
    • Table-level lineage.
    • Column-level lineage.
    • Table lists.
    • Table metadataβ€”column information and data types.
  • Added the following Flex connectors for developing data pipelines:

March 5πŸ”—

DesignerImprovements πŸ”§

  • Updated the AI Note feature with the ability to refine notes. When you click Refine when creating a note, you can elaborate on the note (adding detail), shorten the note's length and detail, or regenerate the note.
  • Added a Load Strategy parameter to connectors that support storage-only outputs. Choose between appending files in folder and overwriting files in folder, based on exact structure matching.

FebruaryπŸ”—

February 29πŸ”—

DesignerNew features πŸŽ‰

  • Added the ability to access the Snowflake Session object from the Python Pushdown component, to allow reading from and writing to the Snowflake database from within a Python script.
  • Added the following Flex connectors for developing data pipelines:
  • Added Azure Blob Storage as a data input type for the Excel Query component.
  • Added Azure Blob Storage as a source and target type for the Data Transfer component.

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.90.16 to 2.90.35.
  • Logging improvements.
  • Library updates and security fixes.

February 28πŸ”—

DesignerNew features πŸŽ‰

  • Added the Query Result to Grid component, which queries a table and returns rows of data that are loaded into a predefined grid variable.

February 27πŸ”—

DesignerNew features πŸŽ‰

  • Improved user interface for easier pipeline creation. Read Designer UI basics for details.

February 21πŸ”—

CDCNew features πŸŽ‰

  • Added Snapshot Overview to display improved metrics when a pipeline is snapshotting.

February 20πŸ”—

DesignerNew features πŸŽ‰

  • Added Azure Blob Storage as a data input type for the File Iterator component.

February 9πŸ”—

DesignerNew features πŸŽ‰


February 8πŸ”—

DesignerNew features πŸŽ‰

  • Added the Mailchimp component, which lets you query the Mailchimp API to retrieve data and load it into a table. You can then use transformation components to enrich and manage the data in permanent tables.
  • Added the following Flex connector for developing data pipelines:
  • Added a toggle to the Schedules tab where users can easily enable or disable their schedules.

February 6πŸ”—

DesignerNew features πŸŽ‰

  • Added the Azure SQL component, which lets you run an SQL query on an Azure SQL database and copy the results to a table. You can then use transformation components to enrich and manage the data in permanent tables.

February 1πŸ”—

DesignerNew features πŸŽ‰

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.89.4 to 2.90.16.
  • Library updates and security fixes.

JanuaryπŸ”—

January 25πŸ”—

DesignerNew features πŸŽ‰

  • Added a Commands for dbt Core component. This component empowers you to run dbt commands as part of your orchestration pipelines.

Data Productivity CloudImprovements πŸ”§

  • Improved the ☰ navigation menu, adding links to manage your custom connectors and agents.

January 23πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Added a Python Pushdown component to Designer. This component lets you execute a Python script using the Snowpark service in your Snowflake account. This component is currently available as a public preview.
  • Improved the schedule feature to include a Standard mode and an Advanced mode.
    • Standard mode lets you define intervals with the units Days (default), Weeks, Hours, and Minutes.
    • Advanced mode retains the Cron expression scheduling feature.

January 17πŸ”—

CDCNew features πŸŽ‰

  • Added option to configure or disable the table name prefix when using Snowflake as a target.

January 15πŸ”—

DesignerNew features πŸŽ‰

  • Added the following Flex connector for developing data pipelines:

January 11πŸ”—

DesignerNew features πŸŽ‰

  • Added the following Flex connector for developing data pipelines:

Improvements πŸ”§

  • Fixed an issue where using the Rename component would reorder columns.
  • Fixed an issue where using the Convert Type component would reorder columns.

January 9πŸ”—

DesignerNew features πŸŽ‰

  • Added the following Flex connector for developing data pipelines:

January 5πŸ”—

DesignerNew features πŸŽ‰

  • Added the following Flex connectors for developing data pipelines: