Skip to content

2023 changelogπŸ”—

Here you'll find the 2023 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.


DecemberπŸ”—

December 12πŸ”—

DesignerNew features πŸŽ‰

  • Added Google Cloud Storage as a "direct to storage" option when using Flex connectors.
  • Added REST API service support to SharePoint Query. This service requires OAuth authentication.
  • Added the following Flex connector for developing data pipelines:

December 11πŸ”—

CDCImprovements πŸ”§

  • Updated CDC agent version from 2.88.3 to 2.89.4.
  • Added option to disable the table name prefix when using Snowflake as a target.
  • Fix support for Snowflake when using the QUOTED_IDENTIFIERS_IGNORE_CASE property.
  • Additional logging when unable to connect to the Matillion Platform.

December 8πŸ”—

DesignerNew features πŸŽ‰


December 7πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • New Update Scalar component. This component allows you to change the value of any pipeline variable or project variable during pipeline execution, to a new value decided at the time the component is configured.
  • New Print Variable component. This component prints the value that a pipeline or project variable has at that specific point in the pipeline execution.
  • Added the following Flex connector for developing data pipelines:
  • Improved the Git Merge workflow to let users merge changes that have been pushed to a remote branch into your local branch. This improvement empowers users to validate and test changes before pushing merged changes to the remote repository.

NovemberπŸ”—

November 28πŸ”—

DesignerNew features πŸŽ‰

  • Added Azure as a staging location to all connectors.
  • Added the following Flex connector for developing data pipelines:

Data Productivity CloudNew features πŸŽ‰

  • Added a Recent pipelines menu to the Data Productivity Cloud UI. The Data Productivity Cloud will dynamically display your most recent data pipelines, with identifying metadata such as the project platform, Git branch, and pipeline folder name.

November 21πŸ”—

DesignerNew features πŸŽ‰


November 14πŸ”—

DesignerNew features πŸŽ‰

  • Added the following Flex connector for developing data pipelines:

November 8πŸ”—

BatchNew features πŸŽ‰Improvements πŸ”§

  • Improved the Set frequency page of the batch pipeline wizard, including adding an advanced mode with support for Quartz cron expressions.

CDCImprovements πŸ”§

  • Fixed issue where data would not load into tables for pipelines with Snowflake as a destination.

November 7πŸ”—

DesignerNew features πŸŽ‰

  • Added the following connectors for developing data pipelines:

  • Workday

  • Pipedrive

November 2πŸ”—

DesignerNew features πŸŽ‰

  • Added the Workday Custom Reports connector to Designer. This component uses the Workday RaaS (Reporting as a Service) API to retrieve and store dataβ€”such as employee, financial, and business-related dataβ€”from Workday, to be loaded into a table.

November 1πŸ”—

CDCNew features πŸŽ‰Improvements πŸ”§

  • Added Copy Table with Soft Deletes as a transformation type for pipelines with Snowflake as a destination.
  • Updated CDC agent version from 2.87.8 to 2.88.2.
  • Improved upload speed when writing to a Snowflake staging table.

OctoberπŸ”—

October 31πŸ”—

DesignerNew features πŸŽ‰

  • Added the following connectors for developing data pipelines:
  • Added Azure Storage as a storage destination for the following connectors:
    • ActiveCampaign
    • Amplitude
    • Anaplan
    • Braze
    • Brevo
    • Chargebee
    • CircleCI
    • Concord
    • Confluence
    • Datadog
    • Delighted
    • Eventbrite
    • Freshdesk
    • Gong
    • Klaviyo
    • LaunchDarkly
    • PagerDuty
    • PayPal
    • Pendo
    • Productboard
    • Recurly
    • Slack
    • Smartsheet
    • Snyk
    • TikTok
    • Toggl
    • Twilio

October 27πŸ”—

DesignerNew features πŸŽ‰


October 26πŸ”—

DesignerImprovements πŸ”§

  • Updated the Add Components panel with clickable filter buttons under the search bar. Click a filter button to only display components tagged for that filter. You can activate multiple filter buttons.
  • Updated the Add component call-to-action (CTA) dialog to include an output connection setting. When adding a component to the canvas, you can now confirm whether that component should be run:
    • On success (green connection line).
    • Unconditionally (grey connection line).
    • On failure (red connection line).
  • You can now sample Query components in orchestration pipelines. To do this:
    • Click on a validated Query component in your orchestration pipeline, e.g. Salesforce Query.
    • Click the Sample tab.
    • Click Sample data.
  • Added row count when sampling components in a transformation pipeline. To view the total rows:
    • Click on a validated transformation component, e.g, Table Input.
    • Click the Sample tab.
    • Click the refresh button.

Custom ConnectorNew features πŸŽ‰

  • The ability to import pre-existing connectors using the Custom Connector import feature. This feature supports two types of connectors:
    • Custom connectors that have been previously exported.
    • Matillion ETL connectors.

October 18πŸ”—

DesignerNew features πŸŽ‰


October 12πŸ”—

DesignerNew features πŸŽ‰

  • Added the Assert View component, which lets you verify that certain conditions are true of a view, or otherwise stop the query.

October 10πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Updated the Add Components panel to un-nest components. We believe this will make finding the component you're looking for much easier. Components are listed alphabetically and include a descriptor keyword, e.g. "Connectors" or "Flow". You can use these descriptors to filter out components that don't match that keyword. For example, typing "flow" in the search bar will return only the Flow components (And, End Failure, End Success, If, Or, Retry, Restart). The search also includes a number of synonym keywords to help filter components. For example, typing "logic" into the search bar will also return the Flow components.
  • Updated the pipeline canvas to include a + call-to-action button. When you select a component on the canvas, the + button will appear to the right. Clicking this button will open the Add component dialog. This dialog works similarly to the Add Component panel.
    • Components are listed alphabetically.
    • Descriptor and synonym keywords help you refine your search.
    • Clicking a component in this dialog will add that component to the canvas in place of the + button and close the dialog. The new component will be connected to the previously selected component.
    • This newly added component will become the selected component, meaning the + button will shift to the right.
  • Added the Table Update component, which lets you update a target table with a set of input rows. The rows to update are based on matching keys.

October 5πŸ”—

DesignerImprovements πŸ”§


SeptemberπŸ”—

September 20πŸ”—

DesignerImprovements πŸ”§

  • The Data Transfer component now supports SFTP as a source type.

September 19πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Oracle is now provided as a driver and a supported database type in the Database Query and RDS Query components.
  • Added support for multi-line secret values for Full SaaS, Matillion hosted projects when creating secrets and secret definitions.
  • Added an Add schedule call-to-action button to the Designer UI to easily navigate from Designer to the Create a new schedule menu.
  • Added the ability to associate newly created cloud credentials to an environment in a single workflow. You can associate credentials from multiple providers, but only one set of credentials for each cloud provider. For example, you can associate both AWS and Azure credentials, but not two different AWS credentials.
  • Added a "hard reset" functionality to Git in Designer. Hard reset lets users reset their branch to the last local commit.

September 18πŸ”—

DesignerNew features πŸŽ‰


September 14πŸ”—

Data Productivity CloudNew features πŸŽ‰Improvements πŸ”§

  • Registered Data Productivity Cloud customers with Account and User Administrator privileges can edit the Account name and Subdomain name for their account.
  • Data Loader Batch pipeline runs are now observable from the Pipeline Observability dashboard.
  • Pipeline error messages are now displayed at the top of the Pipeline run details page.

September 12πŸ”—

CDCNew features πŸŽ‰Improvements πŸ”§

  • Updated CDC agent version from 2.83.5 to 2.87.8.
  • Added "Copy Table With Soft Deletes" as a transformation type for pipelines with Snowflake as a destination.
  • Fixed an issue where Snowflake role names could not contain non-standard characters.
  • Improved connection reuse for Snowflake as a destination.

September 7πŸ”—

DesignerNew features πŸŽ‰


AugustπŸ”—

August 31πŸ”—

DesignerNew features πŸŽ‰

CDCNew features πŸŽ‰Improvements πŸ”§

  • Updated CDC agent version from 2.83.5 to 2.87.1.
  • Added on-demand snapshots for pipelines with Db2 for IBM i as a source.
  • Added Change Log as a transformation type for pipelines with Snowflake as a destination.

August 30πŸ”—

CDCImprovements πŸ”§

  • Added the in-client Help bar with contextual documentation to the "Create Pipeline" pages.

August 24πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Added the following components:
  • Replaced the + button with an Add button that opens a context menu for creating pipelines and folders.

August 23πŸ”—

CDCImprovements πŸ”§

  • Updated CDC Agent version from 2.81.12 to 2.83.5.
  • Fixed issue where on-demand snapshots could fail to trigger.
  • Additional validation of the stage format for the Snowflake destination.
  • Added rs_id and ssn metadata fields to the Oracle change records.

August 22πŸ”—

DesignerNew features πŸŽ‰

  • Added the following components:
  • Added a Getting started wizard for users who load into an instance of Designer with no existing pipelines.

August 16πŸ”—

DesignerNew features πŸŽ‰


August 9πŸ”—

DesignerNew features πŸŽ‰

  • Added new item Invite your Teammates to the help widget's Task checklist, to facilitate collaboration.

August 3πŸ”—

DesignerNew features πŸŽ‰Improvements πŸ”§

  • Added the following components:
  • Added the ability to organize pipelines into folders.
  • Tooltips added to the Designer UI for type of pipeline and type of component.
  • The Designer product tour has been replaced with a checklist to help users with creating a pipeline, adding components, and running a pipeline.

CDCNew features πŸŽ‰Improvements πŸ”§


JulyπŸ”—

July 27πŸ”—

DesignerImprovements πŸ”§

  • Improved the code editor to provide features such as:
    • Syntax highlighting.
    • IntelliSense.
    • Validation support for SQL, Python, and Bash.
    • Command palette menu via F1.
    • Visual Studio Code shortcut keys, e.g. CMD/CTRL for multi-cursor editing.

July 25πŸ”—

DesignerNew features πŸŽ‰

CDCImprovements πŸ”§

  • Updated CDC Agent version from 2.77.1 to 2.81.6.
  • Added validation for Oracle Signal Table on PDBs.
  • Changed "Unavailable Value" placeholderβ€”read PostgreSQL connector to learn more.

July 19πŸ”—

CDCNew features πŸŽ‰

  • Enabled on-demand snapshotting functionality for Oracle, SQL Server, and MySQL databases.

July 18πŸ”—

DesignerNew features πŸŽ‰

  • Added the SharePoint Query component. This component's driver currently supports:
    • Windows SharePoint Services 3.0
    • Microsoft Office SharePoint Server 2007
    • SharePoint Server 2010
    • SharePoint Server 2013
    • SharePoint Server 2016
    • SharePoint Online

CDCNew features πŸŽ‰Improvements πŸ”§

  • Updated CDC Agent version from 2.68.1 to 2.76.1.
  • Updated CDC Agent version from 2.76.1 to 2.77.1.
  • Added on-demand snapshotting functionality for Oracle, SQL Server, and MySQL databases.

July 4πŸ”—

DesignerNew features πŸŽ‰

  • Added the Data Transfer component. Users can transfer files from a chosen source to a chosen target using common network protocols.
    • Currently supported data sources include: FTP, HTTP, HTTPS, Amazon S3, and Windows Fileshare.
    • Currently supported targets: Amazon S3 and Windows Fileshare.

JuneπŸ”—

June 20πŸ”—

CDCNew features πŸŽ‰


June 14πŸ”—

CDCImprovements πŸ”§

  • CDC agent version updated to 2.67.4.
  • Updates to the DB2 for IBM i connector.
    1. Pipelines using this connector will now fail on startup if any of the journals for the tracked tables do not have IMAGES(*BOTH) configured.
    2. Pipelines using this connector will now reconnect to the DB2 instance between snapshotting and streaming. This should rectify issues where the connection times out after a long-running snapshot.

AprilπŸ”—

April 12πŸ”—

BatchNew features πŸŽ‰


April 5πŸ”—

BatchNew features πŸŽ‰

  • Added support for POST API methods to Matillion Custom Connector.
  • Added BODY parameter support for Matillion Custom Connector.

CDCImprovements πŸ”§

  • Improved performance related to journal lookup for the Db2 for i connector.

MarchπŸ”—

March 15πŸ”—

CDCImprovements πŸ”§

  • Improved error handling during the pipeline startup process.
  • Db2 for IBM i connector: Fix for the SQL catalog and system name differences.

Note

This connector only works for a single system. Remote system journaling is not supported.


March 1πŸ”—

CDCImprovements πŸ”§

  • Users can now choose a schema before choosing a table when building a CDC pipeline. Read Create a CDC Pipeline to learn more

FebruaryπŸ”—

February 27πŸ”—

CDCImprovements πŸ”§

  • Introduced a starting status to pipelines that are starting up.
  • Made the request to stop a pipeline non-blocking.
  • Db2 for i connector: Fix handling of tables where the table name and system name differ.

February 13πŸ”—

CDCImprovements πŸ”§


February 1πŸ”—

BatchNew features πŸŽ‰

  • Users can now stop a pipeline. Do this from the Dashboard page or the Pipeline Details page.
  • The Stop pipeline button will display the current status of a running pipeline.

JanuaryπŸ”—

January 25πŸ”—

BatchNew features πŸŽ‰

  • Users can now observe the URL being queried, so they know the endpoint they're working with. This feature is specific to the Custom Connector pipeline.

CDCImprovements πŸ”§

  • Added an improvement so that users will be alerted via email if their CDC pipeline fails.
  • Streaming agent deployment approach has improved. Users can now quick-create an agent within AWS, using the link from the Data Loader UI (it will pre-fill values into the template for you). Read CloudFormation ECS Fargate Advanced Template for more information.
  • Fixed an issue that prevented data from being properly erased while creating destinations. Now, when editing one destination after another or adding new destinations, users should be able to establish pipelines with cleared data.

January 20πŸ”—

CDCImprovements πŸ”§

  • Users can now manually reset their own key pair without needing to contact Matillion support. Read Platform Keys for more information.

January 18πŸ”—

BatchImprovements πŸ”§

  • Users can now download error logs for their pipelines as a .txt file.
  • Improved validation when creating and editing pipelines by adding per-field validation.
  • Users can now set table names when creating a pipeline (as opposed to the table name being assigned on the source name.)

CDCImprovements πŸ”§

  • Column names are now sanitized to ensure conformity with the Avro file format rules.
  • Fixed an issue where special characters in table names would cause issues in the pipeline configuration.
  • Improvement added to now log an error if your Organization ID or Agent ID values are not in the correct format.