1.69 release notes

Overview

Below, you can find release notes for Matillion ETL version 1.69.x

Please also familiarize yourself with the tech note Updating to version 1.69 and above if you wish to upgrade to version 1.69.x of Matillion ETL.

Tech notes

Matillion ETL for Redshift

If you're experiencing out-of-memory (OOM) exceptions or high CPU usage since updating to version 1.68.7 or above, read Tech note - Redshift RingBuffer exceeding expected limits for a workaround.

Matillion ETL version 1.69.1 (interim release)

2023 - March 31st

New features and improvements

Added a drop-down menu for selecting long-term support (LTS) version updates. If you're running the latest LTS version, there will be no available version to update to.
A scheduled job that misses its scheduled time (by default, within 30 seconds of its scheduled time) is declared a misfire. When a misfire is detected, it will now show in the task history decorated with a new "misfire" icon. This should make it clearer when misfires are detected and what that means for misfire handling. These events will not appear when the Ignore Misfire option is selected when creating a schedule.

Bug fixes

Fixed an issue where application performance could be limited by an excessive message task queue size.
Fixed an issue where API Extract requests were not handling null responses as expected.
Fixed an issue where a feature flag was stopping stateless authentication from working as expected.

Driver updates

Salesforce API driver updated 22.0.8356.0 → 22.0.8466.0

Matillion ETL version 1.69.0 (major release)

2023 - March 1st

This update is a migration update. Read Migrate to learn more about migration in Matillion ETL. There will not be an in-place update available for this update.

System-level changes

Updated the Java runtime environment from Java 8 to Java 17, together with all dependent code and libraries.
Updated Matillion ETL from Apache Tomcat version 8.5.75 to version 10.1.2.
- Users will need to use the command tomcat instead of tomcat8 where the command is applicable.
Updated Matillion ETL operating system from CentOS Linux 7 to CentOS Stream 8.
Upgraded the JavaScript engine used by Matillion ETL from Nashorn to GraalVM. Please read Migration Guide from Nashorn to GraalVM JavaScript for more information.

Note the following warnings:

A number of deprecated connector components have been removed in this release, since they were not compatible with Java 17. All connector components have updated versions available. If you experience a break in any jobs with third-party connectors—for example, Facebook Query—please try adding a fresh instance of the component to your jobs.
Users consuming legacy JDBC drivers may find these drivers to be incompatible with Java 17. We recommend that you find upgrades to these drivers, or specify additional compatibility options in the setenv.sh file.
If your Matillion ETL jobs include custom Bash scripts, you may need to confirm that these still work as expected in CentOS Stream 8.
Ensured that Matillion ETL is secured against the following cybersecurity vulnerabilities (CVEs):
- CVE-2018-10237
- CVE-2020-1945
- CVE-2020-14338
- CVE-2021-28165
- CVE-2021-36374
- CVE-2022-34169
- CVE-2022-29546
- CVE-2019-10241
- CVE-2012-0881
- CVE-2020-8908
- CVE-2022-3509
- CVE-2014-7810
- CVE-2019-10247
- CVE-2022-3171
- CVE-2021-22569
- CVE-2021-28169
- CVE-2022-42004
- CVE-2022-2047
- CVE-2022-23437
- CVE-2021-34428
- CVE-2021-36373
- CVE-2022-42003
- CVE-2020-1938
- CVE-2022-24839
- CVE-2021-22573
- CVE-2017-12617
- CVE-2013-4002

New features and improvements

Snowflake

Added a Credentials authentication method to the S3 Load and S3 Unload components.
- When Authentication Method is set to Credentials, the new authentication method parameter will display as initially set to [Environment Default].
- Clicking into the Credentials parameter offers a drop-down list of configured credentials for your cloud provider (AWS, Azure, GCP). Click Manage to open Manage Credentials to add or edit credentials.
- If you set up new credentials, return to the Credentials drop-down and your new credentials will be available for selection.

Snowflake and Amazon Redshift

Improved the Metadata API endpoint responses to include source fully qualified names (FQN) in the metadata payload when using Matillion ETL Query components (for example, Database Query, Salesforce Query) to enhance lineage of Matillion ETL jobs. The FQN will include schema information; however, if no schema can be selected, the environment default schema will be returned.

Delta Lake on Databricks

Added support for Databricks Unity Catalog to Matillion ETL Query components (for example, Salesforce Query) and any other orchestration or transformation component where the user must select a database. Each of these components now includes a Catalog parameter that will display a drop-down list of Databricks catalogs tied to the user's account.

Deprecation

The Matillion ETL Change Data Capture feature is being deprecated from version 1.71. This means that this feature will not be available for new users from this version. However, existing users of Matillion ETL Change Data Capture will still be able to access and use this feature up to December 2024.

Bug fixes

Fixed an issue where the Data Transfer component encountered a "pipe closed" error when transferring a zip file and unpacking it to the target location.
Fixed an issue where the Data Transfer component did not timeout during an SFTP file transfer.
Fixed an issue where a duplicate private key error would occur when the Table Maintenance parameter on the RDS Bulk Output component was set to anything other than None when connecting to a Postgres database.

Driver updates

AWS Redshift JDBC driver updated 1.2.55.1083 → 2.1.0.9
Google Ads API driver updated 22.0.8353.0 → 22.0.8432.0
Google Ad Manager API driver updated 22.0.8257.0 → 22.0.8389.0
Google Sheets API driver updated 20.0.7550.0 → 22.0.8434.0
LinkedIn API driver updated 21.0.8011.0 → 22.0.8431.0
LinkedIn Ads API driver updated 21.0.8011.0 → 22.0.8389.0
Marketo API driver updated 22.0.8322.0 → 22.0.8384.0