Skip to content

MySQL connector

The Matillion CDC agent's MySQL connector can capture changes within tables that are selected for monitoring in a pipeline. These changes are then written to your desired cloud storage location.

Supported configurations

Data Loader supports the following MySQL configurations:

Supported Category Supported Values
Database version 5.7 and above
Database driver version 8.0.29

Prerequisites

  • MySQL version is 5.7 or higher.
  • To check this version, enter the following command in your MySQL instance: mysql -V
  • MySQL database port (usually 3306).
  • SELECT privileges must be granted to the database user.
  • Binary log (BinLog) replication must be enabled.

For the CDC Agent to use MySQL

  • You will need to download appropriate MySQL drivers (Version - 8.0.29).
  • You will need to push the driver package into your cloud storage location.
  • The CDC Agent needs access to the S3 bucket to be able to download the package.

  • Provide S3 bucket and S3 key values using the following environment variables to download the files:

    • MYSQL_DRIVER_S3_BUCKET
      • Example: MYSQL_DRIVER_S3_BUCKET="bucket-name"
    • MYSQL_DRIVER_S3_KEY
      • Example: MYSQL_DRIVER_S3_KEY="path/to/driver.jar"

  • You will need to provide one of the following environment variables:

    • MYSQL_DRIVER_SHA512=<value>
      • Where the value is the SHA512 digest of the driver jar file.
    • Example:MYSQL_DRIVER_SHA512="3eabd70f9a947918f434a44923a8e3ff4c3fbc93e6c90f4992c94d804860ef3d09f09fb4fbf905d53a39b51aab965ecfd65f2ff2aa105387b3c8f49c18d7713c"
  • DISABLE_MYSQL_DRIVER_SHA512="true"
    • Setting this value to true will disable checking the .jar files SHA512 digest.

:::warning{title='Warning'} - Disabling the SHA512 variable will skip the SHA512 check. This is not recommended. - If the MYSQL_DRIVER_S3_BUCKET and MYSQL_DRIVER_S3_KEY environment variables are set, then the agent will attempt to download the file. If the download fails, the agent will fail to start. - If the SHA512 value doesn't match the driver provided, the agent will fail to start. :::

Connect to MySQL

When selecting MySQL as a source during pipeline creation, you will be required to provide the following information:

Property Description
Pipeline Name The name for your new pipeline. Must be unique.
Server address The server address of your MySQL database.
Port The port number used to access your MySQL database.
Username The username used to log in to the specified database.
Secret Provider Choose the provider for your secrets manager that contains your database password.
Secret Name Enter the name of the secret that corresponds to your database password. Read Manage secrets to learn how to add a named secret to your provider's secret manager.