Uploading external drivers to the agent

The AWS and Azure hosted agents natively include drivers for a large number of third-party connectors. There are, however, certain external drivers that aren't supplied by default.

For drivers that aren't supplied by default, you can upload your own drivers to the agent at runtime by specifying a storage location in your infrastructure that will hold the driver files.

Note

Adding additional unsupported drivers to the agent will result in increased startup time with no guarantee that they will work, so we don't recommend doing this. Supported drivers are listed below.
There is no support for importing external drivers to Matillion-hosted agents in a Full-SaaS solution.
When adding new drivers, an agent restart is required to recognize the configuration. Once the agent has restarted, you can choose the driver from the Database Type property in the component.

Valid external driver file types

Only the .jar and .so file types will be recognized as external drivers. Any other file types in the drivers storage location will be ignored by the agent.

AWS-hosted agents

To load external drivers into an AWS-hosted agent, store the driver files in an S3 bucket that's in the same account space as the agent and accessible by the agent.

The IAM role for the account the agent is hosted in will need to have at least the following permissions to access S3 buckets:

s3:ListAllMyBuckets
s3:ListBucket
s3:GetObject
s3:GetBucketLocation

These will be applied automatically if you created the agent using the provided CloudFormation template.

The agent must have the EXTERNAL_DRIVER_LOCATION environment variable added and set to the location of the driver files in the S3 bucket—for example, s3://driver-bucket/my-drivers. You can omit s3:// from this, as the agent will assume the connection is to an S3 bucket and automatically use the correct protocol. For more information on setting this, read Optional agent parameters.

When you launch a new agent with the CloudFormation template, the configuration page in the AWS console will have a field named External Driver Location. Enter the path for the driver location here.

If you are updating an existing agent, you will need to create a new revision of the task definition in use, and add the EXTERNAL_DRIVER_LOCATION environment variable with the appropriate path. Then, restart the service using the new task definition.

Azure-hosted agents

To load external drivers into an Azure-hosted agent, store the drivers in an Azure Blob container in a storage account that is in the same resource group as the agent and accessible by the agent.

The storage account's managed identity associated with the agent's container will need to have at least the following permissions:

Storage Account Contributor
Storage Blob Data Contributor
Storage Blob Data Reader

These will be applied automatically if you create the agent using the provided ARM template.

The agent must have the EXTERNAL_DRIVER_LOCATION environment variable added and set to the location of driver files in the Azure Blob container—for example, https://mystorageaccount.blob.core.windows.net/my-drivers. You can omit https:// from this, as the agent will assume the connection is to a Blob container and automatically use the correct protocol. For more information on this, read Optional agent parameters.

When you launch a new agent with the ARM template, the configuration page in the Azure Portal will have a field named External Driver Location. Enter the path for the driver location here.

If you are updating an existing agent, you will need to edit and deploy the existing container and add the EXTERNAL_DRIVER_LOCATION environment variable, with the appropriate path. Then, create the new revision.

Drivers needed for components

This section lists the Data Productivity Cloud components that need external drivers to be uploaded to the agent.

SAP ODP

For the SAP ODP component, two driver files are required:

sapjco3.jar
libsapjco3.so

You can obtain these drivers as a single ZIP file from Download SAP Java Connector 3.1 SDK, selecting Linux for AArch64 compatible processors. Unzip the file and place the drivers in the storage location you specified in EXTERNAL_DRIVER_LOCATION, as described above. Do not change the driver file names.

JDBC

For the JDBC component, you will need to upload any driver files required by the data source you are using. Consult the data source's documentation to discover which drivers you need and how to obtain them. Place the drivers in the storage location you specified in EXTERNAL_DRIVER_LOCATION, as described above.

This component also requires a driver manifest file to be uploaded. Read Manifest file for details of how to create this file. Name the file jdbc-driver-manifest.json and place it in the storage location you specified in EXTERNAL_DRIVER_LOCATION, as described above.

NetSuite SuiteAnalytics

For the NetSuite SuiteAnalytics component, two driver files are required. You can obtain the required driver file from NetSuite as follows:

Log in to your NetSuite account.
On the homepage of your NetSuite account, under the Settings module, click Set Up SuiteAnalytics Connect.
Select Linux 64-bit from the drop-down.
Click the Download button next to JDBC Driver. This will download a zip file that contains a file called NQjc.jar.
Unzip the file and place the drivers in the storage location you specified in EXTERNAL_DRIVER_LOCATION, as described above. Do not change the driver file names.