API v1 - Credentials

The Credentials API returns AWS, Azure, and GCP credentials data, in your Matillion ETL instance.

Note

The 'credentials' refers to roles and permissions set on the hosting platform of your Matillion ETL instance, and doesn't refer to the database your client connects to.

For context, in your Matillion ETL instance, you can set User Defined Credentials, which are specific to an environment. Therefore, it's possible to use a different set of credentials for each environment. Whereas, Instance Credentials (tied to the instance hosting Matillion ETL) don't change with the environment, and is overruled by User Defined Credentials. For more information about how to set credentials, read Manage credentials.

Some endpoints listed in this document can be used with each of the three available platforms. These identical endpoints will be displayed under one heading, the endpoint URL includes the platform as a parameter.

Prerequisites

This document is part of a series on Manage Credentials.
To use Matillion ETL API services, you must know how to make REST API calls, either by employing a REST API GUI client such as Postman, or employing a command line interface (CLI) like cURL.
If you are new to Matillion ETL's API, read our Matillion API introduction and explore the v1 API map.
Matillion ETL API endpoints require authorization to make any REST API call, so ensure a username and password for the Matillion ETL instance is configured before making any API call.

Using the API

Returns API metadata, including PATH, GET, POST, and DELETE method options. In the example below, the metadata returned shows the PATH options for the cloud credentials available in this Matillion ETL instance.

<InstanceAddress>/rest/v1/credential/<endpoint>

Note

<InstanceAddress>/rest/v1/credential/ is the base URI for the Credentials API endpoints. Add the specific <endpoint> information for each function, documented in the below sections.

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<endpoint> is the name of the API endpoint that accomplishes the task you want to perform. These endpoints are documented in the following sections.

GET credentials

Returns credential information from a Matillion ETL instance by making a GET call.

To return metadata for the cloud platform credentials, make the following GET call:

<InstanceAddress>/rest/v1/credential

Where <InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.

A successful GET call to the endpoint will return the following response from the server, in JSON format:

{
  "endpoints": [
    {
      "httpMethod": "PATH",
      "name": "CredentialsBaseService",
      "children": [...],
      "type": "RestResponse",
      ...
    },
    {
      "httpMethod": "POST",
      ...
    },
    {
      "httpMethod": "DELETE",
      ...
    },
    {
      "httpMethod": "GET",
      ...
    },...
    ]
}

GET [cloud provider]

Returns a list of your chosen cloud provider credentials in this Matillion ETL instance.

To return a list of credentials, make the following GET call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".

A successful GET call to the endpoint will return the following response from the server, in JSON format:

{
    "Instance Credentials",
    "Manual Credentials",
    "[cloud provider credentials name]"
}

Get test

Tests your chosen cloud provider credentials in this Matillion ETL instance.

To test these credentials, make the following GET call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>/test

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".
<credentialName> is the name of your chosen cloud provider credentials in your Matillion ETL instance.

A successful GET call to the endpoint will return the following response from the server, in JSON format:

For AWS credentials:

{
    "class": "com.matillion.bi.emerald.shared.model.command.credentials.UserCredentialsResult",
    "result": "PASS",
    "connectionMessage": "Redshift API: success
    S3 API: success
    SQS API: success
    SNS API: success
    CloudWatch API: success
    RDS API: success
    EC2 API: Check credentials.
    KMS: success
    DMS: success",
    "specifier": null,
    "details": {
    "timestamp": 1595846515047,
    "message": null,
    "ackID": -1
    }
}

For Azure credentials:

{
  "class": "com.matillion.bi.emerald.shared.model.command.credentials.UserCredentialsResult",
  "result": "PASS",
  "connectionMessage": "Blob Storage: success",
  "specifier": null,
  "details": {
    "timestamp": 1595857388819,
    "message": null,
    "ackID": -1
  }
}

For GCP credentials:

{
  "class": "com.matillion.bi.emerald.shared.model.command.credentials.UserCredentialsResult",
  "result": "PASS",
  "connectionMessage": "BigQuery: success
    GoogleCloudStorage: success
    PubSub: success
    KMS: success",
  "specifier": null,
  "details": {
    "timestamp": 1595850444709,
    "message": null,
    "ackID": -1
  }
}

Get export

Export the "name", "secret", "version", and "environment" of your chosen cloud credentials in this Matillion ETL instance.

To export these cloud provider details, make the following GET call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>/export

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".
<credentialName> is the name of your chosen cloud provider credentials in your Matillion ETL instance.

A successful GET call to the endpoint will return the following response from the server, in JSON format:

{
    "objects": [
        {
            "name": "cloud provider name",
            "secret": "gsdjwjh23412jk"
        }
    ],
    "version": "master",
    "environment": "redshift"
}

POST delete

Delete the selected cloud provider credentials from this Matillion ETL instance.

To delete your chosen cloud provider credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>/delete

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".
<credentialName> is the name of your chosen cloud provider credentials in your Matillion ETL instance.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
  "success": true,
  "msg": "<cloud provider name> deleted",
  "id": 784
}

POST create AWS and GCP credentials

Create AWS and GCP cloud credentials. The POST body will need to include "Name" and "secretKey", as a JSON file to create the new object. To create your Azure cloud credentials, read Post Create Azure Credentials.

To create the cloud credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/create

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", or "gcp".

The POST body should contain the following, in JSON format:

{  
    "name": "<credentialName>",
    "secretKey": "<secretKey>"
}

<name> is the name of your AWS or GCP cloud credentials.
<secretKey> is the key name for the selected AWS or GCP credentials.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
    "success": true,
    "msg": "Created [cloud provider] Credential [cloud credentials name]",
    "id": -1
}

POST create Azure credentials

Create Azure cloud credentials. The POST body will need to include "Name", "tenentID", "clientID", and "keyVault", as a JSON file to create the new Azure object. To create your AWS and GCP cloud credentials, read POST create AWS and GCP credentials.

To create Azure credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/azure/create

Where <InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.

The POST body should contain the following, in JSON format:

{
    "name": "<Credential name>",
    "tenantID": "<tenantID>",
    "clientID": "<clientID>",
    "Key Vault": "<keyVault>"
}

<name> is the name of your Azure cloud credentials.
<tenantID> is the "tenantID" name to authenticate your Azure credentials.
<clientID> is the "clientID" name for your Azure credentials object.
<keyVault> is the name of your Azure Key Vault that securely stores your access tokens, passwords, certificates, API and secret keys.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
    "success": true,
    "msg": "Created Azure Credential [Azure credentials name]",
    "id": -1
}

POST update AWS and GCP credentials

Update AWS and GCP cloud credentials in this Matillion ETL instance. To update your Azure cloud credentials, read POST update Azure credentials.

To update your AWS and GCP cloud credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>/update

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", or "gcp".
<credentialName> is the name of your chosen cloud provider credentials in your Matillion ETL instance.

The POST body should contain the following, in JSON format:

{    
    "name": "<credentialName>",
    "secretKey": "<secretkey>"
}

<name> is the name of your AWS or GCP cloud credentials.
<secretKey> is the name that securely stores access tokens, passwords, certificates, API and secret keys, for your AWS and GCP credentials.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
  "success": true,
  "msg": "Successfully updated credential [cloud provider]",
  "id": 782
}

POST update Azure credentials

Update Azure cloud credentials in this Matillion ETL instance. To update AWS and GCP cloud credentials, read POST update AWS and GCP credentials.

To update your Azure cloud credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>/update

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider.
<credentialName> is the name of your Azure cloud provider credentials in your Matillion ETL instance.

The POST body should contain the following, in JSON format:

{
    "name": "<Azure credentials name>",
    "tenantID": "<tenantID>",
    "clientID": "<clientID>",
    "keyVault": "<keyVault>"
}

<name> is the name of your Azure cloud credentials.
<tenantID> is the "tenantID" name to authenticate your Azure credentials.
<clientID> is the "clientID" name for your Azure credentials object.
<keyVault> is the name of your Azure Key Vault that securely stores your access tokens, passwords, certificates, API and secret keys.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
  "success": true,
  "msg": "Successfully updated credential [Azure credentials name]",
  "id": 782
}

POST updateFromExport

Update your exported cloud credentials. For more information, read Get export. Use the exported "name" and "secret" information in the POST body, and update the selected credentials.

To update your exported cloud platform credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>/updateFromExport

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".
<credentialName> is the name of your chosen cloud provider credentials in your Matillion ETL instance.

The POST body should contain the exported environment, and look similar to the following, in JSON format:

{  
    "name": "<credentialname>",
    "secret": "<secretkey>" 
}

<name> is the name of your chosen cloud platform credentials.
<secretKey> is the key name for your chosen cloud platform credentials.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
    "success": true,
    "msg": "Successfully updated credential [cloud platform credentials name]",
    "id": 782
}

POST import

Once you have exported your chosen cloud credentials, import those credentials into your Matillion ETL instance.

Note

When importing, there is no "merge" option. If a resource of the same name already exists, you must delete the existing resource before importing the new one.

To import your exported cloud platform credentials, make the following POST call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/import WITH POST DATA arg0

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".

The POST body should contain the exported credentials, and look similar to the following, in JSON format:

{
    "objects": [
        {
            "name": "[cloud credentials name]",
            "secret": "gadgjs28738kl"
        }
    ],
    "version": "master",
    "environment": "redshift"
}

<objects> is an array containing the "name" and "secret" fields.
<name> is the chosen cloud credentials name.
<secret> is the imported cloud credentials.
<version> is the version of the selected cloud credentials.
<environment> is the environment in which the selected cloud credentials exist.

A successful POST call to the endpoint will return the following response from the server, in JSON format:

{
  "name": "Credentials",
  "statusList": [
    {
      "success": true,
      "name": "[cloud credentials name]"
    }
  ],
  "success": true
}

Delete credential name

Remove your chosen cloud credentials object from this Matillion ETL instance.

To remove your chosen cloud credentials, make the following DELETE call:

<InstanceAddress>/rest/v1/credential/<cloudProvider>/name/<credentialName>

Where:

<InstanceAddress> is the URI of your Matillion ETL instance in the form http://host:port.
<cloudProvider> is the name of your chosen cloud provider - "aws", "azure", or "gcp".
<credentialName> is the name of your chosen cloud provider credentials in your Matillion ETL instance.

A successful DELETE call to the endpoint will return the following response from the server, in JSON format:

{
    "success": true,
    "msg": "[cloud provider] Credential [credential name] deleted",
    "id": 66535
}