API v1 - Secret manager

Overview

This is a guide to providing details on the Secret Manager API endpoints, and how they allow you to Import, Export, Post, and Delete information stored in the Matillion ETL Secret Manager. If you want to make use of these endpoints you will be required to register for an account with your chosen cloud provider, so that you can reference passwords, API tokens, keys, and secrets stored in third-party secret manager platforms, such as AWS, Azure, and GCP. For more information, read Integrating Matillion ETL with Secret Managers.

The secret manager endpoints consist of two PATHs that combine the number of endpoint "resource" data. "Resource" refers to the information returned by an API. These resources have various endpoints, which are combined with multiple HTTP methods: GET, POST and DELETE.

You can manage your secret managers within Matillion ETL, via Project, Manage Project Group Passwords, then switch to the Secret Managers tab.

Important Information

• Users responsible for experimenting with Matillion ETL API services require access to the Matillion ETL instance and ought to know how to make REST API calls either employing a REST API GUI client such as Postman or employing a command-line interface like cURL.
• For more information about accessing the Matillion API, and general information on the Matillion API platform before using the API, please refer to Matillion ETL API - v1.
• Matillion ETL API endpoints require authorisation to make any REST API call, so ensure a username and password for the Matillion ETL instance is configured before making any API call.

---

Secret Manager API Endpoints

API Base URL

http(s)://<InstanceAddress>/rest/v1/group/name/<groupName>/secretManager

API Endpoints and Functions

The Secret Manager API is available on standard REST-based APIs that use HTTP or HTTPS requests to GET, POST, and DELETE data. The OAuth API service is accessed through the Uniform Resource Identifier (URI). All following references in this document will assume the API Base URL has been specified.

The available API endpoints are listed below:

Method	Path	URI	Function
PATH/secretManager
GET	/secretManager	http://<instance address>/rest/v1/group/name/<groupname>/secretManager	Returns all secret manager names in your Matillion ETL instance.
GET	/export	http://<instance address>/rest/v1/group/name/<groupname>/secretManager/export	Exports all secret managers in your Matillion ETL instance.
POST	/import	http://<instanceaddress>/rest/v1/group/name/<groupname>/secretManager/import	Imports all secret managers from the specified Matillion ETL instance.
PATH/name/{secretManager}
DELETE	/<SecretManager>	http://<instance address>/rest/v1/group/name/<groupname>/secretManager/name/<secretManagerName>	Deletes the specified secret manager in your Matillion ETL instance.

---

URL Parameters and Descriptions

Below is the list of endpoint parameters and their descriptions:

Parameter Name	Description
<InstanceAddress>	The server IP address or domain name.
<groupName>	The name of the relevant project for the Secret Manager you wish to reference, within your Matillion ETL instance.
<secretManagerName>	The name of the secret manager within your Matillion ETL instance.
<type>	The third-party platform of your secret manager.
<credentialName>	The cloud provider credentials that are used to connect to your third-party platform secret manager.
<region>	AWS Secret Manager only. The AWS region within which your secrets manager resource has been created.
<version>	Your Matillion ETL build.
<environment>	Details of the Matillion ETL environment used to execute the job, such as the default Cloud Data Warehouse (CDW) database and schema.
<onConflict>	For the POST\\import endpoint only. If there is a conflict importing the intended secret manager in to your Matillion ETL instance, you can use this optional parameter to decide how you want the endpoint to behave. You can choose between the following three values: SKIP, OVERWRITE, ERROR. For more information, read Import conflicts

---

Endpoints and Server Response

This section describes the secret manager endpoints along with some examples of the server response. These APIs offer REST-based web services, ease of use, a flexible choice of programming language, and can also be used to access and analyze the secret managers and their data, stored in your Matillion ETL instance.

Prior to working with any Secret Manager endpoint calls, it is recommended that you learn and understand the concept of PATH used in the Secret Manager API family. A PATH is a unit of a REST API call. A PATH consists of an HTTP method (GET/POST/DELETE) and a URL PATH that, when exposed, is combined with the base PATH URL (<server address>/rest/v1) of the API.

The Secret Manager API family consists of two PATHs:

• PATH/secretManager
• PATH/name/{secretManager}

The PATH is relative to the API base URL. The full requested URL is constructed as follows - <server address>/rest/v1/. The API base URL - <server address>/rest/api is the prefix for all API paths, relative to the host root, for example, the server address or instance address. The PATH and HTTP method combined defines a unique operation for the resources.

To illustrate the secret manager API, endpoints, and methods, the diagram below shows the /secretManager endpoint with its PATH, GET, POST and DELETE options.

Once you have a basic understanding of PATH and its associated HTTP methods, you can begin to make API requests. All the API's listed in this article are available to use with GET/POST/DELETE methods to retrieve the data used to get, add and delete the intended secret manager. The detailed description of each endpoint and associated methods are discussed in the next section of this guide.

---

PATH/secretManager

List of endpoints for the PATH/secretManager:

• GET/secretManager
• GET/export
• POST/import

Below, see detailed descriptions of these endpoints with server response examples.

GET/secretManager

Use this endpoint request to get a list of all the secret managers listed in your Matillion ETL instance.

• Base URL

http://<instanceaddress>/rest/v1/group/name/<groupname>/secretManager

• Server Response

{
    "JiraAPIAccess"
}

GET/export

This endpoint request authorizes you to export your secret manager stored in your Matillion ETL instance. The export endpoint incorporates the following parameters: secretManagerName, secretManagerConfigExport, type, credentialName, region, version and environment.

• Base URL

http://<instanceaddress>/rest/v1/group/name/<groupname>/secretManager/export

• Server Response

{
    "secretManagerName": "YourSecretManagerName",

    "secretManagerConfigExport":
        "type": "YourCloudProvider_secret_manager_config_export",

        "credentialName": "YourCredentialsName",

        "region": "SecretManagerRegion",

    "version": "YourBuildVersion",

    "environment": "Data Warehouse"
}

POST/import

This endpoint allows you to import the secret managers you created in your Matillion ETL instance earlier, into a new instance. You will use a POST method API call. As part of this method, you will be required to attach the following information into the body, as a JSON format.

Please Note

An easy method of importing your secret managers from one Matillion ETL instance into another, is to copy the server response from the GET/export endpoint, and paste it into the body of your POST/import endpoint. Make sure it is JSON formatted. An example of this is displayed in this section further down.

• Base URL

http://<instanceaddress>/rest/v1/group/name/<groupname>/secretManager/import

• POST Body (JSON)

{
    "secretManagerName": "YourSecretManagerName",

    "secretManagerConfigExport":
        "type": "YourCloudProvider_secret_manager_config_export",

        "credentialName": "YourCredentialsName",

        "region": "SecretManagerRegion",

    "version": "YourBuildVersion",

    "environment": "Data Warehouse"
}

The table below shows a description of the fields that are included in the POST body:

Field name	Data type	Description
secretManagerName	String	The name of the secret manager within your Matillion ETL instance. Modify this name before you import it into a new instance.
secretManagerConfigExport	String	Contains parameters of your previously exported secret manager: type, credentialName and region.
type	String	The third-party platform of your secret manager, and the previously exported status of your secret manager.
credentialName	String	The cloud provider credentials that are used to connect to your third-party platform secret manager.
region	String	AWS Secret Manager only. The AWS region within which your secret manager resource has been created.
version	String	Your Matillion ETL build.
environment	String	Details of the Matillion ETL environment used to execute the job, such as the default Cloud Data Warehouse (CDW) database and schema.

• Server Response

{
    "success": true,
    "name": "<secretManagerName>"
}

Import Conflicts

There is an optional parameter you can use when using the POST/import endpoint, called onConflict. It determines what happens when an import clashes with something that already exists, such as when you try to import a secret manager with the same name. The parameter has one of three values - "ERROR", "SKIP", "OVERWRITE" - which are defined below:

• ERROR - Sends an error message back.
• SKIP - Skips importing any clashes, and preserves the existing secret manager.
• OVERWRITE - Removes the existing secret manager and replaces it with the newly imported secret manager.

The example below will show you how to use onConflict in Postman. For this example, we are using SKIP so that when we import a secret manager from one Matillion ETL instance into another, the import process will disregard any conflict and maintain your original secret manager.

Begin with entering the request URL:

http://<instanceaddress>/rest/v1/group/name/<groupname>/secretManager/import?onConflict=SKIP

Use the Params tab to enter onConflict in the Key column, and specify the Value.

---

PATH/name/{secretManager}

PATH/name/{secretManager} combines the HTTP method; DELETE. Please refer to the DELETE/deleteSecretManager endpoint below to see an example of the server response.

DELETE/deleteSecretManager

This endpoint request deletes the intended secret manager from the specified Matillion ETL instance.

• Base URL

http://<instanceaddress>/rest/v1/group/name/<groupname>/secretManager/name/<secretManagerName>

• Server Response

{
    "success": true,
    "msg": "Successfully deleted secret manager [secretManagerName]",
    "id": 0
}