Skip to content

API v1 - Group/project

Overview

This is a guide to providing details on the Group API. The Group endpoint allows users to explore the "resources" in their Matillion ETL instance. By "resources" here, we refer to Jobs, Tasks, Environment, Versions, Password, Projects, and Groups in which they reside. Group is one of the largest and most diverse endpoints with many PATHs that combined with HTTP methods (GET, POST, and DELETE) for unique combinations to get the "resource" information for the Matillion ETL instance.

Important Information

Please take note of the following information before getting started:

  • Users responsible for experimenting with Matillion ETL API services require access to the Matillion ETL instance and ought to know how to make REST API calls either employing a REST API GUI client such as Postman or employing a command-line interface like cURL.
  • For more information about accessing the Matillion API, and general information on the Matillion API platform before using the API, please refer to Matillion ETL API - v1.
  • Matillion ETL API endpoints require authorisation to make any REST API call, so ensure a username and password for the Matillion ETL instance is configured before making any API call.

Group API Endpoints

API Base URL

http(s)://<instance address>/rest/v1/<group>

API Endpoints and Function

Group API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET, POST, and DELETE data. The Group API service is accessed through the Uniform Resource Identifier (URI). All following references in this document will assume the API Base URL has been specified. The available Group API endpoints are listed below:

Method Path URI Function
GET /name http://<InstanceAddress>/rest/v1/group/name/<groupName>/name Get the name of the current group.
GET /id http://<InstanceAddress>/rest/v1/group/name/<groupName>/id Get the id associated with the current group.
GET /export http://<InstanceAddress>/rest/v1/group/name/<groupName>/export Export a Project with its Projects, Passwords, Jobs, Schedules, Environments, Versions and Variables.
POST /import http://<InstanceAddress>/rest/v1/group/import WITH POST DATA <ExportContainer> Import a group with its Projects, Passwords, Jobs, Schedules, Environments, Versions and Variables.
POST /delete http://<InstanceAddress>/rest/v1/group/name/<groupName>/delete Delete the selected group via a HTTP POST request.
DELETE /groupName http://<InstanceAddress>/rest/v1/group/name/<groupName> Delete the selected group via a DELETE request.
Graphical Representation (Group API)

To illustrate the Group API PATHs, endpoints and methods to the further, below is the graphical flow of the /group endpoint showing possible PATH, GET , POST, and DELETE options.

Group API endpoint Flow


Endpoints and Server Response


This chapter describes a few common cases of Group API endpoints . These APIs offers REST-based web service, offering ease of use and a flexible choice of programming language. These APIs can be used to access and analyse the available Groups, Projects, Job, Schedule, Version and more resource endpoints associated within the Matillion ETL instance.

Please Note

The Group API possesses several PATHs and endpoints, which are further examined and described within their particular guides. The goal here is to provide a bite-size, independent, generic example found in the Group endpoint of the v1 API.

Before we start working with any endpoint call, it's recommended that you understand the concept of PATH used in the Group API family.

A PATH is a unit of a REST API that you can call. A PATH comprises an HTTP Method (GET/POST/DELETE) and a URL PATH that, when exposed, is combined with the base PATH URL (<server address>/rest/v1) of the API.

The Group API family comprises of following PATHs:

  • PATH/password
  • PATH/project
    • PATH/project/variables
    • PATH/schedule
    • PATH/version
      • PATH/job
      • PATH/scm
    • PATH/task
      • PATH/id/{taskID:\\\\d+}
      • PATH/instance
    • PATH/environment
      • PATH/variable

The graphical view of the Group API PATHs shown below:

Group API PATH Flow

PATH are relative to the API base URL. The full requested URL is constructed as : <Serveraddress>/rest/v1/group. API base URL <Serveraddress>/rest/v1/ is the prefix for all API paths, relative to the host root (i.e Server address). PATH and a HTTP method combined defines a unique operation for the resource.

To get a illustrative flow of the Group API, endpoints and methods to the further, you can refer the Graphical Represtation section of this guide.

Once, you have basic understanding of PATH and associated HTTP methods, you can start making API requests.
All the APIs listed in this chapter are available to use with GET/POST/DELETE methods to retrieve the data used to import/export, get, post or delete the resources. The detailed description of the endpoint is discussed in the next section of this guide.

The goal here is to provide a bite-sized, independent, generic example of the GET, POST and DELETE options found in the Group endpoint of the v1 API. Whereas the Group API PATHs endpoints can be accessed via their independent guides available at Matillion ETL Support documentation.

Please Note

  • The PATH/project/variable can be accessed via the Project endpoint. For further information of the Projectname/variable API calls, see Import/Export - Project/Variables section.
  • The PATH/password can be accessed via the Group endpoint. For further information of the Password API calls, see API v1 - Password.
  • The PATH/schedule can be accessed via Group and Project endpoint. For further information of the Schedule API calls, see API v1 - Schedule.
  • The PATH/tasks can be accessed via Group and Project endpoint. For further information of the Tasks API calls, see API v1 - Tasks.
  • The PATH/environment can be accessed via Group and Project endpoint. For further information of the Environment API calls, see API v1 - Environment.
  • The PATH/version can be accessed via Group and Project endpoint. For further information of the Version API calls, see API v1 - Version.

Finding Resource

The v1 API can be used to list the names of resources that can then be used to access those resources.

Finding Groups


Accessing Group

To retrieve the list of groups available in the current instance, you could use the URI for the /group endpoint . This is a GET API request that will expose the list of groups available within the Matillion ETL instance.

  • Base URL 
    http://<instance address>/rest/v1/group
  • Server Response 
    [
    "Example-Group ",
    "Example-Group 1",
    "Example-Group 2",
]

​

Accessing a specific Group by name

In the example below, we will be retrieving a single resource information, perform a GET request for that named resource endpoint.

Whenever you reach a named resource endpoint, the API will expose API metadata for that resource, including PATH, GET and POST and DELETE method options available.

  • Base URL 
    http://<instance address>/rest/v1/group/name/<groupName>
  • Server Response 
    {
  "endpoints": [
    {
      "httpMethod": "PATH",
      "name": "GroupInstanceService",
      "children": [],   
    {
      "type": "Job",
      "fields": []
    },
    {
      "type": "Password",
      "fields": []
    },
    {
      "type": "TaskPackage",
      "fields": []
    },
    {
      "type": "VariableExport",
      "fields": []
    },
    {
      "type": "VersionExport",
      "fields": []
    }
  ]
}

Finding Projects within a Group

Accessing Project

To retrieve the list of Projects available within the selected group in the current instance, you could use the endpoint /project using GET request API call.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project
  • Server Response 
    [
    "Example-Project ",
    "Example-Project 1",
    "Example-Project 2",
]

Accessing a specific Project by name

In the example below, we will be retrieving a single project metadata, perform a GET request for that selected project endpoint.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>
  • Server Response 
    {
  "endpoints": [
    {
      "httpMethod": "PATH",
      "name": "ProjectInstanceService",
      "children": [....
      ],
       "type": "ProjectInstanceService"
    }
  ],
  "datatypes": [.....
  ]
}

Finding Versions within a Project

Accessing Version

To retrieve the version of the project within the current instance, you could use the endpoint /version using GET request API call.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version
  • Server Response 
    {
  "default"
}

Accessing a specific Version name

In this example, we will be accessing the specific version metadata of the project within the group in the current instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>
  • Server Response 
    {
  "endpoints": [
    {
      "httpMethod": "PATH",
      "name": "ProjectInstanceService",
      "children": [....
      ],
       "type": "VersionInstanceService"
    }
  ],
  "datatypes": [.....
  ]
}

Finding Jobs within a Version

Accessing Job

To retrieve the Job within the version of the project for current instance, you could use the endpoint /version using GET request API call to fetch the list of jobs within project.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job
  • Server Response 
    [
    "dim_airport_setup",
    "dim_airports",
    "Example-job",
    "docs"
]

Accessing a specific Job by name

In this example, we will be accessing a specific job within the version of the selected project in the current Matillion instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job/name/<jobName>
  • Server Response 
    {
  "endpoints": [
    {
      "httpMethod": "PATH",
      "name": "JobInstanceService",
      "children": [],
    {
      "type": "RestResponse",
      "fields": []
    },
    {
      "type": "RunVariablesContainer",
      "fields": []
    },
    {
      "type": "SingleExportContainer<JobExport>",
      "fields": []
    },
  ]
}

Import/Export all Groups

Export/Import Project Groups

Export all Project Groups

To export the project, provide the group and use the /export endpoint. This will export the group and all associated projects and passwords metadata using GET HTTP method call request.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/export
  • Server Response 
    {
    "objects": [
        {
            "name": "Docs",
            "passwords": {
                "objects": [
                    {
                        "name": "Example-Group",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [T3st].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    },
                    {
                        "name": "Example-Group 1",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [RS].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    }
                ],
                "version": "master",
                "environment": "redshift"
     },

Import all Project Groups

Now that you have an exported project group(see previous example), this time we use the API to import all the projects of the group into the Matillion ETL instance. Note that, when importing, there is no "merge" option. If a resource of the same name already exists, you must delete the existing resource before importing the new or change the name of the resource in POST Body. This will be a POST method API call as we will have to attach the project (in JSON form, as exported), in the body to import into the Matillion ETL instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/import WITH POST DATA <ExportContainer>
  • POST Body (JSON) 
    {
    "objects": [
        {
            "name": "DocsImported",
            "passwords": {
                "objects": [
                    {
                        "name": "Example-Project",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [T3st].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    },
                    {
                        "name": "Example-Project-1",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [RS].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    }
                ],
                "version": "master",
                "environment": "redshift"
     },
  • Server Response 
    {
  "name": "Example-Group",
  "children": [
    {
      "name": "Project Group - DocsImported",
      "children": [
        {
          "name": "Passwords",
          "statusList": [
            {
              "success": true,
              "name": "Example-Project"
            },
            {
              "success": true,
              "name": "Example-Project-1"
            }
          ],
          "success": true
        },
  • URL Parameters

    There is an optional parameter, onConflict, which determines what should happen if an import clashes with something that already exists. For an explanation of this parameter's use, read API Import conflicts - Explanation in Matillion ETL API - v1.


Import/Export Specific Resources


Export a Specific (Single) Project Groups

To export the selected project group, provide the groupName and use the /export endpoint. This will export the selected group and all associated information such as variables, environments, schedules, versions, jobs, and more details, aligned with the project. This is to export a project group with its passwords and projects using GET HTTP method call request.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/export
  • Server Response 
    {
    "objects": [
        {
            "name": "Example-Group",
            "passwords": {
                "objects": [
                    {
                        "name": "TestStuff-T3st",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [T3st].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    },
                    {
                        "name": "Example-Project-RS",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [RS].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    }
                ],
                "version": "master",
                "environment": "redshift"
     },

Import a Specifc (Single) Project Groups

Now that you have an exported group (see previous example), this time we use the API to import that group into a Matillion ETL instance. Note that, when importing, there is no "merge" option. If a resource of the same name already exists, you must delete the existing resource before importing the new. This will be a POST method API call as we will have to attach the project (in JSON form, as exported), in the body to import into the Matillion ETL instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/import WITH POST DATA <ExportContainer>
  • POST Body (JSON) 
    {
    "objects": [
        {
            "name": "Example-Group-Imported",
            "passwords": {
                "objects": [
                    {
                        "name": "TestStuff-T3st",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [T3st].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    },
                    {
                        "name": "Example-Project-RS",
                        "password": "",
                        "masterKey": null,
                        "description": "Password for environment [RS].",
                        "encryptionType": "ENCODED",
                        "extraInfo": null
                    }
                ],
                "version": "master",
                "environment": "redshift"
     },
  • Server Response 
    {
  "name": "Project Groups",
  "children": [
    {
      "name": "Project Group - Example-Group-Imported",
      "children": [
        {
          "name": "Passwords",
          "statusList": [
            {
              "success": true,
              "name": "TestStuff-T3st"
            },
            {
              "success": true,
              "name": "Example-Project-RS"
            }
          ],
          "success": true
        },
  • URL Parameters

    There is an optional parameter, onConflict, which determines what should happen if an import clashes with something that already exists. For an explanation of this parameter's use, read API Import conflicts - Explanation in Matillion ETL API - v1.

Export a specific (single) Project

To export the project, provide the projectName and use the /export endpoint. This will export the selected project and all associated information such as variables, environments, schedules, versions, jobs, and more details, aligned with the project. This is to export a project with its passwords and projects using GET HTTP method call request.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/export
  • Server Response 
    {
    "objects": [
        {
            "name": "Example-Project",
            "variables": [],
            "environmentExports": {
                "objects": [],
                "version": "master",
                "environment": "redshift"
            },
            "scheduleExports": {
                "objects": [],
                "version": "master",
                "environment": "redshift"
            },
            "versionExports": {
                "objects": [],
                "version": "master",
                "environment": "redshift"
            },
            "accessControlList": {
                "userRoles": {},
                "publicAdmins": true,
                "publicWriters": true
            }
        }
    ],
    "version": "master",
    "environment": "redshift"
}

Import a specific (single) Project

Now that you have an exported project (see the previous example), this time we use the API to import that project into a Matillion ETL instance. Note that, when importing, there is no "merge" option. If a resource of the same name already exists, you must delete the existing resource before importing the new or could change the name of the project to be imported. This will be a POST method API call as we will have to attach the project (in JSON form, as exported), in the body to import into the Matillion ETL instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/import WITH POST DATA <ExportContainer>
  • POST Body (JSON) 
    {
    "objects": [
        {
            "name": "ProjectImported",
            "variables": [],
            "environmentExports": {
                "objects": [],
                "version": "master",
                "environment": "redshift"
            },
            "scheduleExports": {
                "objects": [],
                "version": "master",
                "environment": "redshift"
            },
            "versionExports": {
                "objects": [],
                "version": "master",
                "environment": "redshift"
            },
            "accessControlList": {
                "userRoles": {},
                "publicAdmins": true,
                "publicWriters": true
            }
        }
    ],
    "version": "master",
    "environment": "redshift"
}
  • Server Response 
    {
  "name": "Projects",
  "children": [
    {
      "name": "Project - ProjectImported",
      "children": [
        {
          "name": "Environments",
          "statusList": [],
          "success": true
        },
        {
          "name": "Versions",
          "children": [],
                  "success": true
                }
              ],
            "success": true
        },
          "name": "Schedules",
          "success": true
        }
      ],
      "success": true
    }
  ],
  "success": true
}
  • URL Parameters

    There is an optional parameter, onConflict, which determines what should happen if an import clashes with something that already exists. For an explanation of this parameter's use, read API Import conflicts - Explanation in Matillion ETL API - v1.

The API call will create this project in the Matillion ETL instance. Note that we've changed the project name to "ProjectImported" in the JSON, else we could get an error for importing a job whose name already exists (APIProject). This project can now be switched to in the Matillion ETL instance.

Group endpoint Overview

Export/Import - Version


Export a specific (single) Version

To export the version, provide the /versionName and use the /export endpoint. This will export the selected version using GET HTTP method call request.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/export
  • Server Response 
    {
    "objects": [
        {
            "name": "default",
            "description": "Default Version",
            "locked": false,
            "jobs": {
                "objects": [
                    {
                        "jobObject": {....
                        },
                        "info": {
                            "id": 1022,
                            "name": "docs",
                            "type": "TRANSFORMATION",
                            "tag": "ca7d2079-dfc7-4c1d-bd59-69c056ea7f47"
                        },
                        "path": []
                    }
    ],
    "version": "master",
    "environment": "redshift"
}

Import a specific (single) Version

This will import the resource data which is exported as shown in the above example using /import endpoint. If a resource of the same name already exists, you must delete the existing resource before importing the new or could change the name of the resource to be imported in the POST Body file. This will be a POST method API call as we will have to attach the project (in JSON form, as exported), in the body to import into the Matillion ETL instance..

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/import
  • POST Body (JSON) 
    {
    "objects": [
        {
            "name": "default-Imported",
            "description": "Default Version",
            "locked": false,
            "jobs": {
                "objects": [
                    {
                        "jobObject": {....
                        },
                        "info": {
                            "id": 1022,
                            "name": "docs",
                            "type": "TRANSFORMATION",
                            "tag": "ca7d2079-dfc7-4c1d-bd59-69c056ea7f47"
                        },
                        "path": []
                    }
    ],
    "version": "master",
    "environment": "redshift"
}
  • Server Response 
    {
  "name": "Versions",
  "children": [
    {
      "name": "Version - default-Imported",
      "children": [
        {
          "name": "Jobs",
          "statusList": [
            {
              "success": true,
              "name": "dim_airport_setup"
            },
            {
              "success": true,
              "name": "Example-Job"
            },
          ],
          "success": true
}
  • URL Parameters

    There is an optional parameter, onConflict, which determines what should happen if an import clashes with something that already exists. For an explanation of this parameter's use, read API Import conflicts - Explanation in Matillion ETL API - v1.


Export/Import - Job


Export a specific(single) Job

To export the Job resource, provide the /jobName and use the /export endpoint. This will export the selected job using GET HTTP method call request.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job/name/<jobName>/export
  • Server Response 
    {
    "objects": [
        {
            "jobObject": {
                "JobType": ".TransformationJob",
                "id": 1022,
                "revision": 2,
                "created": 1594080796911,
                "timestamp": 1594080796911,
            },
            "info": {
                "id": 1022,
                "name": "docs",
                "type": "TRANSFORMATION",
                "tag": "ca7d2079-dfc7-4c1d-bd59-69c056ea7f47"
            },
            "path": []
        }
    ],
    "version": "master",
    "environment": "redshift"
}

Import a specific(single) Job

This will import the resource data which is exported as shown in the above example using /import endpoint. If a resource of the same name already exists, you must delete the existing resource before importing the new or you can change the name of the imported resource (docsImported)in the POST BODY. This API call is a POST method API call as we will have to attach the resource data, in the body as a JSON file to import into the Matillion ETL instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job/import
  • POST Body (JSON) 
    {
    "objects": [
        {
            "jobObject": {
                "JobType": ".TransformationJob",
                "id": 1022,
                "revision": 2,
                "created": 1594080796911,
                "timestamp": 1594080796911,
            },
            "info": {
                "id": 1022,
                "name": "docsImported",
                "type": "TRANSFORMATION",
                "tag": "ca7d2079-dfc7-4c1d-bd59-69c056ea7f47"
            },
            "path": []
        }
    ],
    "version": "master",
    "environment": "redshift"
}
  • Server Response 
    {
  "name": "Jobs",
  "statusList": [
    {
      "success": true,
      "name": "docsImported"
    }
  ],
  "success": true
}
  • URL Parameters

    There is an optional parameter, onConflict, which determines what should happen if an import clashes with something that already exists. For an explanation of this parameter's use, read API Import conflicts - Explanation in Matillion ETL API - v1.


Export/Import - Schedule


Export a specific(single) schedule

To export a specific schedule resource, provide the /scheduleName and use the /export endpoint. This will export the selected schedule using GET HTTP method call request.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/schedule/name/<scheduleName>/export
  • Server Response 
    {
    "objects": [
        {
            "name": "Example-Schedule",
            "minute": "30",
            "hour": "08:00",
            "dayOfWeek": true,
            "monday": true,
            "tuesday": false,
            "wednesday": false,
            "thursday": false,
            "friday": false,
            "saturday": false,
            "sunday": false,
            "daysOfMonth": "",
            "enabled": true,
            "timezone": "UTC",
            "versionName": "default",
            "jobName": "dim_airport_setup",
            "environmentName": "T3st",
            "preventDuplicateJob": true
        }
    ],
    "version": "master",
    "environment": "redshift"
}

Import a specific(single) schedule

This will import the resource data which is exported as shown in the above example using /import endpoint. If a resource of the same name already exists, you must delete the existing resource before importing the new one, or you can change the name of the imported resource (Schedule-Imported)in the POST BODY. This API call is a POST method API call as we will have to attach the resource data in the body as a JSON file to import into the Matillion ETL instance.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/schedule/import
  • POST Body (JSON) 
    {
    "objects": [
        {
            "name": "Schedule-Imported",
            "minute": "30",
            "hour": "08:00",
            "dayOfWeek": true,
            "monday": true,
            "tuesday": false,
            "wednesday": false,
            "thursday": false,
            "friday": false,
            "saturday": false,
            "sunday": false,
            "daysOfMonth": "",
            "enabled": true,
            "timezone": "UTC",
            "versionName": "default",
            "jobName": "dim_airport_setup",
            "environmentName": "T3st",
            "preventDuplicateJob": true
        }
    ],
    "version": "master",
    "environment": "redshift"
}
  • Server Response 
    {
  "name": "Schedules",
  "statusList": [
    {
      "success": true,
      "name": "Schedule-Imported"
    }
  ],
  "success": true
}
  • URL Parameters

    There is an optional parameter, onConflict, which determines what should happen if an import clashes with something that already exists. For an explanation of this parameter's use, read API Import conflicts - Explanation in Matillion ETL API - v1.


Deleting Resources

Resources can be deleted by accessing the resource via the API then appending the /delete endpoint. For example, deleting a Project:

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/delete
  • Server Response 
    {
  "success": true,
  "msg": "Successfully deleted Project: Example-Project",
  "id": 59889
}

Promoting a Job

Promoting a job from one version to another (for example, from Test to Live) is a common task. To accomplish this, the job (in this example, 'dim_airport_setup') to be promoted must first be exported:

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job/name/<jobName>/export
  • Server Response 
    {
    "objects": [
        {
            "jobObject": {
                "JobType": ".TransformationJob",
                "id": 1022,
                "revision": 2,
                "created": 1594167211446,
                "timestamp": 1594167211446,
                "components": {},
                "connectors": {},
                "variables": {},
                "grids": {}
            },
            "info": {
                "id": 1022,
                "name": "dim_airport_setup",
                "type": "TRANSFORMATION",
                "tag": "d367630d-cf63-4993-82d1-5435eea2e73e"
            },
            "path": []
        }
    ],
    "version": "master",
    "environment": "redshift"
}

Now the job can be imported to the new Version:

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job/import
  • Server Response 
    {
    "objects": [
        {
            "jobObject": {
                "JobType": ".TransformationJob",
                "id": 1022,
                "revision": 2,
                "created": 1594167211446,
                "timestamp": 1594167211446,
                "components": {},
                "connectors": {},
                "variables": {},
                "grids": {}
            },
            "info": {
                "id": 1022,
                "name": "dim_airport_setup-01",
                "type": "TRANSFORMATION",
                "tag": "d367630d-cf63-4993-82d1-5435eea2e73e"
            },
            "path": []
        }
    ],
    "version": "master",
    "environment": "redshift"
}
And finally the old Job can be deleted:
  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupName>/project/name/<projectName>/version/name/<versionName>/job/name/<jobName>/delete
  • Server Response 
    {
  "success": true,
  "msg": "Successfully deleted Job: dim_airport_setup",
  "id": 1013
}

Import/Export - Project/Variables

This section is a part of the series of PATH/project. The PATH/projectname/variable endpoint is further combined with HTTP methods GET and POST for unique combinations to export and import the Variable information within the project.

GET/variable/export

To export the variables available from within the chosen project.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/variable/export
  • Server Response 
    {
    "objects": [
        {
            "name": "tests",
            "variables": [
                {
                    "name": "Example_Variable",
                    "type": "TEXT",
                    "scope": "BRANCH",
                    "description": null,
                    "visibility": "PUBLIC"
                }
            ],
    "version": "master",
    "environment": "redshift"
}

POST/variable/import

To import the variables to the chosen project.

  • Base URL 
    http://<InstanceAddress>/rest/v1/group/name/<groupname>/project/name/<projectName>/variable/import
  • POST Body 
    {
    "objects": [
         "name": "tests",
            "variables": [
                {
                    "name": "Example_Variable",
                    "type": "TEXT",
                    "scope": "BRANCH",
                    "description": null,
                    "visibility": "PUBLIC"
                    "values": {
                           " Dev1": "value1"
                                }
                }
            ],
    "version": "master",
    "environment": "redshift"
}
  • Server Response 
    {
    "name": "tests",
    "statusList": [
        {
            "success": true,
            "name": "Example_Variable"
        }
    ],
    "success": true
}

Secret Manager Integration

There are two endpoints available to configure in relation to integrating Matillion ETL with secret managers.

The relevant endpoints are listed below:

Method Path URI Function
GET /export <host>/rest/v1/group/name/<groupName>/secretManager/export Exports all secret managers.
POST /import <host>/rest/v1/group/name/<groupName>/secretManager/import Import secret managers.

URL Parameters and Description

Below is the list of endpoint parameters (used in the guide) and their brief description:

Parameters Name Description
<InstanceAddress> This is the server IP address or domain name.
<groupName> The name of the group created in the Matillion ETL instance.
<projectName> Get the Project API detail created in the group by its name.
<scheduleName> Get the schedule API by its name.
<versionName> Get the version API detail by its name.
<jobName> Get the job name within the project by its name.
<delete> Delete the resource within the available resources.
© Copyright 2024 Matillion Ltd | Legal | Privacy Policy | Disclaimer | Modern Slavery Statement