API v1 - Shared jobs

Overview

The Shared Job endpoint allows users to explore what shared jobs are available on the instance as well as importing, exporting and deleting them. You can manage your shared jobs in your Matillion ETL instance by clicking the Project Menu, then Manage Shared Jobs.

:::info{title='Note'} - This document is part of a series on shared jobs and the Matillion ETL API - v1. For related articles, refer to Shared jobs, Manage shared jobs. - Users wanting to make use of this endpoint will require your Matillion ETL instance URL, the username and password with appropriate permissions, and an account with the API Role and Shared Jobs permissions. - Users responsible for experimenting with Matillion ETL API services require access to the Matillion ETL instance and should know how to make REST API calls either employing a REST API GUI client such as Postman, or employing a command-line interface like cURL. :::

URL parameters and descriptions

Below is the list of endpoint parameters and descriptions for the shared job API:

Parameter Name	Description
instance_address	The server IP address or domain name.
shared_jobs	The shared jobs in your Matillion ETL instance.
packageName	The name of the package within the shared jobs.
jobName	The name of the job within the package.
revision	The revision listed within the job package of shared jobs.
export	Exports the metadata of the resource.
import	Imports the metadata to the resource.
delete	Deletes the selected resource.

Shared job API endpoints

Base URL

http(s)://{instanceAddress}/rest/v1/sharedjob

API endpoints and function

The shared job endpoint API is available on standard REST-based APIs that uses HTTP or HTTPS request to GET, POST, and DELETE data. The shared job endpoint API service is accessed through the Uniform Resource Identifier (URI). The available API endpoints are listed below:

:::info{title='Note'} A "package" represents a folder that can house both other packages and shared jobs. A "package name" is the concatenated path of packages that leads to the shared job. The "package name" shouldn't be mistaken for the literal name of a single package. Nested packages are denoted by a period/full stop. For example, a package name could be packageA.packageB.packageC. If a single package's name includes a space, replace the space with %20 in your API request. Read Shared job configuration for more information. :::

Method	Path	URI	Function
GET	export	http://{instanceAddress}/rest/v1/sharedjob/export	To export the metadata of the shared jobs within the instance.
POST	import	http://{instanceAddress}/rest/v1/sharedjob/import	To import the data of the shared jobs within the instance.
GET	package	http://{instanceAddress}/rest/v1/sharedjob/package	To get the package available within the shared job of the instance.

{packageName}

Method	Path	URI	Function
GET	packageName/export	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/export	To export the specific package from the shared job within the instance.
GET	packageName/job	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job	To get the list of Jobs available within the current package of the instance.
POST	packageName/delete	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/delete	To delete the selected package from the list using HTTP POST request.
DELETE	packageName	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}	Delete all jobs within a package.

{jobName}

Method	Path	URI	Function
GET	jobName/export	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/export	To export the specific job available within the package.
GET	jobName/revision	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision	To get the revisions available within the selected job.
POST	jobName/delete	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/delete	Delete all revisions of a single job.
DELETE	jobName	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}	Delete all revisions of a single job.

{revisionId}

Method	Path	URI	Function
GET	revisionId/export	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}/export	To export the specific job available within the package.
POST	revisionId	http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}revision/id/{revisionId}/	Delete a single revision.

Graphical representation

To illustrate the API v1 shared job, endpoints and methods, below is the graphical flow of the /sharedjob endpoint showing the PATH, GET, POST, and DELETE options.

Endpoints and server response

This section describes the shared job API endpoints with examples. These APIs offer REST-based web service, offering ease of use and a flexible choice of programming language. These APIs can be used to access and analyze the packages, jobs and revisions within your Matillion ETL instance.

All the APIs listed in this section are available to use with GET/POST/DELETE methods to retrieve the data used to get, update, import/export, delete the resource within the shared jobs.

List of endpoints for the /sharedjob:

GET/export
POST/import
GET/package

Below is the detailed description of these endpoints with example responses from the server.

GET/export

To export the shared jobs metadata from within your Matillion instance, use the /export endpoint after /sharedjob. This example will use the GET method REST API call to export the shared jobs metadata available within the instance.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/export

Server Response:

{
    "objects": [
        {
            "metadata": {
                "identifier": {... }
                    ]
                },...
                "orchestrationJobs": [
                    "Shared Jobs - Unicode"
                ],...
                "transformationJobs": [],       
                    },
                    "variables": {
                        "oauth": {....}
                    },  ....      
    ],
    "version": "master",
    "environment": "redshift"
}

POST/import

Now you have already export the shared job metadata in the example before. This time is to import the data via ExportContainer. Supply the data as already encoded since the shared job will be created as-is. This will be a POST method API call, as we will have to attach the imported data in the body as a JSON file to run into the Matillion ETL instance.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/export

Server Response:

{
  "name": "sharedjobs",
  "statusList": [
    {
      "success": true,
      "name": "matillion.regression.Unicode_1.1"
    },
     {
      "success": true,
      "name": "Matillion.Incremental.Gmail Incremental Load.1",
    }
  ],....
  "success": true
}

GET/package

To get the packages available within the shared job, use the /package endpoint after /sharedjob. This example will use the GET method REST API call to fetch the package details available within the shared jobs.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/export

Server Response:

[
    "TestPackage-1",
    "TestPackage-2",
    "TestPackage-3"
]

PATH/package/name/{packageName}

List of endpoints for the /package/name/{packageName}:

GET/export
GET/job
POST/delete
DELETE/{packageName}

Below is the detailed description of these endpoints with example responses from the server.

GET/export

To export a specific package data from within the shared jobs, provide /{packageName}, and use the /export endpoint after that. This example will use the GET method REST API call to export a package available within your Matillion ETL instance.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/export

Server Response:

{
    "objects": [
        {
            "metadata": {...
            },
                "orchestrationJobs": {...}
                   },
                "transformationJobs": {...},       
                    },
    ],
    "version": "master",
    "environment": "redshift"
}

GET/job

This is an example of the GET method REST API call to retrieve the jobs available within the selected package.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job

Server Response:

[
    "SharedJob"
]

POST/delete

This will be a POST method API call. The /delete endpoint will allow you to delete the selected package from the shared job, indicating the revision id.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/delete

Server Response:

{
  "success": true,
  "msg": "Successfully deleted [3] Shared Job(s).",
  "id": -1
}

DELETE/{packageName}

To remove any resource from the list, we will use the DELETE API request. In this example, we will delete a package from the shared job.

:::info{title='Note'} There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap. :::

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}

Server Response:

        {
          "success": true,
          "msg": "Successfully deleted [4] Shared Job(s).",
          "id": -1
        }

PATH/job/name/{jobName}

List of endpoints for the /job/name/{jobName}:

GET/export
GET/revision
POST/delete
DELETE/{jobName}

Below is the detailed description of these endpoints with example responses from the server.

GET/export

To export a specific job metadata from within the shared job package, provide /{jobName} and just use the /export endpoint after that. This example will use the GET method REST API call to export a job available within the instance.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/export

Server Response:

        {
            "objects": [
                {
                    "metadata": {...
                    },
                        "orchestrationJobs": {...}
                           },
                        "transformationJobs": {...},       
                            },
            ],
            "version": "master",
            "environment": "redshift"
        }

GET/revision

This is an example of the GET method REST API call to get the revision id associated with the current job.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision

Server Response:

[
    1
]

POST/delete

This will be a POST method API call. The /delete endpoint will allow you to delete the selected package from the shared job.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/delete

Server Response:

        {
          "success": true,
          "msg": "Successfully deleted [1] Shared Job(s).",
          "id": -1
        }

DELETE/{packageName}

To remove any resource from the list, we will use the DELETE API request. In this example we will delete a job.

:::info{title='Note'} There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap. :::

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/delete

Server Response:

        {
          "success": true,
          "msg": "Successfully deleted [1] Shared Job(s).",
          "id": -1
        }

PATH/revision/id/{revisionId}

List of endpoints for the /revision/id/{revisionId}:

GET/export
POST/delete
DELETE/revisionId

Below is the detailed description of these endpoints with example responses from the server.

GET/export

To export the job metadata using revision id from within the shared job package, provide /{revisionId} and just use the /export endpoint after that. This example will use the GET method REST API call to export a job metadata, including all of its PATHS and data types within your Matillion ETL instance.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}/export

Server Response:

        {
            "endpoints": [
                {...
                }
                ]'
              "dataTypes": [...
                 ]
        }

POST/delete

This will be a POST method API call. The /delete endpoint will allow you to delete the selected job using its revision id from the shared job.

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}/delete

Server Response:

        {
          "success": true,
          "msg": "Successfully deleted [1] Shared Job(s).",
          "id": -1
        }

DELETE/{revisionId}

To remove any resource from the list, we will use the DELETE API request. In this example we will delete a job .

:::info{title='Note'} There are often multiple ways of achieving the same task within the API as resources and methods branch out and overlap. :::

Base URL:

http://{instanceAddress}/rest/v1/sharedjob/package/name/{packageName}/job/name/{jobName}/revision/id/{revisionId}

Server Response:

        {
          "success": true,
          "msg": "Successfully deleted [1] Shared Job(s).",
          "id": -1
        }