Jira Query example

In this example, we will be outlining some very basic data loading pipelines using the Jira Query component in the Matillion Designer. These examples will result in a table that gives only Jira issues for a specific project.

The Jira Query component is available through the Components tab when editing an orchestration pipeline. This component will connect to your Jira account using a username/password or username/API token, stage the data in a storage area, and then load that data into a Snowflake table.

---

Authentication

The Jira Query component can take a username/password as authentication or a username/API token. We'll be using username/API token for this example and this means registering our Jira API token as a secret on the AWS Secrets Manager. The general process for this is given below.

Note

This example assumes a Hybrid SaaS solution, which uses your AWS Secrets Manager to store secrets. If you are using a Full SaaS solution, the process will be different. Read Secret definitions for more details.

1. Log in to the AWS account that houses your agent.
2. Browse to the Secrets Manager service.
3. Ensure you're in the same AWS Region as your agent.
4. Click Store a new secret.
5. Click Other type of secret.
6. Enter a memorable key name and then your Jira API token as the value.
7. Click Next until the secret creation is completed.

You then need to register a Secret definition in the Designer. Follow these steps:

1. Navigate to Designer in the Hub.
2. Select your Project. Secret definitions are project-specific.
3. Click the Secret definitions tab.
4. Click Add secret definition.
5. Configure your secret definition.
   • Secret definition name: A name for the secret definition. Any whitespace added to the start or end of the secret definition name is automatically trimmed.
   • Description: An optional, referential description for the secret. Max 256 characters.
   • Agent: Select the agent that will be used by the pipelines that use this secret.
   • Secret name: Select the secret you created previously in AWS Secrets Manager.
   • Secret key: Select a named secret key tied to your secret name.
6. Click Create secret.

The value of the selected key will be used when this definition is referenced inside the Designer. For example, when this definition is selected as the password for connector authentication.

Secret definitions don't create secrets, they create a reference to an existing secret for use within the Designer.

---

Loading using the Jira Query component

Our first option to create project-specific Jira datasets is to use the Filter property on the Jira Query component. The component can be called multiple times in a single pipeline with each pointing to a different table.

To filter the data, click the Jira Query component and click to configure the Data Selection property. Ensure the ProjectName column is being loaded by moving it to the Selected Data Selection panel. Next, click the Data Source Filter property and configure with the following details:

Input Column	Qualifier	Comparator	Value
ProjectName	Is	Equal to	[YourProjectName]

Only rows that match this criteria will be kept in the data load.

---

Filtering with the Filter component

Alternatively, the Jira Query component can be used with no filter included to load all Jira issues data with a single load, and then we can let a transformation pipeline do the filtering by using the Filter component and then outputting to different tables using Rewrite Table.

If the Filter component does not validate as expected, try re-validating the pipeline so the Designer can acknowledge your columns from the Table Input component.