Component Exports

Overview

All orchestration components and some transformation components support exporting runtime information into variables during job execution. Component export values are also used to populate the tasks view and the run history. The component's Export tab allows you to edit mappings between runtime information that the component makes available, and variables you have already defined.

---

List of export variables

This tables gives a list of common export variables. Some components will have additional component-specific variables.

Exported Value	Type	Description
Automatic Compression	String	Components such as Table Output allow data to be compressed if not already compressed. This variable indicates whether this compression occurred or not.
Completed At	DateTime	Time the component finished running in the job.
Component	String	Name of the component.
Duration	Integer	Length of time the component ran for.
Iteration Attempted	Integer	The number of iterations that this component attempts to reach (Max Iterations parameter).
Iteration Generated	Integer	The number of iterations that have been initiated. Iterators terminate after failure so this number will be the successful iterations plus any potential failure.
Iteration Successful	Integer	The number of iterations successfully performed. This is the max iteration number, minus failures and any unattempted iterations (since the component terminates after failure).
Message	String	Output message from component (usually empty).
Row Count	Integer	Number of rows the component has run over. Note: BigQuery doesn't always report row counts but attempting to retrieve the count won't cause an error.
Started At	DateTime	Time the component began running in the job.
Status	String	Status of the component.
Table Recreated	String	Will return No if the table already exists and the component is set to Create if not exists. In all other cases, will return Yes.

---

Usage

To export a value, you must map it to an existing job variable or environment variable.

1. Click the component on the job canvas and click the Export tab.
2. Click Edit to open the Export Variables dialog.
3. Click + to create a new mapping, and click in the Source column of the newly added blank row to open the drop-down.
4. Select the desired export variable from the drop-down.
5. Click in the Target Variable column and select a target variable from the drop-down. This drop-down contains both environment variables and job variables.
6. Click OK when you have completed all mappings.

Note

• The type of the target variable (String, Integer, DateTime) must match the type of the source variable given in the table above.
• Runtime information exported to job variables is only available while the job is running, and is no longer available after the job finishes running.

---

Grid variable export

When a job calls a child job with the Run Orchestration component, the values of grid variables can be exported from the child job to the parent job. This is managed from the component's Grid Export tab.

To export a grid variable between jobs:

1. Define separate grid variables in both the child job and the parent job, as described in Creating grid variables. The variables don't have to have the same name, nor do the number or names of the columns in each variable have to be identical.

   The grid variable created in the child job must have its Behaviour set to Shared and its Visibility set to Public.

2. Click the Run Orchestration component on the job canvas and click the Grid Export tab.

3. The Grid Export tab lists the grid variables currently exported from that component. A component can contain multiple different grid variables, and each can be exported to a different grid in the parent job. To configure a new grid export, click +.
4. In the Export Grid Variable dialog, select a grid variable from the Map Into drop-down. This is the grid in the parent job that the component's grid variable will be exported to. If you haven't previously created a suitable grid variable, you can click + to create a new one.
5. Select a grid variable from the Grid Variable drop-down. This is the grid in the child job that will be exported.
6. Under Column Mapping, each column in the parent grid is listed. From the drop-down next to each column, select the column in the child grid that will be exported to it. Every column in the parent grid must be matched with a column in the child grid, but you do not have to use every column in the child grid. You can also map the same column in the child grid to multiple columns in the parent grid if you wish. The data type of the mapped columns must be the same.

When the job runs and the Run Orchestration component is executed, the current values of the mapped grid columns will be exported to the columns of the grid variable in the parent job, over-writing any current value they may have.