Skip to content

Run Transformation

Run a transformation pipeline to completion. After the transformation pipeline completes, other orchestration components can be linked to this one—depending upon success or failure.

We refer to the pipeline containing this component as the parent pipeline, and the pipeline run by this component as the child pipeline.

Properties

Name = string

A human-readable name for the component.

Transformation Pipeline = drop-down

Choose an existing transformation pipeline.

Set Scalar Variables = column editor

Pipeline variables declared in the child pipeline can be given new values in the child pipeline through this property. The values set here are used as the new defaults in the child pipeline when it runs through this property only; they don't change any actual defaults which have been set in the child pipeline.

Read Variables for more information.

  • Variable: Select the variable in the child pipeline that you want to assign a new value to.
  • Value: Enter a new value for the variable. Ensure that the new value is of the correct type.

Set Grid Variables = column editor

Grid variables declared in the parent pipeline with a public scope can be used by the child pipeline. These variables can be given new values in the child pipeline through this property. Read Grid variables for more information.

  • Name: Lists every grid variable in the child pipeline.

  • Status: Defines which values each grid variable will use when the child pipeline runs. Options are:

    • Use defaults: Use the defaults set in the child pipeline's grid variable when the child pipeline runs in this component.
    • Set values: Lets you set new values for the child pipeline's grid variable in this property. The values set here are used as the new defaults in the child pipeline when it runs through this property only; they don't change any actual defaults which have been set in the child pipeline.
    • Use grid: Select a grid variable from the parent pipeline and assign its values to the grid variable in the child pipeline when the child pipeline runs in this component. Use the Column Mapping drop-downs to map each column in the child grid to a column from the parent grid. Take care to ensure you are mapping like-for-like data types.
Snowflake Databricks (preview) Amazon Redshift
✅ ✅ ✅