Skip to content

Update Scalar

This component allows you to change the value of any pipeline variable or project variable during pipeline execution, to a new value decided at the time the component is configured. The value is updated at the point the pipeline reaches this component, meaning that any later components in the pipeline will be executed using the new value.

Note

If you construct a branching pipeline and add an Update Scalar component to one branch, the value of the variable only changes in that branch; the other branch will use the original value of the variable. In effect, each branch of a complex pipeline has its own "local copy" of the variable, each of which can be updated independently by instances of this component.

For a full explanation of variables, read Variables.

Properties

Name = string

A human-readable name for the component.

Variables to Update = column editor

In the Variables to Update dialog, select a variable in the Name drop-down and enter a new value in the Value column. Click + to select additional variables to update.

The Value field allows you to use simple expressions within the field to change the variable value. Some examples of the expressions you can use include:

  • Add a fixed string to the beginning or end of a text variable, for example:

    STAGE_${table_name}

    will add the string "STAGE_" to the front of the text variable table_name.

  • Combine (concatenate) two text variables:

    ${prefix}${table_name}

  • Perform arithmetic on numeric variables. For example:

    ${counter + 1}

    will increment the variable counter by one. You must put the mathematical operator inside the curly brackets (braces), { }, that surround the variable name. ${counter} + 1 won't produce the expected result.

  • Use a JavaScript expression to convert a text variable to upper case:

    ${table_name.toUpperCase()}

  • Use a JavaScript regular expression to replace all characters in a text variable that aren't letters or numbers with an underscore:

    ${table_name.replace(/[^0-9a-zA-Z]/g, "_")}

  • Use a JavaScript expression to set a text variable to the current timestamp in ISO 8601 format:

    ${new Date().toISOString()}

These examples can be combined to perform complex variable manipulation. For example:

STAGE_${stage_name.toUpperCase()}_TABLE_${table_name.toUpperCase()}

Warning

If you enter or calculate a new value of the wrong data type, for example enter non-numeric characters into a variable of type "Number", the component will fail at runtime. This failure will not necessarily cause the pipeline to fail, though you may encounter unexpected results due to the failure to update the variable.

Snowflake Databricks Amazon Redshift
✅ ✅ ✅