Migration: Variables
Your Matillion ETL job variables, environment variables, and grid variables will be migrated. Environment variables become project variables in the Data Productivity Cloud, and job variables become pipeline variables. Grid variables become a type of pipeline variable. Read Variables and Grid variables for details.
Migration path
Most variables will migrate automatically and need no intervention. Note, however, the following key differences:
- Date formats for variables aren't currently supported in the Data Productivity Cloud. The migration tool will automatically convert all
DATETIMEvariables toSTRING. STRUCTformats for variables aren't currently supported in the Data Productivity Cloud.- Scalar and grid pipeline variables can't share the same name, as a grid is now a type of pipeline variable, not a separate class of variable.
If any of the above cases apply in your jobs, you will need to refactor them to work correctly within the Data Productivity Cloud.
Remove from Grid
The Remove from Grid component has been improved in the Data Productivity Cloud so that the Fixed/Grid parameter is no longer needed. This means that when migrating a Remove from Grid component that had Fixed/Grid set to Grid, you will lose your selected grid mappings.
To put the mappings back, select the Use Grid Variable option of the Values Grid property and select the grid mappings again.
If the component had Fixed/Grid set to Fixed, the migrated component will continue to work with no additional mitigation needed.
Automatic variables
Read Migration: Automatic variables for full details of differences.
Export variables
Matillion ETL export variables are called system variables in the Data Productivity Cloud. For a full discussion of export variable migration, read Migration: Export variables.
Variables in iterator components
Currently it isn't possible to control the concurrency behavior of an iterator component with a variable. If your job did this, edit the component to set the Concurrency property manually.