Migration: Automatic variables
The Data Productivity Cloud supports most automatic variables and includes a similar concept to Matillion ETL called System variables. While both serve comparable purposes, they differ in syntax:
- Matillion ETL:
${my_variable_name}. - Data Productivity Cloud:
${sysvar.object.property}.
A clear pathway is established for migrating these variables and mapping them directly using the migration tool.
For more details of variable migration in general, read Migration: Variables.
Migration path
The table below lists all available automatic variables with their corresponding system variable mappings. To view the full list of system variables, read List of system variables. You will need to manually edit pipeline components to use the correct variables.
| Matillion ETL automatic variable | Data Productivity Cloud system variable |
|---|---|
| environment_id | .environment.id |
| environment_name | .environment.name |
| job_name | .thisPipeline.fullName |
| project_id | .project.id |
| run_history_id | .rootPipeline.executionId |
| task_id | .thisComponent.taskId |
| version_name | .artifact.versionName |
Not supported
The following Matillion ETL automatic variables don't have equivalents in the Data Productivity cloud.
- detailed_error
- queued_time
- component_id
- job_id
- project_group_id
- project_group_name
- version_id
- environment_endpoint
- environment_port
Not yet supported
The following Matillion ETL automatic variables don't currently have equivalents in the Data Productivity cloud, but support will be added in a future release. See our Roadmap for details.
- Environment properties such as:
- environment_username
- environment_database
- environment_default_schema
Accessing through scripts
The Data Productivity cloud doesn't support directly accessing automatic variables through the Python Script or Bash Script components.
If you require this functionality, you can use an Update Scalar component to write the values to user-defined variables, which can then be passed to the script.
For more alternative options available when migrating Python and Bash scripts, read Migration: Bash and Migration: Python.