March 2025 changelog

Here you'll find the March 2025 changelog for the Data Productivity Cloud. Just want to read about new features? Read our New Features blog.

5th March

DesignerImprovements 🔧

• System variables and post-processing are now generally available:

  • System Variables can now be used in component parameters, Python, Bash, and SQL scripts, in addition to post-processing.
  • Users can access shared and public scalar variables from a child pipeline using the post-processing Update Scalar property in Run Transformation, Run Orchestration, and Run Shared Pipeline components.
  • System variables are read-only and cannot be modified manually. Instead, system variable values are updated automatically during component executions.
  • System variables can be directly used in Python Pushdown, manually assigned in Bash Pushdown, and used inline in SQL script (orchestration) and SQL (transformation) components, with mapping required for external SQL.
  • New child pipeline system variable syntax is ${sysvar.childPipeline.vars.<varname>} and allows referencing variables from child pipelines, e.g., ${sysvar.childPipeline.vars.var1}.

• Data lineage now includes orchestration pipelines, providing a comprehensive view of data origins. This expansion introduces support for additional connectors and offers key benefits such as audit and compliance, impact analysis, and faster debugging.