Upgrade: Export variables🔗
Matillion ETL export variables are called system variables in Maia, and operate in broadly the same way, with a few notable differences listed below.
For more details of variable migration in general, read Upgrade: Variables.
Upgrade path🔗
Mappings of component exports to job or environment variables are automatically migrated to Maia. For more information, read Upgrade: Variables.
The following table lists Maia equivalents for some commonly used export variables. This list isn't exhaustive—consult Maia documentation for an up-to-date list of system variables.
| Matillion ETL component export | Maia system variable |
|---|---|
| Completed At | .thisComponent.finishedAt |
| Component | .thisComponent.name |
| Duration | .thisComponent.duration |
| Message | .thisComponent.message |
| Row Count | .thisComponent.rowCount |
| Started At | .thisComponent.startedAt |
| Status | .thisComponent.status |
The following table outlines the component-specific export variables that can be migrated from Matillion ETL to Maia:
| Matillion ETL component | Matillion ETL export variable | Maia system variable |
|---|---|---|
| SQL Query | Query ID | .thisComponent.queryId |
| Table update | Rows Deleted | .thisComponent.rowsDeleted |
| Table update | Rows Inserted | .thisComponent.rowsInserted |
| Table update | Rows Updated | .thisComponent.rowsUpdated |
Some Matillion ETL components have specific export variables that currently have no equivalents in Maia, and you will have to refactor to work around these. The following table has a (non-exhaustive) list of some common examples.
| Matillion ETL component | Variables |
|---|---|
| Append to Grid | Rows Added |
| Create Stream | Stream Recreated |
| Create Table | Table Recreated |
| Data Transfer | Bytes Written |
| Extract components (e.g. API Extract, Amplitude Extract, etc...) | Data Structure |
| Iterator components | Iterations Attempted, Iterations Generated (but see below), Iterations Successful |
| JDBC Table Metadata to Grid | Column Count |
| Query components (e.g. Database Query, DynamoDB Query, Email Query, etc...) | Filename, Time Taken to Load, Time Taken to Stage |
| Remove from Grid | Rows Removed |
| Retry | Iterations Attempted, Iterations Generated, Iterations Successful |
For a workaround to replace Iterations Generated in iterator components, read Upgrade: Iterators.
Functionality not supported🔗
The following are not currently supported in Maia. If your Matillion ETL jobs incorporate any of these, you will need to refactor the migrated pipelines:
- Returning grid variables from a child pipeline.
- Component-specific properties, for example "Iterations generated".
- Use of system variables in an If component. This can only be done in Maia using "Advanced" mode.
- User defined variables.