Feature differences in Maia🔗
Some components and features need specific treatment, mitigation, or workarounds when migrated. If you use any of the following features, make sure you understand what specific treatment each one will require.
API trigger🔗
API triggers are supported in Maia, but the API works differently. Read the API documentation for further details.
You may need to update your trigger scripts to work with the Maia API. Evaluate your triggers on a case-by-case basis and update where needed, guided by the API documentation.
Git🔗
Unlike Matillion ETL where Git integration is an optional feature, Maia is built with Git as an integral element, providing pipeline version control and making it simple to collaborate and manage data pipelines within your team. Read Git in Designer to learn more about this feature of Maia.
If you currently use Git in Matillion ETL, we don't recommend that you use the same Git repository for Maia. Maia won't recognize the format of Matillion ETL files stored in Git. Although it is possible to connect to the same repository, you won't be able to access your previous Git history. The process of migrating to Git in Maia should be:
- Create a Maia project using the Connect your own Git repository option.
- Connect the project to a new Git repository with your preferred provider.
- Migrate jobs that use Git.
- Perform any necessary manual changes to the imported pipelines.
- Commit and push the migrated pipelines to Maia Git repository.
If you don't currently use the optional Git feature within Matillion ETL, you simply need to select which type of Git repository you want to use in Maia and configure it prior to migrating your jobs.
OAuths🔗
For security reasons, we do not migrate credentials such as OAuths from Matillion ETL to Maia. Any OAuths you have set up in Matillion ETL will have to be recreated manually in Maia to allow your pipelines to run. Read OAuth for details.
Secrets🔗
For security reasons, we do not migrate credentials such as secrets and passwords from Matillion ETL to Maia. Any secrets or other credentials you have set up in Matillion ETL will have to be recreated manually in Maia to allow your pipelines to run. Read Secrets and secret definitions and Cloud provider credentials for details.
Passwords can't be entered directly into Maia components. This is by design, to enforce security. All passwords must be stored in secrets, which the component references. Secrets are stored in Maia secret manager in a Full SaaS environment, or in your own cloud platform's secret manager in a Hybrid SaaS environment.
- Create secrets with the credentials that your pipelines will need to connect to third-party services. Read Secrets and secret definitions and Cloud provider credentials for details.
- Update components to point to the secrets you have created.
Webhook and queue triggers for pipelines🔗
Maia doesn't natively support triggering pipelines from a webhook or a queue such as SQS or Azure Queue. However, Maia architecture shouldn't suffer from some of the internal queuing, scaling, or availability limitations that can make creating a queuing solution necessary for Matillion ETL, making triggering from a webhook or queue unnecessary in most scenarios.
We recommend using Maia API for running pipelines directly.
If you need to integrate Maia with an existing system based on webhooks or queues (such as triggering a pipeline when a file lands in an S3 bucket), we recommend using AWS Lambda or Azure Functions to implement an API call based on an event.
Variables🔗
Read the following articles to understand how variables will be migrated to Maia:
Snowflake query tags🔗
Maia supports Snowflake query tags, but they work differently than in Matillion ETL and will require manual intervention to configure query tagging for your pipelines. Your Matillion ETL query tag configuration won't be migrated to Maia automatically.
In Maia, a query tag configuration can be set as a Default session parameter in the environment. Add the following parameter:
ALTER SESSION SET QUERY_TAG = '<tag_value>';
This will apply to all queries executed within that session. This means that you can only have a single query tag configuration within any given environment.
Warning
Setting a query tag by adding ALTER SESSION SET QUERY_TAG = '<tag_value>'; in a SQL Script component is possible but not recommended. Due to the connection pooling architecture of Maia, this won't guarantee that all queries executed by the pipeline will have the query tag applied, as different queries may use different connections from the pool.
Components🔗
Some components need specific treatment when migrated. The following articles describe this in detail: