Upgrade: Bash scripts

Unlike Matillion ETL, which runs on your own Linux virtual machine (VM), Maia uses a SaaS operating model, which means there is no underlying VM you can use to run Bash scripts on. Instead, you can use the Script Pushdown component to run your Bash scripts in your own cloud infrastructure.

---

Upgrade path

If you have a job that contains a Bash Script component, follow these steps in Maia to configure the imported pipeline to use Script Pushdown instead:

1. Click Edit preferences in the Importing files panel.
2. Toggle Convert to Script Pushdown to On. The default setting for this option is Off, meaning that no scripts will be converted unless you explicitly enable it.

3. This will present you with a set of parameters to configure the connection to your cloud infrastructure. Complete these as follows:

   • Host: Your SSH host.
   • User name: Your SSH username. Run echo "$USER" to prompt your shell for your username.
   • Connection timeout (ms): The length of time in milliseconds before the connection times out. The default is 3000.
   • Port: The port number used to connect to the SSH host. The default is set to port 22.

   The selected values are applied to all Script Pushdown components in the pipelines being imported. Other Script Pushdown properties must be configured on the individual components.

4. Click Apply & re-run.

5. If you examine the migration report, it will now show "Bash Script component has been converted to a Script Pushdown component."

6. Click Import to complete the import process, and continue as described in Import to Maia.

Alternative approaches

If you choose not to use the Script Pushdown component, you need to refactor any jobs that use Bash scripts to run in a Maia pipeline. The following guidelines can help you decide how to do this:

• See if the functionality has a native component equivalent, such as the Print Variables component.
• See if the workload can be written in a Python script using the Python Pushdown component. This option is only available for Snowflake environments.

If none of the above options are suitable, you can continue to use the Bash Script component without converting to Bash Pushdown, but we don't recommend this option. The script won't run in a Full SaaS deployment, and may run with problems in a Hybrid SaaS deployment. If you wish to continue with this option, we recommend you discuss the matter with Matillion Support.

Automatic variables

Maia doesn't support directly accessing automatic variables through the Bash Pushdown component.

If you require this functionality, you can use an Update Scalar component to write the values to user-defined variables, which can then be passed to the script.