Upgrade: Iterators

All Matillion ETL iterator components are migrated to the equivalent iterator component in Maia.

---

Upgrade path

In many cases, the migration proceeds without manual refactoring of pipelines, but there are some specific differences you may need to address in your migrated pipelines. These are outlined below.

Stop on condition

Iterators with Stop On Condition set to No in Matillion ETL continue to work in Maia without any issues, and perform as expected.

Behavior of iterators in Maia with Stop On Condition set to Yes differs from Matillion ETL behavior. Maia evaluates the stop condition after each iteration completes, unlike Matillion ETL, which evaluates it before each iteration runs. You may need to refactor your pipeline to account for this change in behavior.

Maia supports copied variables with Stop On Condition.

Maximum concurrent iterations

Maia introduces a property called Maximum concurrent iterations for all iterator components. This sets a maximum on the number of concurrent iterations each component attempts. This ensures the workload accommodates any source and target constraints.

Read Iterator components in the Maia documentation for more information on how this property works.

During migration, this property defaults to 16 for all iterators, but you can change this default when you import your jobs into Maia.

To change the default value during the import:

1. Click Edit preferences in the Importing files panel.

2. In the Iterators section of the Edit import preferences dialog, you can:

   • Toggle off the Add a cap for maximum concurrent iterations option. This will allow unlimited concurrent iterations for all iterator components during migration.
   • Set the Maximum concurrent iterations property to your desired value. This will be applied to all iterators during migration.

3. Click Apply & re-run.

4. If you again examine the migration report, it will show the new value for Maximum concurrent iterations for all iterators.
5. Click Import to complete the import process, and continue as described in Import to Maia.

After import, you can manually change the Maximum concurrent iterations property for individual iterator components in your pipelines as needed.