Run Notebook
Overview
Use the Run Notebook component to execute a Databricks notebook from a Matillion ETL orchestration job. You might find this component useful if you wish to conflate ETL/ELT tasks with the subsequent analysis performed in your Databricks notebooks.
Properties
Name = string
A human-readable name for the component.
Cluster = drop-down
Select the Databricks cluster. The special value, [Environment Default], will use the cluster defined in the active Matillion ETL environment.
Notebook = string
The path to a Databricks notebook. Search via a filepath string or browse the list of directories and notebooks based on the connected Workspace in the Matillion ETL environment. Read Copy notebook path (AWS) or Copy notebook path (Azure) to learn how to retrieve your notebook file path.
Execution Mode = drop-down
- Asynchronous: Runs the task and sends a request to Databricks. The status of the task is ignored in terms of continuing the Matillion ETL job.
- Synchronous: Runs the task and polls Databricks for a status update. The Matillion ETL job is delayed until Databricks has returned a status of
TERMINATED. The results of which includeSUCCESS,FAILED,TIMEOUT, orCANCELLED.
The default is asynchronous.
Poll Interval (seconds) = integer
The polling interval to Databricks. The default is 5 seconds. Only available when Execution Mode is synchronous.
| Snowflake | Delta Lake on Databricks | Amazon Redshift | Google BigQuery | Azure Synapse Analytics |
|---|---|---|---|---|
| ❌ | ✅ | ❌ | ❌ | ❌ |