Skip to content

Launching Matillion ETL using an Azure ARM Template

:::warning{title='Warning'} New customers must go through the Hub to select their preferred cloud provider and data warehouse to begin their Matillion ETL journey. :::

Overview

This is a guide to launching your Matillion ETL instance on Azure, using an Azure Resource Manager (ARM) template. The template is a JSON file that defines the infrastructure and configuration for your project (Matillion ETL).

The template uses declarative syntax, which lets you state what you intend to deploy without having to write the sequence of programming commands to create it. In the template—which is provided in this article—you specify both the resources to deploy and the properties for those resources.

In its simplest structure, an ARM template has the following elements:

  • Parameters: Provide values during deployment that allow the same template to be used with different environments.
  • Variables: Define values that are reused in your templates. They can be constructed from parameter values.
  • User-defined functions: Create customized functions that simplify your template.
  • Resources: Specify the resources to deploy.
  • Outputs: Return values from the deployed resources.

Each element has properties you can set. For more information about these, read Understand the structure and syntax of ARM templates.

These steps below describe how to launch your Matillion ETL instance and deploy an ARM template in the Azure portal.

Prerequisites

Prior to launching a Matillion ETL instance you will need to register for a Hub account. You will also require: - Adequate knowledge about the cloud service account (AWS, Azure, GCP) and Cloud Data Warehouse (Snowflake, Redshift or Google BigQuery) you want to launch. - A user with admin permissions who can access the intended cloud service account. - Access to a cloud storage bucket (S3, Azure, Blob Storage or Google Cloud Storage) to house the transient staging files Matillion used to load data to the cloud. - A network path to access the intended data sources. This may involve working with your network team to enable access to on-premise databases.

Deploying the ARM template and launching Matillion ETL for Azure

:::info{title='Note'} Download and save the ARM template to your computer before starting the template deployment process. :::

  1. Access, and log in to the Azure portal.

  2. Click Templates on the Azure services menu. If you do not see the option, type "Templates" into the search bar at the top of the screen, and click Templates in the search results. Either method will open the Templates page.

  1. Click Create, situated at the top-left of the Templates page.

  2. The Add Template page will be displayed, and will open in the General tab. Type an appropriate Name and Description for the template in the respective fields, then click Next: ARM Template at the bottom of the page.

  3. Open the ARM template you downloaded earlier. You will initially see placeholder information in the ARM Template section in Azure. Copy and paste the contents of the ARM template you downloaded earlier. Then, click Add at the bottom of the page.

  4. You will be redirected to the Azure homepage. Click the Templates link to open the Templates page, where you will see the name of your template in the list. Click on your template to open it in a new window.

  1. Your template details will be displayed, showing its name at the top of the page, Description, Publisher and Modified details, with a View Template link if you wish to preview your template before deploying it.

  2. Click Deploy underneath the name of your template.

  3. The Custom Deployment page will be displayed. Enter template property information into the fields provided. For more information about template properties, see Properties.

:::info{title='Note'} When you deploy a template, you will be required to specify a resource group that will contain the resources within the ARM template. For more information about how to create a resource group, read Create a Resource Group. :::

  1. Once you have completed your template's properties, tick that you agree to Azure's terms and conditions at the bottom of the page, and click Purchase.

:::info{title='Note'} If there are validation errors in your ARM template, you must fix these before you can continue. Azure will provide validation error information if this occurs, and a View Details link will be available fo you to select. :::

  1. You will be redirected to your Azure portal's homepage, where you will receive a notification that the deployment of your ARM template is in progress. Click Deployment in progress, upper-right of the homepage.

  2. When the deployment of the template is finished the Deployment succeeded status will appear at the top of the homepage. Click the Deployment details heading on the page to expand the list, and click your new deployment to open a new page where the Public IP address will be displayed for your newly deployed template.

  3. Copy and paste the IP address into your web browser. Your Matillion ETL instance will launch and the login dialog will be displayed. Enter your credentials, and click Log In.

Properties

The ARM template properties are listed below.

Property Setting Description Default Value
vmName String The name of your Matillion ETL virtual machine. vm-matillion
adminUsername String/Select Username for SSH, or if you are logging into a shell for the virtual machine. centos
authenticationType String/Select Type of authentication to use on the virtual machine. SSH key is recommended. sshPublicKey
adminPasswordOrKey Secure String SSH Key or password for the virtual machine. SSH key is recommended. Enter your key or password
userAssignedIdentityMachine String Name of the managed identity for the virtual machine that will be created. matillion-etl-managed-identity
location String Location for all resources. Should be the same as the Resource Group location selected at the top of the template. centralus
VmSize String/Select The size of the Matillion ETL virtual machine. Standard_B2s
publicIpAddressName String The name of the IP address to be created. matillion-etl-ip-address
public_private String/Select Select Public or Private Network IP Configuration. Select from the drop-down menu
networkInterfaceName String The name of the network interface to be created. matillion-etl-network-interface
storageAccount_name String The name of the Storage Account to be created. matillionetlstorageaccount
virtualNetworkName String Name of the virtual network to be created (or the name of an existing network). matillion-etl-network
addressPrefix String Address prefix for new network (if specified). 10.0.0.0/24
subnetName String Name of the subnet in the virtual network to be created (or the name an existing subnet). Subnet
subnetAddressPrefix String Address prefix for new subnetwork (if specified). 10.0.0.0/24
networkSecurityGroupName String Name of the network security group to be created. matillion-etl-nsg
NSG_SourceIP String Source IP for allowed traffic to Matillion ETL virtual machine. 0.0.0.0/0
offer String/Select Matillion ETL product to launch. User preference. For example, matillion:matillion-etl-snowflake:matillion-etl-snowflake-byol:1.53.8