Skip to content

Launching Matillion ETL from AWS Marketplace

Warning

  • This article is specifically for customers who have an existing Matillion ETL installation launched via the AWS Marketplace and wish to launch another of the same kind. If you haven't subscribed to Matillion ETL on AWS in the past, the listings given here won't be accessible.
  • New and existing customers must launch Matillion ETL through the Hub.

The following guide describes how to launch and connect Matillion ETL for Snowflake using Amazon Machine Image (AMI), and launching the software through your EC2 instance. The method is the same for launching Matillion ETL for Amazon Redshift.

To begin launching Matillion ETL, you must browse to the AWS Marketplace listing for the product of your choice:

Note

The above listing will not work if you haven't already subscribed to Matillion ETL on AWS. New customers should launch Matillion ETL via the Hub.

Prerequisites

Prior to launching a Matillion ETL instance you will need to be an existing non-hub customer, and are required to have:

  • Adequate knowledge about the cloud service account (AWS, Azure, GCP), and Cloud Data Warehouse (Snowflake, Redshift or Google BigQuery), you want to launch.
  • A user with admin permissions who can access the intended cloud service account.
  • Access to a cloud storage bucket (S3, Azure, Blob Storage or Google Cloud Storage) to house the transient staging files Matillion used to load data to the cloud.
  • A network path to access the intended data sources. This may involve working with your network team to enable access to on-premise databases.

Launching Matillion ETL from the AWS Marketplace

Note

It is also possible for existing customers to locate the Matillion ETL listing via the EC2 Console. Once on the EC2 Console, navigate to "AMIs" then set the filter to "Private".

To configure and launch Matillion ETL for Snowflake or Amazon Redshift, use the following steps:

  1. Sign in to the AWS Marketplace, and enter Matillion ETL for "Snowflake" or "Redshift", respectively, into the search bar, at the top of the marketplace.

    Note

    Please raise a support case if you're unable to see the listing you require. For more information, read Getting Support

  2. The Subscribe to this software page will be displayed. Next to the Matillion ETL listing, click Continue to Subscribe.

    Note

    At this stage, if you aren't already logged into the AWS Marketplace, you will be prompted for your AWS account credentials before you can continue.

  3. Depending on your current setup, you may need to click Continue to Configuration.

  4. On the Configure this software page, use the Delivery Method drop-down menu to select Amazon Machine Image with 64-bit Amazon Machine Image (AMI) as your Fulfillment Option, then click Continue to Launch.

    Note

    • Where possible, select the most up-to-date product version available.
    • Launching Matillion ETL at this stage will not launch your instance, and will not charge your account.

  5. You will be redirected to the Launch this software. Review your configuration, and use the drop-down menu in the Choose Action section to select Launch through EC2. This will launch the software from the website, then click Launch.

Choosing the Instance Type

The browser will redirect you to the Choose an Instance Type page. Choose one of the supported Instance Types, and click Next: Configure Instance Details. Each instance type is appropriately sized to support a given number of users. Your Matillion ETL instance will automatically scale its capabilities according to your chosen instance size.

Note

The instance type you select affects the number of users that can use Matillion ETL concurrently. For more information, read Changing EC2 Instance Size:

  • t3.medium: For teams of 1-2 data professionals using Matillion ETL concurrently.
  • m5.large: For teams of 3-5 data professionals using Matillion ETL concurrently.
  • r5 type instance: For the production instances that run jobs but need few users.

Configuring Instance Details

  1. This is a manual setup instead of a 1-click setup. There are some settings you can specify on this screen that are unavailable through 1-click that improve the functionality of your Matillion Instance. Please follow the configurations as shown below:

  2. Number of instances: leave as default (unless you want multiple instances).

  3. Purchasing options: leave as default i.e. unchecked.
  4. Network: Choose a VPC. The zone in which this VPC is located should ideally be the same as your Snowflake account (European Snowflake accounts should be paired with an EU-based AWS region, for example).
  5. Subnet: Choose a subnet. Each subnet resides in one availability zone.
  6. Auto-assign public IP: This depends very much on the setup. By default, a new VPC will not have VPN connections or NAT Gateways available, so for Matillion ETL to connect to the internet and for you to access Matillion ETL, this will normally need to be set to Enable.
  7. IAM Role: IAM Roles are used to allow your instances to use the Amazon APIs securely without manual management of security keys. To use all the features of Matillion ETL, you should configure an IAM Role for your instance to use. This procedure assumes you do not already have an appropriate IAM role setup (if you do, select it). For detailed information on creating IAM roles, refer to the Additional Information section in this guide.

  8. Choose below settings for Shutdown behavior: Enable termination protection, Monitoring, and Tenancy:

    • Shutdown behavior: leave as default i.e. Stop.
    • Enable termination protection: we suggest setting it to, Enabled.
    • Monitoring: we suggest setting it to, Enabled.
    • Tenancy: run a shared hardware instance.

  9. Once you have completed all the details, click Next - Add Storage.

  10. On the Add Storage page, choose the default root volume size, and click Next - Tag Instance.

    Note

    You can attach additional EBS volumes and instance store volumes to your instance, or edit the settings of the root volume. You can also attach additional EBS volumes after launching an instance, but not instance store volumes. We recommend increasing volume by the factor of 5.

  11. On the Add Tags page, add any instance tags you require. For example, you could define a tag with key = Name and value = Webserver. A copy of a tag can be applied to volumes, instances, or both. Then, click Next - Configure Security Group.

  12. On the Configure Security Group page, a default security group will be created with the minimum set of ports. If required, name and adjust as to your security requirements. Next, click Review and Launch.

    Note

    The default recommended security group uses SSH (port 22) and HTTP(S) (port 80 and 443) access to the instance. The range of allowed IPs should be tailored to your needs.

  13. Finally, in the Review Instance Launch page, review your settings and configuration details for your Matillion ETL instance, then click Launch.

    Note

    Once the AMI has initialized, which normally takes a few minutes, you can access Matillion ETL by entering the hostname of the instance into a web browser.

  14. Log in to your copy of Matillion ETL with ec2-user as the username and the instance ID i-xxxxxxxx (e.g. i-88ed92c6) as the password. We recommend changing your password as soon as possible. For more information, read Change My Password.

Additional information

Part of configuring instance details includes creating a new IAM role in the AWS Console. Follow these steps to create a new IAM role:

  1. In the Configure Instance Details page, click Create New IAM Role next to the IAM Role field.
  2. The browser will redirect you to the AWS Console, Identity and Access Management (IAM), then Roles. Click Create Role at the top of the page.
  3. The Select type of trusted entity page will be displayed. Under Create role, select AWS Service, this is the preferred trusted entity, and choose a common use case from the available list. The preferred choice is EC2. Click the Next: Permissions button at the bottom of the page.

  4. On the next page, attach one or more Permission Policies. You can choose from the list of existing policies, or you can create a new one if needed, by clicking the Create Policy button at the top of the page. From the list of policies, select the following, and when you have made your selections, Click Next: Tags:

    • AmazonSNSFullAccess
    • AmazonSQSFullAccess
    • CloudwatchFullAccess
    • AmazonRDSReadOnlyAccess
    • AmazonS3FullAccess

    Note

    For more information about permission policies, read Managing Credentials.

  5. The next page requires the addition of IAM Tags. These are key-value pairs you can add to your role. Tags can include user information, such as an email address, description, or a job title. Click Next: Review.

  6. Add information in the empty fields to complete the Review page. Review the details of the IAM role before you create it, then click Create Role.
  7. Finally, the IAM role has been created and you will find the newly created role in the list.

Next Steps

We also recommend the following pages:

© Copyright 2024 Matillion Ltd | Legal | Privacy Policy | Disclaimer | Modern Slavery Statement