Launching troubleshooting (AWS)

Note

New customers must go through the Hub to select their preferred cloud provider and data warehouse to begin their Matillion ETL journey.

The below QandA troubleshooting guide will hopefully resolve any issues you have when setting up Matillion ETL for Redshift for the first time. If a problem persists, please contact Matillion support.

Q. The product cannot get through loading my Project. Why?

A: Matillion ETL relies on websockets for its communications and is therefore reliant on the user having access to a compatible internet browser and a modern, stable internet connection. Weaker connections with either low bandwidth or high latency may experience problems loading Matillion ETL, especially when loading very large Projects.

Q. The product loads, but then immediately says "Connection lost". Why?

A: The "connection" that is being lost refers to a websocket connection. Once the site loads, all further communication is done over a websocket to broadcast changes to other users.

If your browser is not directly connected to the Matillion ETL instance, then this error may occur. There could be an EC2 Load balancer and/or a proxy server between your browser and the Matillion ETL instance.

We recommend using an ALB rather than ELB for most situations.

For EC2 Load Balancers, log in to your AWS account and browse to the EC2 services through Services → EC2 at the upper left corner of the page.

Within EC2, users can select Load Balancers from the lists of options.

If no load balancers are associated with this account, it is likely that a proxy is the cause of this issue. Otherwise, select the applicable load balancer, click the Listeners tab, and re-configure the "Listeners" to use TCP instead of HTTP (and SSL instead of HTTPS, if you want to use SSL connections).

Since websocket connections start as HTTP requests (with some special headers), proxy servers may choose to buffer, compress, or otherwise change the data. If your proxy cannot be configured to support websockets, then enabling SSL and accessing Matillion over SSL (HTTPS) will usually work since the proxy will pass the encrypted data along unchanged.

Q. The Create Project dialog can't find any Redshift clusters. How do I get connected?

A: Matillion ETL uses the AWS API to discover Redshift instances running inside the same region as the Matillion instance.

If the IAM Role attached to the instance does not have the permission to list the Redshift clusters, then the Cluster section will be empty. You can add such a policy.
If you did not attach an IAM role, this can be worked around. Read Manage Credentials to see how to add manual API keys and set them as the active credentials.
If your Redshift cluster is in a different region, you should consider moving Matillion to the same region as it will reduce latency between the instance and Redshift.
Regardless of API availability or being in a different region, you can always connect to any Redshift cluster by manually completing the connection settings during project creation. Most of the settings can be retrieved from the AWS Console for your account. Security groups must still allow connections to Redshift that originate from the Matillion instance.

Q. Matillion ETL cannot connect to Redshift, but I don't know why. Any ideas?

If you encounter a "Connection attempt timed out" error while editing or creating a Project in Matillion ETL, it is likely related to security groups. Often the security group assigned to the Redshift cluster and the security group assigned to the Matillion ETL instance are different groups. To resolve this issue, add the Matillion ETL security group as an inbound rule to the VPC Security Group in Redshift Cluster configuration.

Click Services, then click or type Amazon Redshift.
For the corresponding Redshift cluster, click VPC Security Groups.
Click the Inbound tab and then click Edit and then click Add Rule.
Add a new rule with:
- Type: Redshift
- Set the Source parameter to the name of the security group assigned to the Matillion ETL instance.

Q. I forgot to launch the product with an IAM role attached, but need to use features that require access to the AWS APIs. How do I provide credentials?

A: Read Manage Credentials, which explains how to use an IAM User instead of an IAM Role.

A: From version 1.21.5, new instances are launched with security enabled by default.

On first launch, the username is set to ec2-user and the password is set to the newly created instance ID. Read Change My Password for details on how to change your password.

Since Matillion have no access or control over your own AWS resources, we can't reset or manage passwords for you.

Q. How can I add additional users?

A: Read User Configuration to learn how to add users.

Q. I have set up integration with my corporate directory server, but noone can log in. What could be wrong?

A: Many directory servers will only talk over a secure connection. So if you have configured a connection URL such as:

ldap://server:389

Then try:

ldaps://server:636

You must also provide a username and password for the initial bind, as many servers reject anonymous binds.

Q. When using Matillion over SSL, my browser warns me the site is not secure. Why is this?

A: We generate self-signed SSL certificates for each version, and so the browser cannot validate them. You may use your own certificates provided by AWS or another provider by uploading them. For more information, please see this article for information about managing the server, including how to upload new certificates.

Q. I do not have direct access to Matillion ETL, how can I tunnel Matillion ETL?

A: Matillion ETL treats localhost as a special case, so to tunnel Matillion ETL, you will need to bind to a local adapter. e.g. if you use ssh to create the tunnel use the following syntax.

ssh -i <keyfile>.pem ec2-user@<Matillion IP> -L <local IP>:8080:<Matillion IP>:8080

The <local IP> parameter is optional, but we recommend explicitly specifying a Local IP address. Do not bind to localhost or 127.0.0.1 as this may hinder the websocket connection required by Matillion ETL. Users need to create an entry in the host file to map 127.0.0.1 to an IP to access the Matillion ETL instance. Accessing via localhost will not work.

Then use the local IP to connect, for example:

http://<local IP>:8080/

Alternatively, if you are creating the tunnel in PuTTY, you will need to tick Local ports accept connections from other hosts. See below:

Q. I needed to resize the cluster but after using Matillion, the resize fails. How can I resize my cluster?

A: To fix, remove all views from the cluster prior to resizing. You can remove all views for a given environment by right clicking on it and then clicking Delete Views.

Matillion will automatically recreate the views on the next job validation or run, so be sure to resize the cluster before any jobs are run.

Q. Should I use ELB or ALB?

A: For the end user, the difference can often be that ELB requires additional configuration, while ALB generally works out of the box. A description on ELB and ALB can be found in the Amazon documentation, and a direct comparison between the load balancers can be found here.

We generally recommend using an ALB with Matillion ETL.

Can I get help and support, if I need it?

A: If you require further help launching Matillion ETL for Redshift, contact Matillion Support.