Updating a high availability cluster

This article describes the options available to you when it comes to keeping your Matillion ETL high availability (HA) cluster's software up-to-date on AWS.

There are two ways to update:

Create a new cluster and migrate your work.
In-place update.

These methods are described in the following sections.

Note

This article only applies to Matillion ETL running on AWS.

Read Updating and migrating overview for detailed pros and cons of the different methods.

Create a new cluster and migrate your work

The preferred—and safest—way to update is to launch a new HA cluster, using the same settings as your existing one and running the latest version of Matillion ETL, and migrate your work from the old cluster to the new one. This option gives you a simple, low-risk back out plan.

We recommend that you create a set of re-runnable scripts that will enable you to automate the build of a new HA cluster. For example, using CloudFormation, based on the templates that Matillion make available for download.

Once the new HA cluster has been created:

In your network settings, allow the new load balancer to accept incoming connections on ports 80 and 443 from the load balancer of the existing HA cluster.
Use the migration wizard described in Performing the migration. This will copy your projects, users, and other resources to the new instance.
Verify that the installation on the new cluster is working correctly.
Switch off all schedules on the old HA cluster, replacing them with schedules running on the new cluster.
Turn off the nodes of the old cluster, keeping them for a defined period of time as a backup.

In-place update

Choose a time when schedules aren't due to run, and proceed as follows.

Note

You may need to ensure that the following URLs are allowed first:

artifacts.matillion.com/rpm/matillion-metl/stable
artifacts.matillion.com/rpm/matillion-cdata/stable

Switch off Matillion ETL on all nodes

SSH onto each node in turn and issue this command:
```
sudo service tomcat stop
```
Confirm that Tomcat is actually stopped via the following command. It should return no active processes:
```
ps -ef | grep java
```

Take a backup of all nodes

We strongly recommend you take backups immediately prior to upgrading, as this will give you a roll-back option if you decide to revert to the original configuration for any reason.

Document the current date and time (reference for Point in Time recovery of RDS Postgres instance, if required).
In the EC2 Management Console, locate the instance running Matillion ETL and select it.
Find Root devices and then select the EBS ID of the volume.
Right-click on the volume and select Create Snapshot.
Enter a Name and Description for your snapshot, including the name of the server.
Repeat for the other EC2 node.
In the EC2 dashboard click SNAPSHOTS, and wait for creation to complete on both instances.

Perform the updates

SSH to the first EC2 instance.
Install Matillion ETL updates:
```
sudo yum update matillion-universal
```
Install Linux security updates:
```
sudo yum update --security
```
Repeat for the second EC2 instance.

Restart Matillion ETL

SSH to the first EC2 instance and use this command:
```
sudo service tomcat start
```
Repeat on the second EC2 instance.

Test the update

Confirm that you can open all the following in a web browser, and that you can log on to Matillion ETL:

Address of first server.
Address of second server.
Load balancer.

Back out of the update

If you decide to revert to the original version, the steps are:

Recreate both EC2 instances from the snapshots.
Revert the RDS database to the point-in-time when the backup was taken.
Restart Matillion ETL on both nodes.

Cleanup

Set a calendar reminder to delete the backup snapshots.