Upgrade the Tomcat version
Matillion's Tomcat service has always been a high-priority target for bug fixes and CVEs. Before version 1.78, it was not possible to upgrade Tomcat separately from a Matillion ETL instance-to-instance migration, posing problems to customers wanting to keep Matillion ETL up-to-date.
However, starting with release 1.78, we have implemented an Upgrade Tomcat feature, which lets you upgrade Tomcat as part of an in-place update.
Upgrade Tomcat requires filesystem changes and will be available on new or migrated 1.78+ instances. 1.78 upgrades will continue to work as before, but will not have this feature available.
How to upgrade Tomcat
Note
Users require the Tomcat version updates permission, which is located in Admin → Manage Permissions.
- Click Admin → Upgrade Tomcat.
The Upgrade Tomcat dialog box will appear, showing:
- Which version of Tomcat is currently running on the server.
- Which version will be running on the server following a restart.
- Links to the most recent Tomcat version and all available versions.
A new version of Tomcat can be downloaded by entering its version number into the lower text box and clicking Download. The download files are typically less than 15 MB and unpack to around 20 MB. If an error occurs during the download process, a dialog box will be shown with the error.
Note
We recommend that you use the most recent version of Tomcat to upgrade, although downgrades are supported, too.
Supported Tomcat versions
Matillion ETL currently works with versions of Tomcat in the 10.1 series. This series is still fully supported by Apache and receives security and bug fixes. The 11.0 has been released, but makes substantial changes to configuration and layout. It has not been feasible for us to support both the 10.1 and 11.0 series simultaneously in Matillion ETL, although we are working to do so in a future release.
Following the download, the Upgrade Tomcat dialog will contain both the distributed version and your new version of Tomcat. Select your new version and click "Update"—which should complete in about 1 second. This operation will prompt you that a restart is required. We recommend that you restart the Matillion ETL server as soon as possible to avoid system instability.
Filesystem changes
As a result of a change to how Matillion ETL stores its working files, the following environment variable is now set on all instances:
EMERALD_WORKING_DIR (default value: /usr/share/tomcat, which is the old location).
This environment variable will be set to our preferred new location, /usr/share/local/tomcat, in new 1.78+ instances. This will also make it easier to back up any files that are stored here, as they will no longer be intermingled with system files.
The following files and directories are moved:
| Before | After | Notes |
|---|---|---|
/usr/share/tomcat |
/usr/share/tomcat -> /opt/tomcat |
Previously the location of the Tomcat installation. It is now a symlink to the new locations. |
/usr/share/tomcat |
/usr/share/local/tomcat |
Previously the location of the Emerald working files. These are now stored in a new location. |
/etc/tomcat |
/usr/local/share/tomcat |
Previously the location of the Tomcat configuration files. It is now stored in the "working config" and is shared between all installed Tomcat versions. |
/opt/tomcat |
For example: /opt/tomcat10.1.26 |
The version of Tomcat currently in use is symlinked so that it can be switched easily. |
The "working files" referred to are:
extract_connectorsapi_profilesoauthDriversbilling
Recovering from failed upgrades
Although it is rare, we have found that some versions of Tomcat have been released with regressions and incompatibilities. These versions have mostly been removed from the Tomcat website above and can no longer be downloaded. It is simple to recover from this situation in the event that Matillion ETL ships with such a version. Shell into the instance, and list the /opt/ folder:
# ls -l /opt
total 8
drwxr-xr-x. 9 root root 4096 Mar 28 02:23 google-cloud-sdk
drwxr-xr-x. 6 root root 56 Mar 28 02:12 python-2.7.18
drwxr-xr-x. 6 root root 56 Mar 28 02:18 python-3.12.4
lrwxrwxrwx. 1 root root 18 Mar 28 02:09 tomcat -> /opt/tomcat10.1.36
drwxr-x--- 17 tomcat tomcat 4096 Mar 27 14:12 tomcat10.1.26
drwxr-xr-x. 17 tomcat tomcat 4096 Mar 28 03:08 tomcat10.1.36
This shows that there are two versions of Tomcat present, and that 10.1.36 is linked. Also, there are two versions of Python available, and the Google Cloud SDK is installed.
Adjust the symlink to a known good version and restart.
# rm /opt/tomcat
# ln --symbolic /opt/tomcat10.1.26 /opt/tomcat
# systemctl restart tomcat