Instance sizes guide
This article intends to give a brief overview of how different instance types can affect your Matillion ETL client. The instance types listed here are recommendations, not instructions. For more detailed information on instance sizes, please consult the documentation for your chosen cloud provider:
- Amazon Documentation on instance sizes
- GCP Documentation on instance sizes
- Azure Documentation on instance sizes
Note
- New customers must go through the Hub to select their preferred cloud provider and data warehouse to begin their Matillion ETL journey.
- If you're a registered Hub customer, ignore this article, and follow the advice and instructions in the Hub documentation, and refer to Instance Sizes.
- This licensing restriction only applies to legacy (marketplace) deployments, not the Hub.
- Matillion ETL is tested to run on instances with processors using x86_64 instruction sets and should be used as such. Other instance types, such as those using ARM processors should be avoided and in many cases cannot be selected for use with Matillion ETL.
Matillion ETL is limited by instance size in the following ways:
- Threads: The number of tasks that can execute simultaneously inside the Matillion ETL instance. Dictated by the number of vCPUs available on the instance. Read Job Concurrency for more information. Although we suggest the following instances for use with Matillion ETL, it is possible to run on machines with greater numbers of vCPUs, and concurrency is not capped in this respect.
- Concurrent users: Instances with larger numbers of concurrent users will require larger instance sizes to remain performant:
- For legacy marketplace-billed instances, this number is a hard cap of users that can be logged in concurrently.
- Hub instances have a practical limit to stop the VM from running out of resources. The limits are as follows:
- 2 vCPUs = 16 users
- 4 vCPUs = 32 users
- 8 vCPUs = 64 users
- Environments: The total number of environments across all projects that can be defined in a single instance of Matillion ETL. 1 Environment = 1 Project. Thus, the maximum number of environments is equal to the maximum number of projects connected to a server.
- Enterprise Mode: Enterprise Mode features are only available on certain instance sizes. For Hub editions and their specific features, read Editions.
AWS instance recommendations
| Instance Size | Max. Concurrent Users | Max. Environments | Threads | vCPUs | Memory (GiB) | Suitable for | Enterprise Mode Available? |
|---|---|---|---|---|---|---|---|
| t3.medium | 2 | 6 | 4 | 2 | 4 | Teams of 1 or 2 persons | No |
| m5.large | 5 | 15 | 4 | 2 | 8 | Teams of 2 to 5 persons | Yes |
| m5.xlarge | 12 | 36 | 8 | 4 | 16 | Teams of 6 to 12 persons | Yes |
| m5.2xlarge | 25 | 200 | 16 | 8 | 32 | Teams of 6-25 or a highly available production environment | Yes |
| r5.large | 2 | 15 | 4 | 2 | 16 | Production Environment. Low user count. | Yes |
| r5.xlarge | 2 | 50 | 8 | 4 | 32 | Production Environment. Low user count. | Yes |
| r5.2xlarge | 2 | 200 | 16 | 8 | 64 | Production Environment. Low user count. | Yes |
Azure instance recommendations
| Instance Size | Max. Concurrent User | Max. Environments | Threads | Azure VM Size | vCPUs | Memory (GiB) | Enterprise Mode Available? |
|---|---|---|---|---|---|---|---|
| Medium | 2 | 6 | 4 | B2s | 2 | 4 | No |
| Large | 5 | 15 | 8 | D4s_v3 and D4ds_v3 | 4 | 16 | Yes |
| XLarge | 12 | 36 | 16 | D8s_v3 and D8ds_v3 | 8 | 32 | Yes |
We do not recommend a Medium instance by default for first-time users, as these tend to be for specialist purposes and lack many Enterprise features of the larger instances.
GCP instance recommendations
| Instance Size | Max. Concurrent User | Max. Environments | Threads | GCP VM Size | vCPUs | Memory (GiB) | Enterprise Mode Available? |
|---|---|---|---|---|---|---|---|
| Medium-M | 2 | 6 | 4 | n2-standard-2 | 2 | 8 | No |
| Large-L | 5 | 15 | 8 | n2-standard-4 | 4 | 16 | Yes |
| XLarge-XL | 12 | 36 | 16 | n2-standard-8 | 8 | 32 | Yes |
Snowflake: The size of your instance (per the table above) is based on the number of vCPUs your chosen VM has when launching from the GCP Marketplace. Thus there is only a single marketplace listing for Matillion ETL for Snowflake on GCP and the VM you use is not restricted to those given in the "GCP VM Size" column.
Google BigQuery: It is not possible to change your Matillion ETL for BigQuery instance size. Instead, we advise beginning a new subscription with the instance size you wish to use, migrating resources, and then stopping your old subscription. Please contact Matillion Support for more information or help with this.