Skip to content

Diagnostic data policy🔗

This article describes Matillion's policy and procedures concerning the handling of diagnostic data.

Diagnostic data is any data provided by a customer to the Matillion support team to assist in identifying and resolving a customer's issue while using the Matillion ETL product.

Diagnostic data policy🔗

Matillion ETL includes some self-monitoring capabilities and may capture certain diagnostic data in the event of runtime failures. Matillion support may request that a customer enables extra diagnostic information during problem investigation, and request that a customer sends that diagnostic data for analysis. Examples include:

  • Java heap memory dumps.
  • Component-level log traces.
  • Operating system log files.
  • Matillion ETL database backups.
  • Other data as required by the support process.

Warning

These diagnostics capture the state of a running system at a point in time, and so may contain fragments of personally identifiable information, personal, or sensitive data.

Non Disclosure Agreement (NDA)🔗

  • Customers will be given the option to sign Matillion's mutual, support-specific NDA before sending any diagnostic data.
  • If a customer is not able to sign Matillion's NDA, Matillion Support may not be able to solve problems without diagnostic data. In some circumstances, Matillion may not be able to receive data without an NDA in place.

Note

Matillion's mutual NDA is available for download on the side bar for this article.

Triggering the transfer of diagnostic data to Matillion🔗

  • By default, diagnostic data transfer to Matillion will happen only as a result of customer action. The customer will be given the option to "push" data to Matillion.
  • By default, the software will never automatically push diagnostic data to Matillion or "call home" in any way. We will never (and in fact can never) "pull" diagnostic data from an instance of the software.
  • If diagnostic data pushed to Matillion is ever automated, the customer will have to explicitly choose to opt in.

Data security🔗

  • Matillion will provide appropriate mechanisms to ensure integrity and privacy of data in transit. All communication outside of the infrastructure holding the data is via HTTPS. All data is processed automatically and once uploaded, data does not leave the Amazon cloud environment.
  • Diagnostic data will be encrypted at rest while on Matillion's cloud infrastructure.

  • Matillion uses Amazon S3 to store and receive diagnostic data. Server-side encryption with Amazon S3-managed encryption keys (SSE-S3) uses strong multi-factor encryption. Amazon S3 encrypts each object with a unique key. As an additional safeguard, it encrypts the key itself with a master key that it rotates regularly. Amazon S3 server-side encryption uses 256-bit Advanced Encryption Standard (AES-256).

    Note

    You may need to install the AWS CLI using pip install awscli to move files to S3.

  • No Matillion employees have access to the encryption keys, and policy access to the Amazon S3 bucket is restricted to a limited administration team only.

  • Data will be received, stored, and processed in the EU-west region.

Retention🔗

Matillion will permanently erase data held for diagnostics according to the following rules:

  • Raw diagnostic data—7 days.
  • Some data (such as HPROF files) will be analyzed by automatic tools. Automated analysis outputs—30 days (analysis outputs do not usually contain customer data directly).
  • Sometimes manual analysis is required. Matillion will take steps to ensure that the outputs of manual analysis are deleted daily—1 day.
  • For long running support issues, Matillion will inform the customer if it is necessary to extend the retention period of data.

Purpose and usage of data🔗

  • Matillion will not request more diagnostic data than is needed for analysis.
  • Diagnostic data will only be used for support and problem diagnosis.

Sharing🔗

  • Matillion staff are prevented from easily copying diagnostic data to their company devices.
  • Matillion will always seek the customer's approval before sharing data with subcontractors if it's necessary during problem diagnosis.

Erasure and restriction🔗

  • The customer has the right to request that Matillion stop analysis and delete all diagnostic data. This can be done by contacting Matillion support. In such instances, Matillion may not be able to diagnose or help with any problem associated with that diagnostic data.

Procedure for sending diagnostic data🔗

  1. Raise a case with Matillion Support and get a case reference number.
  2. Review Matillion's diagnostic data policy.
  3. Optionally sign Matillion's mutual NDA.
  4. In the event of a problem that requires analysis, Matillion Support will request that you enable or directly send the relevant diagnostic data files (see details below).
  5. Await instruction from Matillion Support on next steps.

Enabling/finding heap dumps on your instance🔗

For most instances, Heap Dump generation should be enabled by default. If not, or if you are unsure, they can be enabled by finding the file:

/etc/sysconfig/tomcat

Open the file and find the line containing:

JAVA_OPTS

Then, add the following options towards the end. Don't change the rest of the line:

-XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp

For example, below is a "before and after" example of what the line might look like.

Before:

JAVA_OPTS=" -Djavax.net.ssl.trustStore=/usr/lib/jvm/jre/lib/security/cacerts -Djavax.net.ssl.trustStorePassword=changeit -Djava.security.egd=file:/dev/./urandom -XX:+UseG1GC -XX:OnOutOfMemoryError=/usr/share/emerald/WEB-INF/classes/scripts/oom.sh"

After:

JAVA_OPTS=" -Djavax.net.ssl.trustStore=/usr/lib/jvm/jre/lib/security/cacerts -Djavax.net.ssl.trustStorePassword=changeit -Djava.security.egd=file:/dev/./urandom -XX:+UseG1GC -XX:OnOutOfMemoryError=/usr/share/emerald/WEB-INF/classes/scripts/oom.sh -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"

Save and close the file and then restart tomcat.

Finally, if you suspect the Heap Dump is being written but you are not sure where, use the following command to find the location:

sudo find / -name "*.hprof" -type f 2>/dev/null

Upload Heap Dump file to Matillion🔗

Compress the HPROF file before uploading. For example, use the following command to compress the file:

gzip <filename>.hprof

Your Support engineer will have provided a one-time upload key. Replace and below:

curl -H "X-API-Key: <KEY>" \

     -F file=@/tmp/<filename>.hprof.gz \

     https://heap-dump-upload.matillion-sa.matillion.com/upload

You should see a response like the following if the upload was successful:

{

  "status": "ok",

  "bucket": "mtln-heap-dump-staging",

  "key": "20260317T120000Z_a1b2c3d4_java_pid123.hprof.gz",

  "size": 804363055

}

Removing HPROFs from the Heap Dump🔗

The location is determined by the value provided for attribute XXXX in file tomcat.conf.

Assuming it is set to /tmp, use the following command to remove all HPROF files:

rm /tmp/*.hprof
© Copyright 2026 Matillion Ltd | Legal | Privacy Policy | Disclaimer | Modern Slavery Statement