Community
Forums
Blogs
Roadmap & Ideas
User Groups & Events
Stack Overflow
Reddit
Documentation
Data Productivity Cloud Docs
Matillion ETL Docs
YouTube
Connectors
Migration
Support
Knowledge Base
My Support Cases
About Support
Training & Certification
Academy
Data Productivity Cloud
Matillion ETL
Certifications
Services
Professional Services & Training
Consulting Partners
Matillion.com
Support
Data Productivity Cloud
Matillion ETL
Style guide
Data Loader CDC
Matillion Docs
Data Productivity Cloud
Data Productivity Cloud
Home
Get started
Get started
Registration
Free trial
Data Productivity Cloud new user tutorial
Quickstart guides
Quickstart guides
Services
Set up a Matillion managed project
Pipelines tutorial
Setup guides
Setup guides
Setup overview
Hybrid SaaS Snowflake on AWS
Hybrid SaaS Snowflake on Azure
Hybrid SaaS Snowflake on Snowpark
Full SaaS Snowflake
Hybrid SaaS Databricks on AWS
Hybrid SaaS Databricks on Azure
Full SaaS Databricks
Hybrid SaaS Amazon Redshift on AWS
Full SaaS Amazon Redshift
Maia
Maia
Overview of Maia
Start using Maia
Maia prerequisites
Maia pipelines
Maia context files
Maia sessions and tools
Using Maia for root cause analysis
Using Maia for anomaly detection
Create a custom connector with Maia
Video Guides
Video Guides
How-to
Features
Connectors
AI
Core concepts
Core concepts
Data Productivity Cloud overview
Architecture overview
Connectors overview
DataOps overview
Getting support
Supported browsers
Data Productivity Cloud release FAQ
Cloud data platforms
Cloud data platforms
Troubleshooting data warehouse connectivity
Snowflake
Snowflake
Snowflake role privileges
Using Snowflake key-pair authentication
Databricks
Databricks
Databricks
Databricks Partner Connect
Amazon Redshift
Amazon Redshift
Amazon Redshift connection
Amazon Redshift Serverless connection
Managing secrets
Managing secrets
Secrets overview
AWS Secrets Manager
Azure Key Vault
GCP Secret Manager
Security
Security
Overview
Deployment options
Git integration
Cloud environment roles and permissions
Cloud platform security and compliance
Authentication and access control
Data source connectivity
Allowing IP addresses
Software updates and patches
Data gathering
Data lineage
Private link
SSH Tunneling
Administration
Administration
User management
User management
Manage email address and password
Edit user permissions
Create an account
Edit account details
Account regions
Manage account users
Subscriptions, usage, & billing
Subscriptions, usage, & billing
Usage dashboard
Subscriptions
Subscriptions
Editions
Payment and invoicing
Marketplace subscriptions
AWS Marketplace subscription
Azure Marketplace subscription
Authentication
Authentication
Multi-factor authentication
SSO overview
Agents
Agents
Data Productivity Cloud agent
Data Productivity Cloud agent
Agent overview
Create an agent in your infrastructure
Manage agents
Restart an agent
Pause an agent
Refresh agent credentials
Agent updates
Agent logs
AWS agent setup
AWS agent setup
Manual agent setup
Agent installation using a CloudFormation quick-create link
AWS IAM roles
Add agent credentials to AWS Secrets Manager
Connect an agent via AWS PrivateLink
Azure agent setup
Azure agent setup
Agent installation using an ARM template
ARM template permissions
Configuring a key vault for Azure agent
Download and amend the Azure ARM template
Azure managed identity
Azure agent troubleshooting
Additional configuration
Additional configuration
Scaling best practices
Customizing agent networking and connectivity
Uploading external drivers to the agent
Installing Python libraries
Optional agent parameters
Matillion agent for Snowflake
Matillion agent for Snowflake
Installing Matillion agent for Snowflake
Secrets in Matillion agent for Snowflake
Troubleshooting Matillion agent for Snowflake
Streaming agent
Streaming agent
Create a Streaming agent
Sizing streaming agents
Streaming agent logs
Modify access to source databases
AWS installation
AWS installation
AWS IAM roles for Streaming agents
AWS Fargate manual Streaming agent setup
Agent installation using a CloudFormation quick-create link
Azure installation
Azure installation
Streaming agent installation on Azure
Download and amend the Azure ARM Streaming agent template
ARM Streaming agent template permissions
Configuring a key vault for Azure Streaming agents
Communicating with other Azure services
Azure Streaming agent troubleshooting
GCP installation
GCP installation
GCP Streaming agent install
Designer
Designer
Designer overview
Designer UI basics
Pipelines tutorial
Troubleshooting Designer
Your Designer projects
Your Designer projects
Projects
Environments
Environment access
Cloud provider credentials
Branches
Pipelines
Shared pipelines
Getting started with data loading
Secrets and secret definitions
OAuth
Schedules
Artifacts
User access, roles and permissions
Networks
Databricks Jobs Compute
Connecting via AWS PrivateLink
Git version control
Git version control
Git in Designer
Commit changes
Compare changes
Merge from branch
Push local changes
Pull remote changes
Pull requests
Reset branch
Revert changes
Connect your own Git repository
Connect your own Git repository
Overview
Reconnecting your Git user account
GitHub
GitHub
Overview of the Matillion GitHub app
Prerequisites
Installing the Matillion GitHub app
Troubleshooting GitHub repository connection
Azure DevOps
Azure DevOps
Overview of the Matillion Azure app
Prerequisites
Installing the Matillion Azure app
Troubleshooting an Azure DevOps repository connection
GitLab
GitLab
Overview of the Matillion GitLab app
Prerequisites
Troubleshooting GitLab repository connections
Bitbucket
Bitbucket
Overview of the Matillion Bitbucket app
Prerequisites
Troubleshooting Bitbucket repository connections
AI
AI
Overview of AI features
Data Productivity Cloud AI FAQ
AI Note
Pipeline components
Pipeline components
Components overview
Data lineage
Pipeline observability
Pipeline notifications
Sampling output
Variables
Grid variables
System variables
Post-processing
Organizing file storage
Code Editor
Orchestration
Orchestration
AI
AI
Snowflake
Snowflake
Chunk Text
Cortex Finetune
Snowpark Container Prompt
Create an app in Snowpark Container Services
OpenAI
OpenAI
OpenAI Prompt
AWS
AWS
Amazon Bedrock Prompt
Amazon OpenSearch Upsert
Amazon Textract Input
Amazon Transcribe
Azure
Azure
Azure OpenAI Prompt
Azure Document Intelligence
Azure Speech Transcribe
Google
Google
Vertex AI Prompt
Vector database operations
Vector database operations
Snowflake
Snowflake
Snowflake Vector Upsert (preview)
Pinecone
Pinecone
Pinecone Vector Upsert
Pinecone Vector Query
Postgres
Postgres
Postgres Vector Upsert
Connectors
Connectors
Load
Load
ActiveCampaign
Amplitude
Anaplan
Asana
Attio
Azure Blob Storage Load
Azure Cosmos DB for NoSQL
Azure SQL
Bing Ads Query
Box
Braze
Brevo
Bugcrowd
Buildkite
Chargebee
CircleCI
ClickUp
Coalesce
Concord
Confluence
Custom connector (agnostic)
Data Productivity Cloud (Matillion)
Data Transfer
Database Query
Datadog
Databricks
dbt Cloud
Delighted
Dixa
Dropbox
Dynamics 365 Business Central
Dynamics 365 Query
DynamoDB Query
Email Query
Eventbrite
Excel Query
Facebook Ads Query
Facebook Query
Fanatics
FireHydrant
Float
Freshdesk
GitHub
Gmail
Gong
Google Ads Query
Google Analytics Query
Google BigQuery Query
Google Cloud Storage Load
Google Sheets
Google Sheets Query (deprecated)
HubSpot Query
Infobip
Intercom
JDBC
JDBC Load
Jira Load
Jira Query
Jotform
Kafka
Klaviyo
LaunchDarkly
LDAP Query
LinkedIn Ads
Lob
Mailchimp
Mailgun
Mandrill
MariaDB Load
Marketo Load
Marketo Query
Microsoft Dataverse
Microsoft Exchange
Microsoft SQL Server Load
Mixpanel
MongoDB Query
NetSuite Query
NetSuite SuiteAnalytics
New Relic
Notion
OData Query
Ongoing WMS
Orbit
Ortto
PagerDuty
PayPal
Pendo
Persona
Pingdom
Pipedrive
PivotalTracker
PostgreSQL Load
Productboard
RDS Query
Recurly
Rootly
S3 Load
Salesforce Load
Salesforce Marketing Cloud Query
Salesforce Pardot
Salesforce Query
Salesloft
SAP NetWeaver
SAP ODP
SAP SuccessFactors
SeatGeek
SendGrid
ServiceNow Query
SharePoint Query
Shopify
Shopify Load
Slack
Smartsheet
Snapchat
Snyk
Square
SugarCRM Load
SurveyMonkey
Stripe Query
Ticketmaster
TikTok
Toggl
Twilio
Unstructured.io
Volume to Delta Table
Workday
Workday Custom Reports
Workday Load
X Ads Load
X Ads
Yelp
Zendesk Talk
Zendesk Ticketing
Zoom
Zuora
Unload
Unload
Azure Blob Storage Unload
Google Cloud Storage Unload
Microsoft SQL Server Output
Oracle Unload from Snowflake
RDS Bulk Output
S3 Unload
Salesforce Output
Authentication guides
Authentication guides
Create an OAuth connection for a custom connector
Bing Ads Query authentication guide
Dynamics 365 Query authentication guide
Google Query authentication guide
HubSpot Query authentication guide
Jira Query authentication guide
Kafka authentication guide
Mailchimp authentication guide
Marketo authentication guide
NetSuite Query authentication guide
NetSuite SuiteAnalytics authentication guide
Salesforce authentication guide
ServiceNow Query authentication guide
Stripe Query authentication guide
Workday authentication guide
Database transactions
Database transactions
Transactions
Begin
Commit
Rollback
DDL
DDL
Add Partition
Alter Table
Analyze Tables
Append to Grid
Create External Table
Create Stream
Create Table
Delete Tables
JDBC Table Metadata To Grid
Optimize
Refresh External Table
Refresh Materialized View
Refresh Table
Remove from Grid
Schema Copy
SQL Script
Table Metadata To Grid
Truncate Table
Action
Action
Snowflake
Snowflake
Alter Warehouse
Create File Format
AWS
AWS
CloudWatch Publish
SNS Message
SQS Message
Messaging
Messaging
Azure Queue Storage Message
Cloud Pub/Sub
Send Email
Webhook Post
Testing
Testing
Assert External Table
Assert Scalar Variables
Assert Table
Print Variables
Scripting
Scripting
dbt Core
Bash Pushdown
Python Pushdown
Python Script
Query Result to Grid
Query Result to Scalar
Run Notebook
Update Scalar
Utility
Utility
Start Compute
Vacuum
Run pipeline
Run pipeline
Run Orchestration
Run Shared Pipeline
Run Transformation
Logic
Logic
Flow
Flow
And
End Failure
End Success
If
Or
Retry
Start
Iterate
Iterate
File Iterator
Fixed Iterator
Grid Iterator
Loop Iterator
Table Iterator
Stacking and detaching iterators
Transformation
Transformation
AI
AI
Snowflake
Snowflake
Cortex
Cortex
Completions
Embed
Extract Answer
Multi Prompt
Parse Document (preview)
Sentiment
Summarize
Translate
Snowflake Vector Search
ML functions
ML functions
Classification (preview)
Document AI Predict
Databricks AI (preview)
Databricks AI (preview)
AI Analyze Sentiment
AI Classify
AI Extract
AI Fix Grammar
AI Mask
AI Query
AI Similarity
AI Summarize
AI Translate
Databricks Vector Search
Aggregate
Assert View
Calculator
Construct Variant
Convert String to Struct
Convert Type
Create View
Detect Changes
Distinct
Except
Extract Nested Data
Extract Structured Data
Filter
First/Last
Fixed Flow
Flatten Variant
Generate Sequence
Intersect
Join
Lead/Lag
Map Values
Multi Table Input
Pivot
Rank
Rename
Rewrite Table
Split field
SQL
Stream Input
Table Delete Rows
Table Input
Table Output
Table Update
Transpose Columns
Transpose Rows
Unite
Unpivot
Window Calculation
Examples
Examples
Data quality framework
Facebook Ads Query example configurations
Incremental loading
Jira Query
S3 Load and transformation
System tables
Streaming
Streaming
Streaming pipelines
Sources
Sources
Db2 for IBM i
Db2 for IBM i
Db2 for IBM i streaming connector
Db2 for IBM i database configuration
Microsoft SQL Server
Microsoft SQL Server
Microsoft SQL Server streaming connector
Microsoft SQL Server database configuration
MySQL
MySQL
MySQL streaming connector
MySQL database configuration
MySQL streaming agent configuration
Oracle
Oracle
Oracle streaming connector
Oracle database configuration
PostgreSQL
PostgreSQL
PostgreSQL streaming connector
PostgreSQL database configuration
Destinations
Destinations
Snowflake
Amazon S3
Azure Blob Storage
Pre-built pipelines
Pre-built pipelines
Sync All Tables
Data Loader Batch
Data Loader Batch
Matillion Data Loader
Batch Loads Introduction
Getting started
Batch Pipeline UI
Batch loading overview
Batch loading overview
Schema drift
Batch load replication overview
Batch pipeline detailed view
Administration
Administration
Setup requirements
Setup requirements
Technical requirements
Cloud data warehouse permissions
Managing credentials
Managing credentials
Destinations
Passwords
Cloud credentials
OAuth connections
NetSuite batch pipeline authentication guide
Scheduling with Quartz cron expressions
Sources
Sources
Custom Connector Data Loader pipeline
Allowed connection properties
ActiveCampaign
Amazon Redshift
Amplitude
Brevo
Chargebee
Circle CI
Concord
Confluence
Datadog
DB2 for IBM i
Delighted
Dropbox
Dynamics 365 Sales
Elasticsearch
Eventbrite
Excel
Facebook AdAccounts
Facebook AdInsights
Facebook Content Insights
Freshdesk
Google Analytics
Google BigQuery
Google Custom Search
Google Sheets
HubSpot
IBM Db2
Jira
Klaviyo
LaunchDarkly
Mailchimp
MariaDB
Marketo
Microsoft SQL Server
MongoDB
MySQL
NetSuite
OData
Open Exchange Rates
Oracle
Oracle Eloqua
PagerDuty
Pendo
PostgreSQL
Productboard
QuickBooks
Recurly
Sage Intacct
Salesforce
Salesforce Pardot
Shopify
Slack
Smartsheet
Snapchat
Snowflake
Snyk
Stripe
SugarCRM
TikTok
Toggl
Twilio
Xero
Zendesk
Zendesk Talk
Destinations
Destinations
Snowflake
Snowflake
Set up Snowflake
Connect to Snowflake
Delta Lake on Databricks
Delta Lake on Databricks
Set up Delta Lake on Databricks
Connect to Delta Lake on Databricks
Amazon Redshift
Amazon Redshift
Set up Amazon Redshift
Connect to Amazon Redshift
Google BigQuery
Google BigQuery
Set up Google BigQuery
Connect to Google BigQuery
Tech notes
Tech notes
Tech note - failing pipelines
Troubleshooting
Troubleshooting
Destinations
Custom Connector
Custom Connector
Matillion Custom Connector Overview
Export your own connector to Matillion ETL
Create a pipeline with custom connectors
Creating Custom Connectors
Creating Custom Connectors
Custom connector setup
Endpoints
Authentication
Add OAuth
Pagination
Script paging
API parameters
Custom connector example
Flex connectors
Flex connectors
Flex connector setup
Flex connector tutorial
Data Productivity Cloud API
Data Productivity Cloud API
Explanation
Explanation
API overview
Authentication
API reference user guide
Endpoints reference
Endpoints reference
Data Productivity Cloud public API reference
How-to guides
How-to guides
Artifacts and CI/CD
Artifacts and CI/CD
Understanding Artifacts
Creating artifacts with the API
Scheduling artifacts
Audit service
Executing and managing a pipeline
Managing secret references
Streaming pipelines
Changelogs
Changelogs
2025
2025
October
September
August
July
June
May
April
March
February
January
2024
2024
December
November
October
September
August
July
June
May
April
March
February
January
2023
2023
December
November
October
September
August
July
June
May
April
March
February
January
2022
2022
December
November
October
September
August
July
June
May
April
Tech notes
Tech notes
Tech note - Snowflake to block single-factor password authentication
Matillion ETL
Matillion ETL
Home
Matillion ETL product overview
Matillion ETL and the Data Productivity Cloud
Matillion ETL and the Data Productivity Cloud
Explore the Data Productivity Cloud
Matillion ETL instance creation
Associating a Matillion ETL instance
Configuring a connection from Matillion ETL to the Data Productivity Cloud
Launching Matillion ETL
Launching Matillion ETL for GCP
Subscriptions, usage & billing
Subscriptions, usage & billing
Matillion ETL usage
Subscriptions
Subscriptions
Matillion ETL editions
Matillion ETL marketplace subscriptions
Matillion ETL observability
Instance sizes
Matillion ETL basics
Matillion ETL basics
Accessing your instance
Accessing your instance
Accessing the Matillion ETL Client (Amazon EC2)
Accessing the Matillion ETL Client (Google Cloud Platform)
Accessing the Matillion ETL Client (Microsoft Azure)
Components
Components
Assert components in Matillion ETL
Expression editors
Incremental load tools
URL safe characters
Jobs
Jobs
Generate Job Documentation
Jobs
Job concurrency
Notes
Shared jobs
Task management
Environments
Environments
Environments
Manage Stages
Multiple environment connections
Table properties
Creating a Snowflake Zero-Copy Clone
Create External Schema
Variables
Variables
Variables
Component Exports
Date and time methods
Environment Variables
Grid variables
Job Variables
Enterprise mode
Scope of Matillion ETL features
UI and basic functions
Admin menu
Schema
Notices
Search tab
Performance monitor
Views
Project collaboration
Software versions
Glossary
Administration
Administration
Manage Interpreters
Administrative functions
Administrative functions
Audit log
How to place restrictions on Bash and Python components
How to configure Catalina log rotation
Control session timeout expiration
Product improvement metrics
Restart server
Publicly available warning
Backups
Backups
Backups (AWS)
Backups (GCP)
Backups (Azure)
Manage backups
Cloud data platform configuration
Cloud data platform configuration
Snowflake configuration for Matillion ETL
Redshift configuration for Matillion ETL
Delta Lake on Databricks configuration for Matillion ETL
Configuring an AWS VPC
How to generate a new Databricks token
Snowflake key-pair authentication
Connectivity
Connectivity
Configuring Matillion ETL to use a Proxy
Setting up an external connection to a Matillion database
Manage connections
Matillion ETL access ports
Configuring a connection from Matillion ETL to the Data Productivity Cloud
Setting up Matillion ETL in a private VPC
Drivers
Drivers
Adding a third-party JDBC driver
Getting started with the API driver in Matillion ETL
SAP Hana JDBC driver installation for Matillion ETL
Manage Database Drivers
Permissions
Permissions
Groups and Permissions
Permissions list
Preview Labs
Preview Labs
Preview Labs
SSL
SSL
SSL commands
How to add a certificate chain file for SSL configuration
Recreating self-signed SSL certificates on a Matillion ETL instance
SSL Configuration FAQ
Updating and migrating
Updating and migrating
Updating and migrating overview
Migration
In-place update
Updating to a specific release
Updating a high availabilty cluster
User configuration
User configuration
User configuration
Stateless authentication
Stateless authentication
Using stateless authentication with existing users
Configuring stateless authentication
Internal security (stateless authentication)
OpenID integration (stateless authentication)
LDAP integration (stateless authentication)
Stateless authentication rollback steps
OpenID
OpenID
OpenID overview
Microsoft Entra ID OpenID setup
Google OpenID setup
Microsoft Active Directory OpenID setup
Okta OpenID setup
OneLogin OpenID setup
OpenID setup
Troubleshooting OpenID setup
LDAP
LDAP
LDAP integration
Okta LDAP Configuration
Read-only users
Reverting from external to internal security
Matillion ETL security best practices
Using CSRF tokens to safeguard Matillion ETL instances
Installing DBT on Matillion ETL
Connecting to an external PostgreSQL database
Automatic security updates
Manage optional features
Launching Matillion ETL
Launching Matillion ETL
Launching Matillion ETL overview
Launching Cloud Platform Permissions
Launching - AWS
Launching - AWS
Launching Matillion ETL using CloudFormation Templates
List of CloudFormation Templates
Launching Matillion ETL using Amazon Machine Image
Launching Matillion ETL for Delta Lake on AWS
Launching troubleshooting (AWS)
Launching - Azure
Launching - Azure
Overview
Launching Matillion ETL from Azure Marketplace
Launching Matillion ETL using an Azure ARM Template
Launching Matillion ETL for Delta Lake on Azure
Launching troubleshooting (Azure)
Launching - GCP
Launching - GCP
Launching Matillion ETL for Snowflake - GCP
Launching Matillion ETL for BigQuery - GCP
High Availability (HA)
High Availability (HA)
Launching a Matillion ETL HA Cluster via AWS
RPM installations
RPM installations
Installing Matillion ETL using the Universal Installer (RPM install)
Non-Data Productivity Cloud existing customers
Non-Data Productivity Cloud existing customers
Overview
Launching Matillion ETL using CloudFormation Templates
List of Snowflake Launch Templates
Migrate from Snowflake Partner Connect to Matillion ETL for Snowflake
Launching Matillion ETL from AWS Marketplace
Launching Matillion ETL for Redshift from AWS Marketplace
List of Redshift Launch Templates
Converting to be an Annual Customer
Launching Matillion ETL via Azure CLI
Finding and Launching Matillion BYOL Images
Instance Sizes Guide
Applying a licence
Project menu
Project menu
Create Project
Create Project
Create Project (Snowflake)
Create Project (Delta Lake on Databricks)
Create Project (Amazon Redshift)
Create Project (Google BigQuery)
Create Project (Azure Synapse Analytics)
Manage functions
Manage functions
Manage Project
Manage Credentials
Manage Extract Profiles
Manage Passwords
Manage Query Profiles
Manage OAuth
Manage Schedules
Manage Sequences
Manage Shared Jobs
Manage Versions
Manage Webhook Payloads
Integrating Matillion ETL with secret managers
Manage API Profiles wizards
Manage External File Sources
Queue Messaging
Queue Messaging
Azure Queue Message configuration
Manage Pub-Sub configuration
Manage SQS Configuration
CDC
CDC
Manage CDC
Enable Manage CDC
Configuring a source database for CDC
DMS migration instances
Git integration
Git integration
Git Integration with Matillion ETL
Git Integration Frequently Asked Questions
When to choose Git
MergeManager
API Profiles
API Profiles
API Profiles Overview
API Query Profiles
API Extract Profiles
API Connector Wizard
API Profiles - Pagination
API Profiles - Parameters
API Profiles - Authentication
API Profiles - RSDs
API Profiles Example - MongoDB and DynamoDB
API Profiles Example - Jira Cloud
API Profiles Example - Salesforce Lightning
Switch Project
Using data structure variables
Change My Password
Extract to new job
Task History
Import - Export
Input data report
Manage Error reporting
Project user access
Recycle Bin
Job references
Job reference renaming
Databricks job compute configuration
Snowflake query tag configuration
Orchestration
Orchestration
Data staging components
Data stagers - support
Data models
Connecting to external services securely
Testing
Testing
Assert External Table
Assert Scalar Variables
Assert Table
Assert View
Print Variables
Connectors
Connectors
Connectors overview
Acquiring Azure Credentials
Amplitude
Amplitude
Amplitude Extract
Amplitude Extract authentication guide
Anaplan
Anaplan
Anaplan Bulk
Anaplan Bulk authentication guide
API Queries
API Queries
API Query
API Extract
API Query functions
Create your own Matillion ETL Extract connector
Populating parameters with API Query
Apache
Apache
Apache Hive Query
Apache Spark SQL Query
Azure
Azure
Azure SQL Query
Azure Cosmos DB Query
Bing
Bing
Bing Ads Query
Bing Ads Query authentication guide
Bing Search Query
Bing Search Query authentication guide
Box
Box
Box Extract
Box Extract authentication guide
Cassandra
Cassandra
Cassandra Query
CloudWatch Publish
CloudWatch Publish
CloudWatch Publish
Couchbase
Couchbase
Couchbase Query
Data Transfer
Data Transfer
Data Transfer Object
Dropbox
Dropbox
Dropbox Extract
Dropbox Extract authentication guide
Dynamics 365
Dynamics 365
Dynamics 365 Query
Dynamics 365 Query Authentication Guide
Dynamics 365 Business Central Query
Dynamics 365 Business Central Query authentication guide
Dynamics 365 Sales Query
Dynamics 365 Sales Query Authentication Guide
Dynamics CRM
Dynamics CRM
Dynamics CRM Query
Dynamics CRM Query authentication guide
Dynamics 365 NAV
Dynamics 365 NAV
Dynamics NAV Query
DynamoDB
DynamoDB
DynamoDB Load
DynamoDB Query
EMR
EMR
EMR Load
Elasticsearch
Elasticsearch
Elasticsearch Query
Email
Email
Email Query
Excel
Excel
Excel Query
Facebook
Facebook
Facebook Query
Facebook Ads Query
Facebook Authentication Guide
Facebook Connector FAQ
Google
Google
Gmail Query
Gmail Query authentication guide
Google Ad Manager Query
Google Ads Query
Google Ad Manager Query authentication guide
Google Analytics Query
Google Analytics Query - add new data sources
Google BigQuery
Google Custom Search Query
Google Custom Search Query authentication guide
Google Drive Table
Google Sheets Query
Google Query authentication guide
Google third-party OAuth using Service Accounts
How to set your own Google Ads developer token
HubSpot
HubSpot
HubSpot Query
HubSpot Query authentication guide
Instagram
Instagram
Instagram Extract
Instagram Extract authentication guide
Instagram Business Account ID Guide
Intercom
Intercom
Intercom Extract
Intercom Extract authentication guide
Jira
Jira
Jira Query
Jira Query authentication guide
LDAP
LDAP
LDAP Query
LinkedIn
LinkedIn
LinkedIn Query
LinkedIn Query authentication guide
LinkedIn Ads Query
LinkedIn Ads Query authentication guide
Magento
Magento
Magento Query
Mailchimp
Mailchimp
Mailchimp Query
Mailchimp Query authentication guide
Mandrill
Mandrill
Mandrill Extract
Mandrill Extract authentication guide
Marketo
Marketo
Marketo Query
Marketo Query Authentication Guide
Microsoft Exchange
Microsoft Exchange
Microsoft Exchange Query
Microsoft Exchange Query Authentication Guide
MindSphere
MindSphere
MindSphere Extract
MindSphere Extract authentication guide
Mixpanel
Mixpanel
Mixpanel Query
Mixpanel Query authentication guide
MongoDB
MongoDB
MongoDB Query
NetSuite
NetSuite
NetSuite Query
NetSuite Query Authentication Guide
NetSuite SuiteAnalytics Connect
NetSuite SuiteAnalytics Connect Authentication Guide
OData
OData
OData Query
Open Exchange Rates
Open Exchange Rates
Open Exchange Rates Query
Open Exchange Rates Query authentication guide
Oracle Eloqua
Oracle Eloqua
Oracle Eloqua Query
Oracle Eloqua Query authentication guide
Pardot
Pardot
Pardot Extract
Pardot Extract authentication guide
PayPal
PayPal
PayPal Query
PayPal Query Authentication Guide
QuickBooks
QuickBooks
QuickBooks Online Query
QuickBooks Online Query Authentication Guide
RDS
RDS
RDS Query
Redis
Redis
Redis Query
Recurly
Recurly
Recurly Extract
Recurly Extract authentication guide
Sage Intacct
Sage Intacct
Sage Intacct Query
Salesforce
Salesforce
Salesforce Query
Salesforce Incremental Load
Salesforce Query authentication guide
Salesforce Bulk Query
Salesforce Bulk Query authentication guide
Salesforce Marketing Cloud Query
Salesforce Marketing Cloud Query authentication guide
Salesforce Output authentication guide
SAP NetWeaver
SAP NetWeaver
SAP NetWeaver Query
SAP NetWeaver Query - add new data sources
SAP ODP
SAP ODP
SAP ODP Extract
SAP ODP connection options
SAP ODP troubleshooting
SAP ODP FAQ
SendGrid
SendGrid
SendGrid Query
SendGrid Query authentication guide
ServiceNow
ServiceNow
ServiceNow Query
ServiceNow Query authentication guide
SharePoint
SharePoint
SharePoint Query
SharePoint Query authentication guide
Shopify
Shopify
Shopify Query
Shopify Query Authentication Guide
Snapchat
Snapchat
Snapchat Extract
Snapchat Extract authentication guide
Splunk
Splunk
Splunk Query
SQL databases
SQL databases
Database Query
JDBC Incremental Load
Square
Square
Square Query
Square Query authentication guide
Stripe
Stripe
Stripe Query
Stripe Query authentication guide
Sugar CRM
Sugar CRM
Sugar CRM Query
SurveyMonkey
SurveyMonkey
SurveyMonkey Query
SurveyMonkey Query authentication guide
Twilio
Twilio
Twilio Query
Twilio Query authentication guide
Twitter
Twitter
Twitter Query
Twitter Query authentication guide
Twitter Ads Query
Workday
Workday
Workday Extract
Workday Custom Reports
Workday Extract authentication guide
Workday Integration System User setup
Finding a Workday object's integration ID
Managing access to data sources
Xero
Xero
Xero Query
Xero Query authentication guide
YouTube
YouTube
YouTube Query
YouTube Analytics Query
Zendesk
Zendesk
Zendesk Support Query
Zendesk Talk Query
Zendesk Support/Talk Query authentication guide
Zendesk Chat Extract
Zendesk Chat Extract Authentication Guide
Zoho CRM
Zoho CRM
Zoho CRM Query
Zoho CRM Query authentication guide
Zuora
Zuora
Zuora Query
Zuora Bulk Query
Zulk Query and Bulk Query authentication guide
Output components
Output components
Output components overview
Oracle Output
Microsoft SQL Server Output
Salesforce Output
Pardot Output
Intercom Output
Azure SQL Bulk Output
RDS Bulk Output
Cloud Storage
Cloud Storage
Amazon S3
Amazon S3
S3 Load
S3 Manifest Builder
S3 Unload
Azure Blob Storage
Azure Blob Storage
Azure Load Snowflake
Azure Load Delta Lake
Azure Load Synapse
Azure Unload
Google Cloud Storage
Google Cloud Storage
Cloud Storage Load
Cloud Storage Unload
DDL
DDL
External Schema and Tables
Snowflake
Snowflake
Alter External Table
Create External Table
Refresh External Table
Alter Table
Alter Warehouse
Assign Tag
Alter Masking Policy
Create File Format
Create Stream
Amazon Redshift
Amazon Redshift
Add Partition
Alter Session WLM Slots
Analyze Tables
Create External Table
Delete Partition
Nested Data Load
Refresh Materialized View
Schema Copy
Text Output
Delta Lake on Databricks
Delta Lake on Databricks
Delta Live Streaming
Optimize
Refresh Table
Run Delta Live Table
Start Cluster
Google BigQuery
Google BigQuery
Create External Table
Refresh Materialized View
Azure Synapse Analytics
Azure Synapse Analytics
Alter Database
Create External Table
Create Table
Delete Table
SQL Script
Truncate Table
Vacuum Table
Flow components
Flow components
Flow components overview
Iterators
Iterators
Iterator components
File Iterator
Fixed Iterator
Grid Iterator
Loop Iterator
Table Iterator
Transactions
Transactions
Begin
Commit
Rollback
And
End Failure
End Success
If
Or
Retry
Run Orchestration
Run Transformation
Start
Load generators
Load generators
Load generators overview
Azure Blob Load Generator
Cloud Storage Load Generator (Snowflake)
Cloud Storage Load Generator (BigQuery)
S3 Load Generator (Snowflake)
S3 Load Generator (Redshift)
S3 Load Generator (Delta Lake)
Messaging
Messaging
SQS Message
SNS Message
PubSub
Azure Queue Storage Message
Webhook Post
Scripting
Scripting
Bash Script
Python Script
Sync File Source
Commands for dbt Core
Python Script additional settings
Run Notebook
Shared jobs
Shared jobs
CDC shared jobs overview
Sync All Tables shared job
Sync Single Table shared job
Create or Refresh External Table shared job
Drop CDC Tables shared job
Data typing with CDC shared jobs
Tables created
Append metadata
Variables
Variables
Append To Grid
Describe To Grid
Show to Grid
Query Result To Grid
JDBC Table Metadata To Grid
Query Result To Scalar
Remove From Grid
Table Metadata To Grid
Load generators overview
Send Email
Transformation
Transformation
Join
Join
Except
Intersect
Join
Unite
Read
Read
Fixed Flow
Generate Sequence
Multi Table Input
Stream Input
Table Input
Wildcard Table Input
Transform
Transform
Aggregate
Calculator
Construct Struct
Construct Variant
Convert Type
Table Delete Rows
Detect Changes
Distinct
Extract Nested Data
Filter
First-Last
Flatten Variant
Lead-Lag
Map Values
Pivot
Rank
Rename
Replicate
Split Field
SQL
Transpose Columns
Transpose Rows
Transpose Rows (Snowflake)
Unpivot
Window Calculation
Write
Write
Copy Table To External Schema
Create View
External Table Output
Rewrite External Table
Rewrite Table
Table Output
Table Update
Matillion ETL API
Matillion ETL API
Endpoints
Endpoints
API v1 - API extract profile
API v1 - API profile
API v1 - Audit
API v1 - Credentials
API v1 - Driver
API v1 - Environment
API v1 - Git integration for projects
API v1 - Git integration for shared jobs
API v1 - Group/project
API v1 - License
API v1 - Metadata
API v1 - Notice
API v1 - OAuth
API v1 - Passwords
API v1 - Permission
API v1 - Queue
API v1 - Schedules
API v1 - Running jobs
API v1 - Shared jobs
API v1 - Tasks
API v1 - Userconfig
API v1 - Versions
API v1 - Webhookpayloadprofile
API v1 - Secret manager
Matillion ETL API - v1
Maps of Matillion API v1
Getting started with Postman
Getting started with cURL
Managing users, passwords, groups, and permissions
Authenticating Matillion REST API bearer token
Obtaining an API token and passing it to an API Query profile
v0 API
v0 API
Matillion ETL API - v0
v0 examples
Cloud configuration
Cloud configuration
Authorization and authentication
Setting up Let's Encrypt for SSL on a Matillion ETL Instance
Snowflake
Snowflake
Snowflake role privileges with Matillion ETL
Snowflake optimization for Matillion ETL
Amazon Web Services
Amazon Web Services
Attaching AWS IAM roles To EC2 instances
IAM roles & permissions (AWS)
Changing EC2 instance size
Google Cloud Platform
Google Cloud Platform
IAM roles & permissions (GCP)
GCP service accounts
BigQuery data set setup
GCP enabling APIs
Microsoft Azure
Microsoft Azure
Changing Azure instance size
Roles & permissions (Azure)
Configuring a high availability cluster (Azure)
Guides
Guides
AWS services
AWS services
Spectrum
Spectrum
Getting Started with Amazon Redshift Spectrum
Using Spectrum in Matillion ETL
Accessing files in S3 using Pre-signed URLs
Automating Redshift maintenance
AWS S3 lifecycle rule
Cross-account S3 access
Parallelism with Matillion ETL for Redshift
Snowflake AWS Storage Integration setup guide
Azure services
Azure services
Snowflake Azure Storage Integration setup guide
Troubleshooting connection to Azure Blob Storage
Creating secrets in Azure Key Vault
GCP services
GCP services
Snowflake GCP Storage Integration setup guide
Common operations
Common operations
Populating tables
Populating tables
Adding filename as a column to tables (BigQuery)
Adding Filename as a column to new table
Populating an audit table
Building a data vault
Building a data vault (Snowflake)
Data transfer between databases
Ingesting AWS ElasticSearch data via the API Query component
Year-on-year analysis
Designing a job for a high availability cluster
Incremental or high water mark data Loading
Microbatch replication
How to receive emails by subscribing to a cloud Pub/Sub topic
Flattening nested arrays
Installation configuration
Installation configuration
Changing the host file
How to configure SSL protocols
Outbound IP requirements
Triggering Matillion ETL
Triggering Matillion ETL
Triggering ETL from an S3 event via AWS Lambda
Triggering ETL from an email via SES and Lambda
Triggering ETL from Amazon Alexa via AWS Lambda & Amazon SQS
Triggering a Matillion ETL job from your Google Home device
Triggering Matillion ETL from a storage queue via an Azure function
Collibra integration
Connecting to an RDS in a private VPC
Diagnostic data policy
Executing Python scripts outside of Matillion
Helping with the GDPR
Integrating Slack with Matillion ETL
Using grid variables to apply business rules in a transformation job
Making multiple API queries
Matillion data quality framework
NRT replication In Redshift
Pivoting and unpivoting tables
SCM integration
Tracking loaded files
Using incron to automatically copy data to S3
Using KMS encrypted passwords in Python
Using R with Matillion ETL for Redshift
Using Table Metadata to Grid
Managing Python on a Matillion ETL virtual machine (VM)
How to retrieve missing Task History entries after 1.47 upgrade
Matillion Exchange
Migrate to the Data Productivity Cloud
Migrate to the Data Productivity Cloud
Migration considerations
Migration prerequisites
Migration process
Migrating shared jobs
Migration mappings
Migration - Decommission Matillion ETL
Detailed considerations
Detailed considerations
API Extract
API Query
Automatic variables
Bash
Database Query
dbt
Export variables
Extract Nested Data
Iterator components
Python
Replicate
Temporary tables
Text Output
Transactions
Variables
Release notes
Release notes
Release notes index
Support lifecycle
Supported releases
1.78 (LTS) release notes
1.77 release notes
1.76 release notes
1.75 (LTS) release notes
1.74 release notes
1.73 release notes
1.72 release notes
1.71 release notes
1.70 release notes
1.69 release notes
1.68 release notes
1.67 release notes
1.66 release notes
1.65 release notes
1.64 release notes
1.63 release notes
Earlier than version 1.63
Earlier than version 1.63
Matillion ETL for Snowflake release notes
Matillion ETL for Redshift release notes
Matillion ETL for BigQuery release notes
Matillion ETL for Synapse release notes
Matillion ETL for Delta Lake release notes
Release notes advisories
Release notes archive
Tech notes
Tech notes
Critical Advisory - Mandatory update required to address Licence Management Defect
Tech note - Adjusting Tomcat memory for Matillion ETL upgrades
Tech note - Snowflake to block single-factor password authentication
Tech note - Image scanning for CVEs
Tech note - Removal of Manage CDC
Tech note - AWS thread count increases leading to failing instances
Tech note - user configuration and security best practices update
Tech note - has your Matillion ETL instance stopped working?
Tech note - AWS SDK upgrade for Java
Tech note - 1.68 update failure
Tech note - automatic security updates
Tech note - Snowflake GCP update
Tech note - legacy key algorithm constraints
Tech note - OutOfMemory events in version 1.69
Tech note - Redshift RingBuffer exceeding expected limits
Tech note - disk partition sizing for versions 1.69-1.72
Updating to version 1.69 and above
Technology upgrade of PostgreSQL
Tech note - running Query components through a proxy server
Tech note - Shopify Query versioning
Tech note - Splunk Query versioning
Tech note - Google Analytics driver update
Tech note - Postgres driver for Amazon Redshift
Security Advisory - Spring4Shell
Tech Note - Google Ads updates in v1.62
Updating to version 1.60 and above
Security Advisory - v1.59.11
Security Advisory 13th Jan 2022
Security Advisory 17th Dec 2021
Security Advisory 14th Dec 2021
Tech Note 15th June 2021
Tech Note 14th May 2021
Potential credentials in Matillion ETL log file
Security Advisory - 29th April 2021
Tech Note 10th February 2021
Tech Note 28th January 2021
Tech Note 12th November 2020
Tech Note 22nd September 2020
Tech note 22nd June 2020
Getting support
Style guide
Style guide
Style guide overview
Markdown and formatting
Markdown and formatting
Formatting checklist
Formatting text
Capitalization
Headings
Horizontal rules
Images
Links
Lists
Tables
UI elements
Words
MarkUp and reviewing
MarkUp and reviewing
MarkUp.io
Release status
Release status
Release status
Contribute
Contribute
Prerequisites
Contribute to the docs
Data Loader CDC
Data Loader CDC
Matillion Change Data Capture
Getting started
CDC shared jobs
CDC Pipeline UI
CDC Overview
CDC Overview
CDC pipeline overview
CDC pipeline detailed view
Create and manage a CDC pipeline
Schema Drift
CDC Agent
CDC Agent
CDC agent installation
CDC agent setup UI
Sizing CDC agents
Agent versions
Agent update
Platform keys
Environment variables
Lock down outbound traffic
AWS installation
AWS installation
AWS installation overview
Deploying a CDC agent in AWS quick guide
Cloud resources
Cloud resources
Subnets and security groups
CloudWatch Logs
IAM Roles
S3 Buckets
Fargate and ECS
AWS templates
AWS templates
CloudFormation
CloudFormation
CloudFormation ECS Fargate basic template
CloudFormation ECS Fargate advanced template
CloudFormation EC2 ECS proxy templates
Kubernetes templates
Kubernetes templates
Agent (AWS) template
Terraform
Terraform
Terraform ECS Fargate basic template
Terraform ECS Fargate advanced template
Azure installation
Azure installation
Azure installation overview
Deploying a CDC agent in Azure quick guide
Deploying a CDC agent in Azure using the CLI
Cloud resources
Cloud resources
Managed identity
Blob Storage
Log analytics
Container instances
Azure templates
Azure templates
Azure - ARM template
Azure - Terraform template
Kubernetes templates
Kubernetes templates
Agent (Azure) template
GCP installation
GCP installation
GCP installation overview
Deploying a CDC agent in GCP quick guide
Cloud resources
Cloud resources
Project and Service Account
GCP Cloud Storage bucket
IAM permissions
Workload Identity (GKE)
GCP templates
GCP templates
Agent (GCE) template
Agent (GKE) Kubernetes template
Logs
Logs
AWS CloudWatch Logs
Azure Log Analytics
Google Cloud logging
Administration
Administration
Technical Requirements (CDC)
Lifecycle management
Lifecycle management
Amazon S3 lifecycle management
Azure Blob Storage lifecycle management
Google Cloud Storage lifecycle management
Sources
Sources
CDC sources overview
Source database maintenance
Db2 for IBM i
Db2 for IBM i
Db2 for IBM i connector
Configuring Db2 for IBM i
DB2 for IBM i snapshot
PostgreSQL
PostgreSQL
PostgreSQL connector
Configuring PostgreSQL database
PostgreSQL advanced settings
PostgreSQL snapshot
Oracle
Oracle
Oracle connector
Configuring Oracle Database
Oracle advanced settings
Oracle snapshot
Microsoft SQL Server
Microsoft SQL Server
Microsoft SQL Server connector
Configuring Microsoft SQL Server database
Microsoft SQL advanced settings
Microsoft SQL Server snapshot
MySQL
MySQL
MySQL connector
Configuring MySQL database
Configuring the CDC agent for MySQL
MySQL snapshot
Destinations
Destinations
Snowflake
Amazon S3
Azure Blob
Google Cloud Storage
Troubleshooting
Troubleshooting
Agent setup
Destinations
UI setup
Matillion ETL Documentation
Ask a question about
Matillion ETL
and get an answer instantly!
Back to top