Matillion Docs
Pipeline execution API reference
Initializing search
Home
Index
Matillion ETL
Unlimited Scale
Data Productivity Cloud
Legal
Style guide
Azure agent preview
Copilot preview
Matillion Docs
Home
Index
Matillion ETL
Matillion ETL
Using the site
Using the site
Matillion ETL documentation
Getting support
Matillion ETL basics
Matillion ETL basics
Accessing your instance
Accessing your instance
Accessing the Matillion ETL Client (Amazon EC2)
Accessing the Matillion ETL Client (Google Cloud Platform)
Accessing the Matillion ETL Client (Microsoft Azure)
Components
Components
Assert components in Matillion ETL
Expression editors
Incremental load tools
URL safe characters
Jobs
Jobs
Generate Job Documentation
Jobs
Job concurrency
Notes
Shared jobs
Task management
Environments
Environments
Environments
Manage Stages
Multiple environment connections
Table properties
Creating a Snowflake Zero-Copy Clone
Variables
Variables
Variables
Component Exports
Date and time methods
Environment Variables
Grid variables
Job Variables
Matillion ETL product overview
Enterprise mode
Scope of Matillion ETL features
UI and basic functions
Admin menu
Schema
Notices
Search tab
Performance monitor
Views
Project collaboration
Software versions
Glossary
Administration
Administration
Manage Interpreters
Administrative functions
Administrative functions
Audit log
How to place restrictions on Bash and Python components
How to configure Catalina log rotation
Control session timeout expiration
Product improvement metrics
Restart server
Publicly available warning
Backups
Backups
Backups (AWS)
Backups (GCP)
Backups (Azure)
Manage backups
Cloud data platform configuration
Cloud data platform configuration
Snowflake configuration for Matillion ETL
Redshift configuration for Matillion ETL
Delta Lake on Databricks configuration for Matillion ETL
Configuring an AWS VPC
How to generate a new Databricks token
Connectivity
Connectivity
Configuring Matillion ETL to use a Proxy
Setting up an external connection to a Matillion database
Manage connections
Matillion ETL access ports
Configuring a connection from Matillion ETL to Matillion Hub
Setting up Matillion ETL in a private VPC
Drivers
Drivers
Adding a third-party JDBC driver
Getting started with the API driver in Matillion ETL
SAP Hana JDBC driver installation for Matillion ETL
Manage Database Drivers
Permissions
Permissions
Groups and Permissions
Permissions list
Preview Labs
Preview Labs
Preview Labs
SSL
SSL
SSL commands
How to add a certificate chain file for SSL configuration
Recreating self-signed SSL certificates on a Matillion ETL instance
SSL Configuration FAQ
Updating and migrating
Updating and migrating
Updating and migrating overview
Migration
In-place update
Updating to a specific release
Updating a high availabilty cluster
User configuration
User configuration
User configuration
Stateless authentication
Stateless authentication
Configuring stateless authentication
Internal security (stateless authentication)
OpenID integration (stateless authentication)
LDAP integration (stateless authentication)
Stateless authentication rollback steps
OpenID
OpenID
OpenID overview
Azure Active Directory OpenID setup
Google OpenID setup
Microsoft Active Directory OpenID setup
Okta OpenID setup
OneLogin OpenID setup
OpenID setup
Troubleshooting OpenID setup
LDAP
LDAP
LDAP integration
Okta LDAP Configuration
Read-only users
Reverting from external to internal security
SAML in Matillion ETL
Matillion ETL security best practices
Using CSRF tokens to safeguard Matillion ETL instances
Installing DBT on Matillion ETL
Connecting to an external PostgreSQL database
Automatic security updates
Manage optional features
Launching Matillion ETL
Launching Matillion ETL
Launching Matillion ETL overview
Launching Cloud Platform Permissions
Launching - AWS
Launching - AWS
Launching Matillion ETL using CloudFormation Templates
List of CloudFormation Templates
Launching Matillion ETL using Amazon Machine Image
Launching Matillion ETL for Delta Lake on AWS
Launching troubleshooting (AWS)
Launching - Azure
Launching - Azure
Overview
Launching Matillion ETL from Azure Marketplace
Launching Matillion ETL using an Azure ARM Template
Launching Matillion ETL for Delta Lake on Azure
Launching troubleshooting (Azure)
Launching - GCP
Launching - GCP
Launching Matillion ETL for Snowflake - GCP
Launching Matillion ETL for BigQuery - GCP
High Availability (HA)
High Availability (HA)
Launching a Matillion ETL HA Cluster via AWS
RPM installations
RPM installations
Installing Matillion ETL using the Universal Installer (RPM install)
Non-Hub existing customers
Non-Hub existing customers
Overview
Launching Matillion ETL using CloudFormation Templates
List of Snowflake Launch Templates
Migrate from Snowflake Partner Connect to Matillion ETL for Snowflake
Launching Matillion ETL from AWS Marketplace
Launching Matillion ETL for Redshift from AWS Marketplace
List of Redshift Launch Templates
Converting to be an Annual Customer
Launching Matillion ETL via Azure CLI
Finding and Launching Matillion BYOL Images
Instance Sizes Guide
Applying a licence
Project menu
Project menu
Create Project
Create Project
Create Project (Snowflake)
Create Project (Delta Lake on Databricks)
Create Project (Amazon Redshift)
Create Project (Google BigQuery)
Create Project (Azure Synapse Analytics)
Manage functions
Manage functions
Manage Project
Manage Credentials
Manage Extract Profiles
Manage Passwords
Manage Query Profiles
Manage OAuth
Manage Schedules
Manage Sequences
Manage Shared Jobs
Manage Versions
Manage Webhook Paylods
Integrating Matillion ETL with secret managers
Manage API Profiles wizards
Manage External File Sources
Queue Messaging
Queue Messaging
Azure Queue Message configuration
Manage Pub-Sub configuration
Manage SQS Configuration
CDC
CDC
Manage CDC
Enable Manage CDC
Configuring a source database for CDC
DMS migration instances
Git integration
Git integration
Git Integration with Matillion ETL
Git Integration Frequently Asked Questions
When to choose Git
MergeManager
API Profiles
API Profiles
API Profiles Overview
API Query Profiles
API Extract Profiles
API Connector Wizard
API Profiles - Pagination
API Profiles - Parameters
API Profiles - Authentication
API Profiles - RSDs
API Profiles Example - MongoDB and DynamoDB
API Profiles Example - Jira Cloud
API Profiles Example - Salesforce Lightning
Switch Project
Using data structure variables
Change My Password
Extract to new job
Task History
Import - Export
Input data report
Manage Error reporting
Project user access
Recycle Bin
Job references
Job reference renaming
Databricks job compute configuration
Orchestration
Orchestration
Data staging components
Data stagers - support
Data models
Connecting to external services securely
Assert components
Assert components
Assert External Table
Assert Scalar Variables
Assert Table
Assert View
Connectors
Connectors
Connectors overview
Acquiring Azure Credentials
Amplitude
Amplitude
Amplitude Extract
Amplitude Extract authentication guide
Anaplan
Anaplan
Anaplan Bulk
Anaplan Bulk authentication guide
API Queries
API Queries
API Query
API Extract
API Query functions
Create your own Matillion ETL Extract connector
Populating parameters with API Query
Apache
Apache
Apache Hive Query
Apache Spark SQL Query
Azure
Azure
Azure SQL Query
Azure Cosmos DB Query
Bing
Bing
Bing Ads Query
Bing Ads Query authentication guide
Bing Search Query
Bing Search Query authentication guide
Box
Box
Box Extract
Box Extract authentication guide
Cassandra
Cassandra
Cassandra Query
CloudWatch Publish
CloudWatch Publish
CloudWatch Publish
Couchbase
Couchbase
Couchbase Query
Dropbox
Dropbox
Dropbox Extract
Dropbox Extract authentication guide
Dynamics 365
Dynamics 365
Dynamics 365 Query
Dynamics 365 Query Authentication Guide
Dynamics 365 Business Central Query
Dynamics 365 Business Central Query authentication guide
Dynamics 365 Sales Query
Dynamics 365 Sales Query Authentication Guide
Dynamics CRM
Dynamics CRM
Dynamics CRM Query
Dynamics CRM Query authentication guide
Dynamics 365 NAV
Dynamics 365 NAV
Dynamics NAV Query
DynamoDB
DynamoDB
DynamoDB Load
DynamoDB Query
EMR
EMR
EMR Load
Elasticsearch
Elasticsearch
Elasticsearch Query
Email
Email
Email Query
Excel
Excel
Excel Query
Facebook
Facebook
Facebook Query
Facebook Ads Query
Facebook Authentication Guide
Facebook Connector FAQ
Google
Google
Gmail Query
Gmail Query authentication guide
Google Ad Manager Query
Google Ads Query
Google Ad Manager Query authentication guide
Google Analytics Query
Google Analytics Query - add new data sources
Google BigQuery
Google Custom Search Query
Google Custom Search Query authentication guide
Google Drive Table
Google Sheets Query
Google Query authentication guide
Google third-party OAuth using Service Accounts
How to set your own Google Ads developer token
HubSpot
HubSpot
HubSpot Query
HubSpot Query authentication guide
Instagram
Instagram
Instagram Extract
Instagram Extract authentication guide
Instagram Business Account ID Guide
Intercom
Intercom
Intercom Extract
Intercom Extract authentication guide
Jira
Jira
Jira Query
Jira Query authentication guide
LDAP
LDAP
LDAP Query
LinkedIn
LinkedIn
LinkedIn Query
LinkedIn Query authentication guide
LinkedIn Ads Query
LinkedIn Ads Query authentication guide
Magento
Magento
Magento Query
Mailchimp
Mailchimp
Mailchimp Query
Mailchimp Query authentication guide
Mandrill
Mandrill
Mandrill Extract
Mandrill Extract authentication guide
Marketo
Marketo
Marketo Query
Marketo Query Authentication Guide
Microsoft Exchange
Microsoft Exchange
Microsoft Exchange Query
Microsoft Exchange Query Authentication Guide
MindSphere
MindSphere
MindSphere Extract
MindSphere Extract authentication guide
Mixpanel
Mixpanel
Mixpanel Query
Mixpanel Query authentication guide
MongoDB
MongoDB
MongoDB Query
NetSuite
NetSuite
NetSuite Query
NetSuite Query Authentication Guide
NetSuite SuiteAnalytics Connect
NetSuite SuiteAnalytics Connect Authentication Guide
OData
OData
OData Query
Open Exchange Rates
Open Exchange Rates
Open Exchange Rates Query
Open Exchange Rates Query authentication guide
Oracle Eloqua
Oracle Eloqua
Oracle Eloqua Query
Oracle Eloqua Query authentication guide
Pardot
Pardot
Pardot Extract
Pardot Extract authentication guide
PayPal
PayPal
PayPal Query
PayPal Query Authentication Guide
QuickBooks
QuickBooks
QuickBooks Online Query
QuickBooks Online Query Authentication Guide
RDS
RDS
RDS Query
Redis
Redis
Redis Query
Recurly
Recurly
Recurly Extract
Recurly Extract authentication guide
Sage Intacct
Sage Intacct
Sage Intacct Query
Salesforce
Salesforce
Salesforce Query
Salesforce Incremental Load
Salesforce Query authentication guide
Salesforce Bulk Query
Salesforce Bulk Query authentication guide
Salesforce Marketing Cloud Query
Salesforce Marketing Cloud Query authentication guide
Salesforce Output authentication guide
SAP NetWeaver
SAP NetWeaver
SAP NetWeaver Query
SAP NetWeaver Query - add new data sources
SAP ODP
SAP ODP
SAP ODP Extract
SAP ODP connection options
SAP ODP troubleshooting
SAP ODP FAQ
SendGrid
SendGrid
SendGrid Query
SendGrid Query authentication guide
ServiceNow
ServiceNow
ServiceNow Query
ServiceNow Query authentication guide
SharePoint
SharePoint
SharePoint Query
SharePoint Query authentication guide
Shopify
Shopify
Shopify Query
Shopify Query Authentication Guide
Snapchat
Snapchat
Snapchat Extract
Snapchat Extract authentication guide
Splunk
Splunk
Splunk Query
SQL databases
SQL databases
Database Query
JDBC Incremental Load
Square
Square
Square Query
Square Query authentication guide
Stripe
Stripe
Stripe Query
Stripe Query authentication guide
Sugar CRM
Sugar CRM
Sugar CRM Query
SurveyMonkey
SurveyMonkey
SurveyMonkey Query
SurveyMonkey Query authentication guide
Twilio
Twilio
Twilio Query
Twilio Query authentication guide
Twitter
Twitter
Twitter Query
Twitter Query authentication guide
Twitter Ads Query
Workday
Workday
Workday Extract
Workday Custom Reports
Workday Extract authentication guide
Workday Integration System User setup
Finding a Workday object's integration ID
Managing access to data sources
Xero
Xero
Xero Query
Xero Query authentication guide
YouTube
YouTube
YouTube Query
YouTube Analytics Query
Zendesk
Zendesk
Zendesk Support Query
Zendesk Talk Query
Zendesk Support/Talk Query authentication guide
Zendesk Chat Extract
Zendesk Chat Extract Authentication Guide
Zoho CRM
Zoho CRM
Zoho CRM Query
Zoho CRM Query authentication guide
Zuora
Zuora
Zuora Query
Zuora Bulk Query
Zulk Query and Bulk Query authentication guide
Output components
Output components
Output components overview
Oracle Output
Microsoft SQL Server Output
Salesforce Output
Pardot Output
Intercom Output
Azure SQL Bulk Output
RDS Bulk Output
Cloud Storage
Cloud Storage
Amazon S3
Amazon S3
S3 Load
S3 Manifest Builder
S3 Unload
Azure Blob Storage
Azure Blob Storage
Azure Load Snowflake
Azure Load Delta Lake
Azure Load Synapse
Azure Unload
Google Cloud Storage
Google Cloud Storage
Cloud Storage Load
Cloud Storage Unload
Data Transfer Object
DDL
DDL
External Schema and Tables
Snowflake
Snowflake
Alter External Table
Create External Table
Refresh External Table
Alter Table
Alter Warehouse
Assign Tag
Alter Masking Policy
Create File Format
Create Stream
Amazon Redshift
Amazon Redshift
Add Partition
Alter Session WLM Slots
Analyze Tables
Create External Table
Delete Partition
Nested Data Load
Refresh Materialized View
Schema Copy
Text Output
Delta Lake on Databricks
Delta Lake on Databricks
Delta Live Streaming
Optimize
Refresh Table
Run Delta Live Table
Start Cluster
Google BigQuery
Google BigQuery
Create External Table
Refresh Materialized View
Azure Synapse Analytics
Azure Synapse Analytics
Alter Database
Create External Table
Create Table
Delete Table
SQL Script
Truncate Table
Vacuum Table
Flow components
Flow components
Flow components overview
Iterators
Iterators
Iterator components
File Iterator
Fixed Iterator
Grid Iterator
Loop Iterator
Table Iterator
Transactions
Transactions
Begin
Commit
Rollback
And
End Failure
End Success
If
Or
Retry
Run Orchestration
Run Transformation
Start
Load generators
Load generators
Load generators overview
Azure Blob Load Generator
Cloud Storage Load Generator (Snowflake)
Cloud Storage Load Generator (BigQuery)
S3 Load Generator (Snowflake)
S3 Load Generator (Redshift)
S3 Load Generator (Delta Lake)
Messaging
Messaging
SQS Message
SNS Message
PubSub
Azure Queue Storage Message
Webhook Post
Scripting
Scripting
Bash Script
Python Script
Sync File Source
Commands for dbt Core
Python Script additional settings
Run Notebook
Shared jobs
Shared jobs
CDC shared jobs overview
Sync All Tables shared job
Sync Single Table shared job
Create or Refresh External Table shared job
Drop CDC Tables shared job
Data typing with CDC shared jobs
Tables created
Append metadata
Variables
Variables
Append To Grid
Describe To Grid
Show to Grid
Query Result To Grid
JDBC Table Metadata To Grid
Query Result To Scalar
Remove From Grid
Table Metadata To Grid
Load generators overview
Send Email
Transformation
Transformation
Join
Join
Except
Intersect
Join
Unite
Read
Read
Fixed Flow
Generate Sequence
Multi Table Input
Stream Input
Table Input
Wildcard Table Input
Transform
Transform
Aggregate
Calculator
Construct Struct
Construct Variant
Convert Type
Table Delete Rows
Detect Changes
Distinct
Extract Nested Data
Filter
First-Last
Flatten Variant
Lead-Lag
Map Values
Pivot
Rank
Rename
Replicate
Split Field
SQL
Transpose Columns
Transpose Rows
List Aggregate
Unpivot
Window Calculation
Write
Write
Copy Table To External Schema
Create View
External Table Output
Rewrite External Table
Rewrite Table
Table Output
Table Update
Matillion ETL API
Matillion ETL API
Endpoints
Endpoints
API v1 - API extract profile
API v1 - API profile
API v1 - Audit
API v1 - Credentials
API v1 - Driver
API v1 - Environment
API v1 - Git integration for projects
API v1 - Git integration for shared jobs
API v1 - Group/project
API v1 - Metadata
API v1 - Notice
API v1 - OAuth
API v1 - Passwords
API v1 - Permission
API v1 - Queue
API v1 - Schedules
API v1 - Running jobs
API v1 - Shared jobs
API v1 - Tasks
API v1 - Userconfig
API v1 - Versions
API v1 - Webhookpayloadprofile
API v1 - Secret manager
Matillion ETL API - v1
Maps of Matillion API v1
Getting started with Postman
Getting started with cURL
Managing users, passwords, groups, and permissions
Authenticating Matillion REST API bearer token
Obtaining an API token and passing it to an API Query profile
v0 API
v0 API
Matillion ETL API - v0
v0 examples
Cloud configuration
Cloud configuration
Authorization and authentication
Setting up Let's Encrypt for SSL on a Matillion ETL Instance
Snowflake
Snowflake
Snowflake role privileges with Matillion ETL
Amazon Web Services
Amazon Web Services
Attaching AWS IAM roles To EC2 instances
IAM roles & permissions (AWS)
Changing EC2 instance size
Google Cloud Platform
Google Cloud Platform
IAM roles & permissions (GCP)
GCP service accounts
BigQuery data set setup
GCP enabling APIs
Microsoft Azure
Microsoft Azure
Changing Azure instance size
Roles & permissions (Azure)
Configuring a high availability cluster (Azure)
Guides
Guides
AWS services
AWS services
Spectrum
Spectrum
Getting Started with Amazon Redshift Spectrum
Using Spectrum in Matillion ETL
Accessing files in S3 using Pre-signed URLs
Automating Redshift maintenance
AWS S3 lifecycle rule
Cross-account S3 access
Parallelism with Matillion ETL for Redshift
Snowflake AWS Storage Integration setup guide
Azure services
Azure services
Snowflake Azure Storage Integration setup guide
Troubleshooting connection to Azure Blob Storage
GCP services
GCP services
Snowflake GCP Storage Integration setup guide
Common operations
Common operations
Populating tables
Populating tables
Adding filename as a column to tables (BigQuery)
Adding Filename as a column to new table
Populating an audit table
Building a data vault
Building a data vault (Snowflake)
Data transfer between databases
Ingesting AWS ElasticSearch data via the API Query component
Year-on-year analysis
Designing a job for a high availability cluster
Incremental or high water mark data Loading
Microbatch replication
How to receive emails by subscribing to a cloud Pub/Sub topic
Flattening nested arrays
Installation configuration
Installation configuration
Changing the host file
How to configure SSL protocols
Outbound IP requirements
Triggering Matillion ETL
Triggering Matillion ETL
Triggering ETL from an S3 event via AWS Lambda
Triggering ETL from an email via SES and Lambda
Triggering ETL from Amazon Alexa via AWS Lambda & Amazon SQS
Triggering a Matillion ETL job from your Google Home device
Triggering Matillion ETL from a storage queue via an Azure function
Collibra integration
Connecting to an RDS in a private VPC
Diagnostic data policy
Executing Python scripts outside of Matillion
Helping with the GDPR
Integrating Slack with Matillion ETL
Using grid variables to apply business rules in a transformation job
Making multiple API queries
Matillion data quality framework
NRT replication In Redshift
Pivoting and unpivoting tables
SCM integration
Tracking loaded files
Using incron to automatically copy data to S3
Using KMS encrypted passwords in Python
Using R with Matillion ETL for Redshift
Using Table Metadata to Grid
Managing Python on a Matillion ETL virtual machine (VM)
How to retrieve missing Task History entries after 1.47 upgrade
Matillion Exchange
Release notes
Release notes
Release notes index
Support lifecycle
Supported releases
Long-term support (LTS) release notes
1.75 release notes
1.74 release notes
1.73 release notes
1.72 release notes
1.71 release notes
1.70 release notes
1.69 release notes
1.68 release notes
1.67 release notes
1.66 release notes
1.65 release notes
1.64 release notes
1.63 release notes
Earlier than version 1.63
Earlier than version 1.63
Matillion ETL for Snowflake release notes
Matillion ETL for Redshift release notes
Matillion ETL for BigQuery release notes
Matillion ETL for Synapse release notes
Matillion ETL for Delta Lake release notes
Release notes advisories
Release notes archive
Tech notes
Tech notes
Tech note - has your Matillion ETL instance stopped working?
Critical Advisory - Mandatory update required to address Licence Management Defect
Remediation instructions for Licence Management Defect
Tech note - AWS SDK upgrade for Java
Tech note - 1.68 update failure
Tech note - automatic security updates
Tech note - Snowflake GCP update
Tech note - legacy key algorithm constraints
Tech note - OutOfMemory events in version 1.69
Tech note - Redshift RingBuffer exceeding expected limits
Tech note - disk partition sizing for versions 1.69-1.72
Updating to version 1.69 and above
Technology upgrade of PostgreSQL
Tech note - running Query components through a proxy server
Tech note - Shopify Query versioning
Tech note - Splunk Query versioning
Tech note - Google Analytics driver update
Tech note - Postgres driver for Amazon Redshift
Security Advisory - Spring4Shell
Tech Note - Google Ads updates in v1.62
Updating to version 1.60 and above
Security Advisory - v1.59.11
Security Advisory 13th Jan 2022
Security Advisory 17th Dec 2021
Security Advisory 14th Dec 2021
Tech Note 15th June 2021
Tech Note 14th May 2021
Potential credentials in Matillion ETL log file
Security Advisory - 29th April 2021
Tech Note 10th February 2021
Tech Note 28th January 2021
Tech Note 12th November 2020
Tech Note 22nd September 2020
Tech note 22nd June 2020
Unlimited Scale
Unlimited Scale
Unlimited Scale
Unlimited Scale
Editing permissions for Unlimited Scale
Running Matillion ETL jobs with Unlimited Scale
Agent
Agent
Agent overview
Editing user permissions for agents
Agent version tracks
Create an agent
Add agent credentials to AWS Secrets Manager
AWS IAM roles
Agent installation using a CloudFormation quick-create link
Manual agent setup
Data Productivity Cloud
Data Productivity Cloud
Data Productivity Cloud overview
Prerequisites
Architecture overview
Connectors overview
Free trial
Getting support
Supported browsers
Hub
Hub
Basics
Basics
Getting started
Registration
SSO integration
Multi-factor authentication
Administration
Administration
Change email address
Edit user permissions
Create an account
Manage account details
Manage account users
Reset your password
Services
Services
Select a service
Pipeline Observability
Data lineage (preview)
Matillion Data Loader
Matillion ETL
Matillion ETL
Matillion ETL instance creation
Associating a Matillion ETL instance
Configuring a connection from Matillion ETL to the Hub
Launching Matillion ETL
Launching Matillion ETL for GCP
Instance sizes
Subscriptions
Subscriptions
AWS Marketplace subscription
Azure Marketplace subscription
Subscription Management
Subscription Management
Credit consumption dashboard
Editions
Manage payment details and invoices
Manage subscription
Administration
Administration
Cloud credentials
Cloud credentials
Snowflake role privileges
Manage secrets
Manage secrets
Secrets overview
AWS Secrets Manager
Azure Key Vault
GCP Secret Manager
Security
Security
Overview
Deployment options
Git integration
Roles and permissions
Cloud platform security and compliance
Authentication and access control
Data source connectivity
Allowing IP addresses
Software updates and patches
Data gathering
Data lineage
Agents
Agents
Agent for Designer
Agent for Designer
Agent overview
Agent setup
Agent setup
Create an agent in your infrastructure
Manual agent setup
Agent installation using a CloudFormation quick-create link
Additional configuration
Additional configuration
Allowing the Matillion hosted agent
Editing user permissions for agents
Agent version tracks
Add agent credentials to AWS Secrets Manager
AWS IAM roles
Scaling best practices
How to modify access for each source system
Agent for CDC
Agent for CDC
CDC agent installation
CDC Agent setup UI
Sizing CDC agents
Agent versions
Agent update
Platform keys
Environment variables
Lock down outbound traffic
AWS installation
AWS installation
AWS installation overview
Deploying a CDC agent in AWS quick guide
Cloud resources
Cloud resources
Subnets and security groups
CloudWatch Logs
IAM Roles
S3 Buckets
Fargate and ECS
AWS templates
AWS templates
CloudFormation
CloudFormation
CloudFormation ECS Fargate basic template
CloudFormation ECS Fargate advanced template
CloudFormation EC2 ECS proxy templates
Kubernetes templates
Kubernetes templates
Agent (AWS) template
Terraform
Terraform
Terraform ECS Fargate basic template
Terraform ECS Fargate advanced template
Azure installation
Azure installation
Azure installation overview
Deploying a CDC agent in Azure quick guide
Deploying a CDC agent in Azure using the CLI
Cloud resources
Cloud resources
Managed identity
Blob Storage
Log analytics
Container instances
Azure templates
Azure templates
Azure - ARM template
Azure - Terraform template
Kubernetes templates
Kubernetes templates
Agent (Azure) template
GCP installation
GCP installation
GCP installation overview
Deploying a CDC agent in GCP quick guide
Cloud resources
Cloud resources
Project and Service Account
GCP Cloud Storage bucket
IAM permissions
Workload Identity (GKE)
GCP templates
GCP templates
Agent (GCE) template
Agent (GKE) Kubernetes template
Logs
Logs
AWS CloudWatch Logs
Azure Log Analytics
Google Cloud logging
Designer
Designer
Designer overview
Designer UI basics
Getting started
Getting started
Your Designer projects
Your Designer projects
Projects
Environments
Cloud provider credentials
Branches
Pipelines
Connectors overview
Secret definitions
OAuth
Schedules
Access
Cloud data platform guides
Cloud data platform guides
Databricks
Git version control
Git version control
Git in Designer
Commit
Pull remote changes
Push local changes
Merge
Hard reset
Connect your own Git repository
Connect your own Git repository
GitHub
GitHub
Overview of the Matillion GitHub app
Prerequisites
Installing the Matillion GitHub app
Troubleshooting GitHub repository connection
AI
AI
Overview of AI features
Prompt components
Prompt components
OpenAI Prompt
Azure OpenAI Prompt
Amazon Bedrock Prompt
Vector database operation components
Vector database operation components
Pinecone Vector Upsert
Pinecone Vector Query
AI Note
Pipeline components
Pipeline components
Components overview
Data lineage (preview)
Sampling output
Variables
Grid variables
Organizing file storage
Orchestration
Orchestration
AI
AI
Prompt
Prompt
OpenAI Prompt
Azure OpenAI Prompt
Amazon Bedrock Prompt
Vector database operations
Vector database operations
Pinecone Vector Upsert
Pinecone Vector Query
Connectors
Connectors
Load
Load
ActiveCampaign
Amplitude
Anaplan
Asana
Azure Blob Storage Load
Azure Cosmos DB for NoSQL
Azure SQL
Bing Ads Query
Braze
Brevo
Bugcrowd
Chargebee
CircleCI
Concord
Confluence
Data Transfer
Database Query
Datadog
Databricks
Delighted
Dixa
Dynamics 365 Query
DynamoDB Query
Email Query
Eventbrite
Excel Query
Facebook Ads Query
Facebook Query
FireHydrant
Freshdesk
GitHub
Gong
Google Ads Query
Google Analytics Query
Google BigQuery Query
Google Sheets Query
HubSpot Query
Infobip
Intercom
Jira Query
Jotform
Klaviyo
LaunchDarkly
LDAP Query
Lob
Mailchimp
Mailgun
Mandrill
Marketo Query
Microsoft Exchange
Mixpanel
MongoDB Query
NetSuite Query
New Relic
Notion
OData Query
Ongoing WMS
PagerDuty
PayPal
Pendo
Pipedrive
Productboard
RDS Query
Recurly
S3 Load
Salesforce Marketing Cloud Query
Salesforce Query
Salesloft
SAP SuccessFactors
SendGrid
ServiceNow Query
SharePoint Query
Shopify
Slack
Smartsheet
Snyk
Square
Stripe Query
Ticketmaster
TikTok
Toggl
Twilio
Workday
Workday Custom Reports
Yelp
Zendesk Talk
Zendesk Ticketing
Unload
Unload
Azure Blob Storage Unload
Microsoft SQL Server Output
RDS Bulk Output
S3 Unload
Salesforce Output
Authentication Guides
Authentication Guides
Bing Ads Query authentication guide
Dynamics 365 Query authentication guide
Google Query authentication guide
HubSpot Query authentication guide
Jira Query authentication guide
Mailchimp authentication guide
Marketo Query authentication guide
NetSuite Query authentication guide
Salesforce Query authentication guide
ServiceNow Query Authentication Guide
Stripe Query authentication guide
Workday authentication guide
DDL
DDL
Alter Table
Create Stream
Create Table
Delete Tables
Optimize
Refresh External Table
SQL Script
Truncate Table
Action
Action
Snowflake
Snowflake
Alter Warehouse
Create File Format
AWS
AWS
CloudWatch Publish
SNS Message
SQS Message
Messaging
Messaging
Send Email
Webhook Post
Testing
Testing
Assert External Table
Assert Scalar Variables
Assert Table
Print Variables
Scripting
Scripting
dbt Core
Bash Script
Python Pushdown (preview)
Python Script
Query Result to Grid
Query Result to Scalar
Run Notebook
Update Scalar
Utility
Utility
Start Compute
Vacuum
Run pipeline
Run pipeline
Run Orchestration
Run Transformation
Logic
Logic
Flow
Flow
And
End Failure
End Success
If
Or
Retry
Start
Iterate
Iterate
File Iterator
Fixed Iterator
Grid Iterator
Loop Iterator
Table Iterator
Transformation
Transformation
Join
Join
Except
Intersect
Join
Unite
Read
Read
Fixed Flow
Generate Sequence
Multi Table Input
Stream Input
Table Input
Testing
Testing
Assert View
Transform
Transform
Aggregate
Calculator
Construct Variant
Convert Type
Detect Changes
Distinct
Extract Nested Data
Filter
First/Last
Flatten Variant
Lead/Lag
Map Values
Pivot
Rank
Rename
Split field
SQL
Transpose Columns
Transpose Rows
Unpivot
Window Calculation
Write
Write
Create View
Rewrite Table
Table Output
Table Update
Examples
Examples
Incremental loading
Jira Query
S3 Load and transformation
Data Loader Batch
Data Loader Batch
Batch Loads Introduction
Getting started
Batch Pipeline UI
Batch loading overview
Batch loading overview
Schema drift
Batch load replication overview
Batch pipeline detailed view
Administration
Administration
Setup requirements
Setup requirements
Technical requirements
Cloud data warehouse permissions
Managing credentials
Managing credentials
Destinations
Passwords
Cloud credentials
OAuth connections
NetSuite batch pipeline authentication guide
Scheduling with Quartz cron expressions
Sources
Sources
Custom Connector Data Loader pipeline
Allowed connection properties
ActiveCampaign
Amazon Redshift
Amplitude
Brevo
Chargebee
Circle CI
Concord
Confluence
Datadog
DB2 for IBM i
Delighted
Dropbox
Dynamics 365 Sales
Elasticsearch
Eventbrite
Excel
Facebook AdAccounts
Facebook AdInsights
Facebook Content Insights
Freshdesk
Gmail
Google Analytics
Google BigQuery
Google Custom Search
Google Sheets
HubSpot
IBM Db2
Jira
Klaviyo
LaunchDarkly
Mailchimp
MariaDB
Marketo
Microsoft SQL Server
MongoDB
MySQL
NetSuite
OData
Open Exchange Rates
Oracle
Oracle Eloqua
PagerDuty
Pendo
PostgreSQL
Productboard
QuickBooks
Recurly
Sage Intacct
Salesforce
Salesforce Pardot
Shopify
Slack
Smartsheet
Snapchat
Snowflake
Snyk
Stripe
SugarCRM
TikTok
Toggl
Twilio
Xero
Zendesk
Zendesk Talk
Destinations
Destinations
Snowflake
Snowflake
Set up Snowflake
Connect to Snowflake
Delta Lake on Databricks
Delta Lake on Databricks
Set up Delta Lake on Databricks
Connect to Delta Lake on Databricks
Amazon Redshift
Amazon Redshift
Set up Amazon Redshift
Connect to Amazon Redshift
Google BigQuery
Google BigQuery
Set up Google BigQuery
Connect to Google BigQuery
Tech notes
Tech notes
Tech note - failing pipelines
Troubleshooting
Troubleshooting
Destinations
Data Loader CDC
Data Loader CDC
Matillion Change Data Capture
Getting started
CDC shared jobs
CDC Pipeline UI
CDC Overview
CDC Overview
CDC pipeline overview
CDC pipeline detailed view
Create and manage a CDC pipeline
Schema Drift
Administration
Administration
Technical Requirements (CDC)
Lifecycle management
Lifecycle management
Amazon S3 lifecycle management
Azure Blob Storage lifecycle management
Google Cloud Storage lifecycle management
Sources
Sources
CDC sources overview
Source database maintenance
Db2 for IBM i
Db2 for IBM i
Db2 for IBM i connector
Configuring Db2 for IBM i
DB2 for IBM i snapshot
PostgreSQL
PostgreSQL
PostgreSQL connector
Configuring PostgreSQL database
PostgreSQL advanced settings
PostgreSQL snapshot
Oracle
Oracle
Oracle connector
Configuring Oracle Database
Oracle advanced settings
Oracle snapshot
Microsoft SQL Server
Microsoft SQL Server
Microsoft SQL Server connector
Configuring Microsoft SQL Server database
Microsoft SQL advanced settings
Microsoft SQL Server snapshot
MySQL
MySQL
MySQL connector
Configuring MySQL database
Configuring the CDC agent for MySQL
MySQL snapshot
Destinations
Destinations
Snowflake
Amazon S3
Azure Blob
Google Cloud Storage
Troubleshooting
Troubleshooting
Agent setup
Destinations
UI setup
Custom Connector
Custom Connector
Matillion Custom Connector Overview
Export your own connector to Matillion ETL
Create a pipeline with custom connectors
Creating Custom Connectors
Creating Custom Connectors
Custom connector setup
Endpoints
Authentication
Add OAuth
Pagination
API parameters
Custom connector example
Flex connectors
Flex connectors
Flex connector setup
Flex connector tutorial
Changelogs
Changelogs
2024
2024
May
April
March
February
January
2023
2023
December
November
October
September
August
July
June
May
April
March
February
January
2022
2022
December
November
October
September
August
July
June
May
April
Data Productivity Cloud API
Data Productivity Cloud API
API overview
Authentication
API reference
API reference
API reference user guide
Pipeline execution API reference
API Guides
API Guides
Executing and managing a pipeline
API changelog
Getting support
Legal
Legal
Billing details
Credit breakdown
Data processing terms
Matillion SLA
Mission critical support
Software licenses
Terms of service
Style guide
Style guide
Markdown and formatting
Markdown and formatting
Formatting text
Capitalization
Headings
Horizontal rules
Images
Links
Lists
Tables
Words
Azure agent preview
Azure agent preview
Azure agent private preview
Create an Azure agent in your infrastructure
Configuring a key vault for Azure agent
Agent installation using an ARM template
ARM template permissions
Download and amend the Azure ARM template
Troubleshooting
Copilot preview
Copilot preview
Copilot overview
Copilot prerequisites
Start using Copilot
Example uses of Copilot
Pipeline execution API reference
Swagger UI
Back to top