logo
Forums Blogs Roadmap & Ideas User Groups & Events Stack Overflow Reddit
Data Productivity Cloud Docs Matillion ETL Docs YouTube Connectors Migration
Knowledge Base My Support Cases About Support
Academy Data Productivity Cloud Matillion ETL Certifications
Professional Services & Training Consulting Partners
Initializing search
    • Home
    • Matillion ETL
    • Data Productivity Cloud
    • Style guide
    • Data Loader CDC
    • Home
      • Matillion ETL product overview
      • Ask Matillion
        • Explore the Data Productivity Cloud
        • Matillion ETL instance creation
        • Associating a Matillion ETL instance
        • Configuring a connection from Matillion ETL to the Data Productivity Cloud
        • Launching Matillion ETL
        • Launching Matillion ETL for GCP
          • Matillion ETL usage
            • Matillion ETL editions
            • Matillion ETL payment and invoicing
            • Matillion ETL marketplace subscriptions
        • Matillion ETL observability
        • Instance sizes
          • Accessing the Matillion ETL Client (Amazon EC2)
          • Accessing the Matillion ETL Client (Google Cloud Platform)
          • Accessing the Matillion ETL Client (Microsoft Azure)
          • Assert components in Matillion ETL
          • Expression editors
          • Incremental load tools
          • URL safe characters
          • Generate Job Documentation
          • Jobs
          • Job concurrency
          • Notes
          • Shared jobs
          • Task management
          • Environments
          • Manage Stages
          • Multiple environment connections
          • Table properties
          • Creating a Snowflake Zero-Copy Clone
          • Create External Schema
          • Variables
          • Component Exports
          • Date and time methods
          • Environment Variables
          • Grid variables
          • Job Variables
        • Enterprise mode
        • Scope of Matillion ETL features
        • UI and basic functions
        • Admin menu
        • Schema
        • Notices
        • Search tab
        • Performance monitor
        • Views
        • Project collaboration
        • Software versions
        • Glossary
        • Manage Interpreters
          • Audit log
          • How to place restrictions on Bash and Python components
          • How to configure Catalina log rotation
          • Control session timeout expiration
          • Product improvement metrics
          • Restart server
          • Publicly available warning
          • Upgrade Tomcat version
          • Backups (AWS)
          • Backups (GCP)
          • Backups (Azure)
          • Manage backups
          • Snowflake configuration for Matillion ETL
          • Redshift configuration for Matillion ETL
          • Delta Lake on Databricks configuration for Matillion ETL
          • Configuring an AWS VPC
          • How to generate a new Databricks token
          • Snowflake key-pair authentication
          • Configuring Matillion ETL to use a Proxy
          • Setting up an external connection to a Matillion database
          • Manage connections
          • Matillion ETL access ports
          • Configuring a connection from Matillion ETL to the Data Productivity Cloud
          • Setting up Matillion ETL in a private VPC
          • Adding a third-party JDBC driver
          • Getting started with the API driver in Matillion ETL
          • SAP Hana JDBC driver installation for Matillion ETL
          • Manage Database Drivers
          • Groups and Permissions
          • Permissions list
          • Preview Labs
          • SSL commands
          • How to add a certificate chain file for SSL configuration
          • Recreating self-signed SSL certificates on a Matillion ETL instance
          • SSL Configuration FAQ
          • Updating and migrating overview
          • Migration
          • In-place update
          • Updating to a specific release
          • Updating a high availabilty cluster
          • User configuration
            • Using stateless authentication with existing users
            • Configuring stateless authentication
            • Internal security (stateless authentication)
            • OpenID integration (stateless authentication)
            • LDAP integration (stateless authentication)
            • Stateless authentication rollback steps
            • OpenID overview
            • Microsoft Entra ID OpenID setup
            • Google OpenID setup
            • Microsoft Active Directory OpenID setup
            • Okta OpenID setup
            • OneLogin OpenID setup
            • OpenID setup
            • Troubleshooting OpenID setup
            • LDAP integration
            • Okta LDAP Configuration
          • Read-only users
          • Reverting from external to internal security
        • Matillion ETL security best practices
        • Using CSRF tokens to safeguard Matillion ETL instances
        • Installing DBT on Matillion ETL
        • Connecting to an external PostgreSQL database
        • Automatic security updates
        • Manage optional features
        • Launching Matillion ETL overview
        • Launching Cloud Platform Permissions
          • Launching Matillion ETL using CloudFormation Templates
          • List of CloudFormation Templates
          • Launching Matillion ETL using Amazon Machine Image
          • Launching Matillion ETL for Delta Lake on AWS
          • Launching troubleshooting (AWS)
          • Overview
          • Launching Matillion ETL from Azure Marketplace
          • Launching Matillion ETL using an Azure ARM Template
          • Launching Matillion ETL for Delta Lake on Azure
          • Launching troubleshooting (Azure)
          • Launching Matillion ETL for Snowflake - GCP
          • Launching Matillion ETL for BigQuery - GCP
          • Launching a Matillion ETL HA Cluster via AWS
          • Installing Matillion ETL using the Universal Installer (RPM install)
          • Overview
          • Launching Matillion ETL using CloudFormation Templates
          • List of Snowflake Launch Templates
          • Migrate from Snowflake Partner Connect to Matillion ETL for Snowflake
          • Launching Matillion ETL from AWS Marketplace
          • Launching Matillion ETL for Redshift from AWS Marketplace
          • List of Redshift Launch Templates
          • Converting to be an Annual Customer
          • Launching Matillion ETL via Azure CLI
          • Finding and Launching Matillion BYOL Images
          • Instance Sizes Guide
        • Applying a licence
          • Create Project (Snowflake)
          • Create Project (Delta Lake on Databricks)
          • Create Project (Amazon Redshift)
          • Create Project (Google BigQuery)
          • Create Project (Azure Synapse Analytics)
          • Manage Project
          • Manage Credentials
          • Manage Extract Profiles
          • Manage Passwords
          • Manage Query Profiles
          • Manage OAuth
          • Manage Schedules
          • Manage Sequences
          • Manage Shared Jobs
          • Manage Versions
          • Manage Webhook Payloads
          • Integrating Matillion ETL with secret managers
          • Manage API Profiles wizards
          • Manage External File Sources
          • Azure Queue Message configuration
          • Manage Pub-Sub configuration
          • Manage SQS Configuration
          • Manage CDC
          • Enable Manage CDC
          • Configuring a source database for CDC
          • DMS migration instances
          • Git Integration with Matillion ETL
          • Git Integration Frequently Asked Questions
          • When to choose Git
          • MergeManager
          • API Profiles Overview
          • API Query Profiles
          • API Extract Profiles
          • API Connector Wizard
          • API Profiles - Pagination
          • API Profiles - Parameters
          • API Profiles - Authentication
          • API Profiles - RSDs
          • API Profiles Example - MongoDB and DynamoDB
          • API Profiles Example - Jira Cloud
          • API Profiles Example - Salesforce Lightning
        • Switch Project
        • Using data structure variables
        • Change My Password
        • Extract to new job
        • Task History
        • Import - Export
        • Input data report
        • Manage Error reporting
        • Project user access
        • Recycle Bin
        • Job references
        • Job reference renaming
        • Databricks job compute configuration
        • Snowflake query tag configuration
        • Data staging components
        • Data stagers - support
        • Data models
        • Connecting to external services securely
          • Assert External Table
          • Assert Scalar Variables
          • Assert Table
          • Assert View
          • Print Variables
          • Connectors overview
          • Acquiring Azure Credentials
            • Amplitude Extract
            • Amplitude Extract authentication guide
            • Anaplan Bulk
            • Anaplan Bulk authentication guide
            • API Query
            • API Extract
            • API Query functions
            • Create your own Matillion ETL Extract connector
            • Populating parameters with API Query
            • Apache Hive Query
            • Apache Spark SQL Query
            • Azure SQL Query
            • Azure Cosmos DB Query
            • Bing Ads Query
            • Bing Ads Query authentication guide
            • Bing Search Query
            • Bing Search Query authentication guide
            • Box Extract
            • Box Extract authentication guide
            • Cassandra Query
            • CloudWatch Publish
            • Couchbase Query
            • Data Transfer Object
            • Dropbox Extract
            • Dropbox Extract authentication guide
            • Dynamics 365 Query
            • Dynamics 365 Query Authentication Guide
            • Dynamics 365 Business Central Query
            • Dynamics 365 Business Central Query authentication guide
            • Dynamics 365 Sales Query
            • Dynamics 365 Sales Query Authentication Guide
            • Dynamics CRM Query
            • Dynamics CRM Query authentication guide
            • Dynamics NAV Query
            • DynamoDB Load
            • DynamoDB Query
            • EMR Load
            • Elasticsearch Query
            • Email Query
            • Excel Query
            • Facebook Query
            • Facebook Ads Query
            • Facebook Authentication Guide
            • Facebook Connector FAQ
            • Gmail Query
            • Gmail Query authentication guide
            • Google Ad Manager Query
            • Google Ads Query
            • Google Ad Manager Query authentication guide
            • Google Analytics Query
            • Google Analytics Query - add new data sources
            • Google BigQuery
            • Google Custom Search Query
            • Google Custom Search Query authentication guide
            • Google Drive Table
            • Google Sheets Query
            • Google Query authentication guide
            • Google third-party OAuth using Service Accounts
            • How to set your own Google Ads developer token
            • HubSpot Query
            • HubSpot Query authentication guide
            • Instagram Extract
            • Instagram Extract authentication guide
            • Instagram Business Account ID Guide
            • Intercom Extract
            • Intercom Extract authentication guide
            • Jira Query
            • Jira Query authentication guide
            • LDAP Query
            • LinkedIn Query
            • LinkedIn Query authentication guide
            • LinkedIn Ads Query
            • LinkedIn Ads Query authentication guide
            • Magento Query
            • Mailchimp Query
            • Mailchimp Query authentication guide
            • Mandrill Extract
            • Mandrill Extract authentication guide
            • Marketo Query
            • Marketo Query Authentication Guide
            • Microsoft Exchange Query
            • Microsoft Exchange Query Authentication Guide
            • MindSphere Extract
            • MindSphere Extract authentication guide
            • Mixpanel Query
            • Mixpanel Query authentication guide
            • MongoDB Query
            • NetSuite Query
            • NetSuite Query Authentication Guide
            • NetSuite SuiteAnalytics Connect
            • NetSuite SuiteAnalytics Connect Authentication Guide
            • OData Query
            • Open Exchange Rates Query
            • Open Exchange Rates Query authentication guide
            • Oracle Eloqua Query
            • Oracle Eloqua Query authentication guide
            • Pardot Extract
            • Pardot Extract authentication guide
            • PayPal Query
            • PayPal Query Authentication Guide
            • QuickBooks Online Query
            • QuickBooks Online Query Authentication Guide
            • RDS Query
            • Redis Query
            • Recurly Extract
            • Recurly Extract authentication guide
            • Sage Intacct Query
            • Salesforce Query
            • Salesforce Incremental Load
            • Salesforce Query authentication guide
            • Salesforce Bulk Query
            • Salesforce Bulk Query authentication guide
            • Salesforce Marketing Cloud Query
            • Salesforce Marketing Cloud Query authentication guide
            • Salesforce Output authentication guide
            • SAP NetWeaver Query
            • SAP NetWeaver Query - add new data sources
            • SAP ODP Extract
            • SAP ODP connection options
            • SAP ODP troubleshooting
            • SAP ODP FAQ
            • SendGrid Query
            • SendGrid Query authentication guide
            • ServiceNow Query
            • ServiceNow Query authentication guide
            • SharePoint Query
            • SharePoint Query authentication guide
            • Shopify Query
            • Shopify Query Authentication Guide
            • Snapchat Extract
            • Snapchat Extract authentication guide
            • Splunk Query
            • Database Query
            • JDBC Incremental Load
            • Square Query
            • Square Query authentication guide
            • Stripe Query
            • Stripe Query authentication guide
            • Sugar CRM Query
            • SurveyMonkey Query
            • SurveyMonkey Query authentication guide
            • Twilio Query
            • Twilio Query authentication guide
            • Twitter Query
            • Twitter Query authentication guide
            • Twitter Ads Query
            • Workday Extract
            • Workday Custom Reports
            • Workday Extract authentication guide
            • Workday Integration System User setup
            • Finding a Workday object's integration ID
            • Managing access to data sources
            • Xero Query
            • Xero Query authentication guide
            • YouTube Query
            • YouTube Analytics Query
            • Zendesk Support Query
            • Zendesk Talk Query
            • Zendesk Support/Talk Query authentication guide
            • Zendesk Chat Extract
            • Zendesk Chat Extract Authentication Guide
            • Zoho CRM Query
            • Zoho CRM Query authentication guide
            • Zuora Query
            • Zuora Bulk Query
            • Zulk Query and Bulk Query authentication guide
          • Output components overview
          • Oracle Output
          • Microsoft SQL Server Output
          • Salesforce Output
          • Pardot Output
          • Intercom Output
          • Azure SQL Bulk Output
          • RDS Bulk Output
            • S3 Load
            • S3 Manifest Builder
            • S3 Unload
            • Azure Load Snowflake
            • Azure Load Delta Lake
            • Azure Load Synapse
            • Azure Unload
            • Cloud Storage Load
            • Cloud Storage Unload
          • External Schema and Tables
            • Alter External Table
            • Create External Table
            • Refresh External Table
            • Alter Table
            • Alter Warehouse
            • Assign Tag
            • Alter Masking Policy
            • Create File Format
            • Create Stream
            • Add Partition
            • Alter Session WLM Slots
            • Analyze Tables
            • Create External Table
            • Delete Partition
            • Nested Data Load
            • Refresh Materialized View
            • Schema Copy
            • Text Output
            • Delta Live Streaming
            • Optimize
            • Refresh Table
            • Run Delta Live Table
            • Start Cluster
            • Create External Table
            • Refresh Materialized View
            • Alter Database
            • Create External Table
          • Create Table
          • Delete Table
          • SQL Script
          • Truncate Table
          • Vacuum Table
          • Flow components overview
            • Iterator components
            • File Iterator
            • Fixed Iterator
            • Grid Iterator
            • Loop Iterator
            • Table Iterator
            • Begin
            • Commit
            • Rollback
          • And
          • End Failure
          • End Success
          • If
          • Or
          • Retry
          • Run Orchestration
          • Run Transformation
          • Start
          • Load generators overview
          • Azure Blob Load Generator
          • Cloud Storage Load Generator (Snowflake)
          • Cloud Storage Load Generator (BigQuery)
          • S3 Load Generator (Snowflake)
          • S3 Load Generator (Redshift)
          • S3 Load Generator (Delta Lake)
          • SQS Message
          • SNS Message
          • PubSub
          • Azure Queue Storage Message
          • Webhook Post
          • Bash Script
          • Python Script
          • Sync File Source
          • Commands for dbt Core
          • Python Script additional settings
          • Run Notebook
          • CDC shared jobs overview
          • Sync All Tables shared job
          • Sync Single Table shared job
          • Create or Refresh External Table shared job
          • Drop CDC Tables shared job
          • Data typing with CDC shared jobs
          • Tables created
          • Append metadata
          • Append To Grid
          • Describe To Grid
          • Show to Grid
          • Query Result To Grid
          • JDBC Table Metadata To Grid
          • Query Result To Scalar
          • Remove From Grid
          • Table Metadata To Grid
        • Load generators overview
        • Send Email
          • Except
          • Intersect
          • Join
          • Unite
          • Fixed Flow
          • Generate Sequence
          • Multi Table Input
          • Stream Input
          • Table Input
          • Wildcard Table Input
          • Aggregate
          • Calculator
          • Construct Struct
          • Construct Variant
          • Convert Type
          • Table Delete Rows
          • Detect Changes
          • Distinct
          • Extract Nested Data
          • Filter
          • First-Last
          • Flatten Variant
          • Lead-Lag
          • Map Values
          • Pivot
          • Rank
          • Rename
          • Replicate
          • Split Field
          • SQL
          • Transpose Columns
          • Transpose Rows
          • Transpose Rows (Snowflake)
          • Unpivot
          • Window Calculation
          • Copy Table To External Schema
          • Create View
          • External Table Output
          • Rewrite External Table
          • Rewrite Table
          • Table Output
          • Table Update
          • API v1 - API extract profile
          • API v1 - API profile
          • API v1 - Audit
          • API v1 - Credentials
          • API v1 - Driver
          • API v1 - Environment
          • API v1 - Git integration for projects
          • API v1 - Git integration for shared jobs
          • API v1 - Group/project
          • API v1 - License
          • API v1 - Metadata
          • API v1 - Notice
          • API v1 - OAuth
          • API v1 - Passwords
          • API v1 - Permission
          • API v1 - Queue
          • API v1 - Schedules
          • API v1 - Running jobs
          • API v1 - Shared jobs
          • API v1 - Tasks
          • API v1 - Userconfig
          • API v1 - Versions
          • API v1 - Webhookpayloadprofile
          • API v1 - Secret manager
        • Matillion ETL API - v1
        • Maps of Matillion API v1
        • Getting started with Postman
        • Getting started with cURL
        • Managing users, passwords, groups, and permissions
        • Authenticating Matillion REST API bearer token
        • Obtaining an API token and passing it to an API Query profile
          • Matillion ETL API - v0
          • v0 examples
        • Authorization and authentication
        • Setting up Let's Encrypt for SSL on a Matillion ETL Instance
          • Snowflake role privileges with Matillion ETL
          • Attaching AWS IAM roles To EC2 instances
          • IAM roles & permissions (AWS)
          • Changing EC2 instance size
          • IAM roles & permissions (GCP)
          • GCP service accounts
          • BigQuery data set setup
          • GCP enabling APIs
          • Changing Azure instance size
          • Roles & permissions (Azure)
          • Configuring a high availability cluster (Azure)
            • Getting Started with Amazon Redshift Spectrum
            • Using Spectrum in Matillion ETL
          • Accessing files in S3 using Pre-signed URLs
          • Automating Redshift maintenance
          • AWS S3 lifecycle rule
          • Cross-account S3 access
          • Parallelism with Matillion ETL for Redshift
          • Snowflake AWS Storage Integration setup guide
          • Snowflake Azure Storage Integration setup guide
          • Troubleshooting connection to Azure Blob Storage
          • Creating secrets in Azure Key Vault
          • Snowflake GCP Storage Integration setup guide
            • Adding filename as a column to tables (BigQuery)
            • Adding Filename as a column to new table
            • Populating an audit table
          • Building a data vault
          • Building a data vault (Snowflake)
          • Data transfer between databases
          • Ingesting AWS ElasticSearch data via the API Query component
          • Year-on-year analysis
          • Designing a job for a high availability cluster
          • Incremental or high water mark data Loading
          • Microbatch replication
          • How to receive emails by subscribing to a cloud Pub/Sub topic
          • Flattening nested arrays
          • Changing the host file
          • How to configure SSL protocols
          • Outbound IP requirements
          • Triggering ETL from an S3 event via AWS Lambda
          • Triggering ETL from an email via SES and Lambda
          • Triggering ETL from Amazon Alexa via AWS Lambda & Amazon SQS
          • Triggering a Matillion ETL job from your Google Home device
          • Triggering Matillion ETL from a storage queue via an Azure function
        • Collibra integration
        • Connecting to an RDS in a private VPC
        • Diagnostic data policy
        • Executing Python scripts outside of Matillion
        • Helping with the GDPR
        • Integrating Slack with Matillion ETL
        • Using grid variables to apply business rules in a transformation job
        • Making multiple API queries
        • Matillion data quality framework
        • NRT replication In Redshift
        • Pivoting and unpivoting tables
        • SCM integration
        • Tracking loaded files
        • Using incron to automatically copy data to S3
        • Using KMS encrypted passwords in Python
        • Using R with Matillion ETL for Redshift
        • Using Table Metadata to Grid
        • Managing Python on a Matillion ETL virtual machine (VM)
        • How to retrieve missing Task History entries after 1.47 upgrade
        • Matillion Exchange
        • Migration considerations
        • Migration prerequisites
        • Migration process
        • Migrating shared jobs
        • Migration mappings
        • Migration - Decommission Matillion ETL
          • API Extract and API Query
          • Automatic variables
          • Bash
          • Database Query
          • dbt
          • Export variables
          • Extract Nested Data
          • Iterator components
          • Python
          • Replicate
          • Temporary tables
          • Text Output
          • Transactions
          • Variables
        • Release notes index
        • Support lifecycle
        • Supported releases
        • 1.78 (LTS) release notes
        • 1.77 release notes
        • 1.76 release notes
        • 1.75 (LTS) release notes
        • 1.74 release notes
        • 1.73 release notes
        • 1.72 release notes
        • 1.71 release notes
        • 1.70 release notes
        • 1.69 release notes
        • 1.68 release notes
        • 1.67 release notes
        • 1.66 release notes
        • 1.65 release notes
        • 1.64 release notes
        • 1.63 release notes
          • Matillion ETL for Snowflake release notes
          • Matillion ETL for Redshift release notes
          • Matillion ETL for BigQuery release notes
          • Matillion ETL for Synapse release notes
          • Matillion ETL for Delta Lake release notes
        • Release notes advisories
        • Release notes archive
        • Tech note - Snowflake to block single-factor password authentication
        • Tech note - Image scanning for CVEs
        • Tech note - Removal of Manage CDC
        • Tech note - AWS thread count increases leading to failing instances
        • Tech note - user configuration and security best practices update
        • Tech note - has your Matillion ETL instance stopped working?
        • Critical Advisory - Mandatory update required to address Licence Management Defect
        • Remediation instructions for Licence Management Defect
        • Tech note - AWS SDK upgrade for Java
        • Tech note - 1.68 update failure
        • Tech note - automatic security updates
        • Tech note - Snowflake GCP update
        • Tech note - legacy key algorithm constraints
        • Tech note - OutOfMemory events in version 1.69
        • Tech note - Redshift RingBuffer exceeding expected limits
        • Tech note - disk partition sizing for versions 1.69-1.72
        • Updating to version 1.69 and above
        • Technology upgrade of PostgreSQL
        • Tech note - running Query components through a proxy server
        • Tech note - Shopify Query versioning
        • Tech note - Splunk Query versioning
        • Tech note - Google Analytics driver update
        • Tech note - Postgres driver for Amazon Redshift
        • Security Advisory - Spring4Shell
        • Tech Note - Google Ads updates in v1.62
        • Updating to version 1.60 and above
        • Security Advisory - v1.59.11
        • Security Advisory 13th Jan 2022
        • Security Advisory 17th Dec 2021
        • Security Advisory 14th Dec 2021
        • Tech Note 15th June 2021
        • Tech Note 14th May 2021
        • Potential credentials in Matillion ETL log file
        • Security Advisory - 29th April 2021
        • Tech Note 10th February 2021
        • Tech Note 28th January 2021
        • Tech Note 12th November 2020
        • Tech Note 22nd September 2020
        • Tech note 22nd June 2020
      • Getting support
      • New navigation and Designer experience
      • Ask Matillion
        • Registration
        • Free trial
        • Data Productivity Cloud new user tutorial
          • Full SaaS prerequisites
          • Hybrid SaaS prerequisites
          • Services
          • Set up a Matillion managed project
          • Pipelines tutorial
          • Hybrid SaaS Snowflake on AWS
          • Hybrid SaaS Snowflake on Azure
          • Hybrid SaaS Snowflake on Snowpark
          • Full SaaS Snowflake
          • Hybrid SaaS Databricks on AWS
          • Hybrid SaaS Databricks on Azure
          • Full SaaS Databricks
          • Hybrid SaaS Amazon Redshift on AWS
          • Full SaaS Amazon Redshift
        • Overview of Maia
        • Start using Maia
        • Maia prerequisites
        • Maia pipelines
        • Maia context files
        • Maia sessions and tools
        • Using Maia for root cause analysis
        • Using Maia for anomaly detection
        • Create a custom connector with Maia
        • How-to
        • Features
        • Connectors
        • AI
        • Data Productivity Cloud overview
        • Architecture overview
        • Connectors overview
        • DataOps overview
        • Getting support
        • Supported browsers
        • Data Productivity Cloud release FAQ
            • Snowflake role privileges
            • Using Snowflake key-pair authentication
            • Databricks
          • Troubleshooting data warehouse connectivity
          • Secrets overview
          • AWS Secrets Manager
          • Azure Key Vault
          • GCP Secret Manager
        • Overview
        • Deployment options
        • Git integration
        • Cloud environment roles and permissions
        • Cloud platform security and compliance
        • Authentication and access control
        • Data source connectivity
        • Allowing IP addresses
        • Software updates and patches
        • Data gathering
        • Data lineage
        • Private link
        • SSH Tunneling
          • Manage email address and password
          • Edit user permissions
          • Create an account
          • Edit account details
          • Account regions
          • Manage account users
          • Usage dashboard
            • Editions
            • Payment and invoicing
            • Marketplace subscriptions
            • AWS Marketplace subscription
            • Azure Marketplace subscription
          • Multi-factor authentication
          • SSO overview
          • Agent overview
          • Create an agent in your infrastructure
          • Manage agents
          • Restart an agent
          • Pause an agent
          • Refresh agent credentials
          • Agent updates
          • Agent logs
            • Manual agent setup
            • Agent installation using a CloudFormation quick-create link
            • AWS IAM roles
            • Add agent credentials to AWS Secrets Manager
            • Connect an agent via AWS PrivateLink
            • Agent installation using an ARM template
            • ARM template permissions
            • Configuring a key vault for Azure agent
            • Download and amend the Azure ARM template
            • Azure managed identity
            • Azure agent troubleshooting
            • Scaling best practices
            • Customizing agent networking and connectivity
            • Uploading external drivers to the agent
            • Installing Python libraries
            • Optional agent parameters
            • Installing Matillion agent for Snowflake
            • Secrets in Matillion agent for Snowflake
            • Troubleshooting Matillion agent for Snowflake
          • Create a Streaming agent
          • Sizing streaming agents
          • Streaming agent logs
          • Modify access to source databases
            • AWS IAM roles for Streaming agents
            • AWS Fargate manual Streaming agent setup
            • Agent installation using a CloudFormation quick-create link
            • Streaming agent installation on Azure
            • Download and amend the Azure ARM Streaming agent template
            • ARM Streaming agent template permissions
            • Configuring a key vault for Azure Streaming agents
            • Communicating with other Azure services
            • Azure Streaming agent troubleshooting
            • GCP Streaming agent install
        • Designer overview
        • Designer UI basics
        • Pipelines tutorial
        • Troubleshooting Designer
          • Projects
          • Environments
          • Environment access
          • Cloud provider credentials
          • Branches
          • Pipelines
          • Shared pipelines
          • Getting started with data loading
          • Secrets and secret definitions
          • OAuth
          • Schedules
          • Artifacts
          • User access, roles and permissions
          • Networks
          • Databricks Jobs Compute
          • Connecting via AWS PrivateLink
          • Git in Designer
          • Commit changes
          • Push local changes
          • Pull remote changes
          • Merge from branch
          • Compare changes
          • Reset branch
          • Pull requests
            • Overview
            • Reconnecting your Git user account
              • Overview of the Matillion GitHub app
              • Prerequisites
              • Installing the Matillion GitHub app
              • Troubleshooting GitHub repository connection
              • Overview of the Matillion Azure app
              • Prerequisites
              • Installing the Matillion Azure app
              • Troubleshooting an Azure DevOps repository connection
              • Overview of the Matillion GitLab app
              • Prerequisites
              • Troubleshooting GitLab repository connections
              • Overview of the Matillion Bitbucket app
              • Prerequisites
              • Troubleshooting Bitbucket repository connections
          • Overview of AI features
          • Data Productivity Cloud AI FAQ
          • AI Note
          • Components overview
          • Data lineage
          • Pipeline observability
          • Pipeline notifications
          • Sampling output
          • Variables
          • Grid variables
          • System variables
          • Post-processing
          • Organizing file storage
          • Code Editor
                • Chunk Text
                • Cortex Finetune
                • Snowpark Container Prompt
                • Create an app in Snowpark Container Services
                • OpenAI Prompt
                • Amazon Bedrock Prompt
                • Amazon OpenSearch Upsert
                • Amazon Textract Input
                • Amazon Transcribe
                • Azure OpenAI Prompt
                • Azure Document Intelligence
                • Azure Speech Transcribe
                • Vertex AI Prompt
                  • Snowflake Vector Upsert (preview)
                  • Pinecone Vector Upsert
                  • Pinecone Vector Query
                  • Postgres Vector Upsert
                • ActiveCampaign
                • Amplitude
                • Anaplan
                • Asana
                • Attio
                • Azure Blob Storage Load
                • Azure Cosmos DB for NoSQL
                • Azure SQL
                • Bing Ads Query
                • Box
                • Braze
                • Brevo
                • Bugcrowd
                • Buildkite
                • Chargebee
                • CircleCI
                • ClickUp
                • Coalesce
                • Concord
                • Confluence
                • Custom connector (agnostic)
                • Data Productivity Cloud (Matillion)
                • Data Transfer
                • Database Query
                • Datadog
                • Databricks
                • dbt Cloud
                • Delighted
                • Dixa
                • Dropbox
                • Dynamics 365 Business Central
                • Dynamics 365 Query
                • DynamoDB Query
                • Email Query
                • Eventbrite
                • Excel Query
                • Facebook Ads Query
                • Facebook Query
                • Fanatics
                • FireHydrant
                • Float
                • Freshdesk
                • GitHub
                • Gmail
                • Gong
                • Google Ads Query
                • Google Analytics Query
                • Google BigQuery Query
                • Google Cloud Storage Load
                • Google Sheets
                • Google Sheets Query (deprecated)
                • HubSpot Query
                • Infobip
                • Intercom
                • JDBC
                • JDBC Load
                • Jira Query
                • Jotform
                • Kafka
                • Klaviyo
                • LaunchDarkly
                • LDAP Query
                • LinkedIn Ads
                • Lob
                • Mailchimp
                • Mailgun
                • Mandrill
                • MariaDB Load
                • Marketo Query
                • Microsoft Dataverse
                • Microsoft Exchange
                • Microsoft SQL Server Load
                • Mixpanel
                • MongoDB Query
                • NetSuite Query
                • NetSuite SuiteAnalytics
                • New Relic
                • Notion
                • OData Query
                • Ongoing WMS
                • Orbit
                • Ortto
                • PagerDuty
                • PayPal
                • Pendo
                • Persona
                • Pingdom
                • Pipedrive
                • PivotalTracker
                • PostgreSQL Load
                • Productboard
                • RDS Query
                • Recurly
                • Rootly
                • S3 Load
                • Salesforce Load
                • Salesforce Pardot
                • Salesforce Query
                • Salesforce Marketing Cloud Query
                • Salesloft
                • SAP NetWeaver
                • SAP ODP
                • SAP SuccessFactors
                • SeatGeek
                • SendGrid
                • ServiceNow Query
                • SharePoint Query
                • Shopify
                • Slack
                • Smartsheet
                • Snapchat
                • Snyk
                • Square
                • SurveyMonkey
                • Stripe Query
                • Ticketmaster
                • TikTok
                • Toggl
                • Twilio
                • Unstructured.io
                • Volume to Delta Table
                • Workday
                • Workday Custom Reports
                • X Ads Load
                • X Ads
                • Yelp
                • Zendesk Talk
                • Zendesk Ticketing
                • Zoom
                • Zuora
                • Azure Blob Storage Unload
                • Google Cloud Storage Unload
                • Microsoft SQL Server Output
                • Oracle Unload from Snowflake
                • RDS Bulk Output
                • S3 Unload
                • Salesforce Output
                • Create an OAuth connection for a custom connector
                • Bing Ads Query authentication guide
                • Dynamics 365 Query authentication guide
                • Google Query authentication guide
                • HubSpot Query authentication guide
                • Jira Query authentication guide
                • Kafka authentication guide
                • Mailchimp authentication guide
                • Marketo Query authentication guide
                • NetSuite Query authentication guide
                • NetSuite SuiteAnalytics authentication guide
                • Salesforce Query authentication guide
                • ServiceNow Query Authentication Guide
                • Stripe Query authentication guide
                • Workday authentication guide
              • Transactions
              • Begin
              • Commit
              • Rollback
              • Add Partition
              • Alter Table
              • Analyze Tables
              • Append to Grid
              • Create External Table
              • Create Stream
              • Create Table
              • Delete Tables
              • JDBC Table Metadata To Grid
              • Optimize
              • Refresh External Table
              • Refresh Materialized View
              • Refresh Table
              • Remove from Grid
              • Schema Copy
              • SQL Script
              • Table Metadata To Grid
              • Truncate Table
                • Alter Warehouse
                • Create File Format
                • CloudWatch Publish
                • SNS Message
                • SQS Message
                • Azure Queue Storage Message
                • Cloud Pub/Sub
                • Send Email
                • Webhook Post
                • Assert External Table
                • Assert Scalar Variables
                • Assert Table
                • Print Variables
                • dbt Core
                • Bash Pushdown
                • Python Pushdown
                • Python Script
                • Query Result to Grid
                • Query Result to Scalar
                • Run Notebook
                • Update Scalar
                • Start Compute
                • Vacuum
                • Run Orchestration
                • Run Shared Pipeline
                • Run Transformation
                • And
                • End Failure
                • End Success
                • If
                • Or
                • Retry
                • Start
                • File Iterator
                • Fixed Iterator
                • Grid Iterator
                • Loop Iterator
                • Table Iterator
                • Stacking and detaching iterators
                  • Completions
                  • Embed
                  • Extract Answer
                  • Multi Prompt
                  • Parse Document (preview)
                  • Sentiment
                  • Summarize
                  • Translate
                  • Snowflake Vector Search
                  • Classification (preview)
                • Document AI Predict
                • AI Analyze Sentiment
                • AI Classify
                • AI Extract
                • AI Fix Grammar
                • AI Mask
                • AI Query
                • AI Similarity
                • AI Summarize
                • AI Translate
                • Databricks Vector Search
            • Aggregate
            • Assert View
            • Calculator
            • Construct Variant
            • Convert String to Struct
            • Convert Type
            • Create View
            • Detect Changes
            • Distinct
            • Except
            • Extract Nested Data
            • Extract Structured Data
            • Filter
            • First/Last
            • Fixed Flow
            • Flatten Variant
            • Generate Sequence
            • Intersect
            • Join
            • Lead/Lag
            • Map Values
            • Multi Table Input
            • Pivot
            • Rank
            • Rename
            • Rewrite Table
            • Split field
            • SQL
            • Stream Input
            • Table Delete Rows
            • Table Input
            • Table Output
            • Table Update
            • Transpose Columns
            • Transpose Rows
            • Unite
            • Unpivot
            • Window Calculation
          • Data quality framework
          • Facebook Ads Query example configurations
          • Incremental loading
          • Jira Query
          • S3 Load and transformation
          • System tables
        • Streaming pipelines
            • Db2 for IBM i streaming connector
            • Db2 for IBM i database configuration
            • Microsoft SQL Server streaming connector
            • Microsoft SQL Server database configuration
            • MySQL streaming connector
            • MySQL database configuration
            • MySQL streaming agent configuration
            • Oracle streaming connector
            • Oracle database configuration
            • PostgreSQL streaming connector
            • PostgreSQL database configuration
          • Snowflake
          • Amazon S3
          • Azure Blob Storage
          • Sync All Tables
        • Matillion Data Loader
        • Batch Loads Introduction
        • Getting started
        • Batch Pipeline UI
          • Schema drift
          • Batch load replication overview
          • Batch pipeline detailed view
            • Technical requirements
            • Cloud data warehouse permissions
            • Destinations
            • Passwords
            • Cloud credentials
            • OAuth connections
            • NetSuite batch pipeline authentication guide
          • Scheduling with Quartz cron expressions
          • Custom Connector Data Loader pipeline
          • Allowed connection properties
          • ActiveCampaign
          • Amazon Redshift
          • Amplitude
          • Brevo
          • Chargebee
          • Circle CI
          • Concord
          • Confluence
          • Datadog
          • DB2 for IBM i
          • Delighted
          • Dropbox
          • Dynamics 365 Sales
          • Elasticsearch
          • Eventbrite
          • Excel
          • Facebook AdAccounts
          • Facebook AdInsights
          • Facebook Content Insights
          • Freshdesk
          • Google Analytics
          • Google BigQuery
          • Google Custom Search
          • Google Sheets
          • HubSpot
          • IBM Db2
          • Jira
          • Klaviyo
          • LaunchDarkly
          • Mailchimp
          • MariaDB
          • Marketo
          • Microsoft SQL Server
          • MongoDB
          • MySQL
          • NetSuite
          • OData
          • Open Exchange Rates
          • Oracle
          • Oracle Eloqua
          • PagerDuty
          • Pendo
          • PostgreSQL
          • Productboard
          • QuickBooks
          • Recurly
          • Sage Intacct
          • Salesforce
          • Salesforce Pardot
          • Shopify
          • Slack
          • Smartsheet
          • Snapchat
          • Snowflake
          • Snyk
          • Stripe
          • SugarCRM
          • TikTok
          • Toggl
          • Twilio
          • Xero
          • Zendesk
          • Zendesk Talk
            • Set up Snowflake
            • Connect to Snowflake
            • Set up Delta Lake on Databricks
            • Connect to Delta Lake on Databricks
            • Set up Amazon Redshift
            • Connect to Amazon Redshift
            • Set up Google BigQuery
            • Connect to Google BigQuery
          • Tech note - failing pipelines
          • Destinations
        • Matillion Custom Connector Overview
        • Export your own connector to Matillion ETL
        • Create a pipeline with custom connectors
          • Custom connector setup
          • Endpoints
          • Authentication
          • Add OAuth
          • Pagination
          • Script paging
          • API parameters
          • Custom connector example
          • Flex connector setup
          • Flex connector tutorial
          • API overview
          • Authentication
          • API reference user guide
          • Data Productivity Cloud public API reference
          • Executing and managing a pipeline
          • Managing secret references
            • Understanding Artifacts
            • Creating artifacts with the API
            • Scheduling artifacts
          • Streaming pipelines
          • August
          • July
          • June
          • May
          • April
          • March
          • February
          • January
          • December
          • November
          • October
          • September
          • August
          • July
          • June
          • May
          • April
          • March
          • February
          • January
          • December
          • November
          • October
          • September
          • August
          • July
          • June
          • May
          • April
          • March
          • February
          • January
          • December
          • November
          • October
          • September
          • August
          • July
          • June
          • May
          • April
        • Tech note - Snowflake to block single-factor password authentication
      • Style guide overview
        • Formatting checklist
        • Formatting text
        • Capitalization
        • Headings
        • Horizontal rules
        • Images
        • Links
        • Lists
        • Tables
        • UI elements
        • Words
        • MarkUp.io
        • Release status
      • Matillion Change Data Capture
      • Getting started
      • CDC shared jobs
      • CDC Pipeline UI
        • CDC pipeline overview
        • CDC pipeline detailed view
        • Create and manage a CDC pipeline
        • Schema Drift
        • CDC agent installation
        • CDC agent setup UI
        • Sizing CDC agents
        • Agent versions
        • Agent update
        • Platform keys
        • Environment variables
        • Lock down outbound traffic
          • AWS installation overview
          • Deploying a CDC agent in AWS quick guide
            • Subnets and security groups
            • CloudWatch Logs
            • IAM Roles
            • S3 Buckets
            • Fargate and ECS
              • CloudFormation ECS Fargate basic template
              • CloudFormation ECS Fargate advanced template
              • CloudFormation EC2 ECS proxy templates
              • Agent (AWS) template
              • Terraform ECS Fargate basic template
              • Terraform ECS Fargate advanced template
          • Azure installation overview
          • Deploying a CDC agent in Azure quick guide
          • Deploying a CDC agent in Azure using the CLI
            • Managed identity
            • Blob Storage
            • Log analytics
            • Container instances
            • Azure - ARM template
            • Azure - Terraform template
            • Agent (Azure) template
          • GCP installation overview
          • Deploying a CDC agent in GCP quick guide
            • Project and Service Account
            • GCP Cloud Storage bucket
            • IAM permissions
            • Workload Identity (GKE)
            • Agent (GCE) template
            • Agent (GKE) Kubernetes template
          • AWS CloudWatch Logs
          • Azure Log Analytics
          • Google Cloud logging
        • Technical Requirements (CDC)
          • Amazon S3 lifecycle management
          • Azure Blob Storage lifecycle management
          • Google Cloud Storage lifecycle management
        • CDC sources overview
        • Source database maintenance
          • Db2 for IBM i connector
          • Configuring Db2 for IBM i
          • DB2 for IBM i snapshot
          • PostgreSQL connector
          • Configuring PostgreSQL database
          • PostgreSQL advanced settings
          • PostgreSQL snapshot
          • Oracle connector
          • Configuring Oracle Database
          • Oracle advanced settings
          • Oracle snapshot
          • Microsoft SQL Server connector
          • Configuring Microsoft SQL Server database
          • Microsoft SQL advanced settings
          • Microsoft SQL Server snapshot
          • MySQL connector
          • Configuring MySQL database
          • Configuring the CDC agent for MySQL
          • MySQL snapshot
        • Snowflake
        • Amazon S3
        • Azure Blob
        • Google Cloud Storage
        • Agent setup
        • Destinations
        • UI setup

    Cortex component snippets

    To learn more about Snowflake Cortex, such as availability, usage quotas, managing costs, and more, read Large Language Model (LLM) Functions (Snowflake Cortex).

    To use this component, you must use a Snowflake role that has been granted the SNOWFLAKE.CORTEX_USER database role. Read Required Privileges to learn more about granting this privilege.

    © Copyright 2025 Matillion Ltd | Legal | Privacy Policy | Disclaimer | Modern Slavery Statement