RDS Bulk Output
RDS Bulk Output lets users load the contents of a table (or view) into a table in an Amazon RDS database.
If you're using a Matillion Fully-Managed solution, you may need to allow-list these IP address ranges from which Matillion Fully-Managed agents will call out to their source systems or to cloud data platforms.
Currently, you may need to manually enter the endpoint.
Name = string
A human-readable name for the component.
RDS Type = drop-down
Select the database type.
RDS Endpoint = drop-down
Your RDS database endpoint. If the IAM role attached to the instance (or the manually entered credentials associated with the current environment) has the permissions granted to query the RDS endpoints, you may select the RDS endpoint from a list. Otherwise, you must enter it manually—it can be found in the RDS console and is a long dotted-name and port number, separated by a colon. To acquire your database endpoint and provide it manually, follow these steps:
- Log in to the AWS Console.
- In the Find Services search bar, search for RDS.
- In the Amazon RDS navigation column on the left side of your screen, click Databases.
- Select a database.
- Locate the endpoint for that database in the Connectivity & security section.
Users must include the port number when manually typing the endpoint.
Database Name = string
Provide the name of the database within your RDS instance. In the AWS Console, this is the "DB identifier".
Username = string
The RDS connection username.
Password = drop-down
The secret definition denoting your password tied to your username for your RDS database. Your password should be saved as a secret definition before using this component. Only available when users select User/Password in the Authentication Method property.
JDBC Options = columns editor
- Parameter: A JDBC parameter supported by the database driver. The available parameters are explained in the data model. Manual setup is not usually required, since sensible defaults are assumed.
- Value: A value for the given parameter.
Database = drop-down
The Snowflake database. The special value, [Environment Default], will use the database defined in the environment. Read Databases, Tables and Views - Overview to learn more.
Schema = drop-down
The Snowflake schema. The special value, [Environment Default], will use the schema defined in the environment. Read Database, Schema, and Share DDL to learn more.
Source Table = drop-down
The table (or view) on your cluster to copy to RDS.
Target Table = string
A name for the new table.
Load Columns = dual listbox
Choose the columns to load into RDS. If you leave this parameter empty, all columns will be loaded.
Table Maintenance = drop-down
- None: Assume the RDS database already has the table defined with the correct structure.
- Create if not exists: Only create the table if it doesn't already exist.
- Replace: Always drop and recreate the table.
Primary Key = dual listbox
Select one or more columns to be designated as the table's primary key.
Truncate Target Table = yes/no
Whether or not to truncate the target table before loading data.
On Warnings = drop-down
Choose whether to continue with the load if an error is raised, or to fail the run.
Additional Copy Options = column editor
Any additional options that you want to apply to the copy operation. Some of these may conflict with the options the component already sets—in particular, care is taken to escape the data to ensure it loads into the target database even if the data contains row and/or column delimiters, so you should never override the escape or delimiter options.
Batch Size = integer
This is optional, and specifies the number of rows to load to the target between each commit. On a very large export, this may be desirable to keep the size of the Amazon RDS log files from growing very large before the data is committed.