Create File Format
Creates a named file format that can be used for bulk loading data into (and unloading data out of) Snowflake tables. This format can then be used in other components to simplify the component's options and use the custom file format.
Properties
Name = string
A human-readable name for the component.
Create/Replace = drop-down
Select one of these options:
- Create: The default option. Creates a new file format. This will generate an error if a file format with the same name already exists, but will never destroy existing data.
- Create if not exists: This will only create a new format if one of the same name does not already exist.
- Drop: Drop the file format of the given name. This is useful for deleting file formats. This will fail if a file format of the given name does not exist.
- Drop if exists: Drop the file format of the given name. This is useful for deleting file formats. This will succeed even if a file format of the given name does not exist.
- Replace: Creates a new format and replaces any format of the same name, overwriting it.
Database = drop-down
The Snowflake database that the newly created file format will be stored in. The special value, [Environment Default], will use the schema defined in the environment. Read Database, Schema, and Share DDL to learn more.
Schema = drop-down
The Snowflake schema that the newly created file format will be stored in. The special value, [Environment Default], will use the schema defined in the environment. Read Database, Schema, and Share DDL to learn more.
File Format Name = string
The name of the file format being created.
Compression = drop-down
Whether the input file is compressed in gzip format, BROTLI, BZ2, DEFLATE, RAW_DEFLATE, ZSTD, or not compressed at all.
Record Delimiter = string
The delimiter to be used that separates records (rows) in the file. Defaults to newline. \ can also signify a newline. \\r can signify a carriage return.
Field Delimiter = string
The delimiter to be used that separates fields (columns) in the file.
Skip Header = integer
The number of rows at the top of the file to ignore. Defaults to 0.
Skip Blank Lines = boolean
By default, blank lines encountered in data files produce an end-of-record error. Set this property to True to skip blank lines without producing an error.
Date Format = string
Defaults to auto. This can be used to manually specify a date format.
Time Format = string
Defaults to auto. This can be used to manually specify a time format.
Timestamp Format = string
Defaults to auto. This can be used to manually specify a time format.
Escape = string
Single character string used as the escape character for any field values. Accepts common escape sequences, octal values, or hex values. Also accepts a value of NONE (default).
Escape Unenclosed Field = string
Single character string used as the escape character for unenclosed field values. Accepts common escape sequences, octal values, or hex values. Also accepts a value of NONE (default). If a character is specified in the Escape field, it will override this field.
Trim Space = boolean
If True, removes trailing and leading whitespace from the input data.
Field Optionally Enclosed = string
A character that is used to enclose strings. Can be single quote (') or double quote (") or NONE. The default is NONE. Note that the character chosen can be escaped by that same character.
Null If = listbox
Enter one or more strings that will be replaced with NULL if found in the source. Only applies to columns that are nullable.
Error On Column Count Mismatch = boolean
If True, generate an error if the number of delimited columns in the input does not match that of the table. If False, extra columns are not loaded into the table and missing columns are recorded as NULL in the table.
Empty Field as Null = boolean
If True, cast empty fields as NULL. If False, cast empty fields as the corresponding column type with blank data.
Replace Invalid Characters = boolean
If True, will replace invalid UTF-8 characters with the Unicode replacement character.
If False (the default), the load operation produces an error when invalid UTF-8 character encoding is detected.
Encoding Type = drop-down
Select the character set of the source data when loading data into a table.
| Snowflake | Databricks | Amazon Redshift |
|---|---|---|
| ✅ | ❌ | ❌ |