S3 Manifest Builder

Build and upload a manifest file which describes a set of files (S3 objects) to load based upon a pattern using a regular expression.

If the pattern is simply a prefix, this is not required as the S3 Load can use a prefix directly. However, for more complicated file-naming schemes, use this component to build a manifest file first, and then use an S3 Load with the manifest option and a path pointing to the generated manifest file.

---

Properties

Name = string

A human-readable name for the component.

---

Source S3 Path = string

The S3 bucket (and optionally the path within it) to search.

---

Manifest Path = string

The S3 location to write the manifest file to.

---

Manifest File name = string

The file name for the manifest.

---

Filter Regular Expression = string

This is a regular expression. All names found from Source S3 Path are tested against this expression, and if they match they are included in the manifest.

This requires an exact match, so if you are searching for text anywhere within the name you will need to include wildcards before and after the text.

---

Mandatory = drop-down

Yes writes the Mandatory option into the manifest, meaning the subsequent S3 load will fail if the file is not present. No makes the file optional, and no error will be raised if the file is missing.

---

Variable exports

This component makes the following values available to export into variables:

Source	Description
Matched Objects	The number of objects matching the filter pattern and written to the Manifest.

---

Snowflake	Delta Lake on Databricks	Amazon Redshift	Google BigQuery	Azure Synapse Analytics
❌	❌	✅	❌	❌