Understanding artifacts
Artifacts are versioned, read-only packages that contain one or more pipelines, frozen at a specific point in time and ready for production deployment.
This page explains what artifacts are, how to create them using both the Designer UI and Matillion's public API, and how artifacts help ensure consistency and control in your data workflows.
What is an artifact?
An artifact is a versioned, read-only bundle of pipelines that can be executed in a production environment. An artifact includes any selected pipelines, along with any dependencies, into a single, snapshotted version that captures exactly what was present at the time of the build. Artifacts are:
- Immutable: Once built, an artifact never changes.
- Versioned: Each artifact has a unique name or tag (for example,
v2025.06.01-analytics
). - Portable: An artifact can be deployed across environments without relying on editable pipeline source code.
Why artifacts exist
Artifacts help you manage data pipelines in production environments. They provide:
- Reproducibility: Artifacts capture the exact state of pipelines at the time of build, shielding production environments from later edits or unfinished work.
- Isolation and auditability: Each artifact has a unique version tag, enabling teams to promote or roll back versions with confidence.
- Packaging and performance: Artifact creation flattens pipeline call chains, resolves dependencies, and compiles the result into a portable, serialized format optimized for execution.
- CI/CD friendliness: Artifacts can be integrated into automated workflows and stored in common artifact registries like Artifactory or Amazon S3.
How to create an artifact
This section explains how to create an artifact. You can do this either via the Data Productivity Cloud UI or through Matillion's public API in conjunction with your CI/CD pipelines.
UI method
Read Artifacts to learn how to create artifacts in the Data Productivity Cloud UI.
API method
In a CI/CD pipeline, artifact creation is more explicit and customizable:
- Checkout branch on staging runner.
- Your CI job (for example,
git clone && git checkout release/1.7
)
- Your CI job (for example,
- Upload each pipeline to the platform.
- CI job loops:
POST /pipelines/{id}/versions
- Uses the
uploadPipeline
API call.
- CI job loops:
- Instruct platform to build.
- CI job sends
POST /artifacts
with list of pipeline IDs. - Uses the
createArtifact
API call.
- CI job sends
- Artifact ID returned by the platform.
Note
When using the API, the upload happens first, because the server has no direct access to your local repo. You supply the materials explicitly, then ask for the build.
What constitutes an artifact?
A typical artifact contains:
artifact/
├── manifest.yaml # name, version, build‑time stamps, Git SHAs
├── pipelines/
│ ├── ingest.parquet # compiled representation (JSON / protobuf)
│ ├── enrich.parquet
│ └── master-flow.parquet
└── libs/ # optional UDF jars, wheels, or in‑house libs
Note
Everything under artifact/
is read‑only at runtime.
Artifact lifecycle
The lifecycle of an artifact typically follows these steps:
- Build: Create a versioned snapshot from pipelines.
- Store: Save the artifact to the platform's internal registry.
- Promote: Deploy the artifact to test, staging, or production environments.
- Rollback (if needed): Revert to a previous versioned artifact.
FAQ
Question | Answer |
---|---|
Can I edit an artifact? | No. Artifacts are immutable. To make changes, you must edit your pipelines and build a new artifact. |
Does production pick up pipeline edits automatically? | No. You must create and deploy a new artifact. |
Do I need one artifact per pipeline? | Typically, one artifact is built per release and includes all pipelines that should be deployed together. |
Can I deploy the same artifact to multiple environments? | Yes. Artifacts are designed to be portable across environments. |
How can I inspect what's in an artifact? | Check the artifact's manifest.yaml for version, pipeline list, and metadata. |
Where are artifacts stored? | In the platform's internal artifact registry (for example, in Amazon S3 under artifacts/ ). |
Does production automatically update when I edit a pipeline? | No. Only new artifacts trigger changes. |
How do I rollback? | You can deploy a previous artifact version, with no need to revert source code. |
Artifacts help you deliver reliable, reproducible data pipelines in production. Whether you use the Data Productivity Cloud UI or integrate with your CI/CD pipelines via API, artifacts ensure that what you design is exactly what runs.