Amazon S3 lifecycle policy for file management

This page is a step-by-step guide to configuring an Amazon S3 lifecycle policy to delete objects tagged with matillion_cdc_processed = true from new shared jobs, helping to keep your storage organized and to manage the data lifecycle efficiently.

---

Select your bucket

1. Log in to the AWS Console.
2. Navigate to Amazon S3.
3. Choose the S3 bucket where you want to create the lifecycle policy. Ensure you have the necessary permissions.
4. In this bucket's dashboard, click the Management tab.
5. In the Lifecycle rules section, click Create lifecycle rule.
6. Complete the Create lifecycle rule form using the reference material below.

---

Create lifecycle rule

Lifecycle rule name = string

Provide a descriptive name for the lifecycle rule.

---

Choose a rule scope = checkbox

Select Limit the scope of this rule using one or more filters.

---

Prefix = string

Add a filter to limit the scope of this rule to a single prefix.

---

Object tags = key=value pair

Click Add tag.

1. Set the Key as matillion_cdc_processed.
2. Set the Value as true.

This object tagging process should be automated by the shared jobs.

---

Lifecycle action rules = checkbox

Check Expire current versions of objects to delete the current version of objects with the assigned object tag.

---

Days after object creation = integer

Select the number of days after creation to expire an object.

Finish by clicking Create rule.

---

Monitor and manage

Your new lifecycle policy will now be active for the S3 bucket. Objects tagged with matillion_cdc_processed = true will automatically be deleted after the specified number of days.

You can monitor and manage the lifecycle policy by going back to the lifecycle configuration to modify or delete rules as needed.