Implement Bucket Lifecycle

Bucket lifecycle management in object storage refers to automated rules that manage objects over time. These rules help control storage costs, enforce data retention policies, and maintain storage organization by automatically handling tasks such as deleting old objects, managing versions, and cleaning up incomplete uploads.

Lifecycle on Object Storage

Exoscale Simple Object Storage (SOS) supports native bucket lifecycle management for expiration of objects.

Bucket Lifecycle enables:

Cost Optimization: Automatically removing data that exceeds retention periods to reduce storage costs.
Data Governance: Enforcing compliance and retention requirements.
Storage Hygiene: Managing versions, removing incomplete uploads, and avoiding clutter in object storage.

Supported Features

Expiration of Objects: Objects created more than X days ago (user-defined) are deleted.
Retention of Non-current Versions: A specified number of non-current versions (versions that are not the most recent) are retained.
Removal of Delete Markers: when all object versions are deleted and only a delete marker remains, dangling delete markers are removed.
Abortion of Multipart Uploads: Incomplete multipart uploads that started before a specified number of days are aborted.

Usage

Configuration

Configuration shares the same format as AWS S3’s bucket lifecycle configuration, but only a subset of AWS’s lifecycle rules are supported (storage class transition is currently unsupported).

Lifecycle configuration is specified as a JSON document consisting of one or more rules.

{
    "Rules": [
        {
            "ID": "RULE001",
            "Status": "Enabled",
            "Filter":{
                "Prefix": "logs/"
            },
            "Expiration": {
                "Days": 10
            },
            "NoncurrentVersionExpiration": {
                "NewerNoncurrentVersions": 20
            },
            "AbortIncompleteMultipartUpload": {
                "DaysAfterInitiation": 7
            }
        },
        {
            "ID": "RULE002",
            "Status": "Enabled",
            "Expiration": {
                "ExpiredObjectDeleteMarker": true
            },
            "Filter": {}
        },
        {
            "ID": "RULE003",
            "Status": "Enabled",
            "Filter": {
                "And": {
                    "Prefix": "logs/",
                    "ObjectSizeGreaterThan": 1048576,
                    "ObjectSizeLessThan": 104857600
                }
            },
            "Expiration": {
                "Days": 30
            }
        }
    ]
}

Explanation

Rules: An array containing one or more of rules for bucket lifecycle management.
ID: A unique identifier for the rule, such as "RULE001".
Status: The state of the rule: Enabled or Disabled. Disabled rules have no effect
Expiration: Specifies the expiration settings for objects.
- Days: in RULE001, objects created 10 days ago or more are deleted; in RULE003 it’s 30 days
- ExpiredObjectDeleteMarker: when all previous versions of an object are deleted and only a delete marker remains, dangling delete markers are purged.
NoncurrentVersionExpiration: Defines how non-current versions are handled.
- NewerNoncurrentVersions: Retains the 20 most recent non-current versions.
- NoncurrentDays: non-current versions are deleted X days after they became non-current, that is after a newer version or a delete marker replaced them, whatever their creation date. Mutually exclusive with NewerNoncurrentVersions. In a version listing (ListObjectVersions), the LastModified date of a non-current version is its creation or modification date, not the date it became non-current.
AbortIncompleteMultipartUpload: Configures the abortion of incomplete multipart uploads.
- DaysAfterInitiation: Incomplete multipart uploads that started more than 7 days ago are aborted.
Filter: Every rule requires a Filter field, even if it is empty. Narrows the scope of a rule to a subset of objects. When combining multiple conditions, they should be wrapped in an And block.
- Prefix: Restricts the rule to objects whose key begins with the given string (e.g. "logs/").
- ObjectSizeGreaterThan: Restricts the rule to objects strictly larger than the given size in bytes.
- ObjectSizeLessThan: Restricts the rule to objects strictly smaller than the given size in bytes.

Setting Bucket Lifecycle Configuration

Any tooling that supports AWS S3 API for bucket lifecycle should be fully compatible.

$ aws s3api put-bucket-lifecycle-configuration --bucket yourbucket --lifecycle-configuration file://path/to/your/config.json

$ exo storage bucket lifecycle set sos://yourbucket path/to/your/config.json -z your-zone

Where:

path/to/your/config.json is the actual path to your JSON configuration file.
yourbucket is the name of the bucket on which to apply lifecycle rules.
your-zone is the Exoscale zone where the bucket is located.

Scheduling

Bucket Lifecycle evaluation runs every day at midnight UTC. There might be some delay between evaluating a bucket’s configuration and deleting the respective objects.

Lifecycle and Object Lock

A lifecycle rule never deletes an object version that is under retention (Governance or Compliance mode) or under a legal hold. The version is kept until its retention ends and the legal hold is removed, if any applies.

An expiration rule still adds a delete marker on top of a locked current version. The object then no longer appears in object listings and a GET request without a version ID returns 404 Not Found, but the locked version stays in the bucket and can be read with its version ID. See Object Lock.

Limitations

Exoscale SOS only supports bucket lifecycle rules for object deletion/expiration.
AWS S3 requires either a Filter or a Prefix field at the Rule level. Exoscale SOS only supports the Filter field, which is required to be present in all rules (but can be empty).

Last updated on October 29, 2026