# Exoscale Documentation — Object Storage > Simple Object Storage (SOS): highly durable, S3-compatible object storage accessed over HTTP(S) with a wide range of tools, libraries and languages. --- url: https://community.exoscale.com/product/storage/object-storage/ --- # Object Storage - [Quick Start - Object Storage](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) - Get started quickly with a guided setup to achieve your first successful use of the service, including the required steps and configuration. - [Overview - Object Storage](https://community.exoscale.com/product/storage/object-storage/overview/index.md) - Terminology and key features at a glance, learn the core concepts, capabilities, and options of this service before you dive deeper. - [How-To - Object Storage](https://community.exoscale.com/product/storage/object-storage/how-to/index.md) - Step-by-step guides to configure the service and use specific features, with clear instructions for common tasks and problem solving. - [Service Boundaries - Object Storage](https://community.exoscale.com/product/storage/object-storage/service-boundaries/index.md) - Service boundaries: quotas, limits, and the guaranteed service levels (SLA) for this product, including key constraints to plan and operate reliably. - [Reference - Object Storage](https://community.exoscale.com/product/storage/object-storage/reference/index.md) - Reference documentation for API, CLI, and tools, browse comprehensive details, parameters, commands, and integration notes. - [Recipes - Object Storage](https://community.exoscale.com/product/storage/object-storage/recipes/index.md) - Informational, concise guides showing how to integrate with external tools and solve specific scenarios. --- url: https://community.exoscale.com/product/storage/object-storage/quick-start/ --- # Quick Start - Object Storage This guide demonstrates the basics of Exoscale's Simple Object Storage (SOS). With [object storage](https://en.wikipedia.org/wiki/Object_storage), you can store, distribute and manage large quantities of files (or objects). Example uses range from simple backups to complex media delivery. You can store almost anything in SOS: assets, [backups](https://www.exoscale.com/backup/), email attachments, videos, pictures and anything else you may imagine. Your data will stay in the zone in which you store it. Exoscale will never move data on your behalf. For example, if you first upload data to Simple Object Storage in Geneva, then your data is kept in Switzerland, under [Swiss privacy regulations](https://www.fedlex.admin.ch/eli/cc/2022/491/en). Whichever the initial location, it is replicated in that same location in at least three physical distinct copies to ensure maximum safety. All the actions we will demonstrate through the command line can also be performed through our CLI or the Portal, although the real power of SOS is in the tools you script, and in the clients you may use in your application. ## S3 Compatibility SOS is S3 compatible, so it works with most object storage clients and provides library access from a host of languages. You can drop in from other services and you will not be locked in. Most of the time, a change of API credentials and endpoint allows you to switch to our service. Here are a few clients and tools that you can use: * [Exoscale CLI](https://community.exoscale.com/tools/command-line-interface/index.md) * [s3cmd](https://s3tools.org/s3cmd) * [cyberduck](https://cyberduck.io/) * [owncloud](https://owncloud.com/) * [boto](https://docs.pythonboto.org/en/latest/) * [Minio](https://min.io/) We will use `s3cmd` in this guide. ## Setup This guide assumes you never used s3cmd. If you have used s3md, take a look at the `.s3cfg` example below, drop in your Exoscale's API Key and API Secret and you can use SOS. First, install s3cmd with your favorite package manager: * Debian/Ubuntu Linux: `apt-get update && apt-get install s3cmd` * macOS: `brew install s3cmd` Then create a file named `.s3cfg` in your home directory. Copy and paste the following configuration template: ```bash [default] host_base = sos-{zone}.exo.io host_bucket = %(bucket)s.sos-{zone}.exo.io access_key = $EXO_SOS_KEY secret_key = $EXO_SOS_SECRET use_https = True ``` Replace `{zone}` with the zone your bucket belongs to. For example, for a bucket deployed on the `ch-dk-2` zone: ```bash host_base = sos-ch-dk-2.exo.io host_bucket = %(bucket)s.sos-ch-dk-2.exo.io ``` Replace `$EXO_SOS_KEY` and `$EXO_SOS_SECRET` with your API Key and API Secret. You can choose to use your [User Keys](https://community.exoscale.com/product/security/iam/quick-start/index.md#user-keys) or a specific [IAM API key](https://community.exoscale.com/product/security/iam/quick-start/index.md#api-keys). ## Your first Bucket We need to create a container for all of our objects, which is called a bucket. ```bash s3cmd mb s3://my-new-bucket ``` This creates a bucket called my-new-bucket. Bucket names are **unique across the entire platform.** You may encounter an error creating `my-new-bucket` if the name is already taken. Why are bucket names unique? Buckets and their content can be accessed under `https://sos-ch-dk-2.exo.io/my-new-bucket/` or `https://my-new-bucket.sos-ch-dk-2.exo.io`. Buckets become subdomains and prefixed URLs under the `sos-ch-dk-2.exo.io` domain, so names have to be unique. This does not mean objects are publicly accessible, however. You can define precise access control through [IAM Keys](https://community.exoscale.com/product/security/iam/quick-start/index.md#user-keys) or [ACLs](https://community.exoscale.com/product/storage/object-storage/how-to/acl/index.md), and uploaded objects are private by default. You can check your bucket has been created by listing your buckets: ```bash s3cmd ls ``` ## Uploading and downloading assets Now choose file on your computer and upload it to your bucket: ```bash s3cmd put picture.jpg s3://my-new-bucket/picture.jpg ``` Check your file was uploaded successfully: ```bash s3cmd ls s3://my-new-bucket ``` You can also verify the presence of your file in the Portal. Select the `Storage` section from the main navigation on the left. You should see your `my-new-bucket` listed here. Select the bucket and you will find your file. You can try to upload more files in the Portal by drag-and-drop. To get your files: ```bash s3cmd get s3://my-new-bucket/picture.jpg picture_download.jpg ``` ## Getting further Get further with SOS and the S3 compatibility coverage by going over the [overview section](https://community.exoscale.com/product/storage/object-storage/overview/index.md) --- url: https://community.exoscale.com/product/storage/object-storage/overview/ --- # Overview - Object Storage Exoscale's Simple Object Storage (SOS) is a highly durable, S3-compatible service that offers seamless data accessibility across various locations over the HTTP(s) protocol, supporting a diverse range of tools, libraries, and programming languages. ## Terminology In the realm of object storage, particularly when discussing S3-compatible storage, there are several key terms to understand: Bucket : A container for storing objects. Each bucket can store unlimited objects. Object : The fundamental entities stored . Objects consist of data, metadata, and a unique identifier within a bucket. Key : The unique identifier for an object within a bucket. It's essentially the object's name, and each object in a bucket can be uniquely addressed using the combination of the bucket name and the key. Metadata : A set of name-value pairs that describe or configure the object. It can include standard HTTP headers like Content-Type an Content-Length and custom metadata defined by the user. Access Control : Mechanisms to manage permissions for buckets and objects. This includes policies at both the bucket and object level, which defin who can access the data and what they can do with it. Storage Class : Different classes define how the data is stored and charged, each offering different levels of availability, durability, and cost. ## Features The Simple Object Storage service, compatible with S3, offers high data durability, is accessible in multiple locations, and is supported across a variety of tools, libraries, and languages. Use your own S3 compatible tool : Easily integrate our S3 compatible Object Storage into any existing app or workflow, using your existing clients and libraries with no lock-in. A Cloud native Object Storage solution : Object Storage is important when designing architecture for cloud applications for scale. Get better scalability, reliability, and speed than just storing files on the filesystem. For any kind of data, pay only what you use : Use Object Storage for static assets, user uploads or backups. We offer a performant solution with a fair pay-per-use pricing and no hidden costs. Object Storage features for Cloud Native implementations are: S3 compatible API : Easily integrate our Object Storage solution in your applications using S3 compatible libraries. You can also connect with a GUI or CLI tool. Simply set a new endpoint and credentials. H/A Object Storage replication : We replicate data stored in our Object Storage on three different high-available nodes, in order to keep your important data safe. Our Object Storage is covered by a [99.95% availability SLA](https://community.exoscale.com/product/storage/object-storage/service-boundaries/sla/index.md). Bucket Replication : Automatically replicate objects to one or multiple buckets across zones for disaster recovery, backup, or multi-region synchronization. Direct HTTP/s access : Serve files directly from low latency, high bandwidth object storage via HTTP or HTTPS. Keep your own servers free to do more important things. ACL and CORS support : Set per bucket policies to manage permissions and control access. Or configure cross origin headers to use object storage from browser based applications and websites. Metadata support : Store additional information about your objects in key-value pairs and easily access this metadata through HTTP headers in every Object Storage response. Keep your data in your country : All data and replicas of our Object Storage service are stored in the country of your chosen zone. Object Storage is also available in every zone across Europe. Supported Backup Solutions : Various backup solutions are supported to let you do backups as you need - whether it's services with a simple, fast setup, to solutions with enterprise-grade features. Read more Object lock and versioning : Store multiple versions of your object and configure locks for long term retention and archiving of data. Meet the most stringent data compliance policies with the Legal Hold feature. Read more ## Advanced Features * [ACL for Buckets and Objects](https://community.exoscale.com/product/storage/object-storage/how-to/acl/index.md) * [CDN Caching for Buckets](https://community.exoscale.com/product/networking/cdn/quick-start/index.md) * [CORS for Buckets](https://community.exoscale.com/product/storage/object-storage/how-to/cors/index.md) * [Metadata for Objects](https://community.exoscale.com/product/storage/object-storage/how-to/metadata/index.md) * [Bucket Versioning and Object Lock](https://community.exoscale.com/product/storage/object-storage/how-to/versioning/index.md) * [Server Side Encryption](https://community.exoscale.com/product/storage/object-storage/how-to/encryption/index.md) * [Data Integrity Checksums](https://community.exoscale.com/product/storage/object-storage/how-to/checksum/index.md) * [Bucket Replication](https://community.exoscale.com/product/storage/object-storage/how-to/replication/index.md) * [Archival Storage](https://community.exoscale.com/product/storage/object-storage/how-to/archival-storage/index.md) These advanced features are accessible via the API, with ACL, CORS, and Metadata configurations also manageable through the Portal: Identity and Access Management (IAM) : Control programmatic access by creating specific [IAM API Keys](https://community.exoscale.com/product/security/iam/quick-start/index.md#generating-api-access-keys). These keys can be restricted to a single bucket, ensuring precise access management. Access Control Lists (ACLs) : Define precise permissions for buckets and individual objects with ACLs, allowing for granular access policies. Manage access rights such as read, write, and full control without inheritance from parent entities. For detailed information, refer to the [SOS ACL Documentation](https://community.exoscale.com/product/storage/object-storage/how-to/acl/index.md). Cross-Origin Resource Sharing (CORS) : Enable browser-based applications to interact seamlessly with objects in a bucket via CORS, implemented at the bucket level for uniform configuration. More details are available in the [SOS CORS Documentation](https://community.exoscale.com/product/storage/object-storage/how-to/cors/index.md). Metadata : Tag each object in a bucket with key-value pairs, presented through HTTP headers in SOS requests and responses. Comprehensive information and examples can be found in the [SOS Metadata Documentation](https://community.exoscale.com/product/storage/object-storage/how-to/metadata/index.md). Encryption : All new buckets have server-side encryption (SSE-SOS) enabled by default. You can also use client-side encryption or Server-Side Encryption with Customer Keys (SSE-C). For more details, see the [Encryption Documentation](https://community.exoscale.com/product/storage/object-storage/how-to/encryption/index.md). Data Integrity : Verify data integrity using checksums (CRC32, CRC32C, CRC64NVME) during uploads and downloads. See the [Data Integrity Documentation](https://community.exoscale.com/product/storage/object-storage/how-to/checksum/index.md). --- url: https://community.exoscale.com/product/storage/object-storage/how-to/ --- # How-To - Object Storage Step-by-step guides to configure the service and use specific features, with clear instructions for common tasks and problem solving. - [Use ACL](https://community.exoscale.com/product/storage/object-storage/how-to/acl/index.md) - Manage bucket and object access with ACLs in the Portal or s3cmd, including adding, viewing, and removing permissions. - [Use Bucket Policy](https://community.exoscale.com/product/storage/object-storage/how-to/bucketpolicy/index.md) - Secure Exoscale SOS with JSON bucket policies for IP filtering, request conditions, and S3-compatible policy management. - [Configure Replication](https://community.exoscale.com/product/storage/object-storage/how-to/replication/index.md) - Configure SOS bucket replication for leader-follower or bi-directional sync with CLI, Portal, or Terraform. - [Implement Bucket Lifecycle](https://community.exoscale.com/product/storage/object-storage/how-to/bucketlifecycle/index.md) - Emulate SOS bucket lifecycle with Exoscale’s Docker tool: expire objects, retain noncurrent versions, purge delete markers, and abort multipart uploads. - [Enable Archival Storage](https://community.exoscale.com/product/storage/object-storage/how-to/archival-storage/index.md) - Store cold data on your existing buckets at a reduced rate, check whether it pays off for a given bucket, request activation, and understand how the discount appears on the invoice. - [Enable Encryption](https://community.exoscale.com/product/storage/object-storage/how-to/encryption/index.md) - Secure SOS data in transit with TLS and at rest with SSE-SOS by default or SSE-C for customer-managed encryption keys. - [Manage Metadata](https://community.exoscale.com/product/storage/object-storage/how-to/metadata/index.md) - Manage SOS object metadata key-value pairs via HTTP headers in the Portal or with s3cmd to add, view, and delete metadata. - [Enable Versioning](https://community.exoscale.com/product/storage/object-storage/how-to/versioning/index.md) - Enable SOS bucket versioning and object lock to prevent deletes, restore prior versions, and enforce immutable retention. - [Use Data Integrity Checksums](https://community.exoscale.com/product/storage/object-storage/how-to/checksum/index.md) - Verify SOS object integrity with CRC32, CRC32C, and CRC64NVME checksums on upload and retrieval with S3-compatible commands. - [Manage Terraform](https://community.exoscale.com/product/storage/object-storage/how-to/terraform/index.md) - Manage Exoscale SOS with Terraform AWS provider: configure S3 endpoints, then create buckets, ACLs, versioning, and objects. - [Use Conditional Requests](https://community.exoscale.com/product/storage/object-storage/how-to/conditionalrequests/index.md) - Use SOS conditional requests with ETag and timestamp checks to control object uploads, downloads, copies, and deletes safely. - [Configure CORS](https://community.exoscale.com/product/storage/object-storage/how-to/cors/index.md) - Configure SOS bucket CORS rules to allow browser access from specific origins, methods, and headers via Portal or S3 API. - [Use AWS CLI](https://community.exoscale.com/product/storage/object-storage/how-to/awscli/index.md) - Configure AWS CLI for Exoscale SOS, create an IAM key, and manage buckets with s3 and s3api commands. - [Use Mountpoint](https://community.exoscale.com/product/storage/object-storage/how-to/mountpoint/index.md) - Mount Exoscale SOS buckets as a filesystem with AWS Mountpoint S3 for read-heavy workflows, IAM setup, and zone endpoint access. - [Enable Data Events](https://community.exoscale.com/product/storage/object-storage/how-to/data-events/index.md) - Enable SOS Data Events for Audit-Trail to receive signed webhook logs of accepted and rejected bucket API requests. --- url: https://community.exoscale.com/product/storage/object-storage/how-to/acl/ --- # Use ACL With __Access Control Lists__ (ACLs), you can define specific permissions at the bucket level and single object level for a granular access policy. ACLs are not inherited from parent objects. ACLs let you manage access to buckets and objects with read, write, read ACP, write ACP, and full control (read + write) rights. You can make your objects accessible by their public URL, your buckets writable by your team, and so on. When you add an object (bucket, file, folder), you are the owner will always have `FULL_CONTROL` of the object. > [!NOTE] > If you need to restrict API access to a certain bucket, such as issuing an API key allowed to only > list or write objects on a single bucket, see [IAM](https://community.exoscale.com/product/security/iam/quick-start/index.md). ## Add an ACL ### From the Exoscale Portal Select the `Storage` section from the main navigation on the left to display the list of your buckets. click on the `Edit` icon for your object. You are presented with the ACLs for your object. You have two ways of adding ACLs: * Using the Quick ACL menu: these are presets for commonly used ACLs, known as "canned ACLs". For example, setting a file available for Public Read; [see the list below](#canned-acls). * Editing ACLs manually: for each permission you can enter an Organization ID (can be found in IAM section). Note: editing the ACLs manually will remove the Quick ACL previously set. ### From a command line cool We assume for this example that you are using [s3cmd](https://s3tools.org/s3cmd). The `setacl` command modifies the ACLs. You can then use the following options: * `--acl-public` * `--acl-private` * `--acl-grant=PERMISSION:ORGANIZATION_ID` or `USER_CANONICAL_ID` ```bash s3cmd setacl --acl-public s3:/// ``` Output: ```bash s3:///: ACL set to Public [1 of 1] ``` ## View ACLs ### From the Portal To view the ACL associated with an object, need the object: * Select the `Storage` section from the main navigation * Navigate to your object * click on the `Edit` icon for the object. ### From command line tool We assume you are using [s3cmd](https://s3tools.org/s3cmd). The `info` command will return information on an object, including its ACLs. ```bash s3cmd info s3://// ``` Output: ```bash s3://// (object): File size: 0 Last mod: Mon, 02 Nov 2015 13:37:00 UTC MIME type: application/binary MD5 sum: d41d8cd98f00b204e9800998ecf8427e SSE: none policy: none cors: ACL: owner@domain.tld: FULL_CONTROL ACL: *anon*: READ ``` Here the object is readable by anyone, and the owner has FULL_CONTROL. ## Delete an ACL ### From the Portal To delete the ACL associated with an Object, edit the object: * Select the `Storage` section from the main navigation * Navigate to the object * click on the `Edit` icon of the object. Then click on the `Quick ACL` menu and choose `Private`. This will revert to the default settings. If you need to delete a specific entry, you have to use the `Manual Edit` button which lets you specify all fields. ## Canned ACLs ### For buckets * private * public-read * public-read-write * authenticated-read ### For files/folders * private * public-read * public-read-write * authenticated-read * bucket-owner-read * bucket-owner-full-control --- url: https://community.exoscale.com/product/storage/object-storage/how-to/bucketpolicy/ --- # Use Bucket Policy Exoscale's Simple Object Storage (SOS) supports bucket policies, enabling fine-grained access control over your storage resources. This feature aligns with our commitment to offer enhanced security and flexibility for managing access to your data. ## What's New With the introduction of bucket policies, you can define detailed access permissions directly at the bucket level using JSON-based policy documents. These policies provide control over who can access your buckets and objects, ensuring your data is secure and accessible only to the right users. Unlike the current use of Org Policy and IAM policies with IAM keys they are also enforced for unauthenticated users. ## Bucket Policies Bucket policies in Exoscale SOS allow you to specify permissions using our existing Exoscale policy format with Common Expression Language [(CEL)](https://community.exoscale.com/product/security/iam/how-to/policy-guide/index.md#cel-bindings) expressions. These policies can be managed via the SOS API, compatible with common AWS S3 API tools. > [!WARNING] > Please note that in order to allow access to the bucket without authentication, we have to set > the bucket/object to 'all users' with read-only. ### Example Policies _Minimal IP filter policy:_ ```json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "source_ip.inIpRange('89.145.160.0/24')" }, { "action": "allow", "expression": "source_ip.inIpRange('2a04:c46::/32')" } ] } } } ``` The `default-service-strategy` is `deny` which means if no rule matches we deny the operation on the bucket / object in the bucket. Then we specify a rule for the `"sos"` service: if the source IP of the request comes from IPv4 subnet `89.145.160.0/24` or IPv6 subnet `2a04:c46::/32` we allow the operation (tip: if your network interface has an IPv4 and IPv6 address assigned you should specify both). _IP filter only for unauthenticated requests:_ ```json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "identity.key != ''" }, { "action": "allow", "expression": "source_ip.inIpRange('89.145.160.0/24')" }, { "action": "allow", "expression": "source_ip.inIpRange('2a04:c46::/32')" } ] } } } ``` _Only accept object creation via conditional if-none-match: '*":_ This policy is useful to eg: prevent accidental overwrites of existing objects. ```json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "!parameters.object_creation" }, { "action": "allow", "expression": "headers['if-none-match'] == '*'" } ] } } } ``` _Only accept object creation if the client is using SSE-C encryption:_ This can be useful to enforce encryption on sensitive data when using customer-managed keys. Note that for most use cases, [SSE-SOS](https://community.exoscale.com/product/storage/object-storage/how-to/encryption/index.md#sse-sos) (enabled by default on all new buckets) is the recommended encryption method. ```json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "!parameters.object_creation" }, { "action": "allow", "expression": "headers['sse-c-customer-algorithm'] == 'AES256'" } ] } } } ``` For more details on writing policies, refer to the [Exoscale IAM documentation](https://community.exoscale.com/product/security/iam/overview/index.md). ## Testing a bucket policy Before applying the bucket policy to a bucket you can test it on a specific IAM key. To do this you can create a role with the bucket policy you want to test: ![](iam-role.png "Portal create new Role with bucket policy") In the next step you create an IAM api key that uses that role. ![](iam-key1.png "Portal new IAM key") ![](iam-key2.png "Portal new IAM key2") Now you can test the bucket policy with the newly created key. On requests with a different key the bucket policy doesn't have an influence. > [!NOTE] > When you use the example policy it will apply to all buckets since it does not filter for the bucket name. In the next section we will install the policy on a bucket which will have an effect on all requests with or without an IAM api key! ## Policy Management You can use the following S3 API-compatible commands to manage your bucket policies: First let's create an alias so we don't have to repeat the same options to use aws cli (I assume we use a bucket in our Zurich zone `ch-dk-2`): ```bash alias awsexo='aws --profile=exo --endpoint=https://sos-ch-dk-2.exo.io --region=ch-dk-2' ``` ### Get Bucket Policy ```bash awsexo s3api get-bucket-policy --bucket my-bucket ``` If no bucket policy is configured for the bucket an `NoSuchBucketPolicy` error is returned. ### Put Bucket Policy ```bash awsexo s3api put-bucket-policy --bucket my-bucket --policy file://policy.json ``` ### Delete Bucket Policy ```bash awsexo s3api delete-bucket-policy --bucket my-bucket ``` > [!NOTE] > It's possible that you specify a policy that makes it impossible for you to access the bucket or delete > the bucket policy (e.g. wrong IP in the filter). In this case contact our support via the portal to unlock your bucket. --- url: https://community.exoscale.com/product/storage/object-storage/how-to/replication/ --- # Configure Replication Exoscale's Simple Object Storage (SOS) supports live bucket replication, allowing an automatic asynchronous copy of objects between one bucket, to one or multiple buckets across the same or a different zone. This enables the creation of copies of buckets, in the same or a different zone, covering needs for disaster recovery, multi-zone storage synchronization, etc. ## Requirements In order to enable bucket replication, both the source and destination(s) buckets must be within the same Exoscale organization, and both of them need to have object versioning enabled. If the source bucket has Object Locking enabled, the destination one must also have Object Locking enabled to allow a correct replication. ## Configuration Structure The REST API of Exoscale's SOS allows you to configure the bucket replication configuration using XML. The Exoscale Portal or the Exoscale CLI offer a convenience mechanism to define it in the JSON format. A replication configuration is composed of the unique identifier (uuid) of an Exoscale IAM role, and one or more rules. The replication will assume this IAM role to perform operations on your behalf, on both the source and destination buckets. For example: ```xml role-uuid ... ... ... ``` The IAM role must have the following permissions: - `get-object` on the source bucket - `put-object`, `put-object-*` on the destination bucket ## Rule configuration Each rule will be configured with the following format: - `ID`: a unique name - `Priority`: integer used to identifify the precedence between conflicting rules with the same destination bucket. The highest priority will win. - `Status`: `Enabled` or `Disabled`, to disable or enable a given rule while keeping the configuration - `Filter`: section to filter the elements to be replicated - `Prefix`: ensures the rule applies only to objects matching the configured prefix - `DeleteMarkerReplication`: section to configure replication of delete markers - `Status`: `Enabled` or `Disabled`: to disable or enable the replication of delete markers - `Destination`: section to configure the destination of the rule - `Bucket`: Target bucket name You can use the following minimal configuration as an example: ```xml My-Rule-0 0 Enabled Enabled my-exo-destination-bucket ``` ## Examples ### Simple Leader-Follower Setup #### Using the CLI In this example, we will do a simple configuration with one leader bucket, replicating into the follower bucket. Writes in the leader will be visible inside of the follower. 1. Using the following IAM configuration, create an IAM role for the replication: ```json # policy.json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "parameters.bucket == 'my-source' && operation == 'get-object'" }, { "action": "allow", "expression": "parameters.bucket == 'my-destination' && (operation.startsWith('put-object') || operation.startsWith('delete-object') || operation.startsWith('abort-multipart-upload'))" } ] } } } ``` ```bash cat policy.json | exo iam role create --editable=true --description 'sos bucket replication between my-source and my-destination' --policy - replication-my-source-my-destination ``` Output: ```bash ┼─────────────┼─────────────────────────────────────────────────────────────┼ │ ID │ 6f00a21e-a535-4286-86c5-e81eeab3b4ff │ │ Name │ replication-my-source-my-destination │ │ Description │ sos bucket replication between my-source and my-destination │ │ Editable │ true │ │ Labels │ n/a │ │ Permissions │ n/a │ ┼─────────────┼─────────────────────────────────────────────────────────────┼ ``` 2. Create the source and destination buckets, with the correct configuration ```bash exo storage create --zone ch-dk-2 my-source ``` Output: ```bash ┼──────────────────┼────────────────────────────────────────────┼ │ STORAGE │ │ ┼──────────────────┼────────────────────────────────────────────┼ │ Name │ my-source │ │ Zone │ ch-dk-2 │ │ ACL │ │ │ │ Read - │ │ │ Write - │ │ │ Read ACP - │ │ │ Write ACP - │ │ │ Full Control xxxxxxxxxxxxxxxxxxxxxxx │ │ │ │ │ CORS │ │ │ Object Ownership │ BucketOwnerEnforced │ ┼──────────────────┼────────────────────────────────────────────┼ ``` ```bash exo storage create --zone at-vie-1 my-destination ``` Output: ```bash ┼──────────────────┼────────────────────────────────────────────┼ │ STORAGE │ │ ┼──────────────────┼────────────────────────────────────────────┼ │ Name │ my-destination │ │ Zone │ at-vie-1 │ │ ACL │ │ │ │ Read - │ │ │ Write - │ │ │ Read ACP - │ │ │ Write ACP - │ │ │ Full Control xxxxxxxxxxxxxxxxxxxxxxx │ │ │ │ │ CORS │ │ │ Object Ownership │ BucketOwnerEnforced │ ┼──────────────────┼────────────────────────────────────────────┼ ``` ```bash exo storage bucket versioning enable --zone ch-dk-2 my-source exo storage bucket versioning enable --zone at-vie-1 my-destination ``` 3. Using the following replication configuration, enable the replication between the two buckets. ```json # config.json { "Role": "6f00a21e-a535-4286-86c5-e81eeab3b4ff", "Rules": [{ "ID": "vie1-follower", "Priority": 1, "Filter": { "Prefix": "" }, "Status": "Enabled", "DeleteMarkerReplication": { "Status": "Enabled" }, "Destination": { "Bucket": "my-destination" } }] } ``` ```bash exo storage bucket replication set --zone ch-dk-2 sos://my-source ./config.json ``` 4. Confirm it works as expected: ```bash exo storage upload -r ./ sos://my-source/ ``` Output: ```bash config.json [===========================================================================] 372.00 b / 372.00 b | 0s policy.json [===========================================================================] 418.00 b / 418.00 b | 0s ``` Output: ```bash exo storage list sos://my-source/ ``` Output: ```bash 2025-02-20 15:50:12 UTC 372 B config.json 2025-02-20 15:50:13 UTC 418 B policy.json ``` Output: ```bash exo storage list sos://my-destination/ ``` Output: ```bash 2025-02-20 15:50:12 UTC 372 B config.json 2025-02-20 15:50:13 UTC 418 B policy.json ``` You can also check the Replication Status of each object individually: ```bash exo storage show sos://my-source/config.json ``` Output: ```bash [...] │ Path │ config.json │ │ Bucket │ my-source │ │ Replication Status │ COMPLETED │ ``` ```bash exo storage show sos://my-destination/config.json ``` Output: ```bash [...] │ Path │ config.json │ │ Bucket │ my-destination │ │ Replication Status │ REPLICA │ ``` #### Using the Portal 1. In `IAM` -> `Roles` -> `Add`, create a role with the following configuration: 1. `name`: `replication-my-source-my-destination` 2. `description`: `sos bucket replication between my-source and my-destination` 3. `Editable Policy`: true 4. `Policy`: Using the `Advanced mode`, submit the following configuration: ```json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "parameters.bucket == 'my-source' && operation == 'get-object'" }, { "action": "allow", "expression": "parameters.bucket == 'my-destination' && (operation.startsWith('put-object') || operation.startsWith('delete-object') || operation.startsWith('abort-multipart-upload'))" } ] } } } ``` ![](portal-0-role.png "Portal: Creating role") 2. In `Storage` -> `Add`, create the two buckets. 1. In the list of buckets, for your two bucket, click on `...` and open the `details` page. Enable `Versioning` for both buckets. 2. For our source bucket, open the `Replication` tab and write the following configuration: ```json { "Role": "6f00a21e-a535-4286-86c5-e81eeab3b4ff", "Rules": [{ "ID": "vie1-follower", "Priority": 1, "Filter": { "Prefix": "" }, "Status": "Enabled", "DeleteMarkerReplication": { "Status": "Enabled" }, "Destination": { "Bucket": "my-destination" } }] } ``` ![](portal-1-replication.png "Portal: Configuring replication") 3. Finally, we can upload objects in our source bucket, and confirm they eventually appear in the destination bucket. ![](portal-2-validation.png "Portal: Validating the replication") #### Using Terraform 1. Using the following Terraform configuration, we will directly create a IAM role, two buckets and the cross-bucket replication rule : ```terraform # main.tf terraform { required_providers { exoscale = { source = "exoscale/exoscale" } aws = { source = "hashicorp/aws" } } } provider "exoscale" {} provider "aws" { alias = "ch-dk-2" region = "ch-dk-2" endpoints { s3 = "https://sos-ch-dk-2.exo.io" } # Disable AWS-specific features skip_credentials_validation = true skip_region_validation = true skip_requesting_account_id = true } provider "aws" { alias = "at-vie-1" region = "at-vie-1" endpoints { s3 = "https://sos-at-vie-1.exo.io" } # Disable AWS-specific features skip_credentials_validation = true skip_region_validation = true skip_requesting_account_id = true } resource "aws_s3_bucket" "source" { provider = aws.ch-dk-2 bucket = "my-source-terraform" } resource "aws_s3_bucket" "destination" { provider = aws.at-vie-1 bucket = "my-destination-terraform" } resource "aws_s3_bucket_versioning" "source" { provider = aws.ch-dk-2 bucket = aws_s3_bucket.source.id versioning_configuration { status = "Enabled" } } resource "aws_s3_bucket_versioning" "destination" { provider = aws.at-vie-1 bucket = aws_s3_bucket.destination.id versioning_configuration { status = "Enabled" } } resource "exoscale_iam_role" "iam-source-to-destination-tf" { name = "sos-replication:${aws_s3_bucket.source.id}:${aws_s3_bucket.destination.id}" description = "sos bucket replication between ${aws_s3_bucket.source.id} and ${aws_s3_bucket.destination.id}" editable = true policy = { default_service_strategy = "deny" services = { sos = { type = "rules" rules = [ { action= "allow" expression = "parameters.bucket == '${aws_s3_bucket.source.id}' && (operation in ['get-object'])" }, { action= "allow" expression = "parameters.bucket == '${aws_s3_bucket.destination.id}' && (operation.startsWith('put-object') || operation.startsWith('delete-object') || operation.startsWith('abort-multipart-upload'))" } ] } } } } resource "aws_s3_bucket_replication_configuration" "replication" { provider = aws.ch-dk-2 depends_on = [aws_s3_bucket_versioning.source, aws_s3_bucket_versioning.destination] bucket = aws_s3_bucket.source.id role = "arn:aws:iam::third-party:${exoscale_iam_role.iam-source-to-destination-tf.id}" rule { id = "multi-zone" priority = 1 status = "Enabled" filter { prefix = "" } delete_marker_replication { status = "Enabled" } destination { bucket = "arn:aws:s3:::${aws_s3_bucket.destination.id}" } } } ``` 2. You can directly create all of the resources ussing the following command: ```bash terraform apply ``` Output: ``` [...] Plan: 6 to add, 0 to change, 0 to destroy. Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be accepted to approve. Enter a value: yes [...] Apply complete! Resources: 6 added, 0 changed, 0 destroyed. ``` 3. Confirm it works as expected: ```bash exo storage upload -r ./ sos://my-source-terraform/ ``` Output: ```bash .terraform/prov… [==============================================================================] 34.68 KiB / 34.68 KiB | 0s .terraform/prov… [==============================================================================] 16.33 KiB / 16.33 KiB | 0s .terraform/prov… [==============================================================================] 4.86 KiB / 4.86 KiB | 0s .terraform/prov… [==============================================================================] 32.26 MiB / 32.26 MiB | 0s .terraform/prov… [==============================================================================] 16.37 KiB / 16.37 KiB | 0s .terraform/prov… [==============================================================================] 669.88 MiB / 669.88 MiB | 10s .terraform/prov… [==============================================================================] 16.37 KiB / 16.37 KiB | 0s .terraform/prov… [==============================================================================] 664.45 MiB / 664.45 MiB | 9s .terraform.lock… [==============================================================================] 2.52 KiB / 2.52 KiB | 0s main.tf [==============================================================================] 2.64 KiB / 2.64 KiB | 0s terraform.tfsta… [==============================================================================] 9.05 KiB / 9.05 KiB | 0s ``` ```bash exo storage list sos://my-source-terraform/ ``` Output: ```bash DIR .terraform/ 2025-07-09 08:31:57 UTC 2.5 KiB .terraform.lock.hcl 2025-07-09 08:31:57 UTC 2.6 KiB main.tf 2025-07-09 08:31:58 UTC 9.1 KiB terraform.tfstate ``` ```bash exo storage list sos://my-destination-terraform/ ``` Output: ```bash DIR .terraform/ 2025-07-09 08:31:57 UTC 2.5 KiB .terraform.lock.hcl 2025-07-09 08:31:57 UTC 2.6 KiB main.tf 2025-07-09 08:31:58 UTC 9.1 KiB terraform.tfstate ``` You can also check the Replication Status of each object individually: ```bash exo storage show sos://my-source-terraform/main.tf ``` Output: ```bash [...] │ Path │ main.tf │ │ Bucket │ my-source-terraform │ │ Replication Status │ COMPLETED │ ``` ```bash exo storage show sos://my-destination-terraform/main.tf ``` Output: ```bash [...] │ Path │ main.tf │ │ Bucket │ my-destination-terraform │ │ Replication Status │ REPLICA │ ``` ### Bi-Directional Replication You can also synchronize two buckets in both direction. This can be used by example for creating an eventually-consistent multi-active system, or to ensure easy failover and rollback of your application in case of datacenter faults. Writes to any of the two buckets will be eventually replicated to the other. Writes are only replicated once, this will not create any infinite loops in the replication process. 1. Using the following IAM configuration, create an IAM role for the replication: ```json # a-b.json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "parameters.bucket == 'bucket-a' && operation == 'get-object'" }, { "action": "allow", "expression": "parameters.bucket == 'bucket-b' && (operation.startsWith('put-object') || operation.startsWith('delete-object') || operation.startsWith('abort-multipart-upload'))" } ] } } } ``` ```json # b-a.json { "default-service-strategy": "deny", "services": { "sos": { "type": "rules", "rules": [ { "action": "allow", "expression": "parameters.bucket == 'bucket-b' && operation == 'get-object'" }, { "action": "allow", "expression": "parameters.bucket == 'bucket-a' && (operation.startsWith('put-object') || operation.startsWith('delete-object') || operation.startsWith('abort-multipart-upload'))" } ] } } } ``` ```bash cat a-b.json | exo iam role create --editable=true --description 'sos bucket replication between bucket-a and bucket-b' --policy - replication-bucket-a-bucket-b ``` Output: ```bash ┼─────────────┼──────────────────────────────────────────────────────┼ │ ID │ 91390068-c386-4d9d-b0f1-951e2baa818e │ │ Name │ replication-bucket-a-bucket-b │ │ Description │ sos bucket replication between bucket-a and bucket-b │ │ Editable │ true │ │ Labels │ n/a │ │ Permissions │ n/a │ ┼─────────────┼──────────────────────────────────────────────────────┼ ``` ```bash $ cat b-a.json | exo iam role create --editable=true --description 'sos bucket replication between bucket-b and bucket-a' --policy - replication-bucket-b-bucket-a ``` Output: ```bash ┼─────────────┼──────────────────────────────────────────────────────┼ │ ID │ e23c886b-8091-4adc-b567-7bf796bed37b │ │ Name │ replication-bucket-b-bucket-a │ │ Description │ sos bucket replication between bucket-b and bucket-a │ │ Editable │ true │ │ Labels │ n/a │ │ Permissions │ n/a │ ┼─────────────┼──────────────────────────────────────────────────────┼ ``` 2. Create the A and B buckets, with the correct configuration ```bash exo storage create --zone ch-dk-2 bucket-a [...] exo storage create --zone at-vie-1 bucket-b [...] exo storage bucket versioning enable --zone ch-dk-2 bucket-a exo storage bucket versioning enable --zone at-vie-1 bucket-b ``` 3. Using the following replication configuration, enable the replication between the two buckets. ```json # conf-a.json { "Role": "91390068-c386-4d9d-b0f1-951e2baa818e", "Rules": [{ "ID": "to-bucket-b", "Priority": 1, "Filter": { "Prefix": "" }, "Status": "Enabled", "DeleteMarkerReplication": { "Status": "Enabled" }, "Destination": { "Bucket": "bucket-b" } }] } ``` ```json # conf-b.json { "Role": "e23c886b-8091-4adc-b567-7bf796bed37b", "Rules": [{ "ID": "to-bucket-a", "Priority": 1, "Filter": { "Prefix": "" }, "Status": "Enabled", "DeleteMarkerReplication": { "Status": "Enabled" }, "Destination": { "Bucket": "bucket-a" } }] } ``` ```bash exo storage bucket replication set --zone ch-dk-2 sos://bucket-a ./conf-a.json exo storage bucket replication set --zone at-vie-1 sos://bucket-b ./conf-b.json ``` 4. Confirm it works as expected: ```bash exo storage upload *a.json sos://bucket-a ``` Output: ```bash b-a.json [==============================================================================] 410.00 b / 410.00 b | 0s conf-a.json [==============================================================================] 364.00 b / 364.00 b | 0s ``` ```bash exo storage upload *b.json sos://bucket-b ``` Output: ```bash a-b.json [==============================================================================] 410.00 b / 410.00 b | 0s conf-b.json [==============================================================================] 364.00 b / 364.00 b | 0s ``` ```bash exo storage list sos://bucket-a ``` Output: ```bash 2025-02-24 14:06:34 UTC 410 B a-b.json 2025-02-24 14:06:28 UTC 410 B b-a.json 2025-02-24 14:06:28 UTC 364 B conf-a.json 2025-02-24 14:06:34 UTC 364 B conf-b.json ``` Output: ```bash exo storage list sos://bucket-b ``` Output: ```bash 2025-02-24 14:06:34 UTC 410 B a-b.json 2025-02-24 14:06:28 UTC 410 B b-a.json 2025-02-24 14:06:28 UTC 364 B conf-a.json 2025-02-24 14:06:34 UTC 364 B conf-b.json ``` ## Cost Impact Using bucket replication is free, but replicated objects will count towards your total storage usage. If you replicate an entire bucket within the same or across zones, your storage consumption will **double**. ## Limitations * SOS doesn't support Batch Replication. Only new data mutations will be replicated. * Objects encrypted with SSE-C will not be replicated * As SOS doesn't support Object Tagging, filters referencing tags will not be evaluated * Source and destination buckets must be within the same organization --- url: https://community.exoscale.com/product/storage/object-storage/how-to/bucketlifecycle/ --- # Implement Bucket Lifecycle Bucket lifecycle management in object storage refers to automated rules that manage objects over time. These rules help control storage costs, enforce data retention policies, and maintain storage organization by automatically handling tasks such as deleting old objects, managing versions, and cleaning up incomplete uploads. ## Lifecycle on Object Storage Exoscale Simple Object Storage (SOS) supports native bucket lifecycle management for expiration of objects. Bucket Lifecycle enables: - Cost Optimization: Automatically removing data that exceeds retention periods to reduce storage costs. - Data Governance: Enforcing compliance and retention requirements. - Storage Hygiene: Managing versions, removing incomplete uploads, and avoiding clutter in object storage. ### Supported Features 1. __Expiration of Objects__: Objects created more than X days ago (user-defined) are deleted. 2. __Retention of Non-current Versions__: A specified number of non-current versions (versions that are not the most recent) are retained. 3. __Removal of Delete Markers__: when all object versions are deleted and only a delete marker remains, dangling delete markers are removed. 4. __Abortion of Multipart Uploads__: Incomplete multipart uploads that started before a specified number of days are aborted. ### Usage #### Configuration Configuration shares the same format as AWS S3's bucket lifecycle configuration, but only a subset of AWS's lifecycle rules are supported (storage class transition is currently unsupported). Lifecycle configuration is specified as a JSON document consisting of one or more rules. ```json { "Rules": [ { "ID": "RULE001", "Status": "Enabled", "Filter":{ "Prefix": "logs/" }, "Expiration": { "Days": 10 }, "NoncurrentVersionExpiration": { "NewerNoncurrentVersions": 20 }, "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 } }, { "ID": "RULE002", "Status": "Enabled", "Expiration": { "ExpiredObjectDeleteMarker": true }, "Filter": {} }, { "ID": "RULE003", "Status": "Enabled", "Filter": { "And": { "Prefix": "logs/", "ObjectSizeGreaterThan": 1048576, "ObjectSizeLessThan": 104857600 } }, "Expiration": { "Days": 30 } } ] } ``` __Explanation__ * `Rules`: An array containing one or more of rules for bucket lifecycle management. * `ID`: A unique identifier for the rule, such as `"RULE001"`. * `Status`: The state of the rule: `Enabled` or `Disabled`. `Disabled` rules have no effect * `Expiration`: Specifies the expiration settings for objects. * `Days`: in `RULE001`, objects created 10 days ago or more are deleted; in `RULE003` it's 30 days * `ExpiredObjectDeleteMarker`: when all previous versions of an object are deleted and only a delete marker remains, dangling delete markers are purged. * `NoncurrentVersionExpiration`: Defines how non-current versions are handled. * `NewerNoncurrentVersions`: Retains the 20 most recent non-current versions. * `NoncurrentDays`: non-current versions created more than X days ago are deleted. Mutually exclusive with `NewerNoncurrentVersions`. * `AbortIncompleteMultipartUpload`: Configures the abortion of incomplete multipart uploads. * `DaysAfterInitiation`: Incomplete multipart uploads that started more than 7 days ago are aborted. * `Filter`: Every rule requires a `Filter` field, even if it is empty. Narrows the scope of a rule to a subset of objects. When combining multiple conditions, they should be wrapped in an `And` block. * `Prefix`: Restricts the rule to objects whose key begins with the given string (e.g. `"logs/"`). * `ObjectSizeGreaterThan`: Restricts the rule to objects strictly larger than the given size in bytes. * `ObjectSizeLessThan`: Restricts the rule to objects strictly smaller than the given size in bytes. #### Setting Bucket Lifecycle Configuration Any tooling that supports AWS S3 API for bucket lifecycle should be fully compatible. ```bash $ aws s3api put-bucket-lifecycle-configuration --bucket yourbucket --lifecycle-configuration file://path/to/your/config.json ``` ```bash $ exo storage bucket lifecycle set sos://yourbucket path/to/your/config.json -z your-zone ``` Where: * `path/to/your/config.json` is the actual path to your JSON configuration file. * `yourbucket` is the name of the bucket on which to apply lifecycle rules. * `your-zone` is the Exoscale zone where the bucket is located. #### Scheduling Bucket Lifecycle evaluation runs every day at midnight UTC. There might be some delay between evaluating a bucket's configuration and deleting the respective objects. ### Limitations * Exoscale SOS only supports bucket lifecycle rules for object deletion/expiration. * AWS S3 requires either a `Filter` or a `Prefix` field at the `Rule` level. Exoscale SOS only supports the `Filter` field, which is required to be present in all rules (but can be empty). --- url: https://community.exoscale.com/product/storage/object-storage/how-to/archival-storage/ --- # Enable Archival Storage Archival Storage stores cold data on Simple Object Storage at a reduced rate, on the buckets you already run. It is meant for data you keep and rarely read back: cold archives, long-term backups, and retention data. Nothing changes technically. Enrolled buckets keep the same names, the same S3 endpoints, the same credentials, and the same zone. Only the rate applied to stored bytes changes, through a monthly discount on the invoice. ## Pricing | | Standard | Archival Storage | | --- | --- | --- | | Storage price | €0.0184 per GB/month | €0.0042 per GB/month | | Minimum billed per bucket | none | 10 TB (9,313 GiB) | | Retrieval | free | free | | Minimum storage duration | none | none | | Egress traffic | standard rates | standard rates | | Availability SLA | 99.95% | 99.85% | | Object lock and retention | available | not available | Prices are rounded monthly approximations. Billing is per GiB/hour, and the exact rates are on the [Object Storage pricing page](https://www.exoscale.com/pricing/#storage). ## Check whether it pays off for your bucket Every bucket enrolled in Archival Storage is billed for at least 10 TB of storage per month, metered as 9,313 GiB since Object Storage bills in GiB, even if it holds less. That floor is €42 per month. Below roughly 2.3 TB in a bucket, the floor costs more than paying the Standard rate on the real volume. | Data in the bucket | Standard | Archival Storage | Difference | | --- | --- | --- | --- | | 1 TB | €18.40 | €42.00 (10 TB floor) | +€23.60 | | 2.5 TB | €46.00 | €42.00 (10 TB floor) | −€4.00 | | 10 TB | €184.00 | €42.00 | −€142.00 | | 50 TB | €920.00 | €210.00 | −€710.00 | | 100 TB | €1,840.00 | €420.00 | −€1,420.00 | The floor applies per bucket, not per organization. Consolidating several small archives into one bucket is usually cheaper than enrolling each of them separately. ## Requirements - **Archival workloads.** Hot, frequently accessed data stays on the Standard Class. Each request is reviewed against the actual access pattern of the bucket. - **10 TB billed per bucket**, metered as 9,313 GiB. Whatever the bucket holds. - **No object lock or retention.** These features are not available on enrolled buckets. They arrive with the Cold Storage Class. If a workload needs immutability today, keep it on the Standard Class. ## Request activation 1. Open the [activation request form](https://app.formbricks.com/s/cmpy13efand7qsm01pd7oo85s?utm_source=community.exoscale.com&utm_medium=docs&utm_campaign=storage-classes&utm_content=docs_archival). 2. Submit your organization ID and the name of the bucket to enroll. One request per bucket. 3. The request is checked against the requirements above. If the bucket qualifies, the Archival rate is granted. There is no separate confirmation step: you see the rate applied as a discount on your next invoice. No action is required on the bucket itself either, no API call, no flag to set, and no data to move. ## How the discount is billed Usage on an enrolled bucket is metered and invoiced at the Standard rate, and the difference down to the Archival rate comes back as a discount line on the invoice. Billing accrues at the end of each month, and the discount starts with the next full billing period. A request approved on 20 June therefore covers July usage, and the discount appears on the July invoice issued at the end of that month. Once approved, nothing else is required. ## When the Warm and Cold Storage Classes launch The Warm and Cold Storage Classes are planned for early 2027. A 30-day window opens at launch, during which each enrolled bucket is moved to the Storage Class you want it on. Moving a bucket to Cold Storage keeps the same storage price of €0.0042 per GB/month, with the same data, the same buckets, the same S3 API, and the same zone. Two things change in your favour: the 10 TB minimum no longer applies, and object lock and retention become available. One thing is added: Cold Storage charges €0.027 per GiB retrieved and applies a 90-day minimum storage duration, where Archival Storage has neither today. Warm Storage and the Standard Class stay available if either fits the workload better. > [!WARNING] > Buckets not migrated within the 30-day window return to Standard Class pricing. ## See also - [Storage Classes and pricing comparison](https://www.exoscale.com/object-storage/classes/) - [Object Storage overview](../../overview/) - [Implement Bucket Lifecycle](../bucketlifecycle/) - [Object Storage SLA](../../service-boundaries/sla/) --- url: https://community.exoscale.com/product/storage/object-storage/how-to/encryption/ --- # Enable Encryption Encryption with SOS is realized at various levels: - Data in transit - Data at rest ### Encryption in transit By default all data and metadata sent to or retrieved from buckets using SOS endpoints over HTTPs is encrypted in transit using TLS. ### Encryption at rest To protect data at rest, multiple options are available when using SOS: - Client side encryption: data is encrypted before being sent to SOS by the client library, tool or code. With this mode, the encryption process, encryption key and libraries are fully managed by you. - Server side encryption: data is transparently encrypted by SOS upon reception and transparently deciphered at egress, without any extra costs. Exoscale supports two types of server-side encryption at rest: - Server-side encryption with Exoscale-managed keys (`SSE-SOS`), **recommended and enabled by default on all new buckets** - Server-side encryption with customer-provided keys (`SSE-C`), for advanced use cases requiring customer-managed keys. Not recommended for most workloads due to operational complexity and limitations (see [below](#limitations)) ## SSE-SOS All new buckets are created with SSE-SOS encryption enabled by default. Your data is automatically encrypted at rest without any additional configuration required. ### Enabling SSE-SOS on existing buckets For buckets created before this feature was enabled by default, you can manually enable SSE-SOS: ```bash aws --endpoint https://sos-ch-gva-2.exo.io/ s3api put-bucket-encryption --bucket my-bucket --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault": {"SSEAlgorithm": "AES256"}}]}' ``` This can also be configured using Terraform: ```terraform resource "aws_s3_bucket_server_side_encryption_configuration" "my-bucket" { bucket = aws_s3_bucket.my-bucket.id rule { apply_server_side_encryption_by_default { sse_algorithm = "AES256" } } } ``` ### Transferring objects with SSE-SOS Uploading and downloading encrypted objects works seamlessly, nothing has to be changed within your application or tools. The only visible difference being an extra header informing you about the encryption status of an object. You can confirm that encryption at rest is enabled by writing an object to a bucket, and observing the `"ServerSideEncryption"` header in the response: ```bash $ aws --endpoint https://sos-ch-gva-2.exo.io/ s3api put-object --bucket my-bucket --key sse-sos-object.txt --body plain-text-file.txt { "ETag": "\"da17b5a1c4cd2a8b55f98628e1f1089f\"", "ServerSideEncryption": "AES256" } ``` The same header is visible when retrieving object metadata using the GetObject and HeadObject operations: ```bash > aws --endpoint https://sos-ch-gva-2.exo.io/ s3api head-object --bucket my-bucket --key sse-sos-object.txt { "AcceptRanges": "bytes", "LastModified": "2025-09-12T14:38:01+00:00", "ContentLength": 1050, "ETag": "\"da17b5a1c4cd2a8b55f98628e1f1089f\"", "VersionId": "null", "ContentType": "application/binary", "ServerSideEncryption": "AES256", "Metadata": {} } > aws --endpoint https://sos-ch-gva-2.exo.io/ s3api get-object --bucket my-bucket --key sse-sos-object.txt /dev/null { "AcceptRanges": "bytes", "LastModified": "2025-09-12T14:38:01+00:00", "ContentLength": 1050, "ETag": "\"da17b5a1c4cd2a8b55f98628e1f1089f\"", "VersionId": "null", "ContentType": "application/binary", "ServerSideEncryption": "AES256", "Metadata": {} } ``` >[!NOTE] > That objects uploaded before enabling SSE-SOS will not be automatically encrypted > and will not have the `"ServerSideEncryption"` header returned when you try to access them. ## SSE-C > [!WARNING] > **SSE-SOS is the recommended encryption method for most use cases.** SSE-C requires you to manage your own encryption keys and comes with significant operational overhead and limitations. Consider using SSE-SOS (enabled by default) unless you have a specific requirement for customer-managed keys. To use **SSE-C** (Server-Side Encryption with Customer-Provided Keys), you must manage your own encryption keys. Data can only be retrieved by providing the original key used during upload. Therefore, you must store your keys in a durable and highly available manner, as **Exoscale does not store or recover your keys**. The keys must be 256-bit AES keys and provided to the SOS API in base64-encoded format. An example command to generate such a key is: ```bash openssl rand -out my_personal_sse-c.key 32 ``` ### Transferring objects with SSE-C Uploading and downloading encrypted objects works similarly to standard operations. You use the `cp` operation, passing the encryption key as a command-line parameter. It is your responsibility to track which key is used for each object. The following command generates the 256-bit base64-encoded string for use as the encryption key: ```bash cat /dev/urandom | head -c 32 | base64 - ``` Object upload: ```bash mc cp encrypted_file_to_upload.txt myminio/mylockedbucket/encrypted_file_uploaded.txt --encrypt-key "myminio/mylockedbucket/=wMEoiDXcIAh/uxEQ4vAKaoNwrOQmK5yrseIKieUO494=" ``` If you attempt to display the content of the encrypted object without supplying the encryption key, you should receive the following error to confirm that it has been server-side encrypted correctly. ```bash mc cat myminio/mylockedbucket/encrypted_file_uploaded.txt ``` Output: ```bash mc: Unable to read from myminio/mylockedbucket/encrypted_file_uploaded.txt. 400 Bad Request. ``` ### Limitations - No CLI or Portal support: SSE-C is not currently supported through the Exoscale Portal or standard CLI interfaces. Use tools that natively support SSE-C. - No key rotation via copy: SOS does not support copying objects using different keys. To change keys, you must download the object locally and re-upload it with the new key. - No replication: Objects encrypted with SSE-C cannot be replicated using SOS Bucket Replication. > [!NOTE] > All new buckets are created with SSE-SOS enabled by default. > Once enabled, SSE-SOS cannot be disabled. It acts as a default for new uploads, > while still allowing `SSE-C` to be used on specific objects if needed. > [!NOTE] > SSE-KMS, while similar to SSE-SOS, is currently not supported on Exoscale. > A KMS implementation is planned for future release and this page will be updated accordingly when available. ### Blocking SSE-C by default SSE-C is not a commonly used encryption scheme due to the concerns and limitations outlined above. For that reason it is **blocked by default on newly created buckets**. When SSE-C encryption is blocked, SOS will reject all attempts to create SSE-C encrypted objects in the bucket, whether through standard `PUT` or `POST` uploads, copying, or multipart uploads: ``` An error occurred (BlockedEncryption) when calling the PutObject operation (reached max retries: 0): The encryption type used for the object is blocked by the bucket's server-side encryption configuration. ``` If you insist on using SSE-C, you can update your bucket's SSE configuration to allow it: ```bash aws --endpoint https://sos-ch-gva-2.exo.io/ s3api put-bucket-encryption --bucket my-bucket --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"},"BlockedEncryptionTypes":{"EncryptionType":["NONE"]}}]}' ``` If ever necessary, you can always update the SSE configuration to block SSE-C encrypted objects again: ```bash aws --endpoint https://sos-ch-gva-2.exo.io/ s3api put-bucket-encryption --bucket my-bucket --server-side-encryption-configuration '{"Rules":[{"ApplyServerSideEncryptionByDefault":{"SSEAlgorithm":"AES256"},"BlockedEncryptionTypes":{"EncryptionType":["SSE-C"]}}]}' ``` --- url: https://community.exoscale.com/product/storage/object-storage/how-to/metadata/ --- # Manage Metadata Simple Object Storage (or SOS) allows you to separate file metadata from data. Each object in a bucket can be tagged with key-value pairs. This metadata is presented through HTTP headers on SOS requests and responses. ## Add Metadata ### From the Portal: - Click on **STORAGE** in the main navigation. - Select your bucket. - Click on the pencil icon of the object to edit. - Click on the **Metadata** tab. You can add key-value pairs, assigned to the object you chose. ## From the CLI: We assume you are using [S3cmd](https://s3tools.org/s3cmd). The `modify` command modifies the metadata. You can then use the option `--add-header` to add an HTTP header. The headers that start with `x-amz-meta` will be seen as metadata. If you want to add a metadata "foo" with value "bar", the key will be named `x-amz-meta-foo` ```bash s3cmd modify --add-header x-amz-meta-foo:bar s3:/// modify: 's3:///' ``` ## View Metadata ### From the Portal: To view the metadata associated with an object, you need to edit the object: - Click on **STORAGE** in the main navigation. - Select your bucket. - Click on the pencil icon of the object to edit. - Click on the **Metadata** tab. ### From the CLI: We assume for this example you are using [S3cmd](https://s3tools.org/s3cmd). The `info` command will return information on an object, including its metadata. ```bash s3cmd info s3://// ``` Output: ```bash s3://// (object): File size: 0 Last mod: Mon, 02 Nov 2015 13:37:00 UTC MIME type: application/binary MD5 sum: d41d8cd98f00b204e9800998ecf8427e SSE: none policy: none cors: ACL: None: FULL_CONTROL x-amz-meta-test: true x-amz-meta-foo: bar ``` You can see 2 key-value pairs for this object: - `test` was set to `true`. - `foo` was set to `bar`. ## Delete Metadata ### From the Portal: To delete the metadata associated with an object, you need to edit the object: - Click on **STORAGE** in the main navigation. - Select your bucket. - Click on the pencil icon of the object to edit. - Click on the **Metadata** tab. Then, for each key-value pair, you can click on the **X** icon to delete. ## From the CLI: We assume you are using [S3cmd](https://s3tools.org/s3cmd). The `modify` command modifies the metadata. You can then use the option `--remove-header` to delete an HTTP header. The metadata starts with `x-amz-meta`. So as an example, if you want to remove the metadata "foo" with value "bar", you should remove `x-amz-meta-foo`: ```bash s3cmd modify --remove-header x-amz-meta-foo s3:/// ``` ``` modify: 's3:///' ``` --- url: https://community.exoscale.com/product/storage/object-storage/how-to/versioning/ --- # Enable Versioning Simple Object Storage's versioning and lock support lets you protect and archive data in your applications. It prevents data loss and enables you to restore objects that are accidentally deleted or overwritten via the user interface or API. > [!NOTE]"In progress" > Please note that Web portal support for versioning is currently in progress and will be released soon. ## Versioning By default buckets are un-versioned and versioning needs to be activated. Once activated, it will modify the following behaviors for an object: * If an object is deleted, SOS will instead keep the object and add the delete marker. Therefore the object could still be restored. * If an object is overwritten, then SOS will create a new version instead to the object keeping the previous one intact. The previous version can be then restored any time referring to its version ID. ## Object Lock Exoscale SOS implements the S3 compatible object lock feature with the *Legal hold* feature. An object bearing the lock protection cannot be deleted indefinitely until the legal hold has been explicitly removed. ## Pricing considerations Bucket versioning is a free feature and does not trigger additional charges when activated on a bucket whether it is used or not. However there is a cost implied by having multiple versions of an object. For example, if you have 5 versions of an object, each version 1 GB of storage volume, then your charge will be for 5 GB. Object lock is as well a free feature, with the normal SOS charge applying for the duration the object is stored on Exoscale. ## Getting Started This guide will walk you through various operations involving bucket versioning and locking using the Minio client, one of the open-source tools compatible with S3 for managing your buckets. Prerequisites: - Install the Minio client by referring to the official Minio documentation [here](https://min.io/docs/minio/linux/reference/minio-mc.html). - Obtain an access key and a secret key to manage your buckets using the Minio client. For detailed instructions, please refer to our guide [here](https://community.exoscale.com/product/security/iam/quick-start/index.md). ### Configure the Minio client ```bash mc alias set myminio https://sos-de-fra-1.exo.io ``` Storage API Endpoint: https://sos-{zone}.exo.io Replace {zone} with the intended value. Available sos zones are : ch-dk-2, ch-gva-2, de-fra-1, de-muc-1, at-vie-1, at-vie-2, bg-sof-1. ### Create a bucket ```bash mc mb --with-lock myminio/mylockedbucket ``` > [!NOTE] > You can only enable object locking during bucket creation as per S3 behavior. > You cannot enable object locking on a bucket created without locking enabled. > You can then configure object retention rules at any time. > Object locking requires versioning and enables the feature implicitly. ### Listing buckets ```bash mc ls myminio ``` ### Listing bucket content recursively ```bash mc ls --recursive myminio/mylockedbucket ``` ### Verify bucket versioning status ```bash mc stat myminio/mylockedbucket ``` Output: ```bash Name : mylockedbucket Date : 2023-10-26 22:10:26 CET Size : N/A Type : folder Properties: Versioning: Enabled Location: de-fra-1 Anonymous: Disabled ILM: Disabled Usage: Total size: 0 B Objects count: 0 Versions count: 0 ``` ### Upload an object to bucket ```bash mc cp file_to_upload.txt myminio/mylockedbucket ``` ### List all versions of my objects in a bucket ```bash mc ls --versions myminio/mylockedbucket ``` ### Delete a specific version of an object ```bash mc rm myminio/mylockedbucket/file_to_upload.txt --version-id "916619044952888425" ``` ### Remove a bucket and all its content ```bash mc rb --force myminio/mylockedbucket ``` ### Set up the lock configuration for a bucket ```bash mc retention set --default GOVERNANCE "30d" myminio/mylockedbucket ``` For additional details on Retention MODE and DURATION, please refer to this [guide](https://min.io/docs/minio/linux/administration/object-management/object-retention.html#id9). ### Show bucket configuration Now that lock configuration was set, the `stat` command shows its status: ```bash mc stat myminio/mylockedbucket ``` Output: ```bash Name : mylockedbucket Date : 2023-10-26 23:33:34 CET Size : N/A Type : folder Properties: Versioning: Enabled LockConfiguration: RetentionMode: GOVERNANCE Retention Until Date: 30DAYS Location: de-fra-1 Anonymous: Disabled ILM: Disabled Usage: Total size: 0 B Objects count: 0 Versions count: 0 ``` Alternatively: ```bash mc retention info myminio/mylockedbucket ``` Output: ```bash Object locking 'GOVERNANCE' is configured for 30DAYS. ``` --- url: https://community.exoscale.com/product/storage/object-storage/how-to/checksum/ --- # Use Data Integrity Checksums SOS supports data integrity verification using checksums. When uploading objects, you can specify a checksum algorithm and SOS will verify the data was transmitted correctly. ## Supported Algorithms - CRC32 - CRC32C - CRC64NVME ## Uploading with Checksum ```bash aws --endpoint https://sos-ch-gva-2.exo.io/ s3api put-object \ --bucket my-bucket \ --key my-object.txt \ --body my-file.txt \ --checksum-algorithm CRC32 ``` ## Retrieving Checksum Use `--checksum-mode ENABLED` to retrieve an object's checksum: ```bash aws --endpoint https://sos-ch-gva-2.exo.io/ s3api head-object \ --bucket my-bucket \ --key my-object.txt \ --checksum-mode ENABLED ``` --- url: https://community.exoscale.com/product/storage/object-storage/how-to/terraform/ --- # Manage Terraform Simple Object Storage (or SOS) is S3 compatible. You can use the [Terraform AWS provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) to manage your SOS resources. ## Provider configuration In the provider configuration, you need to skip some AWS-specific validations: ```hcl-terraform locals { zone = "ch-gva-2" } provider "aws" { endpoints { s3 = "https://sos-${local.zone}.exo.io" } region = local.zone # access_key = "my-access-key" # secret_key = "my-secret-key" # Disable AWS-specific features skip_credentials_validation = true skip_region_validation = true skip_requesting_account_id = true } ``` You can also set your credentials with these environment variables: ```bash export AWS_ACCESS_KEY_ID="anaccesskey" export AWS_SECRET_ACCESS_KEY="asecretkey" ``` To manage the buckets in multiple zones, you need to create [multiple provider configurations](https://www.terraform.io/language/providers/configuration#alias-multiple-provider-configurations), each with a different `alias`. You can check Terraform's [provider configuration](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#provider-configuration) documentation for more information. ## Manage your buckets ### Create a bucket ```hcl-terraform resource "aws_s3_bucket" "my_bucket" { bucket = "my-terraform-bucket" } ``` ### Manage ACL ```hcl-terraform resource "aws_s3_bucket_acl" "example_bucket_acl" { bucket = aws_s3_bucket.my_bucket.id acl = "private" } ``` ### Enable bucket versioning ```hcl-terraform resource "aws_s3_bucket_versioning" "my_bucket_versioning" { bucket = aws_s3_bucket.my_bucket.id versioning_configuration { status = "Enabled" } } ``` ### Upload a file ```hcl-terraform resource "aws_s3_object" "object" { bucket = aws_s3_bucket.my_bucket.id key = "new_object" source = "myfile" acl = "public-read" etag = filemd5("myfile") } ``` You can find [more S3 resources](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket), but there is a chance we do not support certain features at this time. Make sure to check our list of [S3 unsupported features](https://community.exoscale.com/product/storage/object-storage/overview/index.md#s3-unsupported-features). --- url: https://community.exoscale.com/product/storage/object-storage/how-to/conditionalrequests/ --- # Use Conditional Requests Conditional requests provide a way to control whether an action should be executed, based on certain metadata checks associated with an object. Here are some common parameters used for conditional requests: - **If-Match / If-None-Match**: These parameters compare the object's ETag value before allowing an operation. - *`If-Match`* allows the operation only if the object's ETag matches the given value. - *`If-None-Match`* allows the operation only if the object's ETag does **not** match the given value. - Both parameters also support the wildcard (`*`), which applies the condition to any ETag. - **If-Modified-Since / If-Unmodified-Since**: These parameters compare the object's last modified date or creation date before proceeding. - *`If-Modified-Since`* allows the operation only if the object has been modified after the specified date. - *`If-Unmodified-Since`* allows the operation only if the object has **not** been modified after the specified date. - **If-Match-Size / If-Match-Last-Modified-Time**: These are typically used with DeleteObject operations to check the object’s size or last modified time before deletion. Conditional request logic can also be enforced at the bucket level through bucket policies. For example, you can configure a policy to prevent accidental data overwrites. For detailed guidance, see [Bucket policy How-To](https://community.exoscale.com/product/storage/object-storage/how-to/bucketpolicy/index.md#example-policies). **Use cases for conditional requests include:** - Preventing conflicts during concurrent object updates - Saving bandwidth by avoiding unnecessary data transfers - Enabling advanced architectural patterns and workflows on top of SOS ## Requirements These parameters are generally available, and need only support from the client (eg: AWS S3 cli, Java SDK, etc). In case of multiple conditional parameters for a single request, SOS follows [RFC7232](https://datatracker.ietf.org/doc/html/rfc7232) as closely as possible. ## Operations All examples are assuming you are using the AWS S3 cli, however most S3 clients should have support for conditional requests. Date parameters are formatted as defined by RFC3339 or RFC822: - `2025-06-09T15:40:42+00:00` - `Tue, 05 Aug 2025 11:32:21 +0000` ### get-object Conditional get requests can be useful when the client may expect an existing object to be in place, and may want to save bandwidth if the client has a local copy. When retrieving an object, the following parameters can be specified: - `--if-match ` gets the object if the `ETag` matches the object `ETag` - `--if-none-match ` gets the object if the `ETag` does not match the object `ETag` - `--if-modified-since ` gets the object if the object `LastModified` is equal or after the `Timestamp` - `--if-unmodified-since ` gets the object if the object `LastModified` is before the `Timestamp` ### put-object Conditional put requests can be useful in a high concurrency scenario, where multiple clients can try to upload a given object but only one should succeed. When uploading an object, the following parameters can be specified: - `--if-match ` puts the object if the `ETag` matches the object `ETag` - `--if-none-match ` puts the object if the `ETag` does not match the object `ETag` ### copy-object Copying an object can accept the following conditional parameters: - `--copy-source-if-match ` copy if the source object `ETag` matches the `ETag` - `--copy-source-if-none-match ` copy if the source object `ETag` does not match the `ETag` - `--copy-source-if-modified-since ` copy if the source object `LastModified` is equal or after the `Timestamp` - `--copy-source-if-unmodified-since ` copy if the source object `LastModified` is before the `Timestamp` All the parameters refer to the **source** object attributes, and are similar to `get-object` parameters. ### delete-object Conditional deletes can supply the following parameters: - `--if-match ` object will be deleted if its `ETag` matches the supplied `ETag` - `--if-match-last-modified-time ` object will be deleted if its `LastModified` matches the supplied `Timestamp` - `--if-match-size ` object will be deleted if its `Size` matches the supplied `Size` ## Example Below is an example of using the aws cli to conditionally put or get an object. ```shell # first we put an object $ aws s3api put-object --bucket my-bucket --key mykey --body /tmp/myobj.bin { "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"", "VersionId": "1152004336876784859" } # we can ensure we overwrite an object given its ETag # this can be useful for a client that wants to put only if the object hasn't been modified in the meantime $ aws s3api put-object --bucket my-bucket --key mykey --body /tmp/myobj.bin --if-match "d41d8cd98f00b204e9800998ecf8427e" { "ETag": "\"323bf3388a58620e50b5b22f550236bb\"", "VersionId": "1152004380476574947" } # we can also ensure we overwrite anything $ aws s3api put-object --bucket my-bucket --key mykey --body /tmp/myobj.bin --if-match "*" { "ETag": "\"3cd3e5bf2f0f53862405768788fd17fb\"", "VersionId": "1152004396721115840" } # a non-matching ETag will fail $ aws s3api put-object --bucket my-bucket --key mykey --if-match "" An error occurred (PreconditionFailed) when calling the PutObject operation: Your preconditions were not met. # but a common case is to only put an object if there isn't one $ aws s3api put-object --bucket my-bucket --key mykey --body /tmp/myobj.bin --if-none-match "*" An error occurred (PreconditionFailed) when calling the PutObject operation: Your preconditions were not met. ## # assuming we have a copy of the object, a client may only be interested in getting different or newer versions $ aws s3api get-object --bucket mybuckey --key mykey --if-none-match "" /tmp/mykey { "AcceptRanges": "bytes", "LastModified": "2025-08-06T11:59:15+00:00", "ContentLength": 0, "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"", "VersionId": "1152004396721115840", "ContentType": "application/binary", "Metadata": {} } # we can also request an object only if a more recent modification exists $ aws s3api get-object --bucket mybucket --key mykey --if-modified-since "2025-08-06T11:59:25+00:00" /dev/null An error occurred (304) when calling the GetObject operation: Not Modified ## # we can also ask to delete an object only if we match the ETag # because another client may have updated the object in the meantime $ aws s3api delete-object --bucket my-bucket --key mykey --if-match "" An error occurred (PreconditionFailed) when calling the DeleteObject operation: Your preconditions were not met. # but if the object is still the one the client knows (the etag will match), the object can be safely deleted $ aws s3api delete-object --bucket my-bucket --key mykey --if-match "d41d8cd98f00b204e9800998ecf8427e" { "DeleteMarker": true, "VersionId": "1152006421454263029" } ``` --- url: https://community.exoscale.com/product/storage/object-storage/how-to/cors/ --- # Configure CORS Simple Object Storage's CORS (Cross-Origin Resource Sharing) support lets you configure access to your buckets from browser-based applications and websites. CORS rules are applied to entire buckets and all the objects they contain. ## CORS configuration format In the portal, a CORS configuration is a JSON document containing a set of rules. Each rule defines a set of allowed HTTP origins, methods, and headers. Here is an example CORS configuration: ```json { "CORSRules": [ { "AllowedHeaders": [ "Content-*" ], "AllowedMethods": [ "*" ], "AllowedOrigins": [ "http://www.example.com" ], "ExposeHeaders": [ "x-amz-server-side-encryption" ], "MaxAgeSeconds": 3000 }, { "AllowedMethods": [ "GET", "HEAD" ], "AllowedOrigins": [ "*" ], "MaxAgeSeconds": 3000 } ] } ``` Ts configuration allows all HTTP methods from the `https://example.com` origin, and all headers starting with `Content-`. For all other origins, the `GET` and `HEAD` methods are allowed. You can use wildcards (`*`) in allowed **origins**, **methods** and **headers**. Wildcards in allowed **headers** are limited to prefix matches. ### AllowedMethods `"AllowedMethods"` supports the following values: * HEAD * GET * POST * PUT * DELETE ### AllowedOrigins `"AllowedOrigins"` specifies the origins from which you want to allow cross-domain requests. You can choose an exact match (`https://example.com`) or a match with the wildcard character (`https://*.example.com`). If you set `"AllowedOrigins"` to `*`, it allows all origins. ### Additional elements Each `"CORSRule"` element additionally supports the following elements: * `"MaxAgeSeconds"`: controls the browser's cache for the OPTIONS response. Caching responses helps the browser avoid making repeated OPTIONS calls if the original request is being repeated. To set a 5-minute cache, set `"MaxAgeSeconds"` to `300`. * `"ExposeHeader"`: a comma-separated list of headers that SOS can send back in its responses, making them available to your JavaScript application code. ## Accessing the CORS configuration in the Portal CORS configuration can be edited in the Exoscale Portal. Navigate to the bucket list and click on the "Details" action next to your bucket. In the CORS tab, paste your JSON document. ## Managing CORS configuration within the SOS API The API endpoint for CORS management is `https://.sos-ch-dk-2.exo.io/?cors`. This endpoint uses an XML representation of the CORS rules but has an equivalent semantic for the parameters. See the AWS S3 documentation for more details: * [GET Bucket CORS](https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetBucketCors.html) * [PUT Bucket CORS](https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketCors.html) * [DELETE Bucket CORS](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteBucketCors.html) --- url: https://community.exoscale.com/product/storage/object-storage/how-to/awscli/ --- # Use AWS CLI The AWS commandline interface (CLI) is a unified tool to manage your AWS services, including AWS S3. Here we will show you how to configure and use AWS CLI to manage your SOS buckets. ## Installation You can install AWS CLI using your package manager or by downloading the latest version from the [AWS website](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html). ## Create an IAM key You need an IAM api-key with a policy that allows the management of SOS buckets. A simple policy that allows all operations on all buckets is: ```json { "default-service-strategy": "deny", "services": { "sos": { "type": "allow" } } } ``` You find more information about key management and IAM policies in the [IAM documentation](https://community.exoscale.com/product/security/iam/how-to/key-mgmt/index.md). ## Configure your AWS CLI You can configure your AWS CLI interactively using the `aws configure` command or with a script. Here is an example of a script that you can use to configure a new `exoscale` profile for your AWS CLI: ```bash export AWS_PROFILE=exoscale S3_REGION="ch-gva-2" # Assuming you have stored your Exoscale api-key and api-secret in `pass` password store. aws configure set aws_access_key_id "$(pass exoscale/api_key)" aws configure set aws_secret_access_key -- "$(pass exoscale/api_secret)" # Note the double hyphen! aws configure set region "${S3_REGION}" aws configure set endpoint_url "https://sos-${S3_REGION}.exo.io" ``` The double hyphen is required to prevent the AWS CLI from interpreting a secret that starts with a hyphen as a flag. Exoscale's secrets sometimes start with a hyphen. ## Test your configuration Now you are ready to use your SOS bucket with the AWS CLI. The following commands will create a bucket, list buckets, and delete the bucket using the `exoscale` profile. ```bash export AWS_PROFILE=exoscale BUCKET="aws-cli-bucket-$(uuidgen)" # Must be unique as per S3 specs. aws s3 mb "s3://${BUCKET}" aws s3 ls aws s3 rb "s3://${BUCKET}" aws s3 ls ``` ## Advanced usage The command `aws s3` covers only basic operations. For advanced usage, you can use the `aws s3api` command. For example, you can list the oldest five versions of a bucket's object(s) using the following command: ```bash export AWS_PROFILE=exoscale export AWS_REGION=at-vie-1 export AWS_ENDPOINT_URL="https://sos-${AWS_REGION}.exo.io" BUCKET="your-sos-bucket" TMP=$(mktemp -t s3-delete-XXX) # Prefix, query and output are optional. aws s3api list-object-versions \ --bucket "$BUCKET" \ --prefix "your-prefix/your-object" \ --query '{Objects:Versions | sort_by(@, &LastModified)[:5].{Key:Key,VersionId:VersionId}}' \ --output json | tee "$TMP" ``` The command above will create a temporary file containing a correctly formatted list of versions that you can use to delete them: ```bash aws s3api delete-objects \ --bucket "$BUCKET" \ --delete "file://$TMP" \ --debug # To see what's going on. ``` A list of all available `aws s3/s3api` commands can be found at: [https://docs.aws.amazon.com/cli/latest/reference/s3](https://docs.aws.amazon.com/cli/latest/reference/s3)
[https://docs.aws.amazon.com/cli/latest/reference/s3api](https://docs.aws.amazon.com/cli/latest/reference/s3api) --- url: https://community.exoscale.com/product/storage/object-storage/how-to/mountpoint/ --- # Use Mountpoint As Exoscale's Simple Object Storage (or SOS) is S3 compatible, all tools using the S3 API can be used with SOS. [Mountpoint](https://github.com/awslabs/mountpoint-s3) is an open source S3 client by AWS that you can use to access the objects in your bucket like files on a filesystem. The ideal use case is for read-heavy applications that require large amounts of data (e.g. data lakes, machine learning training data, ETL). ## Installation Install the current version of AWS Mountpoint for your environment. AWS offers precompiled binaries for different platforms [here](https://github.com/awslabs/mountpoint-s3/releases) ## Configuration Create an Exoscale IAM api-key / secret with permissions to access your SOS bucket. Save it in your home directory in `.aws/credentials` under a profile. For example `exomount`: ```bash [exomount] aws_access_key_id = EXO1234567890 aws_secret_access_key = ``` ## Usage Create a directory where you want to mount the bucket: ```bash mkdir mnt ``` Mount the bucket with the endpoint of the zone where the bucket was created (for a complete list of SOS endpoints look [here](https://community.exoscale.com/api/sos/)). In this example the bucket `mybucket` was created in our Zurich zone `ch-dk-2`: ```bash mount-s3 --profile exomount --endpoint-url https://sos-ch-dk-2.exo.io mybucket mnt ``` Now you can copy files to the mounted bucket: ```bash cp backup.tgz mnt ``` and list them: ```bash ls -l mnt ``` Output: ```bash total 1 -rw-r--r-- 1 exouser exouser 524288 Jul 7 11:23 backup.tgz ``` Read the contents: ```bash md5sum mnt/backup.tgz ``` ```bash 45e4be3ff8d14c966c61cca131f8b8e1 mnt/backup.tgz ``` If you try to delete the file you get an error: ```bash rm mnt/backup.tgz ``` ```bash rm: cannot remove 'mnt/backup.tgz': Operation not permitted ``` By default, Mountpoint does not allow to delete files on a mounted bucket. If you want to do this you must add the option `--allow-delete` when mounting the bucket. If you no longer need the mounted bucket you can unmount it: ```bash umount mnt ``` ## Limitations SOS is an object store, and although Mountpoint helps bridge the gap by presenting SOS objects as files, it doesn't create a POSIX-compliant filesystem. This means some filesystem operations might not behave as expected. For example, it's not possible to modify existing files (only overwrite) or rename directories and it does not support symbolic links or file locking. More details can be found in the Mountpoint documentation [here](https://github.com/awslabs/mountpoint-s3/blob/main/doc/SEMANTICS.md). AWS Mountpoint in combination with SOS does not support server-side encryption (SSE). --- url: https://community.exoscale.com/product/storage/object-storage/how-to/data-events/ --- # Enable Data Events > [!NOTE] > SOS Data Events for Audit-Trail is only available for customers with > `Enterprise` support plan and are affected by a specific extra cost. SOS Data Events for Audit-Trail give you detailed logs for all API requests executed against the SOS service, including accepted and rejected requests, information about the querier, and the affected objects. Events are grouped into batches of up to 1'000 events and are submitted as JSON-encoded payloads to a webhook you provide. Calls to the webhook will include an HMAC-SHA256 signature stored in the `exo-audittrail-signature` HTTP header, computed using a key given by Exoscale during the configuration process. ### Guarantees SOS Data Events for Audit-Trail are delivered on a best-effort basis, the exhaustivity and delivery duration of events is not guaranteed. The data event can be used to understand the traffic against your bucket. Some events can be lost, duplicated or significantly delayed. Requests against the wrong region, resulting in a redirect, do not generate any data-events and will not be visible. When the request is executed against the right zone after following the redirect, data events will be generated. ### Configuration The configuration is per-bucket and will include all data events. An HTTPS webhook URL targeting port 443 and handling `POST` requests must be provided to Exoscale support. In return you will be given a symmetric key that you can use to validate the accompanying HMAC-SHA256 signature. ### Sample output Here is an example JSON payload received by the webhook after executing `show-object-cors` followed by `delete-object`: ```json [ { "api-key": null, "handler": "show-object-cors", "request-id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "request-method": "OPTIONS", "resource": "my-bucket", "source-ip": "1.2.3.4", "status": 200, "timestamp": "2026-01-01T01:01:01.123456789Z", "uri": "https://my-bucket.sos-at-vie-1.exo.io", "user-agent": "Mozilla/5.0 ...", "zone": "at-vie-1" }, { "api-key": null, "handler": "delete-object", "iam-role": { "id": "dddddddd-dddd-dddd-dddd-dddddddddddd", "name": "Owner" }, "iam-user": { "id": "cccccccc-cccc-cccc-cccc-cccccccccccc", "role": { "id": "dddddddd-dddd-dddd-dddd-dddddddddddd" } }, "request-id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "request-method": "DELETE", "resource": "my-bucket", "source-ip": "4.3.2.1", "status": 200, "timestamp": "2026-01-01T01:01:01.123456789Z", "uri": "https://my-bucket.sos-at-vie-1.exo.io/my-object-name.txt", "user-agent": "Mozilla/5.0 ...", "zone": "at-vie-1" } ] ``` Here is another example payload received after performing a batch delete using `delete-objects`. Specially for `delete-objects`, the data event includes the `body` field that contains a JSON encoding of the [XML response](https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html#API_DeleteObjects_ResponseSyntax) sent by SOS: ```json [ { "api-key": "EXOaaaaaaaaaaaaaaaaaaaaaaaa", "body": { "DeleteResult": { "Deleted": [ { "Key": "mykey-0acf6939" }, { "Key": "mykey-8ba16fd3" } ] } }, "handler": "delete-objects", "iam-api-key": { "key": "EXOaaaaaaaaaaaaaaaaaaaaaaaa", "name": "my-key", "role-id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa" }, "iam-role": { "id": "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa", "name": "Owner" }, "request-id": "bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb", "request-method": "POST", "resource": "my-bucket", "source-ip": "4.3.2.1", "status": 200, "timestamp": "2026-01-01T00:00Z", "uri": "https://sos-ch-dk-2.exo.io/my-bucket", "user-agent": "Mozilla/5.0 ...", "zone": "ch-dk-2" } ] ``` If your bucket has versioning enabled you will notice that the objects are replaced with delete markers instead of being deleted outright: ```json [ { ... "body": { "DeleteResult": { "Deleted": [ { "Key": "mykey-e3a00bf0", "DeleteMarker": true, "DeleteMarkerVersionId": "1245494379674602496" }, { "Key": "mykey-f7e094ea", "DeleteMarker": true, "DeleteMarkerVersionId": "1245494379674602497" } ] } }, "handler": "delete-objects", ... } ] ``` However, if your deletion requests target specific versions rather than bare keys, then delete markers are not used and the versions are deleted directly: ```json [ { ... "body": { "DeleteResult": { "Deleted": [ { "Key": "mykey-cbff453c", "VersionId": "1245494389376027648" }, { "Key": "mykey-0619ee01", "VersionId": "1245494386049945600" } ] } }, "handler": "delete-objects", ... } ] ``` Note that delete markers themselves can be deleted: ```json [ { ... "body": { "DeleteResult": { "Deleted": [ { "Key": "mykey-aab0954e", "VersionId": "X/1245496299097165824", "DeleteMarker": true }, { "Key": "mykey-dd18b9ee", "VersionId": "X/1245496299097165825", "DeleteMarker": true } ] } }, "handler": "delete-objects", ... } ] ``` Be aware that it's possible for deletion to fail for some objects, for example due to configured object locks mandating a retention period, etc.: ```json [ { ... "body": { "DeleteResult": { "Deleted": [ { "Key": "mykey-cbff453c", "VersionId": "1245494389376027648" }, ], "Errors": [ { "Key": "mykey-bbcf5de5", "VersionId": "1245494399173922816", "Message": "Access denied" } } }, "handler": "delete-objects", ... } ] ``` ### Webhook example As a starting point, here is an example of a webhook which can be used to receive, validate and process data events. ```python3 import json import base64 import hashlib import hmac from http.server import BaseHTTPRequestHandler .... class SimpleHTTPRequestHandler(BaseHTTPRequestHandler): signing_key = None .... def do_POST(self): content_length = int(self.headers.Get("Content-Length", 0)) body = self.rfile.read(content_length) # Verify HMAC-SHA256 signature signature_header = self.headers.get("exo-audittrail-signature") if not signature_header: self.send_response(400) self.end_headers() self.wfile.write(b"Bad Request: Missing signature header") return try: # Decode the signing key from base64 key = base64.b64decode(self.signing_key) # Compute expected signature expected_signature = hmac.new(key, body, hashlib.sha256).hexdigest() if not hmac.compare_digest(signature_header, expected_signature): self.send_response(400) self.end_headers() self.wfile.write(b"Bad Request: Invalid signature") return except Exception as e: self.send_response(400) self.end_headers() self.wfile.write(b"Bad Request: Signature verification error") return data = json.loads(body.decode("utf-8")) .... self.send_response(200) self.end_headers() ``` --- url: https://community.exoscale.com/product/storage/object-storage/service-boundaries/ --- # Service Boundaries - Object Storage - [Limits and Quotas - Object Storage](https://community.exoscale.com/product/storage/object-storage/service-boundaries/limits-and-quotas/index.md) - Object Storage Limitations and Quotas - [SLA - Object Storage](https://community.exoscale.com/product/storage/object-storage/service-boundaries/sla/index.md) - Object Storage Service Level Agreement - [Pricing ↗︎](https://www.exoscale.com/pricing/#object-storage) --- url: https://community.exoscale.com/product/storage/object-storage/service-boundaries/limits-and-quotas/ --- # Limits and Quotas - Object Storage ## Limits The following table shows which limits are applied to SOS compared to AWS. | Usage | Limit | AWS S3 equivalent | | -------------------------------------------------------- | ------------------------ | ------------------------ | | Maximum object size | 4TiB | 5TiB | | Maximum number of parts in a multipart upload | 10000 | 10000 | | Minimum part size | 5MiB * | 5MiB * | | Maximum part size | 5GiB | 5GiB | | Maximum number of objects in a bucket | unlimited | unlimited | | Maximum number of versions per object | 1000 | unlimited | | Minimum billed size | 128kb | 128kb | | Maximum metadata size | 2kb | 2kb | | Maximum tags per object | 10 | 10 | | Maximum tag key size | 128 char | 128 char | | Maximum tag value size | 256 char | 256 char | | Maximum bucket name size | 64 char ** | 64 char ** | | Maximum key name size | 1024 byte (UTF-8) | 1024 byte (UTF-8) | | Maximum number of objects per delete-objects operation | 1000 | 1000 | > *) does not apply to the 'last part' **) DNS compatible ### Unsupported S3 Features While we strive to maintain full compatibility with the S3 API, we do not support all S3 features. Currently, the following functionality cannot be used on Exoscale: - [Object Tagging](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-tagging.html) - [Object Annotations](https://docs.aws.amazon.com/AmazonS3/latest/userguide/annotations-overview.html) - [Object Lifecycle Management](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html) - [Bucket Access Logging](https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerLogs.html) - [Bucket Event Notification](https://docs.aws.amazon.com/AmazonS3/latest/userguide/NotificationHowTo.html) - [Static Website Hosting](https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html) ## Quotas | Usage | Limit | AWS S3 equivalent | | -------------------------------------------------------- | ------------------------ | ------------------------ | | Maximum number of buckets | 20 | 100 | ## Availability Object Storage is available in all zones. --- url: https://community.exoscale.com/product/storage/object-storage/service-boundaries/sla/ --- # SLA - Object Storage These product-specific SLAs define the Service Availability Target for Simple Object Storage (SOS). It applies to each Client using the Service. Capitalized terms used herein but not defined herein shall have the meanings set forth in the Exoscale Terms and Conditions or EUSA, whichever applies. ## SLA Definitons | Metric | Target | Definition | | -------- | -------- | ------------ | | Availability | >= 99.95% | The percentage of time in a calendar month that the service is available for external connectivity. | ### Non-binding service objectives Simple Object Storage (SOS) has the following service level objectives. They are target values that Exoscale will endeavor to meet, but they are not legally binding. If these objectives are missed, no service credits or other compensation will be granted. They are provided for transparency and capacity planning purposes only. | Metric | Target | | -------- | -------- | | Data durability | >= 99.999999999% | | Data accessibility | >= 99.95% | ## Product Specifications For detailed product features, technical specifications, and tutorials, please refer to the [Exoscale documentation](https://community.exoscale.com/product/storage/object-storage/overview/index.md). Key features that contribute to the SLA: - [Bucket-to-Bucket Replication](https://community.exoscale.com/product/storage/object-storage/how-to/replication/index.md) for disaster recovery and multi-zone synchronization - [Bucket Versioning and Object Lock](https://community.exoscale.com/product/storage/object-storage/how-to/versioning/index.md) to protect data from accidental deletion - [Server-Side Encryption](https://community.exoscale.com/product/storage/object-storage/how-to/encryption/index.md) for data protection at rest ## Client Responsibilities While Exoscale is responsible for operating and maintaining the Simple Object Storage service, Client remains responsible for the following but not limited to: - Bucket access control and IAM policy management - Data classification and regulatory compliance for stored objects - Object lock retention time - Monitoring of resource usage, performance, and costs - Client defined encryption and recovery keys - Backup strategy, restore validation, and disaster recovery planning For a complete explanation of the shared responsibility framework between Exoscale and Clients, please see the [Exoscale Shared Responsibility Model documentation](https://community.exoscale.com/platform/compliance/shared-responsability-model/index.md). ## Exclusions For the complete list of SLA exclusions, please refer to the [General Exoscale Terms and Conditions](https://www.exoscale.com/terms/). ## Retribution | Monthly Uptime Percentage | Service Credit Percentage of Monthly Service Fees for Affected Resources | | --------------------------- | ------------------------------------------------------------------------- | | from 99.95% to 98.3% | 50% | | below 98.3% | 100% | ### Claim Service Credit To request a service credit, refer to the Service Unavailability Credit section of the [Exoscale Terms and Conditions](https://www.exoscale.com/terms/) or the [EUSA](https://www.exoscale.com/eusa/), whichever applies. --- url: https://community.exoscale.com/product/storage/object-storage/reference/ --- # Reference - Object Storage - [API Object Storage](https://community.exoscale.com/reference/api/object-storage/index.md) - All operations on object storage (external) - [CLI Object Storage](https://community.exoscale.com/reference/cli/exo/storage/index.md) - All commands on object storage (external) --- url: https://community.exoscale.com/product/storage/object-storage/recipes/ --- # Recipes - Object Storage Informational, concise guides showing how to integrate with external tools and solve specific scenarios. - [Cyberduck](https://community.exoscale.com/product/storage/object-storage/recipes/cyberduck/index.md) - Set up Cyberduck with Exoscale SOS for secure large file transfers using an S3 endpoint, bucket, and IAM key. - [Back up with Duplicati](https://community.exoscale.com/product/storage/object-storage/recipes/duplicati/index.md) - Set up Duplicati with Exoscale SOS for efficient, resumable backups on Linux or Windows, including bucket and IAM key setup. - [Back up with Duplicity](https://community.exoscale.com/product/storage/object-storage/recipes/duplicity/index.md) - Secure Ubuntu 20.04 backups with Duplicity to Exoscale SOS using encrypted incremental archives, plus restore and cleanup commands. - [HYCU Backup](https://community.exoscale.com/product/storage/object-storage/recipes/hycu-backup/index.md) - Configure HYCU Backup to use Exoscale SOS as S3-compatible long-term storage with bucket, endpoint, and IAM key setup. - [Mountain Duck](https://community.exoscale.com/product/storage/object-storage/recipes/mountainduck/index.md) - Mount Exoscale Simple Object Storage as a drive with Mountain Duck on Windows or macOS using a zone endpoint and IAM credentials. - [Back up a QNAP NAS](https://community.exoscale.com/product/storage/object-storage/recipes/qnap/index.md) - Configure QNAP Hybrid Backup Sync 3 to back up your NAS to Exoscale SOS using a bucket, IAM keys, and an S3-compatible target. - [Back up with Restic](https://community.exoscale.com/product/storage/object-storage/recipes/restic/index.md) - Back up and restore data with Restic on Exoscale SOS using S3-compatible storage, snapshots, and retention management. - [Mount a bucket on Linux with s3fs](https://community.exoscale.com/product/storage/object-storage/recipes/s3fs/index.md) - Mount an Exoscale SOS bucket on Linux with s3fs, using IAM credentials, manual mount commands, and persistent /etc/fstab setup. - [Storage Made Easy (SME) File Fabric](https://community.exoscale.com/product/storage/object-storage/recipes/sme/index.md) - Connect Storage Made Easy File Fabric to Exoscale SOS with an OpenS3 provider, zone endpoint, IAM keys, and bucket management. - [Back up a Synology NAS](https://community.exoscale.com/product/storage/object-storage/recipes/synology-backup/index.md) - Configure Synology Hyper Backup to create versioned NAS backups on Exoscale SOS with a zone endpoint, bucket name, and IAM keys. - [Synchronize a Synology NAS](https://community.exoscale.com/product/storage/object-storage/recipes/synology-sync/index.md) - Synchronize a Synology NAS with Exoscale SOS using Cloud Sync, a zone endpoint, IAM keys, and a bucket for selected folders. - [Back up with UrBackup](https://community.exoscale.com/product/storage/object-storage/recipes/urbackup/index.md) - Deploy UrBackup on Exoscale and configure SOS as the S3 backup target, including appliance setup, ports, HTTPS, and storage. - [Veeam Backup](https://community.exoscale.com/product/storage/object-storage/recipes/veeam/index.md) - Set up Veeam with Exoscale SOS for long-term backup storage using a scale-out repository, zone endpoint, bucket, and IAM keys. - [Immutable backups with Veeam Backup](https://community.exoscale.com/product/storage/object-storage/recipes/veeam-12-immu/index.md) - Configure Veeam 12 immutable backups with Exoscale SOS using object lock, an S3 bucket, IAM keys, and a zone endpoint. - [Plakar Backup](https://community.exoscale.com/product/storage/object-storage/recipes/plakar/index.md) - Create encrypted Plakar backups locally or in Exoscale SOS, then browse snapshots in the UI and restore data on demand. --- url: https://community.exoscale.com/product/storage/object-storage/recipes/cyberduck/ --- # Cyberduck This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. [Cyberduck](https://cyberduck.io/) is open-source software that supports a large number of storage systems, including Exoscale Simple Object Storage. Cyberduck is an ideal file transfer tool for large amounts of data. This guide demonstrates how to setup Cyberduck with Exoscale. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Set Up Cyberduck Open Cyberduck and click on **Open Connection**. ![](cyberduck-step1.png "Cyberduck Overview") Select **Amazon S3**. For the endpoint, enter `sos-ZONE.exo.io`. Make sure to substitute `ZONE` with the relevant zone. > [!NOTE] > You can find all Exoscale zones on [our website](https://www.exoscale.com/datacenters/) > or with the CLI command: `exo zones list`. Add your Exoscale API key and secret. Then click **Connect**. ![](cyberduck-step2.png "Cyberduck Connection Window") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/duplicati/ --- # Back up with Duplicati This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. ## Overview Duplicati was designed for online backups from scratch. Not only is Duplicati data efficient, but it also handles network issues nicely. The backup client allows interrupted backups to resume, and regularly tests the content of backups. This allows an early detection of broken backups on corrupt storage systems. A graphical user interface with a wizard-style interface as well as a command-line version for usage in headless environments is provided by Duplicati. Both interfaces use the same core, so they have the same set of features and capabilities. This step-by-step guide will help you to get started with Duplicati on Exoscale. Duplicati is compatible with Linux as well as Windows. ## Setup ### Linux / Ubuntu 1. Create an Exoscale SOS bucket to store your backups. 2. Create an IAM Key to have access to your Exoscale SOS bucket. 3. Install the necessary packages on Ubuntu (see the [Duplicati documentation](https://duplicati.readthedocs.io/en/latest/02-installation/) for details). For this guide, we use the following commands: ```bash wget https://updates.duplicati.com/beta/duplicati_2.0.6.3-1_all.deb apt install ./duplicati_2.0.6.3-1_all.deb ``` Change `/etc/default/duplicati` to add the `webservice` part, and make sure to restrict access to it in your Security Group settings. See our [Security Groups section](https://community.exoscale.com/product/networking/security-group/index.md) for more details. ```bash cat /etc/default/duplicati ``` Output: ```bash # Defaults for duplicati initscript # sourced by /etc/init.d/duplicati # installed at /etc/default/duplicati by the maintainer scripts # # This is a POSIX shell fragment # # Additional options that are passed to the Daemon. DAEMON_OPTS="--webservice-interface=any --webservice-port=8200 --portable-mode" ``` Enable and start the Duplicati service. ```bash vi /etc/default/duplicati systemctl enable duplicati.service ``` Output: ```bash Created symlink /etc/systemd/system/multi-user.target.wants/duplicati.service → /lib/systemd/system/duplicati.service. ``` ```bash systemctl daemon-reload systemctl start duplicati.service systemctl status duplicati.service ``` Output: ```bash duplicati.service - Duplicati web-server Loaded: loaded (/lib/systemd/system/duplicati.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2022-06-01 10:46:52 UTC; 4s ago Main PID: 14187 (mono) Tasks: 16 (limit: 4672) Memory: 33.3M CGroup: /system.slice/duplicati.service ├─14187 DuplicatiServer /usr/lib/duplicati/Duplicati.Server.exe --webservice-interface=any --webservice-port=8200 --portable-mode └─14207 /usr/bin/mono-sgen /usr/lib/duplicati/Duplicati.Server.exe --webservice-interface=any --webservice-port=8200 --portable-mode Jun 01 10:46:52 vm2backup systemd[1]: Started Duplicati web-server. ``` 4. Connect to the Duplicati web interface to configure your backup. ![](duplicati-setup-1.png "Webinterface") ![](duplicati-setup-2.png "Configure new backup") ![](duplicati-setup-3.png "Set name and security") **Change to S3 for destination** ![](duplicati-setup-4.png "S3 Destination") In this example, we are using a bucket in BG-SOF-1 to store our backups. ![](duplicati-setup-5.png "Name bucket") Select **NO**. ![](duplicati-setup-6.png "Decline automatic naming") Select **YES**. ![](duplicati-setup-7.png "Create Folder") > [!NOTE] > If you run backups from multiple instances to the same bucket, > please create separate folders for each instance (such as **vmbackup/"instancename"**) ![](duplicati-setup-8.png "Destination set") Select the folders and files which you want to back up. ![](duplicati-setup-9.png "Source data") Define a schedule. ![](duplicati-setup-10.png "Define a scheduele") Choose your retention. ![](duplicati-setup-11.png "Options") Confirm that the passphrase will be safely stored. ![](duplicati-setup-12.png "Store passphrase safely") Click **Run now** to start the backup immediately. ![](duplicati-setup-13.png "Start backup") Duplicati starts the backup to your bucket. ![](duplicati-setup-14.png "Progress") ## Restore files from the backup Choose the restore source. ![](duplicati-setup-15.png "Choose restore source") Select files to restore. ![](duplicati-setup-16.png "Restore - Select files") Set restore options. ![](duplicati-setup-17.png "Restore - Restore options") The restore starts. ![](duplicati-setup-18.png "Restore starts") **Check restore on vm2backup** ```bash cd /tmp/restore/ ls -ltr ``` Output: ```bash total 15528 -rw-r--r-- 1 root root 15898608 Jun 17 2021 duplicati_2.0.6.3-1_all.deb root@vm2backup:/tmp/restore# ``` ## Additional resources [Duplicati User's Manual](https://duplicati.readthedocs.io/en/latest/) --- url: https://community.exoscale.com/product/storage/object-storage/recipes/duplicity/ --- # Back up with Duplicity This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. Duplicity produces encrypted .TAR format volumes and uploads them to a remote or local file server to backs up directories. By using librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. These archives are encrypted with GnuPG to protect against any spying or modification by the server. ## Provider Details ### Setup In this example, we test an installation on Ubuntu 20.04. 1. Download the latest [stable version of Duplicity](https://duplicity.gitlab.io/) from their website. 2. Install the following packages: ```bash apt-get install gettext python3-future python3-fasteners python3-boto3 ``` 3. Run the installer to configure Duplicity. ```bash duplicity-0.8.23] # python setup.py install ``` 4. Create an Exoscale SOS bucket. 5. Create specific IAM Keys for the bucket. - By default, all backups are encrypted using GnuPG. - We will use `${HOSTNAME} to enable backups of multiple instances to the same bucket. 6. Run the first backup. ```bash export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= export PASSPHRASE=SECUREPASSWORD duplicity /var s3://duplicity-backup/${HOSTNAME}/var --s3-endpoint-url=https://sos-de-fra-1.exo.io ``` 7. Run a full backup from the instance. ```bash duplicity --exclude /mnt --exclude /tmp --exclude /proc --exclude /run --exclude /sys / s3://duplicity-backup/${HOSTNAME} --s3-endpoint-url=https://sos-de-fra-1.exo.io ``` Output: ```bash Local and Remote metadata are synchronized, no sync needed. Last full backup date: none No signatures found, switching to full backup. --------------[ Backup Statistics ]-------------- StartTime 1655462673.30 (Fri Jun 17 10:44:33 2022) EndTime 1655463010.13 (Fri Jun 17 10:50:10 2022) ElapsedTime 336.83 (5 minutes 36.83 seconds) SourceFiles 171645 SourceFileSize 5369568365 (5.00 GB) NewFiles 171645 NewFileSize 5369568365 (5.00 GB) DeletedFiles 0 ChangedFiles 0 ChangedFileSize 0 (0 bytes) ChangedDeltaSize 0 (0 bytes) DeltaEntries 171645 RawDeltaSize 5275730169 (4.91 GB) TotalDestinationSizeChange 2157358512 (2.01 GB) Errors 0 ``` ### List Files in Backup target** ```bash duplicity list-current-files s3://duplicity-backup/${HOSTNAME} --s3-endpoint-url=https://sos-de-fra-1.exo.io ``` ### Restore a file from the backup ```bash duplicity restore --file-to-restore var/log/kern.log s3://duplicity-backup/${HOSTNAME} /tmp/kern.log --s3-endpoint-url=https://sos-de-fra-1.exo.io ``` ## Tips and Tricks To remove backups that are older than 1 month: ```bash duplicity remove-older-than 1M s3://duplicity-backup/${HOSTNAME} --s3-endpoint-url=https://sos-de-fra-1.exo.io ``` Here is simple backup script that you can modify as needed: ```bash #!/usr/bin/bash if [ -f /root/.duplicity.env ] then . ./.duplicity.env else echo "/root/.duplicity.env not found with needed environment" exit 1 fi /usr/local/bin/duplicity cleanup --force s3://${BACKUP_BUCKET_NAME}/${HOSTNAME} --s3-endpoint-url=${S3_ENDPOINT} /usr/local/bin/duplicity remove-older-than --force ${BACKUP_RETENTION} s3://${BACKUP_BUCKET_NAME}/${HOSTNAME} --s3-endpoint-url=${S3_ENDPOINT} /usr/local/bin/duplicity --full-if-older-than ${FULL_EVERY} --exclude /mnt --exclude /tmp --exclude /proc --exclude /run --exclude /sys / s3://${BACKUP_BUCKET_NAME}/${HOST NAME} --s3-endpoint-url=${S3_ENDPOINT} .duplicity.env File export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= export PASSPHRASE= export BACKUP_BUCKET_NAME= export BACKUP_RETENTION=1M export S3_ENDPOINT=https://sos-de-fra-1.exo.io export FULL_EVERY=7D ``` For this guide, we are using a bucket in DE-FRA-1, or Frankfurt. You can find all zone endpoints in our [API SOS guide](https://community.exoscale.com/api/sos/). ## Additional Resources [Duplicity Documentation](https://duplicity.gitlab.io/docs.html) --- url: https://community.exoscale.com/product/storage/object-storage/recipes/hycu-backup/ --- # HYCU Backup This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. The backup software HYCU is popular in the enterprise area. Amazon S3-compatible object storage can be used as long-term storage. Exoscale offers such S3-compatible storage, which is perfect for HYCU. This guide shows you how to set up HYCU to use the Exoscale Simple Object Storage. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Add an S3 Bucket as a Backup Target Log into the HYCU portal: ![](hycu-backup-step1.png "Overview of the HYCU Dashboard") Click on *Targets* in the main navigation. In the Targets view, click *Add*. ![](hycu-backup-step2.png "Click on Targets") Enter a name. You can also enter an optional description. Select the **AWS S3/Compatible** type and enter the service endpoint (Such as https://sos-de-muc-1.exo.io). Then enter the bucket name and login information. There are additional parameters which can be adjusted, such as the size (or quota), compression, or encryption before you upload. ![](hycu-backup-step4.png "Add Target: Properties") Depending on where the bucket has been created, you will have to enter the appropriate service endpoint. The endpoint is `https://sos-ZONE.exo.io`. Make sure to substitute `ZONE` with the relevant zone. > [!NOTE] > You can find all Exoscale zones on > [our website](https://www.exoscale.com/datacenters/) > or with the CLI command: `exo zones list`. --- url: https://community.exoscale.com/product/storage/object-storage/recipes/mountainduck/ --- # Mountain Duck This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. [Mountain Duck](https://mountainduck.io/) is commercial software you can use to mount Simple Object Storage on Windows and Mac as a storage extension to their local machine. This guide shows you how to set up Mountain Duck for Exoscale. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Set Up Mountain Duck Open the Mountain Duck app. Select **Amazon S3**. For the endpoint, enter `sos-ZONE.exo.io`. Make sure to substitute `ZONE` with the relevant zone. > [!NOTE] > You can find all Exoscale zones on > [our website](https://www.exoscale.com/datacenters/) > or with the CLI command: `exo zones list`. Enter your credentials (the IAM key you have previously created). Then click **Connect**. ![](mountainduck-step1.png "Connection Properties") Exoscale Simple Object Storage is now visible in the Explorer as a drive. ![](mountainduck-step2.png "Windows drives") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/qnap/ --- # Back up a QNAP NAS This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. QNAP NAS systems are some of the most user-friendly storage systems for private people and small-to-medium businesses. These systems come with a large number of applications and can upload backups to an Amazon S3-compatible storage system, including Simple Object Storage. This guide shows how to set up Hybrid Backup Sync 3 to upload backups to SOS. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Install Hybrid Backup Sync Open the **App Center** and install **HBS 3 Hybrid Backup Sync**. ![](qnap-step3.5.jpg "Search for Hybrid Sync in the App Center") ## Set Up Hybrid Backup Sync Open the main menu and start Hybrid Backup Sync, then open the **Storage Spaces** menu. Select **Amazon S3 & S3 Compatible** as a storage medium. ![](qnap-step4.jpg "Choice of Storage Space provider") Complete the following fields: - Name: Enter the storage name. - Service Provider: Select **S3 Compatible**. - Server address: Enter `https://sos-ZONE.exo.io`. Make sure to substitute `ZONE` with the relevant zone. - Access Key: Please enter the `Key` from the Portal. - Secret Key: Please enter the `Secret` from the Portal. > [!NOTE] You can find all Exoscale zones on [our website](https://www.exoscale.com/datacenters/) or with the CLI command: `exo zones list`. Make sure that **Use SSL connection** remains enabled to ensure that the connection to Exoscale is encrypted (see the screenshots below). ![](qnap-step5.jpg "Creation of Storage Space") Create the backup and select the target **Amazon S3 & S3 compatible**. Select the target you just created. ![](qnap-step6.jpg "Storage Space Backup selection") Select the bucket from the dropdown menu. Then select a dedicated folder for your backups. ![](qnap-step7.jpg "Storage Space Backup Properties") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/restic/ --- # Back up with Restic This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. ## Overview Restic is a simple and efficient backup program you can use to easily back up your data to an Exoscale Simple Object Storage bucket. Restic is available for all official Exoscale Templates from Linux, Windows to BSD. Benefits Include: - One bucket for multiple instances - Windows VSS Support - One binary without any dependencies However, Restic comes with some limitations: - No bare-metal restore is possible - No central management is possible - No scheduler is included ## Provider Details ## Setup 1. Create an S3 Bucket on Exoscale. Restic can use Exoscale SOS as backend to store your backups from multiple instances. The [SOS Quick Start Guide](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) gives you more details for using s3cmd or the Exoscale Portal to create a bucket. 2. Create an IAM Key which will give Restic access to the S3 Bucket. We recommend you create a restricted key that only allows access to the bucket. Find more details in our [IAM Quick Start Guide](https://community.exoscale.com/product/security/iam/quick-start/index.md). 3. Get Restic either from the [official Restic website](https://restic.net/) or directly from [Github](https://github.com/restic/restic/releases/latest) and install or copy it to your instance. 4. Initialize your Exoscale SOS Bucket with Restic. You need to define a password for your Restic repository. Make sure to **store your password safely**. If you lose it, you will not be able to access data stored in the repository. ### Linux export AWS_ACCESS_KEY_ID= export AWS_SECRET_ACCESS_KEY= restic -r s3:sos-.exo.io/ init ### Windows set AWS_ACCESS_KEY_ID= set AWS_SECRET_ACCESS_KEY= restic -r s3:sos-.exo.io/ init Output on Linux (Windows also shows a similar result): [root@vm2backup ~]# export AWS_ACCESS_KEY_ID=EXO35ac0e9ba39cf65XXXXXXXXX [root@vm2backup ~]# export AWS_SECRET_ACCESS_KEY=SECRETKEY-XXXXXX [root@vm2backup ~]# restic -r s3:sos-de-fra-1.exo.io/restic-de-fra-1 init enter password for new repository: enter password again: created restic repository a94a0d041a at s3:sos-de-fra-1.exo.io/restic-de-fra-1 Please note that knowledge of your password is required to access the repository. Losing your password means that your data is irrecoverably lost. 5. Run some test backups to check that everything is functional: [root@vm2backup ~]# restic -r s3:sos-de-fra-1.exo.io/restic-de-fra-1 backup /tmp enter password for repository: repository 296fefeb opened successfully, password is correct no parent snapshot found, will read all files Files: 122 new, 0 changed, 0 unmodified Dirs: 209 new, 0 changed, 0 unmodified Added to the repo: 466 B processed 122 files, 1.791 GiB in 0:06 snapshot 6557d88b saved 6. List all backups (in Restic they are called Snapshots): [root@vm2backup restic]# restic -r s3:sos-de-fra-1.exo.io/restic-de-fra-1 snapshots enter password for repository: repository 296fefeb opened successfully, password is correct ID Time Host Tags Paths ------------------------------------------------------------------------------------------------------------- 6557d88b 2022-05-30 12:46:40 vm2backup /tmp 4e188111 2022-05-30 12:50:03 vm2backup /tmp ------------------------------------------------------------------------------------------------------------- 2 snapshots ## Restore files In this example, we will restore the whole /tmp from the previous test to /tmp/restore-tmp using the Snapshot ID: [root@vm2backup ~]# restic -r s3:sos-de-fra-1.exo.io/restic-de-fra-1 restore 6557d88b --target /tmp/restore-tmp enter password for repository: repository 296fefeb opened successfully, password is correct restoring to / tmp/restore-tmp 1. As mentioned in the [first step of the setup](#setup), you can backup multiple instances to the same SOS bucket: C:\Users\Administrator\Downloads>restic backup -r s3:sos-de-fra-1.exo.io/restic-de-fra-1 C:\users\Administrator\Downloads repository 296fefeb opened successfully, password is correct using parent snapshot 0dc43c19 Files: 0 new, 0 changed, 112 unmodified Dirs: 0 new, 0 changed, 9 unmodified Added to the repo: 0 B processed 112 files, 365.439 MiB in 0:22 snapshot 6b3f4585 saved 3. Listing Restic Snapshots again, you can see multiple hosts: C:\Users\Administrator\Downloads>restic snapshots -r s3:sos-de-fra-1.exo.io/restic-de-fra-1 repository 296fefeb opened successfully, password is correct ID Time Host Tags Paths --------------------------------------------------------------------- 6557d88b 2022-05-30 12:46:40 vm2backup /tmp 4e188111 2022-05-30 12:50:03 vm2backup /tmp 6b3f4585 2022-05-30 14:05:49 win2backup C:\users\Administrator\Downloads --------------------------------------------------------------------- 3 snapshots You can use the following snippets as a baseline to run backups on Linux or Windows instances. Please change the values for your Access Key, Secret Key, Restic Repository and your Restic Password. Your password can also be retrieved from an external program using “--password-command” as an option. ### Linux [root@vm2backup restic]# more .restic_env export AWS_ACCESS_KEY_ID="EXO35ac0e9ba39cf65XXXXXXXXX” export AWS_SECRET_ACCESS_KEY="SECRETKEY-XXXXXX" export RESTIC_REPOSITORY="s3:sos-.exo.io/” export RESTIC_PASSWORD="YourVerySecurePassword" [root@vm-9e7977cd-755e-4479-9605-0c2f80fd5254 restic]# more backup.sh #!/usr/bin/bash if [ -f /root/restic/.restic_env ] then . /root/restic/.restic_env else echo "No Restic environment found" exit 1 fi /usr/bin/restic backup -q / --exclude="/proc" --exclude="/sys" ### Windows C:\Users\Administrator\Downloads>more backup.cmd @echo off set AWS_ACCESS_KEY_ID=EXO35ac0e9ba39cf65XXXXXXXXX set AWS_SECRET_ACCESS_KEY=SECRETKEY-XXXXXX set RESTIC_REPOSITORY=s3:sos-.exo.io/ set RESTIC_PASSWORD= YourVerySecurePassword C:\windows\system32\restic backup C:\users\Administrator\Downloads **Restic Housekeeping** Restic includes a housekeeping function to keep only the needed snapshots as defined in the guideline. restic forget --prune --keep-hourly 6 --keep-daily 7 --keep-weekly 4 --keep-monthly 12 In this example, Restic will keep: - 6 hourly Snaphosts per Host - 7 daily Snapshots per Host - 4 weekly Snapshots per Host - 12 monthly Snapshots per Host and will remove all other unused data This command needs to run on one instance only, with a regular (daily) interval. ## Additional Resources See the [Restic documentation](https://restic.readthedocs.io/en/latest/). --- url: https://community.exoscale.com/product/storage/object-storage/recipes/s3fs/ --- # Mount a bucket on Linux with s3fs This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. Linux brings the advantage of having a large number of open source solutions to access the Simple Object Storage. One of these is S3FS. > [!WARNING] > The Object Storage is not a disk drive! Using it as a storage backend with > a database or similar from multiple machines may have unintended consequences. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Installation S3FS is supplied as a package in most Linux distributions, which makes the installation simple. Execute the following commands on a root console. > [!NOTE] S3FS could not show the file sizes before version 1.84. If you see only 0-byte files, update your S3FS version. ### Ubuntu Linux 16.04 or higher, Debian Linux 9 or higher ```bash apt install s3fs ``` ### RHEL 7 or newer, using EPEL: ```bash yum install epel-release yum install s3fs-fuse ``` ### SUSE 12 and openSUSE 42.1 or newer: ```bash zypper install s3fs ``` ## Storing your access key Save your access key as the root user: ```bash echo API-KEY-HERE:API-SECRET-HERE > /root/.passwd-s3fs ``` Change its permissions: ```bash chmod 0600 /root/.passwd-s3fs ``` ## Create a Folder To mount the bucket, create an empty folder. This can be located in `/mnt/sos`: ```bash mkdir /mnt/sos ``` ## Mount by Hand Before we add the mount point permanently, you can try it by hand: ```bash s3fs BUCKET-NAME-HERE /TARGET/DIRECTORY -o use_path_request_style -o url=https://sos-EXOSCALE-ZONE-ID.exo.io/ ``` For example: ```bash s3fs my-testbucket /mnt/sos -o use_path_request_style -o url=https://sos-at-vie-1.exo.io/ ``` ## Automated Mount To mount the object storage automatically at system start, edit the `/etc/fstab` file. > [!WARNING] > This file is required for a correct system start. If you make a mistake, your system may not boot. > Make sure to read up on the restore procedure and create a backup (such as a snapshot) before you proceed. Insert the following line followed by a new line character: ```bash s3fs#BUCKET-NAME /TARGET/DIRECTORY fuse _netdev,allow_other,use_path_request_style,url=https://sos-EXOSCALE-ZONE-ID.exo.io/ 0 0 ``` For example: ```bash s3fs#my-testbucket /mnt/sos fuse _netdev,allow_other,use_path_request_style,url=https://sos-at-vie-1.exo.io/ 0 0 ``` Now you can mount the folder. ```bash mount -a ``` --- url: https://community.exoscale.com/product/storage/object-storage/recipes/sme/ --- # Storage Made Easy (SME) File Fabric This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. The File Fabric data management platform unifies file and object data into a single, easily managed infrastructure. With the File Fabric end users can access Exoscale object storage as they would any network filesystem with desktops and mobile devices. The File Fabric platform is available as hosted service or as an enterprise software appliance that can be deployed to Exoscale servers. See [Exoscale Marketplace - Enterprise File Fabric](https://www.exoscale.com/marketplace/) for more information. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. For access to a File Fabric account, you can launch your own instance from Exoscale (sign up for an Enterprise File Fabric trial) or sign up for our SaaS. ## Set Up File Fabric Choose to add the Open S3 provider to File Fabric, either as a first step on activation of your account, or later from the **Dashboard** which is accessible from the top menu. ![](sme-choose-provider.png "Choose a provider") The next step grants File Fabric access to your Exoscale Object Storage. To do this, you need to enter a zone-specific endpoint, your API Key and a secret. Use an endpoint of the format substituting ZONE with the zone your buckets are located in. For example, [https://sos-ch-dk-2.exo.io](https://sos-ch-dk-2.exo.io). Use the API key as an Access Key and the secret that you created from the portal. ![](sme-exoscale-opens3-addprovider.png "Add OpenS3 provider") After you enter your authentication details, File Fabric will list all Exoscale buckets visible for the account. Choose which buckets you wish to manage through the File Fabric platform, that are in the zone you selected. (You can choose any bucket as a default – it can be changed later). ![](sme-choose-buckets.png "Choose buckets") If you wish to manage buckets in other zones, follow the steps above using a different endpoint, with the applicable zone information. As your data is indexed, it will be accessible through the Cloud File Manager. ![](sme-exoscale-filemanager.png "Illustration of exoscale filemanager") ## Additional Resources For more information see [Adding an S3 Compatible Cloud Provider](https://docs.storagemadeeasy.com/cloudproviders/s3compatible) and [Getting Started with Exoscale](https://docs.storagemadeeasy.com/exoscale-gettingstarted) on the File Fabric documentation. --- url: https://community.exoscale.com/product/storage/object-storage/recipes/synology-backup/ --- # Back up a Synology NAS This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. The Synology NAS is a very user-friendly storage system for private persons and small to medium businesses. It comes with a large number of applications and can create backups to an Amazon S3-compatible storage like Exoscale Simple Object Storage. This guide shows how to set up backups to SOS with the Hyper Backup application. This method creates a versioned backup of a Synology NAS to the Exoscale Object Storage. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Install Hyper Backup Open the **Package Center**. ![](synology-backup-step1.png "Package Center Icon") Search for **Hyper Backup (produced by Synology Inc.)** and install it. ![](synology-backup-step2.png "Hyper Backup Package") ## Set Up Hyper Backup Open the main menu and start Hyper Backup: ![](synology-backup-step3.png "Main Overview") Follow now the description in the following images. In the section with the target destination selection, select **Custom Server URL** and enter `https://sos-ZONE.exo.io` in the server field. Make sure to substitute `ZONE` with the relevant zone. > [!NOTE] You can find all Exoscale zones on [our website](https://www.exoscale.com/datacenters/) or with the CLI command: `exo zones list`. Enter the API key and secret, as well as the bucket name. ![](synology-backup-step4.png "Target destination selection: Select S3") ![](synology-backup-step5.png "Backup-destination properties") ![](synology-backup-step6.png "Select the source folder for the backup") ![](synology-backup-step7.png "Select the applications, you want to backup") ![](synology-backup-step8.png "Select the backup schedule") ![](synology-backup-step9.png "Configure a retention policy") ![](synology-backup-step10.png "Done!") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/synology-sync/ --- # Synchronize a Synology NAS This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. The Synology NAS is a very user-friendly storage system for private persons and small to medium businesses. It comes with a large number of applications and can synchronize with an Amazon S3-compatible storage, like Exoscale Simple Object Storage. This guide shows how to set synchronization with Exoscale SOS with the Cloud Sync application. > [!NOTE] This method synchronizes the Synology NAS with Exoscale SOS. It does not create a versioned backup. If you want to create a versioned backup, see the Hyper Backup guide. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Install Cloud Sync Open the **Package Center**. ![](synology-sync-step1.png "Package Center Icon") Search for **Hyper Backup (produced by Synology Inc.)** and install it. ![](synology-sync-step2.png "Cloud Sync package") ## Set up Cloud Sync Open the main menu and start Cloud Sync. Then follow the screenshots below. When asked for account settings, select **Customer Server URL** and enter `sos-ZONE.exo.io` in the server field. Make sure to substitute `ZONE` with the relevant zone. > [!NOTE] You can find all Zones on [our website](https://www.exoscale.com/datacenters/) and with the cli: `exo zones list` Also enter your IAM Key, secret and bucket name. ![](synology-sync-step3.png "Main Overview") ![](synology-sync-step4.png "Click on +") ![](synology-sync-step5.png "Select S3 storage") ![](synology-sync-step6.png "Account setting") ![](synology-sync-step7.png "Select the folder that you want to synchronize") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/urbackup/ --- # Back up with UrBackup This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. ## urbackup-Infscape Backup Guide This guide can be used to deploy a Infscape Backup appliance on Exoscale and configure SOS as an S3 backup endpoint. You can access the [full UrBackup documentation](https://www.urbackup.org/admin_guide_app.pdf) for additional information. ## Installation Follow the [UrBackup guide](https://www.infscape.com/urbackup-appliance-download/) to install a cloud instance. ### Prepare the Security Group The Infscape instance requires the following ports to be open in the security-group: * TCP/22 for SSH access (only required during installation, can be removed later) * TCP/80 for HTTP * TCP/443 for https ### Prepare the Compute instance Create a new instance with the following settings: * Debian 10 Bullseye as template * At least 4 GB RAM (a medium or higher instance required) * At least 50 GB disk (required for local caching of backup data chunks before uploading to SOS) ### Prepare the SOS Bucket and API Key Create a SOS bucket in your preferred zone. Note that traffic between [Exoscale zones](https://www.exoscale.com/datacenters/) is free. You can set different zones for the bucket and for the Infscape appliance to achieve geo-redundancy with your backups. * Create a bucket with a unique name. * Create an API key restricted to the service SOS and the created bucket. Write down the API key and secret. ![](urbackup-infscape_apikey.png "API Key") ### Install Infscape * SSH into the created Debian system. * Initiate the installation process: ```bash wget https://dl3.infscape.com/images/urbackup-app-10-13.sh && bash urbackup-app-10-13.sh ``` * Agree to the installation. A password will be shown in the terminal. Make sure to save the password, as it will be used to access the web management interface. ![](urbackup-infscape_install.png "Installation") * After a short time, the appliance will reboot itself. Due to a small bug, we need to reboot it again: in the instance detail view in the Portal, click on the `...` more actions button and then click on `Reboot`. This is how the console looks before the appliance is rebooted: ![](urbackup-infscape_nonetwork.png "Before reboot") * You can use the console to validate the appliance has the public IP configured. ![](urbackup-infscape_network.png "Network settings") ## Configure Infscape and the License * Open a browser and navigate to the public IP of the Infscape appliance using HTTP. For example, . * Provide the password generated during installation process ![](urbackup-infscape_firstlogin.png "First Login") * If you do not yet own a license key, login at: [infscape.com](https://www.infscape.com/). If you just want to start, select a free Community Edition license key. * If you do not have a UrBackup account, register with a valid email address and password. This account will be used to access this instance's web interface. You will also receive event-based email notifications (for example, if your Infscape appliance is offline). * Provide the appliance name and the license key. * Use the public IP or a DNS pointing to the public IP for clients to connect. ![](urbackup-infscape_register.png "Infscape Register") ## Configure HTTPS (Optional but Highly Recommended) Note that you require a DNS entry to enable HTTPS. * Login with the new password to the appliance. * Navigate to **Settings**, then **System** and click on **Setup SSL/HTTPS**. ![](urbackup-infscape_setupssl.png "Setup SSL") * Provide the DNS name of the appliance. Then enable the redirect and accept the **Let's Encrypt Subscriber Agreement**. * Click on **Save** to enable HTTPS. The web interface will be unresponsive for about 30 seconds while HTTPS is configured. ![](urbackup-infscape_sslsettings.png "SSL settings") ## Configure Infscape Server Settings to use SSL for Backups (Optional but Highly Recommended) Note that SSL must be enabled. * Navigate to **Settings**, then **Internet**. * Set **Internet Server name/IP** to `<127.0.0.1>`. * Let "**Internet Server port** stay on the preconfigured value of `55415`. * Set **Connect via HTTP(S) proxy** to **your DNS entry**. ![](urbackup-infscape_serversettings.png "Server settings") ### Configure Cloud Storage * Navigate to the **Status** page. * Click on **Use system disk as cache and setup cloud storage** (a large blue button). ![](urbackup-infscape_setupsos.png "Setup SOS") * Agree and enter the cloud storage information. ![](urbackup-infscape_detailsos.png "Details SOS") * **Note**: Write down the cloud storage encryption key and store it safely. This key is **mandatory to decrypt your backups** if the appliance is lost. * Click on **Select/Confirm cloud storage** to finish. * Navigate to **Settings**, then **Storage** to validate that the cloud storage is used. ![](urbackup-infscape_verifysos.png "Verify SOS") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/veeam/ --- # Veeam Backup This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. Veeam is predominantly used in the enterprise segment as a backup software. Amazon S3-compatible object storages are often used for long term storage for backups. Exoscale offers an S3-compatible object storage that works with Veeam. This guide shows how to set up Veeam for backups. ## Prerequisites Before you start this guide, you will need to: - Create a SOS Bucket, either via the [Portal](https://portal.exoscale.com/) or the [CLI](https://community.exoscale.com/tools/command-line-interface/index.md). See the [Simple Object Storage documentation](https://community.exoscale.com/product/storage/object-storage/quick-start/index.md) for more information. - Create an IAM key, also via the Portal or the CLI. See the [IAM documentation](https://community.exoscale.com/product/security/iam/quick-start/index.md) for more information. ## Create a Scale-Out Repository If you have already set up your Veeam backups, click **Scale Out Repositories** on the right side of the top menu, then **Add Scale-out Backup Repository**. For a service endpoint, enter `https://sos-ZONE.exo.io`. Make sure to substitute `ZONE` with the relevant zone. > [!NOTE] You can find all Exoscale zones on [our website](https://www.exoscale.com/datacenters/) or with the CLI command: `exo zones list`. ![](veeam-step1.png "Select this, if Veeam backups were already set up") ![](veeam-step2.png "Wizard: Enter a repository name and click next") ![](veeam-step3.png "Wizard: Performance Tier, click on next") ![](veeam-step4.png "Placement Policy: Select Data locality") ![](veeam-step5.png "Capacity Tier: Click on Add to add a new Object Storage") ![](veeam-step6.png "Add: Select S3 compatible") ![](veeam-step7.png "Object Storage: Choose a name") ![](veeam-step8.png "Add an Service Endpoint") ![](veeam-step9.png "IAM: Insert your Access Key and Secret. Click OK then") ![](veeam-step10.png "Account Overview: Check the inserted data and continue") ![](veeam-step11.png "Select the previously created bucket") ![](veeam-step12.png "Select the specific target") ![](veeam-step13.png "New Object Storage: Summary") ![](veeam-step14.png "Please select after how many days the backups should be moved to the object storage") ![](veeam-step15.png "Done!") --- url: https://community.exoscale.com/product/storage/object-storage/recipes/veeam-12-immu/ --- # Immutable backups with Veeam Backup This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. Exoscale SOS with object lock can be used with software products from Veeam to create an immutable, secure and safe backup environment. This can be used to substitute WORM off-site backup architectures as well as to actively store backups from clients, mobile devices, O365 and workloads running on Exoscale. ## Prerequisites As prerequisites you'll have to: * access to Exoscale environment * an Exoscale API key (unrestricted or restricted to storage) * [s3cmd](https://s3tools.org/s3cmd) installed Available Exoscale zones can be found [here](https://www.exoscale.com/datacenters/). and have a mapping like below: ```bash Geneva, Switzerland: ch-gva-2 Zurich, Switzerland: ch-dk-2 Vienna 1, Austria: at-vie-1 Vienna 2, Austria: at-vie-2 Frankfurt, Germany: de-fra-1 Munich, Germany: de-muc-1 Sofia, Bulgaria: bg-sof-1 Zagreb, Croatia: hr-zag-1 ``` ## Create SOS bucket with object lock enabled Use s3cmd to create a bucket with object lock: ```bash s3cmd --add-header=x-amz-bucket-object-lock-enabled:true mb s3:// --access_key= --secret_key= --host=sos-.exo.io --region= --host-bucket=.sos-.exo.io ``` An example with Frankfurt as zone and veeam-immutable-backup as bucket name: ```bash s3cmd --add-header=x-amz-bucket-object-lock-enabled:true mb s3://veeam-immutable-backup --access_key=EXO0123456789XXXX --secret_key=XXXX --host=sos-de-fra-1.exo.io --region=de-fra-1 --host-bucket=veeam-immutable-backup.sos-de-fra-1.exo.io ``` Alternatively you can spawn a default virtual compute (just leave all settings as is) and use the cloud-init to let it create your bucket. Once that bucket is created you can delete the virtual compute and the API key used to create the bucket. To do so copy and paste the following script into the "User data" field on the virtual compute creation page: ```bash #!/bin/bash key= secret= zone= bucket= apt install s3cmd -y s3cmd --add-header=x-amz-bucket-object-lock-enabled:true mb s3://$bucket --access_key=$key --secret_key=$secret --host=sos-$zone.exo.io --region=$zone --host-bucket=$bucket.sos-$zone.exo.io ``` Check in the UI that the bucket is created and visible. If you used a virtual compute to create the bucket you can now delete the virtual compute and the API key used to create the bucket if it was a dedicated key # Use cases with Exoscale SOS and Veeam using Exoscale SOS and Veeam for different use cases ## Veeam 12 SOS backup repository creating Veeam 12 backup repository on Exoscale SOS to provide a valid endpoint for different use cases with Veeam ### Prerequisites * access to Veeam 12 Backup&Replication server * Exoscale SOS bucket with object lock enabled * API key for bucket to use (can be restricted to storage and the bucket to use) ### Configure exoscale SOS repository in Veeam * Open the Veeam backup and Replication console * Navigate to "Backup Infrastructure" * Navigate to "Backup Repositories" * Click on "Add Repository" * Select the last option "Object storage" * Select the first option "S3 Compatible * Give the repository a name * Enter "Service point", "Region" and use the API Key to create credentials for this bucket ![](veeam-12-1.png "Veeam service point region and Exoscale") * click on Browse and select the bucket * click on Browse and create a new folder for this process * check the "Make recent backups immutable for:" box and configure days backups should be immutable ![](veeam-12-2.png "Immutable Veeam 12 and Exoscale") * configure as you need and finish the process * The repository is visible in "Backup Repositories" and can now be used in Veeam 12 ![](veeam-12-3.png "Veeam 12 repository with Exoscale") ## Using Exoscale SOS repository as immutable Scale-out for existing performance tier ### Prerequisites * access to Veeam 12 Backup&Replication server * Configured backup repository with object lock enabled on Exoscale SOS ### Configuration * Open the Veeam backup and Replication console * Navigate to "Backup Infrastructure" * Navigate to "Scale-out Repositories" * Click on "Add Scale-out Repository" * Give the repository a name * Add the Performance Tier the backups are initially written to * Choose either "Data locality" or "Performance" mode * Check the "Extend scale-out backup repository capacity with object storage" * Click on "Choose..." and select the Exoscale SOS based repository * Configure when backups should be scaled out from the configured Performance Tier to the Scale-out repository * We highly recommend enabling encryption of the backups * Finish the process ## Using Exoscale SOS repository as Performance Tier to backup workloads running on Exoscale or move clients over the internet ### Prerequisites * access to Veeam 12 Backup&Replication server running on Exoscale with a minimum of 200GB disk * access to potential systems to be backed up * Configured backup repository with object lock enabled on Exoscale SOS ### Configuration * Open the Veeam backup and Replication console * Navigate to "Home" * Click on "Backup Job" * Select the job you want to create * Note: For Exoscale this is "Windows computer" or "Linux computer" * Configure the backup job as documented from Veeam 12 * Run the backups --- url: https://community.exoscale.com/product/storage/object-storage/recipes/plakar/ --- # Plakar Backup This guide is provided for informational purposes. Because it involves third party tooling we cannot guarantee things still work exactly as described here. [Plakar](https://www.plakar.io/) is an innovative backup solution. It has many extensions that allow you to create backups from various sources, store them in multiple locations, and restore them to different destinations. Under the hood, Plakar uses `Kloset`, a flexible storage layer that lets you store data locally, on S3-compatible services, or even through tools like rclone. Plakar is available on Linux, macOS, and FreeBSD/OpenBSD. We can install it following [these instructions](https://www.plakar.io/docs/v1.0.4/quickstart/#installing:installing-2). In this how-to, we'll start by creating a backup stored on the local filesystem. Next we'll detail how to store backups on [Exoscale SOS](https://www.exoscale.com/object-storage/), then we'll show the restore procedure. ## Creating our first local backup Before creating a backup, we need to define a Kloset store on your filesystem. The following commands create such a store in `$HOME/my-backups`. ```bash plakar at $HOME/my-backups create ``` > [!NOTE] > Plakar requires a strong passphrase for each new Kloset store you create. Make sure not to forget it. In the background, Kloset uses the following folder hierarchy to store the backup we'll create. ```bash $ tree $HOME/my-backups /Users/lucjuggery/my-backups ├── CONFIG ├── locks ├── packfiles │ ├── 00 │ ├── 01 │ ├── 02 │ ... │ ├── fe │ └── ff └── states ├── 00 ├── 01 ├── 02 ... ├── fe └── ff 516 directories, 1 file ``` Before we create our first backup. Let's first create a demo folder. ```bash mkdir $HOME/mydocuments echo "my first doc" > $HOME/mydocuments/doc-1.md echo "my second doc" > $HOME/mydocuments/doc-2.md ``` Then, we create a backup of this folder in our local store. ```bash plakar at $HOME/my-backups backup $HOME/mydocuments ``` > [!NOTE] > Before creating the backup, Plakar requires you to enter the passphrase defined during the store creation step. Under the hood, Plakar created encrypted files in our local store. ```bash $ find $HOME/my-backups -type f /Users/lucjuggery/my-backups/CONFIG /Users/lucjuggery/my-backups/packfiles/9e/9e469cc05c6e1f09b54136ef52632c930fc5e52d928a1b15909a8fecb15f436d /Users/lucjuggery/my-backups/packfiles/d0/d05fb8833b67c0db0673a684eb13393a24f86bf62d10400930012db8959b9220 /Users/lucjuggery/my-backups/packfiles/44/44171fddf3c94cc46687a2befee8a3e4391a729db223067e8f226d1e72361297 /Users/lucjuggery/my-backups/packfiles/3f/3f48ecbb8b6908cb277f2b1443902e040738cde4b156758d41184920260a7985 /Users/lucjuggery/my-backups/packfiles/ba/ba906f475abbd46a724ac266553594deb09548f4284ac9b8e1dc3a6e9cd24bf5 /Users/lucjuggery/my-backups/packfiles/b0/b01d01d1eb99e0459bbc79dc7bb68d821dbeda9db90bd2bce2b78fa1a503ba38 /Users/lucjuggery/my-backups/states/28/28d73b52795c92ef19a53a1cd7bd293486635c59f571da3d4541e47f851ec376 ``` Plakar has a nice UI that lets you browse backups and more, as we'll see below. ## Accessing the backup from the UI We can access Plakar's UI with the following command, which opens your web browser automatically. ```bash plakar at $HOME/my-backups ui ``` > [!NOTE] > This UI allows you to access the backup created in the Kloset store you specified after the `at` command. ![dashboard](./dashboard-local-backup.png) The dashboard gives an overview of the backups we've created so far. We currently only have one. Each backup is a snapshot we can browse and restore as needed. The screenshot below shows the content of `doc-1.md` existing in the snapshot we've created. ![snapshot](./snapshot.png) At the bottom left of the dashboard, we have information about the Kloset store in use: its location, `$HOME/my-backups` on the local filesystem, and the compression and encryption algorithms used. We'll now create a new snapshot. First, we change a file in our source folder and add a new one. ```bash echo "this is my first doc" > $HOME/mydocuments/doc-1.md echo "another document" > $HOME/mydocuments/doc-3.md ``` Next, we back up the folder using the same command. ```bash plakar at $HOME/my-backups backup $HOME/mydocuments ``` Then, the UI shows that a new snapshot was created. ![snapshot](./snapshot-2.png) In this section, we stored backups on the local filesystem. In a production environment, we would consider a more robust storage solution, such as S3-compatible object storage, which is the focus of the next section. ## Creating a backup in Exoscale SOS Plakar offers various integrations that let you specify additional locations for sources, destinations, and storage. ![integrations](./integrations.png) In this section, we'll focus on the `S3` integration, which allows us to store backups in [Exoscale SOS](https://www.exoscale.com/object-storage/), a S3-compatible object storage. Before we can install the `S3` integration, we need to sign in to Plakar using an email address or via GitHub. ![sign in](./signin.png) > [!NOTE] > Signing in to Plakar also gives you access to additional features, such as Alerting. Once we are signed in, we can install the `S3` integration. ![S3 integration](./integrations-s3.png) Before we can use this integration, we need to create our backup bucket in SOS and create access keys so Plakar can access it. First, from the [Exoscale portal](https://portal.exoscale.com/u/exodemos/storage/add), we create a new bucket named `my-backups`. ![Bucket creation](./bucket-creation.png) Next, we create a role `plakar-backup` with access to the `Object Storage` Service class. ![Role creation](./role-creation.png) Then, we create a pair of keys for that Role. ![Key creation](./keys-creation-1.png) ![Key creation](./keys-creation-2.png) We store these keys in the ACCESS_KEY and SECRET_ACCESS_KEY environment variables. > [!NOTE] > Those keys are temporary ones, they will be removed after this how-to is published. We can now create a new Kloset store using the location of our SOS bucket. First, we add this new store. > [!WARNING] > The following command includes credentials on the command line. Depending on your shell, it may be saved in your command history. Before running it, make sure to use your shell’s mechanism to prevent sensitive commands from being recorded. ```bash plakar store add MyBackups s3://sos-ch-gva-2.exo.io/my-backups access_key=$ACCESS_KEY secret_access_key=$SECRET_ACCESS_KEY use_tls=true ``` Next, we ask Plakar to initialize it, which requires us to specify a passphrase. ```bash plakar at @MyBackups create ``` Then, we create a backup of `$HOME/mydocuments` in this new store. ```bash plakar at @MyBackups backup $HOME/mydocuments ``` Our SOS bucket now has additional folders and files containing an encrypted backup of our local folder. ![SOS backup](./SOS-backup.png) Using the `ui` command, we can access this remote backup and browse its content as we've done previously for the local one. ```bash plakar at @MyBackups ui ``` ![Dashboard](./dashboard-sos-backup-1.png) ## Restoring a backup In the previous sections, we have created 3 backups of our local folder: - 2 on the local filesystem - 1 on the SOS bucket Let's now illustrate the restoration process by restoring our SOS backup into a local folder. First, we need the identifier of the snapshot to restore (0ca68a95) ![Dashboard](./dashboard-sos-backup-2.png) Next, we run the restoration command, indicating: - the Kloset store to use: our SOS bucket referenced by @MyBackups - the destination: `$HOME/restore` on the local filesystem - the snapshot identifier ```bash plakar at @MyBackups restore -to $HOME/restore 0ca68a95 ``` Then, we can verify the restoration was successful. ```bash $ find $HOME/restore /Users/lucjuggery/restore /Users/lucjuggery/restore/mydocuments /Users/lucjuggery/restore/mydocuments/doc-1.md /Users/lucjuggery/restore/mydocuments/doc-2.md /Users/lucjuggery/restore/mydocuments/doc-3.md ``` ## Key takeaways This how-to only scratches the surface of Plakar. It illustrates an everyday use case: backing up a local folder to an S3-compatible object storage service. Feel free to browse [the documentation](https://www.plakar.io/docs/main/) to get a more detailed view of Plakar's capabilities.