This guide demonstrate the basics of Exoscale’s Simple Object Storage (SOS).

With object storage, you can store, distribute and manage large quantities of files (or objects). Example uses range from simple backups to complex media delivery.

You can store almost anything in SOS: assets, backups, email attachments, videos, pictures and anything else you may imagine.

Your data will stay in the zone in which you store it. Exoscale will never move data on your behalf. For example, if you first upload data to Simple Object Storage in Geneva, then your data is kept in Switzerland, under Swiss privacy regulations. Whichever the initial location, it is replicated in that same location in at least three physical distinct copies to ensure maximum safety.

All the actions we will demonstrate through the command line can also be performed through our CLI or the Portal, although the real power of SOS is in the tools you script, and in the clients you may use in your application.

S3 Compatibility

SOS is S3 compatible, so it works with most object storage clients and provides library access from a host of languages. You can drop in from other services and you will not be locked in. Most of the time, a change of API credentials and endpoint allows you to switch to our service.

Here are a few clients and tools that you can use:

We will use s3cmd in this guide.

S3 Unsupported Features

While strive to maintain full compatibility with the S3 API, we do not support all S3 features. Currently, the following functionality cannot be used on Exoscale:

Setup

This guide assumes you never used s3cmd. If you have used s3md, take a look at the .s3cfg example below, drop in your Exoscale’s API Key and API Secret and you can use SOS.

First, install s3cmd with your favorite package manager:

  • Debian/Ubuntu Linux: apt-get update && apt-get install s3cmd
  • OS X: brew install s3cmd

Then create a file named .s3cfg in your home directory. Copy and paste the following configuration template:

[default]
host_base = sos-{zone}.exo.io
host_bucket = %(bucket)s.sos-{zone}.exo.io
access_key = $EXO_SOS_KEY
secret_key = $EXO_SOS_SECRET
use_https = True

Replace {zone} with the zone your bucket belongs to. For example, for a bucket deployed on the ch-dk-2 zone:

host_base = sos-ch-dk-2.exo.io
host_bucket = %(bucket)s.sos-ch-dk-2.exo.io

Replace $EXO_SOS_KEY and $EXO_SOS_SECRET with your API Key and API Secret. You can choose to use your User Keys or a specific IAM API key.

Your first Bucket

We need to create a container for all of our objects, which is called a bucket.

s3cmd mb s3://my-new-bucket

This creates a bucket called my-new-bucket.

Bucket names are unique across the entire platform. You may encounter an error creating my-new-bucket if the name is already taken.

Why are bucket names unique? Buckets and their content can be accessed under https://sos-ch-dk-2.exo.io/my-new-bucket/ or https://my-new-bucket.sos-ch-dk-2.exo.io. Buckets become subdomains and prefixed URLs under the sos-ch-dk-2.exo.io domain, so names have to be unique.

This does not mean objects are publicly accessible, however. You can define precise access control through IAM Keys or ACLs, and uploaded objects are private by default.

You can check your bucket has been created by listing your buckets:

s3cmd ls

Uploading and downloading assets

Now choose file on your computer and upload it to your bucket:

s3cmd put picture.jpg s3://my-new-bucket/picture.jpg

Check your file was uploaded successfully:

s3cmd ls s3://my-new-bucket

You can also verify the presence of your file in the Portal. Select the Storage section from the main navigation on the left. You should see your my-new-bucket listed here. Select the bucket and you will find your file.

You can try to upload more files in the Portal by drag-and-drop.

To get your files:

s3cmd get s3://my-new-bucket/picture.jpg picture_download.jpg

Advanced Options

Buckets and objects support advanced options:

Those options are available through the API, but also through the Portal. When you edit a bucket or an object, you can open a tab with the configurable options.

IAM

It is possible to limit programmatic access to a specific bucket by creating dedicated IAM API Keys and limit them to a single bucket.

ACL

With Access Control Lists (ACLs), you can define specific permissions at the bucket level and single object level for a granular access policy. ACLs are not inherited from parent objects.

ACLs let you manage access to buckets and objects with read, write, read ACP, write ACP, and full control (read + write) rights. You can make your objects accessible by their public URL, your buckets writable by your team, and so on.

For in-depth details about ACLs, you can look at the SOS ACL documentation.

CORS

With CORS (Cross-Origin Resource Sharing) your browser-based applications can perform actions on buckets and objects contained in a bucket. CORS is available only at the bucket level, and CORS configuration is applied to all the objects contained in a bucket.

The SOS CORS documentation has a breakdown and examples.

Metadata

Each object contained in a bucket can be tagged with key-value pairs. This metadata is presented through HTTP headers on SOS requests and responses.

You can read the SOS Metadata documentation for more information and examples.