Using SOS with AWS Mountpoint
As Exoscale’s Simple Object Storage (or SOS) is S3 compatible, all tools using the S3 API can be used with SOS. Mountpoint is an open source S3 client by AWS that you can use to access the objects in your bucket like files on a filesystem. The ideal use case is for read-heavy applications that require large amounts of data (e.g. data lakes, machine learning training data, ETL).
Installation
Install the current version of AWS Mountpoint for your environment. AWS offers precompiled binaries for different platforms here
Configuration
Create an Exoscale IAM api-key / secret with permissions to access your SOS bucket. Save it in your home directory in .aws/credentials
under a profile. For example exomount
:
[exomount]
aws_access_key_id = EXO1234567890
aws_secret_access_key = <your api-secret here>
Usage
Create a directory where you want to mount the bucket:
$ mkdir mnt
Mount the bucket with the endpoint of the zone where the bucket was created (for a complete list of SOS endpoints look here). In this example the bucket mybucket
was created in our Zurich zone ch-dk-2
:
$ mount-s3 --profile exomount --endpoint-url https://sos-ch-dk-2.exo.io mybucket mnt
Now you can copy files to the mounted bucket:
$ cp backup.tgz mnt
and list them:
$ ls -l mnt
total 1
-rw-r--r-- 1 exouser exouser 524288 Jul 7 11:23 backup.tgz
Read the contents:
$ md5sum mnt/backup.tgz
45e4be3ff8d14c966c61cca131f8b8e1 mnt/backup.tgz
If you try to delete the file you get an error:
$ rm mnt/backup.tgz
rm: cannot remove 'mnt/backup.tgz': Operation not permitted
By default, Mountpoint does not allow to delete files on a mounted bucket. If you want to do this you must add the option --allow-delete
when mounting the bucket.
If you no longer need the mounted bucket you can unmount it:
$ umount mnt
Limitations
SOS is an object store, and although Mountpoint helps bridge the gap by presenting SOS objects as files, it doesn’t create a POSIX-compliant filesystem. This means some filesystem operations might not behave as expected.
For example, it’s not possible to modify existing files (only overwrite) or rename directories and it does not support symbolic links or file locking. More details can be found in the Mountpoint documentation here.
AWS Mountpoint in combination with SOS does not support server-side encryption (SSE).