How to migrate cloud provider with rclone
rclone is a command-line program to migrate between a large number of cloud storage providers. It can be used to migrate data from a local environment or other cloud providers to the Exoscale Object Storage.
rclone is available as a package in most Linux distributions and is also available for Windows and MacOS. Since it is a stand-alone executable it can simply be downloaded and started.
On Windows it is recommended to set the
PATH environment variable to point to the folder rclone is located in.
As prerequisite for the following documentation, you’ll have to:
Create an Object Storage Bucket, either via the portal or the CLI. You can refer to the Object Storage documentation
Create an IAM key, either via the portal or the CLI. You can refer to the IAM documentation
Before you can use rclone you will have to configure your accounts first. To do that please enter the following command:
# rclone config
You will be shown the following options:
n) New remote s) Set configuration password q) Quit config
n to set up your Exoscale Account and enter an account name. (e.g.
Exoscale) This name will be used to start
n) New remote s) Set configuration password q) Quit config n/s/q> n name> Exoscale
Next, you will be shown a list of supported cloud storage providers. Please select
Amazon S3 since the Exoscale Object Storage implements the Amazon S3 standard.
Next, you will be asked for the access key and secret source. Please enter
Choose a number from below, or type in your own value 1 / Enter AWS credentials in the next step \ "false" 2 / Get AWS credentials from the environment (env vars or IAM) \ "true"
Next, enter your access key and secret:
AWS Access Key ID - leave blank for anonymous access or runtime credentials. access_key_id> EXO... AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. secret_access_key> ...
Next, you will have to select the region:
/ If using an S3 clone that only understands v2 signatures 12 | eg Ceph/Dreamhost | set this and make sure you set the endpoint. \ "other-v2-signature"
other-v2-signature option. For the endpoint enter
sos-ZONE.exo.io, taking care of substituting
ZONE with the proper zone.
You can find all Zones on our website and with the cli:
exo zones list
Endpoint for S3 API. Leave blank if using AWS to use the default endpoint for the region. Specify if using an S3 clone such as Ceph. endpoint> sos-at-vie-1.exo.io
For the last four questions please enter the following:
- Location constraint:
Empty for US Region, Northern Virginia or Pacific Northwest.
- Canned ACL:
Owner gets FULL_CONTROL. No one else has access rights (default).
- Server-Side Encryption:
- Storage class:
Then save the configuration with
You will now have to configure the second provider.
Now the cloud providers are configured, so you can synchronize the files. To synchronize from a local folder to the Exoscale Object Storage enter the following command:
rclone sync /LOCAL/FOLDER Exoscale:BUCKET-NAME-HERE/
Exoscale please enter the name you have given your configuration. To synchronize from a second cloud provider you can use the following command:
rclone sync OtherProvider:SOURCE-DIRECTORY Exoscale:BUCKET-NAME-HERe/