How to Migrate Cloud Providers with Rclone
Rclone is a command-line program to migrate between a large number of cloud storage providers. Rclone can be used to migrate data from a local environment or other cloud providers to Simple Object Storage.
Rclone is available as a package in most Linux distributions and is also available for Windows and MacOS. Since Rclone is a stand-alone executable, it can simply be downloaded and started.
On Windows, it is recommended to set the
PATH environment variable to point
to the folder Rclone is located in.
Before you start this guide, you will need to:
Create an IAM key, also via the Portal or the CLI. See the IAM documentation for more information.
Before you can use Rclone, you need to configure your accounts. Enter the following command:
# rclone config
You will be shown the following options:
n) New remote s) Set configuration password q) Quit config
n to set up your Exoscale account and enter an account name (such as
Exoscale). This account name will be used to start the sync.
n) New remote s) Set configuration password q) Quit config n/s/q> n name> Exoscale
You will see a list of supported cloud storage providers. Please select
Amazon S3, since the Exoscale SOS implements the Amazon S3 standard.
You will then be asked for the access key and secret source. Please enter
Choose a number from below, or type in your own value 1 / Enter AWS credentials in the next step \ "false" 2 / Get AWS credentials from the environment (env vars or IAM) \ "true"
Enter your access key and secret:
AWS Access Key ID - leave blank for anonymous access or runtime credentials. access_key_id> EXO... AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials. secret_access_key> ...
Select the region:
/ If using an S3 clone that only understands v2 signatures 12 | eg Ceph/Dreamhost | set this and make sure you set the endpoint. \ "other-v2-signature"
other-v2-signature option. For the endpoint, enter
sos-ZONE.exo.io. Make sure to substitute
ZONE with the relevant zone.
Endpoint for S3 API. Leave blank if using AWS to use the default endpoint for the region. Specify if using an S3 clone such as Ceph. endpoint> sos-at-vie-1.exo.io
For the last four fields, enter the following:
- Location constraint:
Empty for US Region, Northern Virginia or Pacific Northwest.
- Canned ACL:
Owner gets FULL_CONTROL. No one else has access rights (default).
- Server-Side Encryption:
- Storage class:
Then save the configuration with
You now need to configure the second provider.
The cloud providers are now configured, so you can synchronize the files. To synchronize from a local folder to the Exoscale SOS, enter the following command:
rclone sync /LOCAL/FOLDER Exoscale:BUCKET-NAME-HERE/
Exoscale, enter the name you have given your configuration. To synchronize from a second cloud provider, you can use the following command:
rclone sync OtherProvider:SOURCE-DIRECTORY Exoscale:BUCKET-NAME-HERe/