How to migrate cloud provider with rclone
rclone is a command-line program to migrate between a large number of cloud storage providers. It can be used to migrate data from a local environment or other cloud providers to the Exoscale Object Storage.
Installation
rclone is available as a package in most Linux distributions and is also available for Windows and MacOS. Since it is a stand-alone executable it can simply be downloaded and started.
On Windows it is recommended to set the PATH
environment variable to point to the folder rclone is located in.
Prerequisites
As prerequisite for the following documentation, you’ll have to:
-
Create an Object Storage Bucket, either via the portal or the CLI. You can refer to the Object Storage documentation
-
Create an IAM key, either via the portal or the CLI. You can refer to the IAM documentation
Configuration
Before you can use rclone you will have to configure your accounts first. To do that please enter the following command:
# rclone config
You will be shown the following options:
n) New remote
s) Set configuration password
q) Quit config
Select n
to set up your Exoscale Account and enter an account name. (e.g. Exoscale
) This name will be used to start
the sync.
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> Exoscale
Next, you will be shown a list of supported cloud storage providers. Please select Amazon S3
since the Exoscale Object Storage implements the Amazon S3 standard.
Next, you will be asked for the access key and secret source. Please enter 1
:
Choose a number from below, or type in your own value
1 / Enter AWS credentials in the next step
\ "false"
2 / Get AWS credentials from the environment (env vars or IAM)
\ "true"
Next, enter your access key and secret:
AWS Access Key ID - leave blank for anonymous access or runtime credentials.
access_key_id> EXO...
AWS Secret Access Key (password) - leave blank for anonymous access or runtime credentials.
secret_access_key> ...
Next, you will have to select the region:
/ If using an S3 clone that only understands v2 signatures
12 | eg Ceph/Dreamhost
| set this and make sure you set the endpoint.
\ "other-v2-signature"
Select the other-v2-signature
option. For the endpoint enter
sos-ZONE.exo.io
, taking care of substituting ZONE
with the proper zone.
Note
You can find all Zones on our website and with the cli: exo zones list
Endpoint for S3 API.
Leave blank if using AWS to use the default endpoint for the region.
Specify if using an S3 clone such as Ceph.
endpoint> sos-at-vie-1.exo.io
For the last four questions please enter the following:
- Location constraint:
Empty for US Region, Northern Virginia or Pacific Northwest.
- Canned ACL:
Owner gets FULL_CONTROL. No one else has access rights (default).
- Server-Side Encryption:
None
- Storage class:
Default
Then save the configuration with Y
.
Note
You will now have to configure the second provider.
Synchronization
Now the cloud providers are configured, so you can synchronize the files. To synchronize from a local folder to the Exoscale Object Storage enter the following command:
rclone sync /LOCAL/FOLDER Exoscale:BUCKET-NAME-HERE/
Instead of Exoscale
please enter the name you have given your configuration. To synchronize from a second cloud provider you can use the following command:
rclone sync OtherProvider:SOURCE-DIRECTORY Exoscale:BUCKET-NAME-HERe/