Duplicity

Overview

Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. By using librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. These archives are encrypted and/or signed with GnuPG, to protect them against spying and/or modification by the server.

Provider Details

Setup

Installation tested on Ubuntu 20.04

  1. Download the latest stable version from Duplicity

  2. Install the following packages

    apt-get install gettext   python3-future python3-fasteners python3-boto3

  3. Run the “installer” to configure Duplicity

    [root@vm2backup duplicity-0.8.23]# python setup.py install

  4. Create an Exoscale SOS Bucket

  5. Create specific IAM Keys for this Bucket

    By default, all backups are encrypted using GnuPG. We will use ${HOSTNAME} to enable backups of multiple Instances to the same bucket

  6. Run the first backup

    export AWS_ACCESS_KEY_ID=<YOUR-ACCESS-KEY>
export AWS_SECRET_ACCESS_KEY=<YOUR-SECRET-KEY>
export PASSPHRASE=SECUREPASSWORD
duplicity /var s3://duplicity-backup/${HOSTNAME}/var --s3-endpoint-url=https://sos-de-fra-1.exo.io

  7. Run a full backup from the instance

    duplicity --exclude /mnt --exclude /tmp --exclude /proc --exclude /run --exclude /sys / s3://duplicity-backup/${HOSTNAME} --s3-endpoint-url=https://sos-de-fra-1.exo.io
Local and Remote metadata are synchronized, no sync needed.
Last full backup date: none
No signatures found, switching to full backup.
--------------[ Backup Statistics ]--------------
StartTime 1655462673.30 (Fri Jun 17 10:44:33 2022)
EndTime 1655463010.13 (Fri Jun 17 10:50:10 2022)
ElapsedTime 336.83 (5 minutes 36.83 seconds)
SourceFiles 171645
SourceFileSize 5369568365 (5.00 GB)
NewFiles 171645
NewFileSize 5369568365 (5.00 GB)
DeletedFiles 0
ChangedFiles 0
ChangedFileSize 0 (0 bytes)
ChangedDeltaSize 0 (0 bytes)
DeltaEntries 171645
RawDeltaSize 5275730169 (4.91 GB)
TotalDestinationSizeChange 2157358512 (2.01 GB)
Errors 0

    List Files in Backup target

    duplicity list-current-files s3://duplicity-backup/${HOSTNAME} --s3-endpoint-url=https://sos-de-fra-1.exo.io

Restore a file from the backup

    duplicity restore --file-to-restore var/log/kern.log s3://duplicity-backup/${HOSTNAME} /tmp/kern.log --s3-endpoint-url=https://sos-de-fra-1.exo.io

Remove Backups older than 1 Month

    duplicity remove-older-than 1M s3://duplicity-backup/${HOSTNAME} --s3-endpoint-url=https://sos-de-fra-1.exo.io

A simple Backup script which can be adapted to your needs

    #!/usr/bin/bash
    if [ -f /root/.duplicity.env ]
    then
    . ./.duplicity.env
    else
    echo "/root/.duplicity.env not found with needed environment"
    exit 1
    fi

    /usr/local/bin/duplicity cleanup --force s3://${BACKUP_BUCKET_NAME}/${HOSTNAME} --s3-endpoint-url=${S3_ENDPOINT}
    /usr/local/bin/duplicity remove-older-than --force ${BACKUP_RETENTION} s3://${BACKUP_BUCKET_NAME}/${HOSTNAME} --s3-endpoint-url=${S3_ENDPOINT}
    /usr/local/bin/duplicity --full-if-older-than ${FULL_EVERY} --exclude /mnt --exclude /tmp --exclude /proc --exclude /run --exclude /sys / s3://${BACKUP_BUCKET_NAME}/${HOST
    NAME} --s3-endpoint-url=${S3_ENDPOINT}

    .duplicity.env File
    export AWS_ACCESS_KEY_ID=<YOUR-ACCESS-KEY>
    export AWS_SECRET_ACCESS_KEY=<YOUR-SECRET-KEY>
    export PASSPHRASE=<YOURVERYSECUREPASSWORD>
    export BACKUP_BUCKET_NAME=<YOUR-SOS-BUCKET>
    export BACKUP_RETENTION=1M
    export S3_ENDPOINT=https://sos-de-fra-1.exo.io
    export FULL_EVERY=7D

For this guide we are using a bucket in de-fra-1. You can find all endpoints in our API SOS community guide.

Additional resources

Duplicity Documentation