Duplicity Backblaze



Duplicity backs up files by producing encrypted, digitally signed, versioned, TAR-format volumes and uploading them to a remote location, including Backblaze B2 Cloud Storage. Released under the terms of the GNU General Public License (GPL), Duplicity is free software. Backblaze makes personal backup to the cloud easy and fast. By automatically selecting all user files to backup Backblaze is the point and shoot of cloud backup. With desktop apps for Windows and Mac, users can quickly and easily have their data backed up to the Backblaze data centers and rest easy knowing their data is safe in the event of a.

When running Duplicity to backup to B2 (Backblaze) it fails with the following error: Attempt 1 failed. AttributeError: B2ProgressListener instance has no attribute 'exit' Attempt 2 failed. Backblaze on the back foot after 'inadvertently' beaming customer data to Facebook. Backup specialist Backblaze has fixed an issue where a Facebook advertising pixel was 'inadvertently' included on signed-in web pages – but users are concerned private filenames and sizes were also sent to the social media giant.

Recently, I’ve been thinking more and more about backups for my small (but growing) homelab. The golden rule is to follow the 3-2-1 method for backups:

  • 3 backups
  • 2 different types of media
  • 1 backup offsite

Current setup#

Currently, I keep an encrypted external HDD at home and another at work. Every couple weeks, I perform a backup to both and rotate the drives (this covers a 2-1-1 backup).

Planned setup#

I’d like to add cloud storage for a full 3-2-1 backup. My idea is to centralize all my backups to one location, then send the backups offsite to a cloud storage provider. The setup below is my final goal and will fulfill my 3-2-1 requirement.

Storage providers#

Duplicity Backblaze

For this, I was looking for a raw storage endpoint with some sort of API or command line interface. I was not interested in a file syncing service (e.g., Google Drive or Dropbox) or a cloud backup solution (e.g., Crashplan or Carbonite). While looking for cloud storage providers, I compared the following:

I ended up choosing Backblaze B2 storage. They seemed to be the cheapest, had the most straight-forward pricing, and were the easiest to setup with the backup program I was using.

Duplicity Backblaze

Full disclosure, I was already a Backblaze fanboy. I was already subscribed to their great blog where they post yearly stats on their hard drives. But, if that’s not enough, they offer free restores via USB flash drive or external HDD if your data is too big to download. And if you need to upload up to 40TB of data, you can request a Fireball (not free, but still cool).

Backblaze

Backup programs#

While looking for backup programs, I compared the following:

I ended up choosing Duplicity. It seemed to be the most popular program, it supports incremental backups and B2 storage, and supports encryption with GPG.

Sign up and install B2#

Duplicity Backblaze B2

Sign up for a B2 account if you don’t have one already. You can download the official B2 command line tool from these instructions, but I’m installing the package from the AUR using pacaur. Note - You can create a bucket from the website if you don’t want to install the B2 command line tool.

Setup a bucket#

Start by authorizing your account (substitute your account ID as needed). You will be prompted for your Application Key, which you can get in the B2 control panel.

Now, create a bucket (make sure it is allPrivate). The bucket name must be globally unique to all of Backblaze, not just your account. You can have up to 100 buckets per account.

Finally, list your available buckets.

I highly recommend you encrypt your backups using GPG. It’s integrated into Duplicity and will protect your files from prying eyes. I won’t be covering it here, but check out my other guide on how to create a GPG key. For this setup, I will be using a separate key for encryption and signing.

Disclaimer - Don’t lose the keys or the passphrases to the keys. For example, don’t backup the GPG keys using Duplicity, then have your hard drive crash, which would require the GPG keys to unlock Duplicity. Virtualbox mac os x 64 bit download. Store the keys on a separate backup by themselves.

First, install Duplicity.

Duplicity basics#

The basic syntax for Duplicity is below.

To backup directly to a server via SFTP, use a command similar to the one below.

To backup a folder to your B2 bucket, use a command similar to the one below. Substitute your account ID, application key, and bucket name as needed.

Duplicity also handles rotating backups. Here, I’m remove backups older than 3 months.

Duplicity script#

Because Duplicity has so many command line options, it’s easier to setup a script and run it via cron.

Hope this helps!

-Logan

Amazon S3 has been around for more than ten years now and I have been happily using it for offsite backups of my servers for a long time. Backblaze’s cloud backup service has been around for about the same length of time and I have been happily using it for offsite backups of my laptop, also for a long time.

In September 2015, Backblaze launched a new product, B2 Cloud Storage, and while S3 standard pricing is pretty cheap (Glacier is even cheaper) B2 claims to be “the lowest cost high performance cloud storage in the world”. The first 10 GB of storage is free as is the first 1 GB of daily downloads. For my small server backup requirements this sounds perfect.

My backup tool of choice is duplicity, a command line backup tool that supports encrypted archives and a whole load of different storage services, including S3 and B2. It was a simple matter to create a new bucket on B2 and update my backup script to send the data to Backblaze instead of Amazon.

Here is a simple backup script that uses duplicity to keep one month’s worth of backups. In this example we dump a few MySQL databases but it could easily be expanded to back up any other data you wish.

2
4
6
8
10
12
14
16
18
20
22
24
26
28
30
32
34
36
38
40
42
44
46
48
50
########################################################################
# Uses duplicity (http://duplicity.nongnu.org/)
# Run this daily and keep 1 month's worth of backups
########################################################################
# b2 variables
B2_APPLICATION_KEY=application_key
ENCRYPT_KEY=gpg_key_id
DB_USER='root'
DATABASES=(my_db_1 my_db_2 my_db_3)
# Working directory
########################################################################
# Make the working directory
# Dump the databases
fordatabasein${DATABASES[@]};do
mysqldump-u$DB_USER-p$DB_PASS$database>$WORKING_DIR/$database.sql
duplicity--full-if-older-than7D--encrypt-key='$ENCRYPT_KEY'$WORKING_DIRb2://$B2_ACCOUNT_ID:$B2_APPLICATION_KEY@$BUCKET
# Verify
duplicity verify--encrypt-key='$ENCRYPT_KEY'b2://$B2_ACCOUNT_ID:$B2_APPLICATION_KEY@$BUCKET$WORKING_DIR
# Cleanup
duplicity remove-older-than30D--force--encrypt-key=$ENCRYPT_KEYb2://$B2_ACCOUNT_ID:$B2_APPLICATION_KEY@$BUCKET
# Remove the working directory

Run this via a cron job, something like this: