blog.lazkani.io/content/posts/automating-borg.md

+++ title = "Automating Borg" author = ["Elia el Lazkani"] date = 2020-02-02T21:00:00+01:00 lastmod = 2021-06-28T00:00:27+02:00 tags = ["borgmatic", "borgbackup", "borg"] categories = ["backup"] draft = false +++

In the previous blog post entitle [BorgBackup]({{< relref "borgbackup" >}}), I talked about borg. If you read that post, you would've noticed that borg has a lot of features. With a lot of features come a lot of automation.

If you were thinking about using borg, you should either make a simple cron or you're gonna have to write an elaborate script to take care of all the different steps.

What if I told you there's another way ? An easier way ! The Borgmatic way... What would you say ?

Borgmatic

Borgmatic is defined on their website as follows.

borgmatic is simple, configuration-driven backup software for servers and workstations. Protect your files with client-side encryption. Backup your databases too. Monitor it all with integrated third-party services.

If you go down to it, borgmatic uses borg's API to automate a list of configurable tasks. This way, it saves you the trouble of writing your own scripts to automate these steps.

Borgmatic uses a YAML configuration file. Let's configure a few tasks.

Location

First, let's start by configuring the locations that borg is going to be working with.

location:
    source_directories:
        - /home/

    repositories:
        - user@backupserver:sourcehostname.borg

    one_file_system: true

    exclude_patterns:
        - /home/*/.cache
        - '*.pyc'

This tells borg that we need to backup our /home directories excluding a few patterns. Let's not forget that we told borg where the repository is located at.

Storage

We need to configure the storage next.

storage:
    # Recommended
    # encryption_passcommand: secret-tool lookup borg-repository repo-name

    encryption_passphrase: "ReallyStrongPassphrase"
    compression: zstd,15
    ssh_command: ssh -i /path/to/private/key
    borg_security_directory: /path/to/base/config/security
    archive_name_format: 'borgmatic-{hostname}-{now}'

In this section, we tell borg a little big of information about our repository. What are the credentials, where it can find them, etc.

The easy way is to go with a passphrase, but I recommend using an encryption_passcommand instead. I also use zstd for encryption instead of lz4, you better do your research before you change the default. I also recommend, just as they do, the use of a security directory as well.

Retention

We can configure a retention for our backups, if we like.

retention:
    keep_hourly: 7
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 6
    keep_yearly: 2

    prefix: "borgmatic-"

The part of what to keep from hourly to daily is self explanatory. I would like to point out the prefix part as it is important. This is the prefix that borgmatic uses to consider backups for pruning.

warning

Watch out for the retention prefix

Consistency

After the updates, we'd like to check our backups.

consistency:
    checks:
        - repository
        - archives

    check_last: 3

    prefix: "borgmatic-"

warning

Watch out, again, for the consistency prefix

Hooks

Finally, hooks.

I'm going to talk about hooks a bit. Hooks can be used to backup MySQL, PostgreSQL or MariaDB. They can also be hooks for on_error, before_backup, after_backup, before_everything and after_everything. You can also hook to third party services which you can check on their webpage.

I deployed my own, so I configured my own.

Borgmatic Configuration

Let's put everything together now.

location:
    source_directories:
        - /home/

    repositories:
        - user@backupserver:sourcehostname.borg

    one_file_system: true

    exclude_patterns:
        - /home/*/.cache
        - '*.pyc'

storage:
    # Recommended
    # encryption_passcommand: secret-tool lookup borg-repository repo-name

    encryption_passphrase: "ReallyStrongPassphrase"
    compression: zstd,15
    ssh_command: ssh -i /path/to/private/key
    borg_security_directory: /path/to/base/config/security
    archive_name_format: 'borgmatic-{hostname}-{now}'

retention:
    keep_hourly: 7
    keep_daily: 7
    keep_weekly: 4
    keep_monthly: 6
    keep_yearly: 2

    prefix: "borgmatic-"

consistency:
    checks:
        - repository
        - archives

    check_last: 3

    prefix: "borgmatic-"

Now that we have everything together, let's save it in /etc/borgmatic.d/home.yaml.

Usage

If you have borg and borgmatic already installed on your system and the borgmatic configuration file in place, you can test it out.

You can create the repository.

 # borgmatic init -v 2

You can list the backups for the repository.

 # borgmatic list --last 5
 borgmatic-home-2020-01-30T22:01:30 Thu, 2020-01-30 22:01:42 [0000000000000000000000000000000000000000000000000000000000000000]
 borgmatic-home-2020-01-31T22:02:12 Fri, 2020-01-31 22:02:24 [0000000000000000000000000000000000000000000000000000000000000000]
 borgmatic-home-2020-02-01T22:01:34 Sat, 2020-02-01 22:01:45 [0000000000000000000000000000000000000000000000000000000000000000]
 borgmatic-home-2020-02-02T16:01:22 Sun, 2020-02-02 16:01:32 [0000000000000000000000000000000000000000000000000000000000000000]
 borgmatic-home-2020-02-02T18:01:36 Sun, 2020-02-02 18:01:47 [0000000000000000000000000000000000000000000000000000000000000000]

You could run a check.

 # borgmatic check -v 1
 /etc/borgmatic.d/home.yaml: Pinging Healthchecks start
 /borg/home: Running consistency checks
 Remote: Starting repository check
 Remote: Starting repository index check
 Remote: Completed repository check, no problems found.
 Starting archive consistency check...
 Analyzing archive borgmatic-home-2020-02-01T22:01:34 (1/3)
 Analyzing archive borgmatic-home-2020-02-02T16:01:22 (2/3)
 Analyzing archive borgmatic-home-2020-02-02T18:01:36 (3/3)
 Orphaned objects check skipped (needs all archives checked).
 Archive consistency check complete, no problems found.

 summary:
 /etc/borgmatic.d/home.yaml: Successfully ran configuration file

But most of all, if you simply run borgmatic without any parameters, it will run through the whole configuration and apply all the steps.

At this point, you can simply add the borgmatic command in a cron to run on an interval. The other options would be to configure a systemd timer and service to run this on an interval. The latter is usually provided to you if you used your package manager to install borgmatic.

Conclusion

If you've checked borg and found it too much work to script, give borgmatic a try. I've been using borgmatic for few weeks now with no issues at all. I recently hooked it to a monitoring system so I will have a better view on when it runs, how much time each run takes. Also, if any of my backups fail I get notified by email. I hope you enjoy borg and borgmatic as much as I am.