If you have any source code repositories hosted online, you probably don’t want to lose those. Just yesterday I converted my professional resume from the OpenDocument Format to a version controlled Asciidoctor project. This prompted me to do an important task I’d been putting off for some time, backing up my Git repositories hosted on GitHub. Below is my solution.

Tutorial

The reference system will of course be the latest Ubuntu LTS, 20.04 at the time of this writing. You will need to be familiar with Git and Unix shells. The fish shell in particular is used here. This tutorial will demonstrate how to automate these backups with systemd.

  1. Install the fish shell.

    sudo apt -y install fish
  2. Create a backup directory for storing your Git repositories.

    mkdir ~/Source
  3. Create mirrors for each repository you wish to backup in this directory, making sure each repository’s name is suffixed with .git.

    Here, I mirror this blog’s repository in the Source directory in my home folder.

    ➜ git clone --mirror https://github.com/jwillikers/blog.git ~/Source/blog.git
  4. Place the following update script in /etc/fish/functions where it will be autoloaded by fish.[1]

    /etc/fish/functions/update_git_mirrors.fish
    function update_git_mirrors -d "For each directory given, non-recursively update each Git mirror repository directory suffixed with .git"
      for dir in $argv
        if not test -d $dir
          continue
          echo "Argument '$dir' is not a directory" 1>&2
        end
    
        for mirror in $dir/*.git
          if test -d $mirror
             git -C $mirror remote update --prune >/dev/null
             echo "Updated $mirror"
          end
        end
      end
    end

    This script takes a number of directories as arguments. Each of these directories is searched for directories ending with .git in their name. Each of these is treated as a Git mirror and updated appropriately.

    Placing the function definition in /etc/fish/functions provides a stronger guarantee for reproducibility compared to placing the function in the user’s directory ~/.config/fish/functions. If you don’t have root access or use systemd-homed and want to migrate this function with your home directory, it makes more sense to place the function in ~/.config/fish/functions.

  5. Test the script by executing update_git_mirrors from within a fish shell.

    Since I use fish as my default shell, it’s as easy as running the function directly from my shell.

    ➜ update_git_mirrors ~/Source
    Updated /home/jordan/Source/blog.git

    If you don’t use fish as your shell - and don’t want to bother converting this code for your shell - you can test the function by calling it with fish -c.

    ➜ fish -c 'update_git_mirrors ~/Source'
    Updated /home/jordan/Source/blog.git
  6. Create the systemd user configuration directory.

    mkdir -p ~/.config/systemd/user
  7. Create a systemd unit to refresh the mirrors.

    ~/.config/systemd/user/update-git-mirrors.service
    [Unit]
    Description=Update my Git mirrors
    
    [Service]
    Environment=fish_function_path=/etc/fish/functions
    ExecStart=/usr/bin/fish -c 'update_git_mirrors /home/jordan/Source'
    Nice=19
    Type=oneshot
    
    [Install]
    WantedBy=default.target

    The command-line here calls the fish function just created, update_git_mirrors to update the mirrors found in the directory /home/jordan/Source. The Environment setting protects the function from being overloaded by a function of the same name placed in another autoloaded directory, such as the user’s ~/.config/fish/functions directory. Remove this line if you placed the function definition in the ~/.config/fish/functions directory instead of /etc/fish/functions. The Nice directive designates a low scheduling priority, 14, for the CPU.

    Be aware of what protocols your repositories are using to authenticate when connecting to private repositories. If you use SSH with an encrypted private key to access any private repositories, your key must be unlocked and available in your SSH agent before running this unit. When using the timer described below, you will want your directory to automatically be unlocked at login for this to work.

    Configure a dedicated backup key with read-only access to your Git repositories for extra safety. You could even use a dedicated user account for these backups to isolate this functionality, but I’ve kept this simple for users that just want to get backups working.

  8. Test run the new systemd unit.

    ➜ systemctl --user start update-git-mirrors.service
  9. Check the output of the command to make sure everything worked.

    ➜ systemctl --user status update-git-mirrors.service
    ● update-git-mirrors.service - Update my Git mirrors
         Loaded: loaded (/home/jordan/.config/systemd/user/update-git-mirrors.service; disabled; vendor preset: enabled)
         Active: inactive (dead)
    
    Dec 07 06:28:27 latitude fish[56735]: Updated /home/jordan/Source/blog.git
    Dec 07 06:28:31 latitude systemd[4148]: update-git-mirrors.service: Succeeded.
    Dec 07 06:28:31 latitude systemd[4148]: Finished Update my Git mirrors.
  10. Add a systemd timer to update the mirrors every day.

    ~/.config/systemd/user/update-git-mirrors.timer
    [Unit]
    Description=Regularly refresh my Git mirrors
    
    [Timer]
    Persistent=true
    OnCalendar=daily
    
    [Install]
    WantedBy=timers.target

    This timer has Persistent=true to account for the situation when the timer would fire but the user has no session running. When this happens, the timer will just fire the next time the user logs on.

  11. Activate the timer automatically when logging in.[2]

    ➜ systemctl --user enable update-git-mirrors.timer
    Created symlink /home/jordan/.config/systemd/user/timers.target.wants/update-git-mirrors.timer → /home/jordan/.config/systemd/user/update-git-mirrors.timer.
  12. Check when your timer’s schedule with the systemctl --user list-timers command.

    ➜ systemctl --user list-timers update-git-mirrors
    NEXT                        LEFT     LAST PASSED UNIT                     ACTIVATES
    Tue 2020-12-08 00:00:00 CST 16h left n/a  n/a    update-git-mirrors.timer update-git-mirrors.service
    
    1 timers listed.
    Pass --all to see loaded but inactive timers, too.

    The above output indicates that the timer should fire for the first time tomorrow.

Make sure to regularly verify that your backups are running properly. Tools like Nagios can make this monitoring easier.

Conclusion

You should now have a good idea as to how to go about backing up your Git repositories locally and automating the task with systemd.


1. See the Autoloading functions documentation for more details.