Atomic site updates

Whenever I have new photos to add to my website, it takes a while for rsync to upload them. It leaves a short time window, during which loading my website used to result in it being loaded only partially, with placeholders for photos which failed to load, because the HTML got updated before the new photos were uploaded. To solve this issue, I decided to dig into how to atomically update a static site. It turns out to be pretty easy.

The first step is to make the webroot a symlink. Neither Caddy, nor Nginx cache the canonical path this symlink resolves to, which is very fortunate. You place one version of your website in a directory next to it and point the symlink at that directory. When you want to update the website, you upload it into yet another directory next to it, create a new symlink, which points to the new directory, and overwrite the original symlink with the new one, using the rename() syscall (which is atomic). You then delete the old directory.

To optimise the upload itself:

1. You may use the rsync --link-dest option to avoid transferring, or indeed even writing new files, when they’re unchanged from the previous version. Instead rsync will create hardlinks to the old files in the new directory. It makes the transfer faster and avoids a temporary doubling of disk use.
2. You may set the rsync --modify-window option to 50 years to avoid creating new files instead of hardlinks when only their mtime is changed. --ignore-times doesn’t achieve that.

Now the code.

Caddyfile:

yakubin.com, www.yakubin.com {
    root * /website/current
    file_server
}

Contents of /website:

$ ls -l
total 8
drwxr-xr-x  6 yakubin  yakubin  512 Jan 15 11:22 I7wyFs
lrwxr-xr-x  1 yakubin  yakubin    6 Jan 15 11:22 current -> I7wyFs
-rwxr-xr-x  1 yakubin  yakubin  180 Jan  6 01:45 switch.sh

/website/switch.sh:

#!/usr/bin/env sh

set -euxo pipefail

ln -s "$1" new-current
mv -fh new-current current # If you're on Linux, replace "-fh" with "-fT"

# Remove all directories except the one with the current website version
find . -mindepth 1 -maxdepth 1 -type d | grep -Fxv "./$(readlink current)" | xargs rm -rf

FreeBSD’s mv has the -h option, which is documented in its man page as follows:

-h      If the target operand is a symbolic link to a directory, do not
        follow it.  This causes the mv utility to rename the file source
        to the destination path target rather than moving source into the
        directory referenced by target.

On Linux the same thing is done with -T.

And now the upload script. It accepts a path to a local directory with the new website version as its first and only command line argument:

#!/usr/bin/env sh

set -eux

# Setting pipefail breaks this script.

directory="$(</dev/urandom LC_ALL=C tr -cd 'A-Za-z0-9' | head -c 6)"
mwindow=1577847600 # 50 years in seconds
rsync -rlDivz --modify-window=$mwindow --link-dest=../current/ "$1"/ yakubin.com:"/website/$directory/"
ssh yakubin.com "cd /website; ./switch.sh $directory"