The panel's snapshot button is the most used feature in our customer area and the most misunderstood. A snapshot is a seatbelt: it saves you from your own next command. It is stored on the same host as the server, so it does not save you from losing the host, the site, or your account. This is the three-tier plan we recommend, with a restic recipe that costs a few dollars a month, and the monthly test that makes it a backup rather than a hope.
Three tiers, three failure modes
| Tier | Protects against | Does not protect against | Restore time |
|---|---|---|---|
| Panel snapshot | a bad upgrade, a wrong rm, a broken config | host loss, site loss, account compromise | 2 to 5 minutes |
| Off-site restic repository | everything above plus host and site loss | an attacker with your restic key deleting the repo | minutes for files, an hour for a full rebuild |
| Second copy, different provider, append-only | everything above plus a compromised server | losing both keys | same, from a colder place |
Tier one: snapshots, used well
Take one before every upgrade or risky change; they are incremental and fast. Keep two or three rolling ones, not thirty: a snapshot is a point in time on the same NVMe, and old ones are just disk you pay attention to. Restoring rolls the whole disk back, so anything written after the snapshot is gone, including database rows. That is what you want during a botched upgrade and not what you want as a “backup from last Tuesday”.
Tier two: restic to an off-site bucket
restic deduplicates, encrypts on the client, and speaks to any S3-compatible bucket. Any object storage works; for a CheapServ server in Frankfurt, a bucket in another provider's European region is ideal. Install and initialise:
apt install -y restic
cat > /etc/restic.env <<'EOF'
export RESTIC_REPOSITORY=s3:https://s3.example-storage.com/backup-web01
export RESTIC_PASSWORD=long-random-passphrase-stored-in-your-password-manager
export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
EOF
chmod 600 /etc/restic.env && . /etc/restic.env
restic initBack up files and a database dump together, then prune on a schedule:
cat > /usr/local/bin/backup.sh <<'EOF'
#!/bin/bash
set -euo pipefail
. /etc/restic.env
mkdir -p /var/backups/db
sudo -u postgres pg_dumpall --clean | gzip > /var/backups/db/pg-$(date +%u).sql.gz
restic backup /etc /srv /var/backups/db /home --exclude-caches --exclude '/srv/**/node_modules' --tag daily
restic forget --keep-daily 14 --keep-weekly 8 --keep-monthly 12 --prune
restic check --read-data-subset=5%
EOF
chmod +x /usr/local/bin/backup.sh
systemd-run --on-calendar='03:15' --unit=restic-backup /usr/local/bin/backup.sh # or a proper timer unitThree details that matter: dump the database rather than backing up its files (a file copy of a running PostgreSQL is not consistent), --exclude-caches and the node_modules rule keep the repository small, and restic check --read-data-subset reads back a slice of the data every night so corruption is found before you need the restore.
A typical web server produces 1 to 5 GB of unique data a month after deduplication. At object-storage prices that is well under a dollar; the whole tier costs less than a coffee.
Tier three: a copy the server cannot delete
If an attacker gets root, they get /etc/restic.env and can run restic forget --prune on your only backup. Two fixes: give the server's key write-only permissions on the bucket (most providers support a policy without DeleteObject), or run restic copy from a separate machine into a second repository whose key the server never sees. A $3 VPS-1 in another site, pulling nightly with its own credentials, is a perfectly good tier three.
The monthly test
An untested backup is a hope. Once a month, on a calendar reminder:
- Deploy a scratch VPS-1 in the panel (a minute, a few cents of balance).
- Install restic, copy
/etc/restic.env, runrestic restore latest --target /restore. - Load the database dump into a fresh PostgreSQL and run one real query; open one restored file.
- Note how long it took. That number is your real recovery time.
- Cancel the scratch server.
The first time, the test finds something: a path you forgot, a dump that was empty because the cron user lacked permissions, an excluded directory that turned out to matter. That is the test working. Fix it, and the second month is boring.
What not to do
- Do not keep the only backup on the same server, in
/backups. It dies with the disk. - Do not rsync live database files. Dump them.
- Do not store the restic passphrase only on the server. If the server is gone, so is the key to the backup.
- Do not back up everything.
/var/lib/dockerimages, caches and package archives are reproducible; back up the data and the configuration that makes them.
With the three tiers in place, the worst case, a server that vanishes, becomes an afternoon of work instead of a company-ending event. The panel snapshot handles the rest of the week.