Raspberry Pi Backup Scripts

This repository provides two Bash scripts for backing up specific folders from multiple Raspberry Pis (running Ubuntu or Debian variants) to a central location on a local device (e.g., rpi5-1.example.com). The scripts are optimized for a Docker-centric environment, ideal for Raspberry Pi 3B, 4, 5, Zero, and Zero 2 setups running containerized workloads.

Overview

• backup_script.sh: Performs daily backups of specified folders (e.g., /home/{your_username}/docker/ and /var/lib/docker/volumes/) to /home/{your_username}/backup/<host>/docker/ and /home/{your_username}/backup/<host>/volumes/. It preserves all files (no deletion), supports large file transfers, and creates a .last_backup timestamp file for retention tracking.
• backup_cleanup.sh: Runs monthly to delete files in the backup that are absent from the source, but only if the last backup is older than a configurable retention period (default: 7 days). It also removes empty directories.

Features

• Multi-Host Support: Backs up both remote Pis (via SSH) and the local device (direct filesystem access).
• Docker Focus: Designed for Dockerized environments, backing up Docker configuration and volume data.
• Webhook Alerts: Sends notifications via the Signal CLI REST API (or other endpoints like Discord/Slack with minor changes) for skips, errors, and successes.
• Retention Policy: Retains deleted source files for at least 7 days (configurable).
• Robust Error Handling: Skips nonexistent directories, logs errors, and supports partial transfers for large files.
• Compatibility: Tested on Ubuntu, should work on Debian and Debian-based systems (Raspberry Pi OS, etc.).

Prerequisites

• Operating System: Ubuntu (tested on newer versions) or Debian-based systems (e.g., Raspberry Pi OS).
• Hardware: Raspberry Pi 3B, 4, 5, Zero, or Zero 2, running Docker for containerized workloads.
• Tools: rsync and curl installed (sudo apt install rsync curl).
• SSH Setup: For remote Pis:
  • Generate an SSH key on the local device: ssh-keygen -t rsa -f /home/{your_username}/.ssh/id_rsa.
  • Copy to remote Pis: ssh-copy-id your_user@remote-host.
  • Set key permissions: chmod 600 /home/{your_username}/.ssh/id_rsa, chmod 700 /home/{your_username}/.ssh/.
• Sudo: Passwordless sudo for the your_user user on all Pis (local and remote):
  • Edit /etc/sudoers with sudo visudo and add: your_user ALL=(ALL) NOPASSWD: ALL (or limit to /usr/bin/rsync, /usr/bin/test).
  • Test: sudo -n ls /var/lib/docker/volumes/.
• Webhook: A Signal CLI REST API instance (or alternative like Discord/Slack) for alerts:
  • Follow signal-cli-rest-api setup to run a Docker container.
  • Example: sudo docker run -d -p 8080:8080 -v $HOME/.local/share/signal-api:/home/.local/share/signal-cli -e 'MODE=native' bbernhard/signal-cli-rest-api.
  • Register/link a Signal number and use the /v2/send endpoint (e.g., http://localhost:8080/v2/send).
• Disk Space: Ensure sufficient space in /home/{your_username}/backup/ (monitor with df -h).

Installation

1. Clone the Repository:

   git clone https://github.com/bjorngluck/raspberry-pi-backup-scripts.git ~/backup-scripts
   cd ~/backup-scripts
   chmod +x backup_script.sh backup_cleanup.sh

   Replace your-username with your GitHub username.

2. Configure Scripts:

   • Edit backup_script.sh and backup_cleanup.sh (e.g., nano backup_script.sh).
   • Update variables:
     • REMOTES: List of hostnames, including the local device (e.g., "rpi5-3.example.com" "rpi3-1.example.com" "rpi5-1.example.com").
     • FOLDERS_TO_BACKUP: Source folders (e.g., "/home/{your_username}/docker/ /var/lib/docker/volumes/").
     • BACKUP_ROOT: Destination (e.g., "/home/{your_username}/backup").
     • SSH_KEY: SSH private key (e.g., "/home/{your_username}/.ssh/id_rsa").
     • WEBHOOK_URL: Signal API endpoint (e.g., "http://localhost:8080/v2/send" for a local Signal CLI container).
     • WEBHOOK_NUMBER, WEBHOOK_RECIPIENTS: Signal numbers (e.g., "+27123456789").
     • RETENTION_DAYS: Retention period in backup_cleanup.sh (default: 7).
   • Example:

     REMOTES=("pi4.example.com" "pi5.example.com" "rpi5-1.example.com")
     WEBHOOK_URL="http://localhost:8080/v2/send"
     RETENTION_DAYS=30

3. Set Up Signal CLI REST API (for webhooks):

   • Create a config directory: mkdir -p $HOME/.local/share/signal-api.
   • Run the Docker container:

     sudo docker run -d --name signal-api --restart=always -p 8080:8080 \
       -v $HOME/.local/share/signal-api:/home/.local/share/signal-cli \
       -e 'MODE=native' bbernhard/signal-cli-rest-api

   • Link a Signal number: Open http://localhost:8080/v1/qrcodelink?device_name=signal-api in a browser, scan the QR code with Signal on your phone (Settings > Linked Devices > +).
   • Test webhook:

     curl -X POST -H "Content-Type: application/json" \
       -d '{"message": "Test backup alert", "number": "+27123456789", "recipients": ["+27123456789"]}' \
       http://localhost:8080/v2/send

   • Alternative webhooks (e.g., Discord, Slack): Update WEBHOOK_URL and adjust the JSON payload as needed (e.g., {"text": "Backup failed..."} for Discord).

Usage

1. Daily Backup:

   • Schedule with cron: crontab -e and add:

     0 2 * * * /home/{your_username}/backup-scripts/backup_script.sh >> /home/{your_username}/backup.log 2>&1

   • Backs up folders to /home/{your_username}/backup/<host>/docker/ and /home/{your_username}/backup/<host>/volumes/ without deleting files.
   • Creates .last_backup timestamp files for retention tracking.
   • Sends Signal alerts for skips, errors, and successes.

2. Monthly Cleanup:

   • Schedule with cron: crontab -e and add:

     0 3 1 * * /home/{your_username}/backup-scripts/backup_cleanup.sh >> /home/{your_username}/backup_cleanup.log 2>&1

   • Deletes files absent from the source if the last backup is older than 7 days, removes empty directories, and sends Signal alerts.

3. Manual Testing:

   • Backup: ./backup_script.sh
   • Cleanup: ./backup_cleanup.sh
   • Check logs: tail -f /home/{your_username}/backup.log or /home/{your_username}/backup_cleanup.log.
   • Example: Delete a file on rpi5-1.example.com (rm /home/{your_username}/docker/frigate/some_file), run backup (file remains), then test cleanup after setting old timestamp:

     touch -d "8 days ago" /home/{your_username}/backup/rpi5-1.example.com/docker/.last_backup
     ./backup_cleanup.sh

Script Details

• backup_script.sh:
  • Handles local (rpi5-1.example.com) and remote hosts.
  • Uses rsync -avz -P (preserves timestamps, shows progress, keeps partial transfers).
  • Local: Direct rsync with sudo for /var/lib/docker/volumes/.
  • Remote: SSH with sudo via --rsync-path="sudo -n rsync".
  • Creates .last_backup for each backup folder.
• backup_cleanup.sh:
  • Deletes files absent from the source if .last_backup is older than RETENTION_DAYS.
  • Uses rsync --delete --dry-run to identify files, find to remove empty directories.
  • Supports local and remote hosts like backup_script.sh.

Compatibility

• OS: Tested on Ubuntu (newer versions). Should work on Debian and variants (e.g., Raspberry Pi OS) with rsync and curl.
• Hardware: Optimized for Raspberry Pi 3B, 4, 5, Zero, and Zero 2 running Docker containers.
• Docker: Designed for Dockerized setups, backing up Docker configs (/home/{your_username}/docker/) and volumes (/var/lib/docker/volumes/).
• Webhooks: Uses Signal CLI REST API by default. Easily adaptable for Discord, Slack, or other endpoints by modifying WEBHOOK_URL and JSON payload.

Contributing

Contributions are welcome! Fork the repo, make changes, and submit a pull request. Report issues or suggest features via GitHub Issues.

License

MIT License - free to use, modify, and distribute.