Don't discover backup failure during a restore
A backup that's never verified isn't a backup. It's an illusion of security. Too many companies discover their backups failed at the precise moment they need them - during a server crash, ransomware attack, or catastrophic human error.
The nightmare scenario is classic: the production server goes down. The technical team calmly initiates a restore from the latest backup. And then, surprise: the backup is 3 weeks old because the script had been silently failing all that time. Or worse: the file exists but it's corrupted.
Heartbeat monitoring for backups eliminates this risk. By actively verifying that each backup runs, succeeds, and produces a valid file, you have the guarantee that your data is protected. And if something goes wrong, you know immediately.
Without proactive monitoring, many problems can go unnoticed:
A reliable backup must be verified at multiple levels:
Each backup type has monitoring particularities:
MySQL, PostgreSQL, MongoDB - database dumps are critical. Verify the dump completed without error, the file isn't empty, and its size is consistent with data volume. For large databases, also monitor execution duration.
Directory backups, configurations, media. Use tools like rsync or tar. Verify the number of files copied, total size, and absence of errors. Watch for permissions that may block certain files.
Cloud uploads can fail for multiple reasons: expired credentials, exceeded quota, network issues. Verify the file is present in the bucket and accessible.
Here's an example bash script integrating all recommended verifications:
#!/bin/bash
# backup_mysql.sh - MySQL backup with complete monitoring
HEARTBEAT_URL="https://monitao.com/ping/YOUR_TOKEN"
BACKUP_DIR="/backups/mysql"
DB_NAME="production"
MIN_SIZE_MB=100 # Expected minimum size in MB
# Startup ping
curl -s "$HEARTBEAT_URL/start" > /dev/null
# Execute backup
BACKUP_FILE="$BACKUP_DIR/${DB_NAME}_$(date +%Y%m%d_%H%M%S).sql.gz"
mysqldump -u backup_user -p"$DB_PASSWORD" "$DB_NAME" | gzip > "$BACKUP_FILE" 2>/tmp/backup_error.log
# Verification 1: File exists
if [ ! -f "$BACKUP_FILE" ]; then
curl -s "$HEARTBEAT_URL/fail?error=file_not_created" > /dev/null
exit 1
fi
# Verification 2: Size is consistent
FILE_SIZE_MB=$(($(stat -f%z "$BACKUP_FILE" 2>/dev/null || stat -c%s "$BACKUP_FILE") / 1024 / 1024))
if [ "$FILE_SIZE_MB" -lt "$MIN_SIZE_MB" ]; then
curl -s "$HEARTBEAT_URL/fail?error=file_too_small&size=${FILE_SIZE_MB}MB" > /dev/null
exit 1
fi
# Verification 3: Gzip integrity test
if ! gzip -t "$BACKUP_FILE" 2>/dev/null; then
curl -s "$HEARTBEAT_URL/fail?error=corrupted_gzip" > /dev/null
exit 1
fi
# All good - success ping
curl -s "$HEARTBEAT_URL?size=${FILE_SIZE_MB}MB" > /dev/null
echo "Backup completed: $BACKUP_FILE (${FILE_SIZE_MB} MB)"This script verifies each critical step: file creation, minimum size, compression integrity. The success ping is only sent if all verifications pass.
Here's how to set up monitoring for your backups:
Good alerts notify you at the right time:
These mistakes reduce your backup monitoring effectiveness:
For compressed files (gzip, zip), use built-in verification tools (gzip -t, unzip -t). For SQL dumps, you can attempt a dry-run restore. For complete verification, calculate a checksum (sha256sum) and store it with the backup.
Configure a 5-6h timeout (normal duration + 50%). Send a startup ping and an end ping. For very long backups, add progress pings (e.g., every hour) to detect blockages before the global timeout.
Integrate size verification in your script. Define an acceptable minimum size based on history. If the file is below threshold, send a fail ping with "error=file_too_small&size=XXX".
Absolutely! Cloud isn't magic. Credentials can expire, quotas can be reached, regions can have incidents. Verify the upload succeeded (return code from aws s3 cp or gsutil cp) and the file is present and accessible.
These tools often generate logs or hooks. Create a cron script that checks for the latest backup presence, its size, and sends the ping accordingly. You can also use webhooks from these tools if available.
Incremental backups produce files of variable size (sometimes very small if few changes). Adjust your thresholds or disable size verification for incrementals. The important thing is verifying the complete chain (full + incrementals) remains consistent.
Your backups are your last line of defense against data loss. They deserve monitoring commensurate with their importance. An unverified backup is as dangerous as no backup - you think you're protected when you're not.
With MoniTao, you can set up complete backup monitoring in minutes. Execution, verification, alerts - everything is covered. Never again discover a backup failure at the moment you need it.
Start free, no credit card required.