Production operations
Operations health
Единая operational-панель: свежесть backup, restore-check, heartbeat worker, состояние очереди, диск, systemd services и Postfix queue.
Backup freshness
ok=True
latest_backup=guru-send-backup-20260630T170720Z.sqlite3
latest_backup_age_hours=22.841815349592103
max_age_hours=26.0
restore_check=ok
backup_dir=/opt/guru-send/data/backups
Worker heartbeat
ok=False
last_run_at=
last_run_age_minutes=None
stale_after_minutes=15.0
Heartbeat пишется при каждом /api/worker/run-once.
Queue health
ok=True
pending_total=0
pending_overdue=0
sending_total=0
failed_total=0
cancelled_total=4
Host resources
ok=True
path=/opt/guru-send/data
free_percent=51.98
free_bytes=4772114432
min_free_percent=10.0
Systemd services
guru-send-web: not_checked ok=True
guru-send-worker: not_checked ok=True
postfix: not_checked ok=True
opendkim: not_checked ok=True
dovecot: not_checked ok=True
Set GURU_OPS_ENABLE_SYSTEMCTL=1 on Linux server to run live systemctl checks.
Postfix queue
status=not_checked
ok=True
queue_lines=0
enabled=False
Set GURU_OPS_ENABLE_POSTQUEUE=1 to check postqueue -p.
Systemd timer install hints
sudo cp deploy/systemd/guru-send-backup.* /etc/systemd/system/
sudo cp deploy/systemd/guru-send-ops-watchdog.* /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now guru-send-backup.timer guru-send-ops-watchdog.timer
Machine-readable status
/api/ops/health{"backups": {"backup_dir": "/opt/guru-send/data/backups", "latest_backup": "/opt/guru-send/data/backups/guru-send-backup-20260630T170720Z.sqlite3", "latest_backup_age_hours": 22.841815349592103, "latest_restore_check": {"ok": true, "quick_check": "ok", "table_count": 34}, "max_age_hours": 26.0, "ok": true}, "checks": {"backups": true, "host": true, "queue": true, "worker": false}, "database": {"backups": {"backup_dir": "/opt/guru-send/data/backups", "latest_backup": "/opt/guru-send/data/backups/guru-send-backup-20260630T170720Z.sqlite3", "latest_backup_size_bytes": 278528, "latest_restore_check": {"ok": true, "quick_check": "ok", "table_count": 34}}, "cutover_plan": ["Announce maintenance/read-only window and pause workers.", "Create a fresh SQLite backup and verify restore_check=ok.", "Provision PostgreSQL and set GURU_POSTGRES_URL/DATABASE_URL.", "Run schema migration and data copy from SQLite into PostgreSQL.", "Run smoke tests against PostgreSQL, then switch app config.", "Restart web/worker, verify /health and admin/database status.", "Keep the SQLite backup for rollback until production checks pass."], "engine": "sqlite", "postgres": {"configured": false, "dsn_redacted": "", "status": "not_configured"}, "sqlite_exists": true, "sqlite_path": "/opt/guru-send/data/guru_send.sqlite3", "sqlite_size_bytes": 385024}, "host": {"disk": {"free_bytes": 4772114432, "free_percent": 51.98, "min_free_percent": 10.0, "ok": true, "path": "/opt/guru-send/data", "total_bytes": 9181077504, "used_bytes": 4392185856}, "ok": true, "postfix_queue": {"enabled": false, "ok": true, "queue_lines": 0, "status": "not_checked"}, "services": {"dovecot": {"enabled": false, "ok": true, "status": "not_checked"}, "guru-send-web": {"enabled": false, "ok": true, "status": "not_checked"}, "guru-send-worker": {"enabled": false, "ok": true, "status": "not_checked"}, "opendkim": {"enabled": false, "ok": true, "status": "not_checked"}, "postfix": {"enabled": false, "ok": true, "status": "not_checked"}}}, "queue": {"cancelled_total": 4, "failed_total": 0, "ok": true, "pending_overdue": 0, "pending_total": 0, "sending_total": 0}, "status": "warn", "worker": {"last_run_age_minutes": null, "last_run_at": "", "ok": false, "stale_after_minutes": 15.0}}