Uptime monitoring (Uptime Kuma)

Dashboard: https://status.rafaelgonzalezalbes.com

Centralized on the infra VPS — one Kuma instance monitors platform services, both VPSes (ping/TCP 443), and per-app HTTPS URLs on the apps VPS. Do not deploy a second Kuma on the apps VPS or IONOS.

For host resource metrics (disk, RAM, CPU, Docker containers), use Netdata at https://metrics.rafaelgonzalezalbes.com.

Kuma runs on the VPS as part of the devops Docker stack (louislam/uptime-kuma:2). Monitor definitions live in git at uptime-kuma/monitors.manifest.json and are applied with scripts/setup-kuma-monitors.sh.

Multi-user (v2)

Uptime Kuma 2.4 supports multiple logins in the database, but there is no Settings → Users UI yet (still coming in a future release). Add accounts from the VPS:

ssh bioscan-vps
cd /opt/devops
bash scripts/kuma-add-user.sh alex 'choose-a-strong-password'
bash scripts/kuma-add-user.sh discord-bot 'choose-a-strong-password'

User	Purpose
rafael.gonzalez.albes	Primary admin (migrated from v1)
alex	Team member — own login
discord-bot	Discord alert bot (Socket.IO)

List users: docker exec uptime-kuma sqlite3 /app/data/kuma.db 'SELECT id, username, active FROM user;'

What is monitored

Monitor definitions live in uptime-kuma/monitors.manifest.json (groups: Platform (infra VPS) and Apps VPS). Applied with scripts/setup-kuma-monitors.sh (reads local/apps-vps.env for apps IP when present).

Group	Monitor	Type	Target
Platform	Forgejo (Git)	HTTP	https://git.{domain}
Platform	Woodpecker (CI)	HTTP	https://ci.{domain}
Platform	Uptime Kuma (self)	HTTP	http://127.0.0.1:3001 (internal; avoids Traefik hairpin)
Platform	Platform Docs	HTTP	https://docs.{domain}
Platform	DefectDojo (Security)	HTTP	https://security.{domain}/login?next=/
Platform	Netdata (Metrics)	HTTP	https://metrics.{domain}/v3/
Platform	Infra VPS (ping)	Ping	Infra VPS IP
Platform	Infra VPS HTTPS (TCP 443)	TCP	Infra VPS :443
Apps VPS	Apps VPS (ping)	Ping	Apps VPS IP
Apps VPS	Apps VPS HTTPS (TCP 443)	TCP	Apps VPS :443
Apps VPS	WearLab (HTTPS)	HTTP	https://wearlab.{domain}
Apps VPS	DroneColonies (HTTPS)	HTTP	https://dronecolonies.{domain} (planned)
Apps VPS	Cat tree (HTTPS)	HTTP	https://cats.{domain} (planned)

Removed monitors — listed in manifest removed_monitors (e.g. legacy BioScanCheck API on infra :3100). Prune on VPS:

ssh bioscan-vps
cd /opt/devops
bash scripts/prune-kuma-removed-monitors.sh

Plane / bridge / MCP monitors are only seeded when ENABLE_PLANE_STACK=true (Plane stack is off by default in deploy-platform.sh). See Architecture overview.

Edit uptime-kuma/monitors.manifest.json when you add apps or DNS names, then re-run the setup script.

First-time setup

1. Open https://status.rafaelgonzalezalbes.com and complete the Kuma admin account.

2. Quick seed (no API password) — on the VPS:

ssh bioscan-vps
cd /opt/devops
bash scripts/seed-kuma-monitors.sh

1. API-based apply — set credentials in local/.env:

KUMA_USER=rafael.gonzalez.albes
KUMA_PASSWORD=your-kuma-login-password

Then on the VPS after sync-to-server.sh:

KUMA_PASSWORD='...' bash scripts/setup-kuma-monitors.sh

Both scripts are idempotent: existing monitor names are skipped.

Upgrade v1 → v2 (one-way)

Only needed once when moving from louislam/uptime-kuma:1. The stack now pins :2.

ssh bioscan-vps
cd /opt/devops
bash scripts/backup-kuma.sh
bash scripts/upgrade-kuma-v2.sh   # watch logs until migration completes — do not interrupt

Official guide: Migration From v1 To v2

Restore from backup if migration is interrupted: extract kuma-data-*.tar.gz into volume devops_uptime_kuma_data.

Notifications (recommended)

Discord alert bot (git-managed)

A custom bot in uptime-kuma/discord-bot/ connects to Kuma over Socket.IO and posts rich embeds to Discord (down/up, degraded, critical rollups, daily digest). It deploys with the platform stack when secrets are configured.

One-time setup:

1. Create a Discord bot and invite it to your server (Send Messages, Embed Links).
2. Copy the bot token and target channel ID.
3. Use your Kuma admin account that owns the monitors (in v2.4 this is usually rafael.gonzalez.albes). The dedicated discord-bot user cannot see monitors owned by another user until Kuma adds sharing in a future release.
4. Edit secrets/plain/kuma-discord-bot.env:

Variable	Required	Notes
DISCORD_BOT_TOKEN	Yes	Bot token from Developer Portal
DISCORD_CHANNEL_ID	Yes	Default channel (fallback for unrouted monitors)
DISCORD_CHANNELS_JSON	No	Route name → channel ID map (see below)
DISCORD_ROUTING_RULES_JSON	No	Monitor name regex → route rules
DISCORD_CRITICAL_ROUTE	No	Route for critical rollups (default default)
DISCORD_DIGEST_ROUTE	No	Route for daily digest (default default)
DISCORD_ALERT_ROLE_ID	No	Role pinged on critical outages
KUMA_URL	Yes	http://uptime-kuma:3001 on VPS (internal)
KUMA_USERNAME	Yes	Monitor owner username (e.g. rafael.gonzalez.albes)
KUMA_PASSWORD	Yes	Password for that Kuma user
CRITICAL_THRESHOLD	No	Default 3
DIGEST_CRON	No	Default 0 9 * * * (09:00)
TZ	No	Default Europe/Madrid

1. Encrypt and deploy:

bash scripts/secrets-init.sh --encrypt-only
git add secrets/kuma-discord-bot.enc.env
git commit -m "chore: add kuma discord bot secrets"
git push origin main

1. Verify on VPS:

bash scripts/kuma-health-check.sh       # no Discord spam — container + Kuma link
bash scripts/kuma-verify-bot-access.sh  # optional: sends one test message to #alerts
bash scripts/kuma-test-alert-types.sh # full matrix: DOWN/UP/DEGRADED per channel + critical/digest
docker logs kuma-discord-bot --tail 20

No new Woodpecker secrets — uses existing sops_age_key + deploy_ssh_host / deploy_ssh_user / deploy_ssh_key on the infra/devops repo.

Production checklist

Item	Status
Secrets in secrets/kuma-discord-bot.enc.env only (SOPS); plain files gitignored	Required
Decrypted to /run/infra-secrets/ (tmpfs, chmod 600) on deploy	Automatic
Bot uses monitor-owner Kuma account (rafael.gonzalez.albes)	Required
restart: unless-stopped on bot container	Automatic
Health file refreshed on Kuma monitorList (/app/data/health.json)	Automatic
State persisted in Docker volume kuma-discord-bot-data (downtime tracking)	Automatic
Woodpecker push to main → rsync + deploy-platform.sh rebuilds bot	Automatic

Re-run alert routing tests after channel ID or routing rule changes:

bash scripts/kuma-test-alert-types.sh

Multi-channel routing (per-app Discord channels)

Set one channel ID per app. The bot maps Kuma monitor names automatically — no JSON required.

Env variable	Discord channel	Kuma monitor(s)
DISCORD_CHANNEL_DOCS	#docs	Platform Docs
DISCORD_CHANNEL_MCP	#mcp	Plane MCP
DISCORD_CHANNEL_GIT	#git	Forgejo (Git)
DISCORD_CHANNEL_CI	#ci	Woodpecker (CI)
DISCORD_CHANNEL_PM	#pm	Plane (PM), Plane Forge Bridge
DISCORD_CHANNEL_SECURITY	#security	DefectDojo (Security), vulnerability/CI alerts
DISCORD_CHANNEL_INFRA	#infra	Woodpecker failures for infra/devops
DISCORD_CHANNEL_PORTFOLIO	#portfolio	Woodpecker failures for */portfolio
DISCORD_CHANNEL_ID	fallback	Uptime Kuma, VPS ping/443, BioScan, external APIs

DISCORD_CHANNEL_ID=1234567890123456789
DISCORD_CHANNEL_DOCS=...
DISCORD_CHANNEL_MCP=...
DISCORD_CHANNEL_GIT=...
DISCORD_CHANNEL_CI=...
DISCORD_CHANNEL_PM=...
DISCORD_CHANNEL_SECURITY=...
DISCORD_CHANNEL_INFRA=...
DISCORD_CHANNEL_PORTFOLIO=...

Inbound webhooks (DefectDojo findings, Woodpecker CI failures) hit the bot on the internal Docker network:

Endpoint	Source
http://kuma-discord-bot.internal:8787/webhook/defectdojo	DefectDojo scan_added
http://kuma-discord-bot.internal:8787/webhook/ci-security	Woodpecker notify-discord-security (security-gate only)
http://kuma-discord-bot.internal:8787/webhook/ci-failure	Woodpecker notify-discord-ci (general pipeline failures)

Set SECURITY_WEBHOOK_SECRET in the bot env and matching Woodpecker repo secrets (security_webhook_secret, shared with ci_webhook_url bearer). Woodpecker also needs ci_webhook_url = http://kuma-discord-bot.internal:8787/webhook/ci-failure. Wire with bash scripts/wire-defectdojo-woodpecker.sh. See Security scanning.

1. Enable Developer Mode in Discord → right-click each channel → Copy Channel ID.
2. Invite the bot to every channel above.
3. Encrypt and deploy (bash scripts/secrets-init.sh --encrypt-only → push main).

Critical and daily digest go to DISCORD_CHANNEL_ID by default. Override with DISCORD_CRITICAL_ROUTE=pm or DISCORD_DIGEST_ROUTE=pm (route name, not channel ID).

Advanced: custom regex rules via DISCORD_ROUTING_RULES_JSON (replaces auto-mapping).

Kuma built-in notifications (optional backup)

In the Kuma UI → Settings → Notifications:

1. Add Email, Discord, Telegram, or Webhook
2. Attach the notification to all platform monitors
3. Set retry / resend intervals per your on-call preference

Public status page (optional — off by default)

Status pages are unpublished after lockdown-platform.sh.

bash scripts/seed-kuma-status-page.sh   # PUBLISH=0 by default

Add a monitor for a new app

1. Deploy the app on the apps VPS and DNS ({app}.{domain} → apps VPS IP)
2. Add an HTTP entry under Apps VPS in uptime-kuma/monitors.manifest.json
3. Re-run bash scripts/setup-kuma-monitors.sh (or seed-apps-vps-kuma-monitors.sh on VPS)
4. Optional: route Discord alerts via DISCORD_CHANNEL_* in kuma-discord-bot secrets

Troubleshooting

Symptom	Check
mcp.* down	Run bash scripts/bootstrap-plane-platform.sh on VPS
docs.* down / 403	Stale nginx bind mount after Woodpecker site/ rsync — docker restart platform-docs or re-run deploy (--force-recreate fixes this automatically). Also check site/index.html exists on VPS
bridge.* shows 404	Normal for root path — monitor accepts 404
Bot not tracking monitors	docker logs kuma-discord-bot — expect Tracking 10+ monitors. Wrong Kuma user → 0 monitors
Bot unhealthy / restarting	bash scripts/kuma-health-check.sh; check /app/data/health.json age inside container
Recovery alert missing	Bot needs downSince in state — fixed for DOWN → PENDING → UP path in heartbeat handler
BioScanCheck API down on infra :3100	Legacy — run bash scripts/prune-kuma-removed-monitors.sh
WearLab down	Check DNS → apps VPS IP, Traefik on apps VPS, docker ps in /opt/apps/wearlab
Cannot reach Kuma from PC	Run scripts/fix-dns-local.ps1 as Administrator

List current monitors:

bash scripts/kuma-query-monitors.sh   # on VPS, or via infra-ops MCP