Skip to content

Server metrics (Netdata)

Dashboard: https://metrics.rafaelgonzalezalbes.com/v3/

Netdata runs on the infra VPS as the parent (netdata/netdata:v2.5.0). A child agent on the apps VPS streams host + Docker metrics to the parent — one login URL covers both servers. IONOS static hosting cannot run Netdata.

Tool URL What it tracks
Netdata metrics.{domain} Infra + apps VPS CPU, RAM, disk, Docker containers
Uptime Kuma status.{domain} HTTP / ping / TCP uptime

Access

  • URL: https://metrics.{domain}/v3/
  • Login: Traefik HTTP basic auth — username metric, password in secrets/netdata.enc.env (METRICS_AUTH_PASSWORD, default metric2026)
  • DNS: A record metrics → infra VPS IP in IONOS (see DNS)

Netdata v2’s built-in sign-in is tied to Netdata Cloud. We keep Cloud disabled and protect the dashboard at the Traefik edge instead (same pattern as locking down other platform URLs).

First-time login setup

cp secrets/plain/netdata.env.example secrets/plain/netdata.env
# optional: set NETDATA_DISCORD_WEBHOOK_URL
bash scripts/setup-netdata-secrets.sh   # generates passwords + encrypts
sops -d secrets/netdata.enc.env         # read passwords to share with team
git add secrets/netdata.enc.env
bash scripts/sync-to-server.sh
ssh bioscan-vps 'cd /opt/devops && bash scripts/deploy-platform.sh'

After deploy, open https://metrics.{domain}/v3/ — the browser asks for username/password, then the Netdata dashboard loads (no “Skip anonymously” splash).

What you see

  • System overview — CPU, RAM, swap, disk per mount, load, network (infra parent + apps child node)
  • Docker — per-container CPU/RAM/network on both VPSes (child streams to parent)
  • Docker containers dashboard — sortable table with Host column (infra vs apps VPS):
    https://metrics.{domain}/docker-containers.html
  • Alerts — threshold breaches in the Netdata UI; optional Discord posts to #infra

Apps VPS child streaming

Metrics from the apps VPS are not duplicated at metrics-apps.*. Instead:

  1. Parent (infra) — docker-compose.yml netdata + stream.conf rendered at deploy (accepts stream from apps IP and 127.0.0.1 for tunnel mode)
  2. Child (apps) — templates/apps-vps/netdata/ deployed to /opt/apps/netdata-child/ with host networking (reaches tunnel on 127.0.0.1:19998)
  3. Tunnel (infra) — IONOS blocks raw TCP 19999 between VPSes (same as Woodpecker gRPC). Systemd netdata-stream-tunnel on infra forwards parent :19999 to apps 127.0.0.1:19998.

Setup:

# 1. Generate stream API key (included in setup-netdata-secrets.sh)
bash scripts/setup-netdata-secrets.sh
git add secrets/netdata.enc.env && git push   # deploy platform on infra

# 2. Reverse SSH tunnel on infra (exposes parent stream on apps localhost)
bash scripts/setup-netdata-stream-tunnel.sh

# 3. Deploy child on apps VPS (from local machine; runs tunnel setup if needed)
bash scripts/deploy-apps-netdata-child.sh

# 4. (Optional) Remove legacy direct-stream UFW if it was added earlier
# ssh bioscan-vps 'cd /opt/devops && bash scripts/revoke-apps-vps-netdata-stream.sh'

Secrets in secrets/netdata.enc.env: NETDATA_STREAM_API_KEY, INFRA_VPS_IP, APPS_VPS_IP, NETDATA_STREAM_DESTINATION (default 127.0.0.1:19998).

bootstrap-apps-vps.sh attempts child deploy automatically when Netdata secrets exist.

Built-in Netdata views

In the main Netdata UI (/v3/):

  1. Open the chart navigation menu (right side)
  2. Go to cgroups / Containers
  3. On any chart, set Group by → Instance to split by container name

The custom /docker-containers.html page is a quicker “who is eating RAM/CPU?” view for all containers at once.

Alert thresholds (git-managed)

Custom health rules live in netdata/health.d/:

Alert Warn Critical
Disk space 80% 90%
RAM usage 85% 95%
Swap usage 50% 80%

Discord notifications are rendered at deploy time from secrets/netdata.enc.env into tmpfs (never plain on disk).

First-time Discord setup

  1. Create a webhook in Discord → #infra → Integrations → Webhooks
  2. Locally:
cp secrets/plain/netdata.env.example secrets/plain/netdata.env
# edit NETDATA_DISCORD_WEBHOOK_URL
bash scripts/setup-netdata-secrets.sh
git add secrets/netdata.enc.env
  1. Sync to VPS and redeploy:
bash scripts/sync-to-server.sh
ssh bioscan-vps 'cd /opt/devops && bash scripts/deploy-platform.sh'
  1. In Netdata UI → Alerts → send a test notification, or check logs:
docker logs netdata 2>&1 | grep -i discord

Deploy / restart

Netdata is started with the core stack:

ssh bioscan-vps
cd /opt/devops
bash scripts/deploy-platform.sh
# or only netdata:
docker compose up -d netdata

Runtime Discord config is written to /run/infra-secrets/netdata-runtime/ during deploy.

Add Alex

Share the metrics login (metric + METRICS_AUTH_PASSWORD from secrets/netdata.enc.env). Also listed in access control.

Future: app-level metrics

Netdata can be extended without replacing the stack:

  • Traefik — enable Prometheus metrics; Netdata scrapes them
  • Postgres — Woodpecker / Plane DB collectors
  • Grafana — optional later via Netdata → Prometheus export

Troubleshooting

Symptom Check
Browser login prompt / 401 Expected — user metric, password from METRICS_AUTH_PASSWORD in secrets
No login prompt (public dashboard) METRICS_AUTH_* missing — run setup-netdata-secrets.sh and redeploy
metrics.* DNS fails Add A record in IONOS; wait 15–60 min
TLS / certificate warning DNS must exist before first deploy; restart Traefik: docker restart traefik
Empty Docker charts docker.sock mounted on netdata container; on apps VPS check netdata-child is running
Apps VPS not in dashboard Tunnel down: systemctl status netdata-stream-tunnel on infra; nc -zv 127.0.0.1 19998 on apps host; docker logs netdata-child
Stream port exposed publicly Parent should bind 127.0.0.1:19999 only (tunnel mode). Revoke direct UFW: bash scripts/revoke-apps-vps-netdata-stream.sh
Traefik 404 docker ps --filter name=netdata; confirm metrics.${DOMAIN} label

List containers Netdata sees:

docker exec netdata curl -s 'http://127.0.0.1:19999/api/v1/charts?chart_type=docker' | head