Skip to content

Secrets management (SOPS + Woodpecker)

Platform secrets are encrypted in git and decrypted only at deploy time into tmpfs (/run/infra-secrets) or process memory. CI/deploy credentials for apps stay in Woodpecker only — never on the VPS disk.

Threat model

Risk Mitigation
Leaked git clone secrets/*.enc.env encrypted with age
Casual cat /opt/devops/.env No persistent plain .env on VPS
Backups copying passwords tmpfs excluded; plain .env removed after migration
Stolen Woodpecker DB Per-repo secrets; rotate SOPS_AGE_KEY separately
Root on VPS Cannot fully hidedocker inspect / /proc/*/environ still show runtime env

Root access always wins at runtime. The goal is no long-lived plain files and encrypted at rest in git.

Where secrets live

Secret type Storage Edit with
Woodpecker DB, OAuth, Traefik auth secrets/vps-stack.enc.env sops edit
Plane ↔ Forgejo bridge tokens secrets/plane-bridge.enc.env sops edit
Plane MCP OAuth secrets/plane-mcp.enc.env sops edit
Kuma Discord bot secrets/kuma-discord-bot.enc.env sops edit
DefectDojo (admin, API key) secrets/defectdojo.enc.env sops edit
Ship / Idea→Done worker LLM keys secrets/ship-workers.enc.env (when created) sops edit — never plain; see Idea→Done FinOps
Per-app SFTP (reference only) secrets/projects/<repo>.enc.env sops edit
App deploy (SFTP, Plane notify) Woodpecker repo secrets Woodpecker UI / onboard-project.sh
Decryption key (private) secrets/.age/key.txt (local, gitignored) Never commit
Auto-deploy key Woodpecker sops_age_key on infra/devops Copy private age line once

First-time setup

# 1. Generate age key + encrypt templates (already done if secrets/*.enc.env exist)
bash scripts/secrets-init.sh

# 2. Install private key for editing (your PC only)
mkdir -p ~/.config/sops/age
cp secrets/.age/key.txt ~/.config/sops/age/keys.txt
chmod 600 ~/.config/sops/age/keys.txt

# 3. Woodpecker secrets on infra-devops repo
#    sops_age_key     = AGE-SECRET-KEY-1... line from secrets/.age/key.txt
#    deploy_ssh_host  = 85.215.32.166
#    deploy_ssh_user  = deploy
#    deploy_ssh_key   = base64 of ~/.ssh/bioscan_vps (one line)

CI must SSH as the non-root deploy user (created by scripts/setup-vps.sh). Never use root for Woodpecker deploy.

Edit a secret

export SOPS_AGE_KEY_FILE=secrets/.age/key.txt   # or use ~/.config/sops/age/keys.txt
sops edit secrets/plane-bridge.enc.env
git add secrets/plane-bridge.enc.env
git commit -m "chore: update bridge secrets"
git push   # Woodpecker deploys if pipeline enabled

Deploy manually

bash scripts/sync-to-server.sh   # from PC with SSH config (see load-server-env.sh)
ssh bioscan-vps "cd /opt/devops && SOPS_AGE_KEY='...' bash scripts/deploy-platform.sh"

deploy-platform.sh:

  1. Decrypts bridge keys to /run/infra-secrets/ (tmpfs)
  2. Runs docker compose up -d with decrypted stack env (no .env file on disk)
  3. Starts bridge with tmpfs secret files
  4. Optionally starts MCP from tmpfs env file
  5. Optionally builds and starts Kuma Discord bot from tmpfs env file
  6. Starts docs nginx container

Remove plain files on VPS (after first successful deploy)

bash scripts/sync-to-server.sh
ssh bioscan-vps 'cd /opt/devops && SOPS_AGE_KEY=... bash scripts/deploy-platform.sh'
bash scripts/purge-plain-env-on-vps.sh

See also scripts/rotate-exposed-secrets.md if tokens were exposed in chat.

Onboard a new app

  1. Deploy creds → Woodpecker only (onboard-project.sh sets sftp_*, plane_api_key, etc.)
  2. Optional encrypted referencesops encrypt local/projects/myapp.env > secrets/projects/myapp.enc.env

Backup rules

scripts/backup.sh takes an encrypted restic snapshot: logical pg_dumps of the Postgres databases, sqlite3 .backup snapshots of the SQLite databases, the Docker volumes, and non-git host state. See Backup & restore.

It deliberately does not include:

  • /run/infra-secrets (tmpfs, decrypted secrets)
  • secrets/.age/key.txt (never on the VPS)
  • secrets/*.plain.env, secrets/*.env.tmp (excluded by name as defence in depth)
  • /opt/devops-secrets/ (holds the restic password — backing up the key inside the repository it unlocks is pointless)

A database dump alone is not a restore

Woodpecker secrets are encrypted at rest with WOODPECKER_ENCRYPTION_KEY, and DefectDojo credentials with DD_CREDENTIAL_AES_256_KEY. Restoring those databases without the matching keys from secrets/*.enc.env yields undecryptable rows. The age key is therefore part of your recovery plan, not just your deploy path.

Infisical (secrets manager)

Infisical runs at https://secrets.<DOMAIN> (infisical/docker-compose.yml).

The split of responsibility is deliberate:

Tier Holds Why
SOPS + age (secrets/*.enc.env) Bootstrap ring only — Infisical's own DB password, ENCRYPTION_KEY, AUTH_SECRET, and RESTIC_PASSWORD These must be reachable when Infisical is down or being rebuilt
Woodpecker repo secrets CI credentials (sops_age_key, deploy_ssh_*) Pipelines must run even if Infisical is unavailable
Infisical Per-project application secrets, migrated over time Central rotation, audit log, per-environment scoping

Never move RESTIC_PASSWORD into Infisical

Restoring Infisical from backup would then require a backup you cannot decrypt. The same reasoning keeps sops_age_key in Woodpecker rather than Infisical — a platform must not depend on itself to recover.

CI consumption. Apps authenticate with an Infisical Universal Auth machine identity: infisical_client_id / infisical_client_secret / infisical_project_id are Woodpecker repo secrets, and a pipeline step wraps its command as infisical run --projectId=$INFISICAL_PROJECT_ID --env=prod -- <command>. Existing from_secret: blocks keep working untouched — migration is per-project and incremental, currently scoped to portfolio, aifeatures-site and this repo.

Rotate age key (advanced)

  1. age-keygen -o secrets/.age/key-new.txt
  2. sops updatekeys -y secrets/*.enc.env (with both keys in .sops.yaml temporarily)
  3. Update Woodpecker sops_age_key
  4. Remove old key from .sops.yaml

MCP note

Secret rotation and SOPS edit operations are not exposed via the infra-ops MCP in v1. Use this runbook manually.