Architecture¶

System Context (C4 Level 1)¶

         Household users
         (mail, cloud, social,
          identity)
              │
              ▼
    ┌─────────────────────────────────────┐
    │        philipp.info platform        │
    │    (self-hosted, Hetzner AX52)      │
    └──────┬──────────────┬───────────────┘
           │              │
           ▼              ▼
      Fediverse      Hetzner StorageBox
      (ActivityPub,  · backup target (Borg)
       Diaspora,     · legacy SSHFS mounts
       Matrix fed.)    (Nextcloud, Paperless)

Users: mail (6 domains), Nextcloud, Friendica (×3 instances), Matrix, Keycloak SSO Fediverse peers: external ActivityPub/Diaspora/Matrix servers federating with this platform

Container Diagram (C4 Level 2)¶

Internet
  │ HTTPS 443/8993
  │ SMTP 25/465/587, IMAP 143/993
  ▼
┌──────────────────────────────────────────────────────────────┐
│                   phil-app (157.90.134.159)                  │
│                                                              │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────────────┐  │
│  │   Traefik   │  │   mailcow    │  │   Friendica ×3      │  │
│  │  (ingress)  │  │  Postfix     │  │   philipp.info      │  │
│  │  CoreDNS    │  │  Dovecot     │  │   opensocial.at     │  │
│  │  Shorewall  │  │  SOGo/Rspamd │  │   friendica.me      │  │
│  └──────┬──────┘  └──────────────┘  └─────────────────────┘  │
│         │                                                    │
│  ┌──────▼──────┐  ┌──────────────┐  ┌─────────────────────┐  │
│  │  Keycloak   │  │    Matrix    │  │    Monitoring       │  │
│  │  (OIDC SSO) │  │  Synapse+MAS │  │  Prometheus+Loki    │  │
│  │  OpenLDAP   │  │  5 bridges   │  │  Grafana+Alertmgr   │  │
│  │  Step-CA    │  │              │  │  Borgmatic (11jobs) │  │
│  └─────────────┘  └──────────────┘  └─────────────────────┘  │
│                                                              │
│  ┌─────────────┐  ┌──────────────┐  ┌─────────────────────┐  │
│  │  Nextcloud  │  │  Forgejo     │  │    Docs             │  │
│  │  Paperless  │  │  Renovate    │  │  MkDocs + nginx     │  │
│  │             │  │  Actions     │  │  docs.philipp.info  │  │
│  └─────────────┘  └──────────────┘  └─────────────────────┘  │
└──────────────────────────┬───────────────────────────────────┘
                           │ WireGuard 10.42.10.4 ↔ 10.42.10.3
┌──────────────────────────┴───────────────────────────────────┐
│                   phil-db (88.198.7.144)                     │
│  MariaDB — friendica_opensocial (157 GB)                     │
│            friendica_friendicame (95 GB)                     │
│            keycloak (small)                                  │
└──────────────────────────────────────────────────────────────┘

Key Communication Flows¶

From	To	Protocol	Notes
Internet	Traefik	HTTPS 443, 8993	All web services; Traefik terminates TLS
Internet	Postfix / Dovecot	SMTP 25/465/587, IMAP 143/993	mailcow handles TLS directly — not via Traefik
Browser	Keycloak	HTTPS / OIDC	SSO login for all services
Keycloak	OpenLDAP	LDAP 389 (internal)	User federation — LDAP is backend only, no service uses LDAP directly
Services (web)	MariaDB	TCP 3306 → WireGuard	Friendica, Keycloak, mailcow reach `phil-db` at `10.42.10.3` via `extra_hosts`
Services	CoreDNS	UDP 53 → `172.21.0.53`	Resolves `phil-db` → `10.42.10.3`; used by services on `traefik-resolver` network
Services	Step-CA	HTTPS → `127.0.0.1:9000`	Internal cert issuance and renewal
cert-renewer@.timer	Step-CA	HTTPS	Renews 24h certs for Keycloak, MariaDB; systemd units on both servers
Alloy	Loki	HTTP	Log shipping from both servers
Prometheus	Exporters	HTTP scrape	node_exporter, mysqld_exporter, cAdvisor, xfs_quota_exporter, blackbox, etc.
Alertmanager	Matrix	HTTP → `matrix-alertmanager`	Alert notifications to Matrix room
Borgmatic / Borg	StorageBox	SSH (Borg protocol)	Backup storage; 2 hourly / 7 daily / 4 weekly / 6 monthly

Internal PKI — Step-CA¶

All internal TLS is issued by a self-hosted CA, not Let's Encrypt. Let's Encrypt is used only for public-facing Traefik certificates.

Container: secure-ca-1 (smallstep/step-ca), exposed at 127.0.0.1:9000
Root CA: /etc/step-ca/certs/root_ca.crt — valid until 2033
Certificate lifetime: 24 hours — automatically renewed by cert-renewer@.timer systemd units
Issued to: Keycloak (stack/secure/config/docker/certs/), MariaDB (via Ansible 36_pki.yml)
Failure cascade: Step-CA cert expired → Keycloak TLS fails → OIDC discovery returns 404 → Synapse, Paperless, and other OIDC clients fail. See operations/runbooks.md.

Key Architectural Decisions¶

Concern	Decision	Rationale
Ingress	Traefik via Docker labels	Zero-config TLS, no static config per service
Internal DNS	CoreDNS at `172.21.0.53`	Resolves `phil-db` → `10.42.10.3` inside containers
DB location	Separate phil-db server	Isolates 80 GB InnoDB buffer pool; protects DB from phil-app restarts
Auth	Keycloak OIDC + OpenLDAP federation	Single SSO for all services; LDAP is backend only
Internal TLS	Step-CA (24h certs)	Short-lived certs; no manual cert management; auto-renewed
Secrets	Docker secrets (`_FILE` env vars) where supported	See ADR-001
Firewall	Shorewall (host) + Docker iptables chains	Docker bypasses Shorewall — port bindings must use `127.0.0.1` or WireGuard IP
Backups	Borgmatic (phil-app) + systemd Borg (phil-db) → StorageBox	See services/backup.md

Ansible Playbooks (`ansible/`)¶

Manages base OS configuration for both servers:

Playbook	Scope	Purpose
`10_baseline.yml`	both	Packages, SSH, fail2ban, Shorewall firewall
`20_hardening.yml`	both	Kernel hardening, sysctl
`30_wireguard.yml`	both	WireGuard tunnel between phil-app and phil-db
`36_pki.yml`	phil-db	Internal PKI (CA + trust) for MariaDB TLS
`40_mariadb.yml`	phil-db	MariaDB config, TLS, performance tuning, mysql_snapshot
`50_docker.yml`	phil-app	Docker CE, daemon config, cert-renewer overrides
`55_monitoring.yml`	both	Alloy, cert-expiry, xfs_quota_exporter, reboot-required
`60_backup.yml`	phil-db	Borg backup agent

Not in scope for Ansible: LDAP runs as a Docker container on phil-app (stack/ldap/), not as a system service.

Secrets Migration Status¶

See ADR-001 for strategy.

Migrated (native _FILE support):

Stack	Service	Env var pattern	Secret file
matrix	db (postgres)	`POSTGRES_PASSWORD_FILE=/run/secrets/...`	`config/secrets/postgres_password.txt`
woodpecker	opensocial-agent	`WOODPECKER_AGENT_SECRET_FILE=/run/secrets/...`	`config/secrets/agent_secret.txt`
paperless	app	`PAPERLESS_DBPASS_FILE=/run/secrets/...`	`config/secrets/db_password.txt`
paperless	app	`PAPERLESS_EMAIL_HOST_PASSWORD_FILE=/run/secrets/...`	`config/secrets/email_password.txt`
itop	hcloud-exporter	`HCLOUD_EXPORTER_TOKEN=file:///run/secrets/...` (promhippie)	`config/secrets/hcloud_token.txt`
nextcloud	nc-exporter	`NEXTCLOUD_AUTH_TOKEN=@/run/secrets/...` (xperimental)	`config/secrets/exporter_token.txt`
ldap	app (nfrastack)	`ADMIN_PASS_FILE=...`, `CONFIG_PASS_FILE=...`	`config/secrets/ldap_{admin,config}_password.txt`

Not migrated (no native _FILE support):

Stack	Secret	Reason
secure	Keycloak `KC_DB_PASSWORD`	Keycloak image does not support `_FILE` env vars
itop	matrix-alertmanager `MATRIX_TOKEN`, `APP_ALERTMANAGER_SECRET`	Image does not support `_FILE`
paperless	OIDC client secret in `PAPERLESS_SOCIALACCOUNT_PROVIDERS`	Embedded in JSON blob
friendica	Nextcloud stats token	Third-party exporter, no `_FILE` support