Friendica (opensocial.at)¶
Stack:
stack/opensocial/· Host:phil-app· Updated: 2026-03-01
Largest Friendica instance — 157 GB DB, 80M post-user rows, split PHP-FPM pools for UI / media / fediverse traffic.
Overview¶
High-traffic instance requiring careful pool tuning and DB connection management. Primary performance focus: post-user table at 80M rows dominates all query times.
Architecture¶
PHP-FPM Pools¶
| Pool | Traffic type | PM mode | max_children | Container limit | Replicas |
|---|---|---|---|---|---|
| app-ui | Web UI requests | dynamic | 20 | 10G | 2 |
| app-media | Photo/media serving | dynamic | 20 | 10G | 2 |
| app-fediverse | ActivityPub + Diaspora federation | dynamic | 40 | 8G | 1 |
| cron | Worker daemon | static | 50 | 8G | 1 |
Why dynamic (not ondemand) for fediverse: See ADR-004.
Containers¶
web-ui(Nginx, 2 replicas),web-media(Nginx, 2 replicas),web-fediverse(Nginx),web-avatar(static Nginx)app-ui(PHP-FPM, 2 replicas),app-media(PHP-FPM, 2 replicas),app-fediverse(PHP-FPM)cron(worker daemon)redis,php-fpm-exporter,redis-exporter
Networks¶
compose-opensocial-network(stack-internal)traefik-ingress(HTTP routing)prometheus-network(metrics)mail-network(outgoing mail via Postfix)
Traefik Routing Split¶
| Router | Paths | Backend |
|---|---|---|
opensocial-ui |
Everything else | web-ui → app-ui (2 replicas) |
opensocial-static |
/view, /images, /fonts |
web-ui (no PHP) |
opensocial-media |
/photo/media |
web-media → app-media (2 replicas) |
opensocial-federation |
/inbox, /outbox, /actor, /followers, /following, /receive |
web-fediverse → app-fediverse |
opensocial_avatar |
/avatar |
web-avatar (static) |
Both opensocial.at and www.opensocial.at must be in the federation router rule. Missing www. causes federation requests via www. to bypass the fediverse pool.
PHP Configuration¶
| File | Container | memory_limit |
|---|---|---|
config/app/www.ui.conf |
app-ui, app-media | FPM pool config |
config/app/www.fediverse.conf |
app-fediverse | FPM pool config |
config/app/app.ini |
all FPM pools | 512M |
config/app/opensocial.ini |
cron | 2G |
Per-task Friendica limit in config/app/local.config.php:
'system' => [
'worker_memory_limit' => 768,
'default_socket_timeout' => 10, // caps remote photo fetch to 10s
],
Cron tmpfs¶
The cron container uses a 1G tmpfs at /tmp/opensocial.at for Guzzle temp files (sys_temp_dir in cron.ini, TMPDIR env var). Default 256M was too small — 57k+ ENOSPC errors observed.
MariaDB¶
- DB host:
phil-db(10.42.10.3), userfriendica_opensocial(UI),friendica_opensocial_cron(cron) max_user_connections: 100 per user (was 20 — too low for multiple FPM pools)max_statement_time: 30s (UI user), 120s (cron user) — kills runaway circle widget queries
PHP-FPM Exporter¶
Scrape URIs must name each replica explicitly:
- opensocial-app-ui-1, opensocial-app-ui-2
- opensocial-app-media-1, opensocial-app-media-2
- opensocial-app-fediverse-1
Round-robin DNS aliases like app-media only hit one replica.
Performance Data (Feb 2026)¶
| Metric | Value |
|---|---|
| DB size | 157 GB |
post-user rows |
80 Million |
contact rows |
1.2 Million |
storage (photos in DB) |
764k rows / 22 GB |
| Slow requests >2s (10 min window) | ~9 (1%) |
Top Slow Requests¶
| Path | Count/day | Root cause |
|---|---|---|
GET /photo/preview/640/{id} |
252x | Remote image proxy blocks FPM worker |
POST /inbox |
99x | AP inbox (routed to fediverse pool) |
POST /receive/public |
28x | Diaspora federation |
GET /search |
11x | Full-text scan, 80M rows |
Fix: default_socket_timeout = 10 in app.ini caps photo fetch at 10s.
Operations¶
Health Check¶
# Pool status
sudo docker compose -f /opt/docker/stack/opensocial/docker-compose.yml ps
# Worker queue
sudo docker exec opensocial-cron-1 php /var/www/html/bin/console friendica worker:status
# DB connections
# On phil-db: SHOW PROCESSLIST WHERE user = 'friendica_opensocial';
Upgrade¶
cd /opt/docker/stack/opensocial
sudo docker compose pull && sudo docker compose up -d
sudo docker exec opensocial-cron-1 php /var/www/html/bin/console dbstructure update
Backup¶
Borgmatic config: borgmatic.d/opensocial.yaml. Backs up opensocial_data and opensocial_storage volumes.
Retention: 2 hourly / 7 daily / 4 weekly / 6 monthly.
Heartbeat: Uptime Kuma push monitor "Backup - Opensocial".
Note: The opensocial storage volume is large (~22 GB binary photos). First backup is slow; incremental runs fast due to Borg deduplication.
Common Commands¶
# Check ImageMagick policy
sudo docker exec opensocial-app-ui-1 cat /etc/ImageMagick-6/policy.xml | grep area
# Fix stuck user deletion (ExpireAndRemoveUsers)
# See pitfalls section below
# Check DB statement timeout
# On phil-db: SELECT user, priv->>'$.max_statement_time' FROM mysql.global_priv WHERE user LIKE 'friendica_opensocial%';
Pitfalls {#pitfalls}¶
Cron container needs REDIS_HOST¶
Must have REDIS_HOST=redis set even with FRIENDICA_DISTRIBUTED_CACHE_DRIVER=redis configured. Without it, worker cache coordination breaks → backlog growth → FriendicaWorkerStale alerts.
worker_memory_limit overrides PHP ini per task¶
PHP Fatal error: Allowed memory size of 536870912 bytes exhausted in Image.php despite opensocial.ini memory_limit=2G means Friendica's per-task ini_set is 512 MiB (Friendica default). Set 'worker_memory_limit' => 768 in local.config.php → system.
ImageMagick area policy allows oversized image allocations {#imagick-area-policy}¶
Symptom: PHP Fatal error: ... (tried to allocate 185015736 bytes) in Image.php:228 when federation contacts have large avatars (e.g., 7000×6600 px ≈ 46 MP).
Root cause: Default 128MP area limit → ImageMagick allocates 185 MB raw bitmap in PHP heap → combined with normal worker state (~327 MB) → exceeds 512 MiB per-task limit → PHP Fatal Error (NOT caught by Friendica's exception handlers).
Fix: area = 32MP in imagemagick-policy.xml. ImageMagick evaluates policy in the C layer, throws ImagickException before PHP heap allocation — caught gracefully.
Circle widget query pile-up (CPU saturation)¶
The circle/group sidebar widget runs a correlated subquery for each circle scanning the full post-user table (80M rows, 60-120s per execution). On page reload, new PHP workers fire the same query → 30+ parallel full table scans → CPU saturation (load 60+, 1200% CPU observed Feb 2026).
Fix: max_statement_time = 30 on UI DB users kills runaway queries automatically. Set via:
UPDATE mysql.global_priv SET priv = JSON_SET(priv, '$.max_statement_time', 30)
WHERE user = 'friendica_opensocial';
FLUSH PRIVILEGES;
ExpireAndRemoveUsers stuck in FK constraint loop¶
User marked account_removed=1 but worker can't delete — post-user.psid FK references a permissionset from the deleted user.
Diagnosis:
SELECT pu.uid, pu.id, pu.psid, ps.uid AS ps_uid
FROM `post-user` pu JOIN permissionset ps ON pu.psid = ps.id
WHERE ps.uid = <deleted_uid>;
Fix: Remap psid to global public permissionset:
UPDATE `post-user` SET psid = 0 WHERE id = <row_id> AND psid = <orphaned_psid>;
Friendica Traefik routing must include /receive and all Host variants¶
Missing /receive in the fediverse router causes UI workers to block on slow federation requests. Federation router must cover both opensocial.at and www.opensocial.at.
Nginx set $variable does not load-balance across FPM replicas¶
set $php_backend ...; fastcgi_pass $php_backend; only uses the first IP returned by Docker DNS. Use a proper upstream block for round-robin across replicas.
Photo preview delays from slow remote servers¶
/photo/preview/{size}/{id} proxies external images in real-time — worker blocks for up to default_socket_timeout.
Fix: default_socket_timeout = 10 in app.ini.
MariaDB max_user_connections too low¶
With 5 FPM pools × 20-40 workers each, max_user_connections=20 is exhausted immediately under load. Set to 100.