Add Nextcloud cloud storage role with split Redis caching strategy

## New Features

- **Nextcloud Role**: Complete cloud storage deployment using Podman Quadlet
  - FPM variant with Caddy reverse proxy and FastCGI
  - PostgreSQL database via Unix socket
  - Valkey/Redis for app-level caching and file locking
  - Automatic HTTPS with Let's Encrypt via Caddy
  - Dual-root pattern: Caddy serves static assets, FPM handles PHP

- **Split Caching Strategy**: Redis caching WITHOUT Redis sessions
  - Custom redis.config.php template for app-level caching only
  - File-based PHP sessions for stability (avoids session lock issues)
  - Prevents cascading failures from session lock contention
  - Documented in role README with detailed rationale

## Infrastructure Updates

- **Socket Permissions**: Update PostgreSQL and Valkey to mode 777
  - Required for containers that switch users (root → www-data)
  - Nextcloud container loses supplementary groups on user switch
  - Security maintained via password authentication (scram-sha-256, requirepass)
  - Documented socket permission architecture in docs/

- **PostgreSQL**: Export client group GID as fact for dependent roles
- **Valkey**: Export client group GID as fact, update socket fix service

## Documentation

- New: docs/socket-permissions-architecture.md
  - Explains 777 vs 770 socket permission trade-offs
  - Documents why group-based access doesn't work for user-switching containers
  - Provides TCP alternative for stricter security requirements

- Updated: All role READMEs with socket permission notes
- New: Nextcloud README with comprehensive deployment, troubleshooting, and Redis architecture documentation

## Configuration

- host_vars: Add Nextcloud vault variables and configuration
- site.yml: Include Nextcloud role in main playbook

## Technical Details

**Why disable Redis sessions?**

The official Nextcloud container enables Redis session handling via REDIS_HOST env var,
which causes severe performance issues:

1. Session lock contention under high concurrency (browser parallel asset requests)
2. Infinite lock retries (default lock_retries=-1) blocking FPM workers
3. Timeout orphaning: reverse proxy kills connection, worker keeps lock
4. Worker pool exhaustion: all 5 default workers blocked on same session lock
5. Cascading failure: new requests queue, more timeouts, more orphaned locks

Solution: Use file-based sessions (reliable, fast for single-server) while keeping
Redis for distributed cache and transactional file locking via custom config file.

This provides optimal performance without the complexity of Redis session debugging.

Tested: Fresh deployment on arch-vps (69.62.119.31)
Domain: https://cloud.jnss.me/
This commit is contained in:
2025-12-14 22:07:08 +01:00
parent 8e8aabd5e7
commit 4f8da38ca6
24 changed files with 1379 additions and 8 deletions

View File

@@ -0,0 +1,71 @@
# Nextcloud Cloud Storage Service
# Caddy reverse proxy to FPM container with FastCGI transport
# Based on official Caddy php_fastcgi Docker example and Nextcloud NGINX config
{{ nextcloud_domain }} {
# Caddy root - host path where static files exist for serving
# This allows Caddy to find files to serve directly (CSS, JS, images)
root * {{ nextcloud_html_dir }}
# .well-known redirects for CalDAV/CardDAV (must be before php_fastcgi)
redir /.well-known/carddav /remote.php/dav 301
redir /.well-known/caldav /remote.php/dav 301
# Handle .well-known requests that aren't explicitly redirected above
# Let Nextcloud's API handle all other /.well-known/* URIs
redir /.well-known/* /index.php{uri} 301
# Block access to sensitive directories (adapted from NGINX config)
# Match both the directory itself and anything under it
@forbidden {
path /build /build/*
path /tests /tests/*
path /config /config/*
path /lib /lib/*
path /3rdparty /3rdparty/*
path /templates /templates/*
path /data /data/*
path /.* /autotest* /occ* /issue* /indie* /db_* /console*
}
respond @forbidden 404
# PHP-FPM with container root for SCRIPT_FILENAME
# The nested 'root' directive tells FPM where files are in the container
# Per official Caddy docs: https://caddyserver.com/docs/caddyfile/directives/php_fastcgi
php_fastcgi 127.0.0.1:{{ nextcloud_fpm_port }} {
root /var/www/html
env front_controller_active true
env modHeadersAvailable true
}
# Serve static files directly (CSS, JS, images, fonts, etc.)
# Disable index serving to let php_fastcgi handle / and /index.php
# This prevents index.html from being served instead of routing to index.php
file_server {
index off
}
# Security headers (adapted from Nextcloud NGINX config)
header {
# HSTS with preload
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
# Prevent embedding in frames from other origins
X-Frame-Options "SAMEORIGIN"
# Prevent MIME type sniffing
X-Content-Type-Options "nosniff"
# XSS protection
X-XSS-Protection "1; mode=block"
# Referrer policy
Referrer-Policy "no-referrer"
# Disable FLoC tracking
Permissions-Policy "interest-cohort=()"
# Remove server header
-Server
}
# Logging
log {
output file {{ caddy_log_dir }}/nextcloud.log
level INFO
format json
}
}

View File

@@ -0,0 +1,44 @@
[Unit]
Description=Nextcloud Cloud Storage Container (FPM)
After=network-online.target postgresql.service valkey.service
Wants=network-online.target
[Container]
ContainerName=nextcloud
Image={{ nextcloud_image }}:{{ nextcloud_version }}
EnvironmentFile={{ nextcloud_home }}/.env
# Note: Container runs as root initially for entrypoint scripts,
# then switches to www-data (UID 33) for PHP-FPM process.
# This is the default behavior of the official Nextcloud image.
# Socket access works via 777 permissions (see infrastructure role docs)
# Volume mounts
# Application files (world-readable for Caddy to serve static assets)
Volume={{ nextcloud_html_dir }}:/var/www/html:Z
# User data (private - only container can access)
Volume={{ nextcloud_data_dir }}:/var/www/html/data:Z
# Configuration (private - contains secrets)
Volume={{ nextcloud_config_dir }}:/var/www/html/config:Z
# Custom apps (world-readable)
Volume={{ nextcloud_custom_apps_dir }}:/var/www/html/custom_apps:Z
# Redis session configuration override (zz- prefix ensures it loads last)
Volume={{ nextcloud_home }}/redis-session-override.ini:/usr/local/etc/php/conf.d/zz-redis-session-override.ini:Z,ro
# Infrastructure sockets (mounted with world-readable permissions on host)
Volume={{ postgresql_unix_socket_directories }}:{{ postgresql_unix_socket_directories }}:Z
Volume={{ valkey_unix_socket_path | dirname }}:{{ valkey_unix_socket_path | dirname }}:Z
# Expose FPM port to localhost only (Caddy will reverse proxy)
PublishPort=127.0.0.1:{{ nextcloud_fpm_port }}:9000
[Service]
Restart=always
TimeoutStartSec=300
[Install]
WantedBy=multi-user.target

View File

@@ -0,0 +1,50 @@
# Nextcloud Environment Configuration
# Generated by Ansible Nextcloud role
# =================================================================
# Database Configuration (PostgreSQL via Unix Socket)
# =================================================================
POSTGRES_HOST={{ postgresql_unix_socket_directories }}
POSTGRES_DB={{ nextcloud_db_name }}
POSTGRES_USER={{ nextcloud_db_user }}
POSTGRES_PASSWORD={{ nextcloud_db_password }}
# =================================================================
# Admin Account (Auto-configured on first run)
# =================================================================
NEXTCLOUD_ADMIN_USER={{ nextcloud_admin_user }}
NEXTCLOUD_ADMIN_PASSWORD={{ nextcloud_admin_password }}
# =================================================================
# Trusted Domains
# =================================================================
NEXTCLOUD_TRUSTED_DOMAINS={{ nextcloud_trusted_domains }}
# =================================================================
# Redis/Valkey Cache Configuration
# =================================================================
# Note: Nextcloud uses REDIS_* variables even for Valkey (Redis-compatible)
# Socket access works because infrastructure sockets use 777 permissions
# Note: These are disabled since we've encountered slowdowns and issues with redis sessions. Instead nextcloud now uses file sessions.
# REDIS_HOST={{ valkey_unix_socket_path }}
# REDIS_HOST_PASSWORD={{ valkey_password }}
# =================================================================
# Reverse Proxy Configuration
# =================================================================
# These settings tell Nextcloud it's behind a reverse proxy (Caddy)
OVERWRITEPROTOCOL={{ nextcloud_overwriteprotocol }}
OVERWRITEHOST={{ nextcloud_domain }}
TRUSTED_PROXIES=127.0.0.1
# =================================================================
# PHP Configuration
# =================================================================
PHP_MEMORY_LIMIT={{ nextcloud_php_memory_limit }}
PHP_UPLOAD_LIMIT={{ nextcloud_php_upload_limit }}
# =================================================================
# Application Settings
# =================================================================
# Enable automatic updates during container restart
NEXTCLOUD_UPDATE=1

View File

@@ -0,0 +1,16 @@
; Redis Session Lock Override for Nextcloud
; Prevents orphaned session locks from causing infinite hangs
;
; Default Nextcloud container settings:
; redis.session.lock_expire = 0 (locks NEVER expire - causes infinite hangs)
; redis.session.lock_retries = -1 (infinite retries - causes worker exhaustion)
; redis.session.lock_wait_time = 10000 (10 seconds per retry - very slow)
;
; These settings ensure locks auto-expire and failed requests don't block workers forever:
; - Locks expire after 30 seconds (prevents orphaned locks)
; - Max 100 retries = 5 seconds total wait time (prevents infinite loops)
; - 50ms wait between retries (reasonable balance)
redis.session.lock_expire = 30
redis.session.lock_retries = 100
redis.session.lock_wait_time = 50000

View File

@@ -0,0 +1,34 @@
<?php
/**
* Redis/Valkey Caching Configuration for Nextcloud
*
* This file provides Redis caching for Nextcloud application-level operations
* (distributed cache, file locking) WITHOUT enabling Redis for PHP sessions.
*
* IMPORTANT: This overrides the default /usr/src/nextcloud/config/redis.config.php
* which checks for REDIS_HOST environment variable. We deploy this custom version
* to enable Redis caching while keeping PHP sessions file-based for stability.
*
* Why not use REDIS_HOST env var?
* - Setting REDIS_HOST enables BOTH Redis sessions AND Redis caching
* - Redis session handling can cause severe performance issues:
* * Session lock contention under high concurrency
* * Infinite lock retries blocking FPM workers
* * Timeout orphaning leaving locks unreleased
* * Worker pool exhaustion causing cascading failures
*
* This configuration provides the benefits of Redis caching (fast distributed
* cache, reliable file locking) while avoiding the pitfalls of Redis sessions.
*
* Managed by: Ansible Nextcloud role
* Template: roles/nextcloud/templates/redis.config.php.j2
*/
$CONFIG = array(
'memcache.distributed' => '\OC\Memcache\Redis',
'memcache.locking' => '\OC\Memcache\Redis',
'redis' => array(
'host' => '{{ valkey_unix_socket_path }}',
'password' => '{{ valkey_password }}',
),
);