Add Nextcloud cloud storage role with split Redis caching strategy
## New Features - **Nextcloud Role**: Complete cloud storage deployment using Podman Quadlet - FPM variant with Caddy reverse proxy and FastCGI - PostgreSQL database via Unix socket - Valkey/Redis for app-level caching and file locking - Automatic HTTPS with Let's Encrypt via Caddy - Dual-root pattern: Caddy serves static assets, FPM handles PHP - **Split Caching Strategy**: Redis caching WITHOUT Redis sessions - Custom redis.config.php template for app-level caching only - File-based PHP sessions for stability (avoids session lock issues) - Prevents cascading failures from session lock contention - Documented in role README with detailed rationale ## Infrastructure Updates - **Socket Permissions**: Update PostgreSQL and Valkey to mode 777 - Required for containers that switch users (root → www-data) - Nextcloud container loses supplementary groups on user switch - Security maintained via password authentication (scram-sha-256, requirepass) - Documented socket permission architecture in docs/ - **PostgreSQL**: Export client group GID as fact for dependent roles - **Valkey**: Export client group GID as fact, update socket fix service ## Documentation - New: docs/socket-permissions-architecture.md - Explains 777 vs 770 socket permission trade-offs - Documents why group-based access doesn't work for user-switching containers - Provides TCP alternative for stricter security requirements - Updated: All role READMEs with socket permission notes - New: Nextcloud README with comprehensive deployment, troubleshooting, and Redis architecture documentation ## Configuration - host_vars: Add Nextcloud vault variables and configuration - site.yml: Include Nextcloud role in main playbook ## Technical Details **Why disable Redis sessions?** The official Nextcloud container enables Redis session handling via REDIS_HOST env var, which causes severe performance issues: 1. Session lock contention under high concurrency (browser parallel asset requests) 2. Infinite lock retries (default lock_retries=-1) blocking FPM workers 3. Timeout orphaning: reverse proxy kills connection, worker keeps lock 4. Worker pool exhaustion: all 5 default workers blocked on same session lock 5. Cascading failure: new requests queue, more timeouts, more orphaned locks Solution: Use file-based sessions (reliable, fast for single-server) while keeping Redis for distributed cache and transactional file locking via custom config file. This provides optimal performance without the complexity of Redis session debugging. Tested: Fresh deployment on arch-vps (69.62.119.31) Domain: https://cloud.jnss.me/
This commit is contained in:
71
roles/nextcloud/templates/nextcloud.caddy.j2
Normal file
71
roles/nextcloud/templates/nextcloud.caddy.j2
Normal file
@@ -0,0 +1,71 @@
|
||||
# Nextcloud Cloud Storage Service
|
||||
# Caddy reverse proxy to FPM container with FastCGI transport
|
||||
# Based on official Caddy php_fastcgi Docker example and Nextcloud NGINX config
|
||||
{{ nextcloud_domain }} {
|
||||
# Caddy root - host path where static files exist for serving
|
||||
# This allows Caddy to find files to serve directly (CSS, JS, images)
|
||||
root * {{ nextcloud_html_dir }}
|
||||
|
||||
# .well-known redirects for CalDAV/CardDAV (must be before php_fastcgi)
|
||||
redir /.well-known/carddav /remote.php/dav 301
|
||||
redir /.well-known/caldav /remote.php/dav 301
|
||||
|
||||
# Handle .well-known requests that aren't explicitly redirected above
|
||||
# Let Nextcloud's API handle all other /.well-known/* URIs
|
||||
redir /.well-known/* /index.php{uri} 301
|
||||
|
||||
# Block access to sensitive directories (adapted from NGINX config)
|
||||
# Match both the directory itself and anything under it
|
||||
@forbidden {
|
||||
path /build /build/*
|
||||
path /tests /tests/*
|
||||
path /config /config/*
|
||||
path /lib /lib/*
|
||||
path /3rdparty /3rdparty/*
|
||||
path /templates /templates/*
|
||||
path /data /data/*
|
||||
path /.* /autotest* /occ* /issue* /indie* /db_* /console*
|
||||
}
|
||||
respond @forbidden 404
|
||||
|
||||
# PHP-FPM with container root for SCRIPT_FILENAME
|
||||
# The nested 'root' directive tells FPM where files are in the container
|
||||
# Per official Caddy docs: https://caddyserver.com/docs/caddyfile/directives/php_fastcgi
|
||||
php_fastcgi 127.0.0.1:{{ nextcloud_fpm_port }} {
|
||||
root /var/www/html
|
||||
env front_controller_active true
|
||||
env modHeadersAvailable true
|
||||
}
|
||||
|
||||
# Serve static files directly (CSS, JS, images, fonts, etc.)
|
||||
# Disable index serving to let php_fastcgi handle / and /index.php
|
||||
# This prevents index.html from being served instead of routing to index.php
|
||||
file_server {
|
||||
index off
|
||||
}
|
||||
|
||||
# Security headers (adapted from Nextcloud NGINX config)
|
||||
header {
|
||||
# HSTS with preload
|
||||
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
|
||||
# Prevent embedding in frames from other origins
|
||||
X-Frame-Options "SAMEORIGIN"
|
||||
# Prevent MIME type sniffing
|
||||
X-Content-Type-Options "nosniff"
|
||||
# XSS protection
|
||||
X-XSS-Protection "1; mode=block"
|
||||
# Referrer policy
|
||||
Referrer-Policy "no-referrer"
|
||||
# Disable FLoC tracking
|
||||
Permissions-Policy "interest-cohort=()"
|
||||
# Remove server header
|
||||
-Server
|
||||
}
|
||||
|
||||
# Logging
|
||||
log {
|
||||
output file {{ caddy_log_dir }}/nextcloud.log
|
||||
level INFO
|
||||
format json
|
||||
}
|
||||
}
|
||||
44
roles/nextcloud/templates/nextcloud.container
Normal file
44
roles/nextcloud/templates/nextcloud.container
Normal file
@@ -0,0 +1,44 @@
|
||||
[Unit]
|
||||
Description=Nextcloud Cloud Storage Container (FPM)
|
||||
After=network-online.target postgresql.service valkey.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Container]
|
||||
ContainerName=nextcloud
|
||||
Image={{ nextcloud_image }}:{{ nextcloud_version }}
|
||||
EnvironmentFile={{ nextcloud_home }}/.env
|
||||
|
||||
# Note: Container runs as root initially for entrypoint scripts,
|
||||
# then switches to www-data (UID 33) for PHP-FPM process.
|
||||
# This is the default behavior of the official Nextcloud image.
|
||||
# Socket access works via 777 permissions (see infrastructure role docs)
|
||||
|
||||
# Volume mounts
|
||||
# Application files (world-readable for Caddy to serve static assets)
|
||||
Volume={{ nextcloud_html_dir }}:/var/www/html:Z
|
||||
|
||||
# User data (private - only container can access)
|
||||
Volume={{ nextcloud_data_dir }}:/var/www/html/data:Z
|
||||
|
||||
# Configuration (private - contains secrets)
|
||||
Volume={{ nextcloud_config_dir }}:/var/www/html/config:Z
|
||||
|
||||
# Custom apps (world-readable)
|
||||
Volume={{ nextcloud_custom_apps_dir }}:/var/www/html/custom_apps:Z
|
||||
|
||||
# Redis session configuration override (zz- prefix ensures it loads last)
|
||||
Volume={{ nextcloud_home }}/redis-session-override.ini:/usr/local/etc/php/conf.d/zz-redis-session-override.ini:Z,ro
|
||||
|
||||
# Infrastructure sockets (mounted with world-readable permissions on host)
|
||||
Volume={{ postgresql_unix_socket_directories }}:{{ postgresql_unix_socket_directories }}:Z
|
||||
Volume={{ valkey_unix_socket_path | dirname }}:{{ valkey_unix_socket_path | dirname }}:Z
|
||||
|
||||
# Expose FPM port to localhost only (Caddy will reverse proxy)
|
||||
PublishPort=127.0.0.1:{{ nextcloud_fpm_port }}:9000
|
||||
|
||||
[Service]
|
||||
Restart=always
|
||||
TimeoutStartSec=300
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
50
roles/nextcloud/templates/nextcloud.env.j2
Normal file
50
roles/nextcloud/templates/nextcloud.env.j2
Normal file
@@ -0,0 +1,50 @@
|
||||
# Nextcloud Environment Configuration
|
||||
# Generated by Ansible Nextcloud role
|
||||
|
||||
# =================================================================
|
||||
# Database Configuration (PostgreSQL via Unix Socket)
|
||||
# =================================================================
|
||||
POSTGRES_HOST={{ postgresql_unix_socket_directories }}
|
||||
POSTGRES_DB={{ nextcloud_db_name }}
|
||||
POSTGRES_USER={{ nextcloud_db_user }}
|
||||
POSTGRES_PASSWORD={{ nextcloud_db_password }}
|
||||
|
||||
# =================================================================
|
||||
# Admin Account (Auto-configured on first run)
|
||||
# =================================================================
|
||||
NEXTCLOUD_ADMIN_USER={{ nextcloud_admin_user }}
|
||||
NEXTCLOUD_ADMIN_PASSWORD={{ nextcloud_admin_password }}
|
||||
|
||||
# =================================================================
|
||||
# Trusted Domains
|
||||
# =================================================================
|
||||
NEXTCLOUD_TRUSTED_DOMAINS={{ nextcloud_trusted_domains }}
|
||||
|
||||
# =================================================================
|
||||
# Redis/Valkey Cache Configuration
|
||||
# =================================================================
|
||||
# Note: Nextcloud uses REDIS_* variables even for Valkey (Redis-compatible)
|
||||
# Socket access works because infrastructure sockets use 777 permissions
|
||||
# Note: These are disabled since we've encountered slowdowns and issues with redis sessions. Instead nextcloud now uses file sessions.
|
||||
# REDIS_HOST={{ valkey_unix_socket_path }}
|
||||
# REDIS_HOST_PASSWORD={{ valkey_password }}
|
||||
|
||||
# =================================================================
|
||||
# Reverse Proxy Configuration
|
||||
# =================================================================
|
||||
# These settings tell Nextcloud it's behind a reverse proxy (Caddy)
|
||||
OVERWRITEPROTOCOL={{ nextcloud_overwriteprotocol }}
|
||||
OVERWRITEHOST={{ nextcloud_domain }}
|
||||
TRUSTED_PROXIES=127.0.0.1
|
||||
|
||||
# =================================================================
|
||||
# PHP Configuration
|
||||
# =================================================================
|
||||
PHP_MEMORY_LIMIT={{ nextcloud_php_memory_limit }}
|
||||
PHP_UPLOAD_LIMIT={{ nextcloud_php_upload_limit }}
|
||||
|
||||
# =================================================================
|
||||
# Application Settings
|
||||
# =================================================================
|
||||
# Enable automatic updates during container restart
|
||||
NEXTCLOUD_UPDATE=1
|
||||
16
roles/nextcloud/templates/redis-session-override.ini.j2
Normal file
16
roles/nextcloud/templates/redis-session-override.ini.j2
Normal file
@@ -0,0 +1,16 @@
|
||||
; Redis Session Lock Override for Nextcloud
|
||||
; Prevents orphaned session locks from causing infinite hangs
|
||||
;
|
||||
; Default Nextcloud container settings:
|
||||
; redis.session.lock_expire = 0 (locks NEVER expire - causes infinite hangs)
|
||||
; redis.session.lock_retries = -1 (infinite retries - causes worker exhaustion)
|
||||
; redis.session.lock_wait_time = 10000 (10 seconds per retry - very slow)
|
||||
;
|
||||
; These settings ensure locks auto-expire and failed requests don't block workers forever:
|
||||
; - Locks expire after 30 seconds (prevents orphaned locks)
|
||||
; - Max 100 retries = 5 seconds total wait time (prevents infinite loops)
|
||||
; - 50ms wait between retries (reasonable balance)
|
||||
|
||||
redis.session.lock_expire = 30
|
||||
redis.session.lock_retries = 100
|
||||
redis.session.lock_wait_time = 50000
|
||||
34
roles/nextcloud/templates/redis.config.php.j2
Normal file
34
roles/nextcloud/templates/redis.config.php.j2
Normal file
@@ -0,0 +1,34 @@
|
||||
<?php
|
||||
/**
|
||||
* Redis/Valkey Caching Configuration for Nextcloud
|
||||
*
|
||||
* This file provides Redis caching for Nextcloud application-level operations
|
||||
* (distributed cache, file locking) WITHOUT enabling Redis for PHP sessions.
|
||||
*
|
||||
* IMPORTANT: This overrides the default /usr/src/nextcloud/config/redis.config.php
|
||||
* which checks for REDIS_HOST environment variable. We deploy this custom version
|
||||
* to enable Redis caching while keeping PHP sessions file-based for stability.
|
||||
*
|
||||
* Why not use REDIS_HOST env var?
|
||||
* - Setting REDIS_HOST enables BOTH Redis sessions AND Redis caching
|
||||
* - Redis session handling can cause severe performance issues:
|
||||
* * Session lock contention under high concurrency
|
||||
* * Infinite lock retries blocking FPM workers
|
||||
* * Timeout orphaning leaving locks unreleased
|
||||
* * Worker pool exhaustion causing cascading failures
|
||||
*
|
||||
* This configuration provides the benefits of Redis caching (fast distributed
|
||||
* cache, reliable file locking) while avoiding the pitfalls of Redis sessions.
|
||||
*
|
||||
* Managed by: Ansible Nextcloud role
|
||||
* Template: roles/nextcloud/templates/redis.config.php.j2
|
||||
*/
|
||||
|
||||
$CONFIG = array(
|
||||
'memcache.distributed' => '\OC\Memcache\Redis',
|
||||
'memcache.locking' => '\OC\Memcache\Redis',
|
||||
'redis' => array(
|
||||
'host' => '{{ valkey_unix_socket_path }}',
|
||||
'password' => '{{ valkey_password }}',
|
||||
),
|
||||
);
|
||||
Reference in New Issue
Block a user