Add comprehensive devigo deployment documentation

- Complete deployment guide with architecture overview
- Documented all deployment steps and current status
- Added lessons learned from deployment challenges
- Comprehensive troubleshooting guide
- Maintenance procedures and security considerations
- Migration notes from arch-vps to mini-vps

Key sections:
- OAuth flow and authentication
- CI/CD deployment pipeline
- MIME type handling strategy
- Podman authentication for systemd
- Future improvements and technical debt tracking

docs/devigo-deployment-guide.md

@@ -0,0 +1,414 @@
+# Devigo Deployment Guide
+
+**Status**: ✅ **DEPLOYED AND OPERATIONAL**  
+**Environment**: Production (mini-vps)  
+**Last Updated**: 2025-12-16  
+**Deployment Date**: 2025-12-15
+
+## Overview
+
+This guide documents the complete deployment of devigo.no (sales training website) from the legacy arch-vps server to the production mini-vps environment. The deployment uses:
+
+- **Infrastructure Management**: Ansible (rick-infra repository)
+- **Application Code**: Hugo static site (rustan repository)  
+- **CI/CD**: GitHub Actions
+- **Container Runtime**: Podman with Quadlet/systemd integration
+- **Web Server**: Caddy (reverse proxy with automatic HTTPS)
+- **Content Management**: Decap CMS with GitHub OAuth
+
+## Architecture
+
+### Request Flow
+```
+Internet → Caddy (443) → nginx container (9080) → Static files
+           ↓
+       TLS termination
+       Security headers
+       Access logging
+```
+
+### Services
+
+| Service | Domain | Container | Purpose |
+|---------|--------|-----------|---------|
+| devigo-site | devigo.no | ghcr.io/jnschaffer/rustan:prod | Main website |
+| devigo-site | www.devigo.no | (redirects to apex) | WWW redirect |
+| devigo-decap-oauth | decap.jnss.me | alukovenko/decapcms-oauth2 | OAuth proxy for CMS |
+
+### OAuth Flow
+```
+1. User visits devigo.no/admin
+2. Clicks "Login with GitHub"
+3. Popup opens → decap.jnss.me/auth
+4. Redirects to GitHub OAuth
+5. GitHub redirects → decap.jnss.me/callback
+6. Callback validates origin (TRUSTED_ORIGINS)
+7. postMessage sends token to parent window
+8. Popup closes, CMS authenticated ✅
+```
+
+### Deployment Flow
+```
+Developer → git push origin/prod
+           ↓
+      GitHub Actions
+           ↓
+    Build Hugo site → Docker image
+           ↓
+    Push to GHCR (ghcr.io/jnschaffer/rustan:prod)
+           ↓
+    SSH to mini-vps → podman pull (uses auth.json)
+           ↓
+    systemctl restart devigo-site
+           ↓
+    Podman Quadlet pulls latest image
+           ↓
+    Container starts with new code ✅
+```
+
+## Completed Deployment Steps ✅
+
+### Infrastructure Configuration
+- ✅ Created Ansible role `roles/devigo/` with Podman Quadlet support
+- ✅ Updated `group_vars/production/main.yml` with devigo configuration
+- ✅ Encrypted OAuth credentials in `group_vars/production/vault.yml`
+- ✅ Updated `playbooks/production.yml` to include devigo role
+- ✅ Deployed infrastructure to mini-vps
+
+### Application Configuration  
+- ✅ Changed `rustan/hugo.toml` baseURL from `https://www.devigo.no/` to `https://devigo.no/`
+- ✅ Updated `rustan/.github/workflows/deploy.yml` for Podman + systemctl deployment
+- ✅ Fixed nginx.conf MIME type handling (location-specific YAML override)
+- ✅ Implemented JavaScript-based Decap CMS initialization
+
+### Authentication & Security
+- ✅ Created GitHub Personal Access Token for GHCR
+- ✅ Configured Podman authentication (`/etc/containers/auth.json`)
+- ✅ Added `REGISTRY_AUTH_FILE` environment to Quadlet
+- ✅ Fixed OAuth `TRUSTED_ORIGINS` environment variable (plural form)
+- ✅ Generated SSH key for GitHub Actions deployment
+- ✅ Updated GitHub repository secrets
+
+### DNS & Networking
+- ✅ Updated DNS records to point to mini-vps (72.62.91.251)
+- ✅ Configured Caddy reverse proxy
+- ✅ Obtained TLS certificates via Let's Encrypt
+- ✅ Configured GitHub OAuth app callback URLs
+
+### Testing & Verification
+- ✅ Verified MIME types (HTML: text/html, CSS: text/css, YAML: text/yaml)
+- ✅ Tested Decap CMS initialization
+- ✅ Verified OAuth authentication flow
+- ✅ Confirmed container image pulls work
+- ✅ Tested automated GitHub Actions deployment
+- ✅ Confirmed site accessible at devigo.no
+
+## Lessons Learned
+
+### Critical Fixes Applied
+
+#### 1. TRUSTED_ORIGINS Variable Name (OAuth)
+
+**Issue**: OAuth showing "Origin not trusted: https://devigo.no" error
+
+**Root Cause**: The decapcms-oauth2 service source code expects:
+- `TRUSTED_ORIGIN` (singular) for a single origin
+- `TRUSTED_ORIGINS` (plural) for comma-separated multiple origins
+
+We were using `TRUSTED_ORIGIN=https://devigo.no,https://www.devigo.no` which treated the entire string as ONE origin.
+
+**Solution**: Changed to `TRUSTED_ORIGINS=https://devigo.no,https://www.devigo.no` (plural)
+
+**Files Modified**:
+- `roles/devigo/templates/devigo-decap-oauth.env.j2`
+- `roles/devigo/defaults/main.yml`
+
+#### 2. MIME Type Handling Strategy (nginx)
+
+**Issue**: Adding `types { text/yaml yml yaml; }` in nginx.conf broke all MIME types:
+- HTML → `application/octet-stream`
+- CSS → `application/octet-stream`  
+- JS → `application/octet-stream`
+
+**Root Cause**: Server-level `types {}` block **replaces** all MIME types, not extends them.
+
+**Solution**: Use location-specific override instead:
+```nginx
+location ~ \.(yml|yaml)$ {
+    default_type text/yaml;
+}
+```
+
+**Outcome**: Standard MIME types work, YAML gets special handling.
+
+#### 3. Decap CMS Initialization Method
+
+**Issue**: Decap CMS failing to load config with `<link href="config.yml" type="text/yaml">` approach
+
+**Root Cause**: Even with JavaScript initialization, Decap CMS validates Content-Type header when fetching via link tag.
+
+**Solution**: Switched to JavaScript manual initialization:
+```javascript
+window.CMS_MANUAL_INIT = true;
+CMS.init({ config: { load_config_file: '/admin/config.yml' } });
+```
+
+**Benefits**:
+- Eliminates YAML MIME type dependency for link element
+- More portable across server configurations
+- Matches working configuration from previous deployment
+
+#### 4. Podman Authentication for systemd Services
+
+**Issue**: `podman pull` worked manually but failed when systemd restarted service
+
+**Root Cause**: Podman wasn't finding `/etc/containers/auth.json` when run by systemd
+
+**Solution**: Added environment variable to Quadlet:
+```ini
+[Service]
+Environment=REGISTRY_AUTH_FILE=/etc/containers/auth.json
+```
+
+**Outcome**: Automated deployments now pull private images successfully.
+
+## Current Infrastructure State
+
+### Services Status
+
+| Service | Status | Container ID | Image Version |
+|---------|--------|--------------|---------------|
+| devigo-site | ✅ Running | a61788a8a673 | 2025-12-15T23:02:XX |
+| devigo-decap-oauth | ✅ Running | 91fc8c4b21e3 | latest |
+| caddy | ✅ Running | N/A | Native service |
+
+### Verification Commands
+
+```bash
+# Check all services
+ansible mini-vps -m shell -a "systemctl status devigo-site devigo-decap-oauth caddy --no-pager"
+
+# View recent logs
+ansible mini-vps -m shell -a "journalctl -u devigo-site -n 20 --no-pager"
+
+# Check containers
+ansible mini-vps -m shell -a "podman ps"
+
+# Test endpoints
+curl -I https://devigo.no
+curl -I https://www.devigo.no  # Should 301 redirect to apex
+curl -I https://decap.jnss.me
+
+# Verify MIME types
+curl -I https://devigo.no/ | grep content-type  # Should be text/html
+curl -I https://devigo.no/admin/config.yml | grep content-type  # Should be text/yaml
+```
+
+## Maintenance Guide
+
+### Updating Content via Decap CMS
+
+1. Visit https://devigo.no/admin
+2. Click "Login with GitHub"
+3. Edit content in the CMS interface
+4. Changes automatically commit to GitHub
+5. GitHub Actions builds and deploys new version
+6. Site updates within 2-3 minutes
+
+### Updating Application Code
+
+```bash
+# In rustan repository
+cd ~/rustan
+git checkout prod
+# Make changes to code
+git add .
+git commit -m "Description of changes"
+git push origin prod
+
+# GitHub Actions automatically:
+# 1. Builds Hugo site
+# 2. Creates Docker image
+# 3. Pushes to GHCR
+# 4. SSHs to mini-vps
+# 5. Pulls latest image
+# 6. Restarts service
+```
+
+### Rotating GitHub PAT for GHCR
+
+When the Personal Access Token expires:
+
+```bash
+# 1. Create new token at https://github.com/settings/tokens/new?scopes=read:packages
+
+# 2. Update Ansible vault
+cd ~/rick-infra
+ansible-vault edit group_vars/production/vault.yml
+# Update: vault_github_token: "ghp_NEW_TOKEN_HERE"
+
+# 3. Deploy updated authentication
+ansible-playbook playbooks/production.yml --tags podman
+
+# 4. Verify
+ansible mini-vps -m shell -a "podman login ghcr.io --get-login"
+# Should show: jnschaffer
+```
+
+### Updating OAuth Trusted Origins
+
+If you need to add/change trusted origins for OAuth:
+
+```bash
+# 1. Update defaults
+cd ~/rick-infra
+# Edit roles/devigo/defaults/main.yml
+# Update: devigo_oauth_trusted_origins: "https://devigo.no,https://new-domain.com"
+
+# 2. Deploy changes
+ansible-playbook playbooks/production.yml --tags devigo
+
+# 3. Restart OAuth service
+ansible mini-vps -m shell -a "systemctl restart devigo-decap-oauth"
+
+# 4. Test OAuth flow at https://devigo.no/admin
+```
+
+### Manual Container Updates
+
+```bash
+# Pull latest image manually
+ansible mini-vps -m shell -a "podman pull ghcr.io/jnschaffer/rustan:prod"
+
+# Restart service to use new image
+ansible mini-vps -m shell -a "systemctl restart devigo-site"
+
+# Check auto-update is working
+ansible mini-vps -m shell -a "podman auto-update --dry-run"
+```
+
+## Troubleshooting
+
+See the [Devigo Role README](../roles/devigo/README.md#troubleshooting) for comprehensive troubleshooting steps.
+
+### Quick Reference
+
+**OAuth not working?**
+```bash
+# Check TRUSTED_ORIGINS format
+ansible mini-vps -m shell -a "cat /opt/devigo-oauth/decap-oauth.env | grep TRUSTED"
+# Should be: TRUSTED_ORIGINS=https://devigo.no,https://www.devigo.no
+```
+
+**MIME types wrong?**
+```bash
+# Check config.yml MIME type
+curl -I https://devigo.no/admin/config.yml | grep content-type
+# Should be: content-type: text/yaml
+
+# Check homepage MIME type  
+curl -I https://devigo.no/ | grep content-type
+# Should be: content-type: text/html
+```
+
+**Can't pull images?**
+```bash
+# Test authentication
+ansible mini-vps -m shell -a "podman login ghcr.io --get-login"
+# Should show: jnschaffer (your GitHub username)
+```
+
+## Security Considerations
+
+### Credentials Storage
+
+All sensitive credentials are encrypted in Ansible Vault:
+- GitHub OAuth Client ID & Secret
+- GitHub Personal Access Token for GHCR
+- Deployed files have restrictive permissions (0600 for secrets)
+
+### Network Security
+
+- TLS certificates automatically renewed via Caddy + Let's Encrypt
+- All HTTP traffic redirected to HTTPS
+- Security headers applied by Caddy:
+  - X-Frame-Options: SAMEORIGIN
+  - X-Content-Type-Options: nosniff
+  - X-XSS-Protection: 1; mode=block
+  - Referrer-Policy: strict-origin-when-cross-origin
+  - Permissions-Policy: geolocation=(), microphone=(), camera=()
+
+### Container Security
+
+- Containers run as non-root where possible
+- `NoNewPrivileges=true` in Quadlet files
+- Minimal attack surface (nginx:alpine base image)
+- Regular updates via GitHub Actions rebuild
+
+## Related Documentation
+
+- [Devigo Role README](../roles/devigo/README.md) - Role configuration and usage
+- [Podman Role](../roles/podman/README.md) - Container runtime setup
+- [Caddy Role](../roles/caddy/README.md) - Reverse proxy configuration
+- [Service Integration Guide](./service-integration-guide.md) - How services work together
+
+## Migration Notes
+
+### Differences from arch-vps Deployment
+
+| Aspect | arch-vps (old) | mini-vps (new) |
+|--------|----------------|----------------|
+| Container | Docker + docker-compose | Podman + Quadlet |
+| Service Mgmt | docker-compose | systemd |
+| Deployment | Manual / docker-compose pull | GitHub Actions + systemctl |
+| OAuth | Same service | Same, but TRUSTED_ORIGINS fixed |
+| CMS Init | Link tag (fragile) | JavaScript (reliable) |
+| MIME Types | Not documented | Location-specific override |
+
+### Why We Migrated
+
+1. **Better systemd integration** - Quadlet generates native systemd units
+2. **Ansible consistency** - Declarative infrastructure management
+3. **Improved security** - Podman rootless capable, no daemon
+4. **Auto-updates** - Podman auto-update support
+5. **Better logging** - Native journald integration
+
+## Success Metrics
+
+- ✅ Site accessible at https://devigo.no
+- ✅ WWW redirect working (www.devigo.no → devigo.no)
+- ✅ TLS certificates valid and auto-renewing
+- ✅ OAuth authentication functional
+- ✅ Decap CMS loads and allows content editing
+- ✅ GitHub Actions automated deployment working
+- ✅ All MIME types correct (HTML, CSS, JS, YAML)
+- ✅ Zero downtime during migration
+- ✅ All content preserved from previous deployment
+
+## Future Improvements
+
+### Potential Enhancements
+
+1. **Monitoring**: Add Prometheus metrics collection
+2. **Backups**: Automated backup of Decap CMS content (GitHub is primary backup)
+3. **Staging Environment**: Deploy to separate staging domain for testing
+4. **CDN**: Consider Cloudflare CDN for improved global performance
+5. **Image Optimization**: Automated image optimization in build pipeline
+6. **A/B Testing**: Framework for testing landing page variations
+
+### Technical Debt
+
+None identified - deployment is clean and well-documented.
+
+## Contact & Support
+
+For issues or questions:
+1. Check this guide's troubleshooting section
+2. Review role README files
+3. Check systemd/container logs
+4. Consult Ansible playbook output
+
+---
+
+**Deployment Status**: ✅ **COMPLETE AND OPERATIONAL**  
+**Next Review**: When rotating GitHub PAT (1 year from creation)