rick-infra/docs/gitea-deployment-guide.md

Gitea Deployment and Testing Guide

Comprehensive guide for deploying and testing Gitea Git service with SSH access on rick-infra.

Deployment

1. Prerequisites

Ensure you have the required vault variables set in your host_vars:

# host_vars/arch-vps/vault.yml (encrypted)
vault_gitea_db_password: "your_secure_password_here"

2. Deploy Gitea Role

Run the rick-infra playbook with Gitea role:

# Deploy Gitea to arch-vps
ansible-playbook -i inventory/hosts.yml rick-infra.yml --limit arch-vps

# Or deploy only Gitea role
ansible-playbook -i inventory/hosts.yml rick-infra.yml --tags gitea --limit arch-vps

3. Verify Deployment

Check that all services are running:

# SSH into the server
ssh root@arch-vps

# Check Gitea service status
systemctl status gitea

# Check if Gitea is listening on HTTP port
ss -tlnp | grep 3000

# Check if Gitea SSH is listening
ss -tlnp | grep 2222

# Verify firewall rules
nft list ruleset | grep 2222

# Check fail2ban status
fail2ban-client status gitea-ssh

Expected output:

• Gitea service: active (running)
• HTTP port 3000: listening on 127.0.0.1:3000
• SSH port 2222: listening on 0.0.0.0:2222
• nftables: Rule allowing tcp dport 2222
• fail2ban: gitea-ssh jail active

Testing Guide

Test 1: Web Interface Access

Purpose: Verify HTTPS access through Caddy reverse proxy

# From your local machine
curl -I https://git.jnss.me

Expected result:

• HTTP/2 200 OK
• Redirects to login page
• Valid TLS certificate

Action: Open browser to https://git.jnss.me and verify web interface loads

---

Test 2: Firewall Port Verification

Purpose: Confirm port 2222 is accessible from external networks

# From your local machine (not from the server)
nc -zv git.jnss.me 2222

Expected result:

Connection to git.jnss.me 2222 port [tcp/*] succeeded!

If this fails: The firewall rule is not active or nftables service is not running.

Troubleshooting:

# On the server
ssh root@arch-vps

# Check if nftables service is running
systemctl status nftables

# List all firewall rules
nft list ruleset

# Verify Gitea rule file exists
cat /etc/nftables.d/gitea.nft

# Manually reload nftables
systemctl reload nftables

---

Test 3: SSH Connection Test

Purpose: Verify Gitea SSH server accepts connections

# From your local machine
ssh -T -p 2222 git@git.jnss.me

Expected result (before adding SSH key):

Hi there, You've successfully authenticated, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.

OR (if authentication fails):

git@git.jnss.me: Permission denied (publickey).

This is normal - it means Gitea SSH server is responding, you just need to add your SSH key.

If connection times out: Port 2222 is blocked or Gitea SSH is not running.

---

Test 4: SSH Key Setup and Authentication

Purpose: Add SSH key to Gitea and test authentication

Step 4.1: Create Gitea admin account

1. Visit https://git.jnss.me
2. Click "Register" (if registration is enabled) or use initial admin setup
3. Create your user account

Step 4.2: Generate SSH key (if needed)

# On your local machine
ssh-keygen -t ed25519 -C "your_email@example.com"

# View your public key
cat ~/.ssh/id_ed25519.pub

Step 4.3: Add SSH key to Gitea

1. Log into Gitea web interface
2. Click your profile → Settings
3. Click "SSH / GPG Keys" tab
4. Click "Add Key"
5. Paste your public key (id_ed25519.pub contents)
6. Give it a name and click "Add Key"

Step 4.4: Test SSH authentication

# From your local machine
ssh -T -p 2222 git@git.jnss.me

Expected result:

Hi there, <your_username>! You've successfully authenticated with the key named <key_name>

---

Test 5: Repository Operations

Purpose: Test actual Git operations over SSH

Step 5.1: Create a test repository in Gitea web interface

1. Click "+" → "New Repository"
2. Name: test-repo
3. Click "Create Repository"

Step 5.2: Clone the repository

# From your local machine
git clone ssh://git@git.jnss.me:2222/your_username/test-repo.git
cd test-repo

Expected result: Repository clones successfully

Step 5.3: Make a commit and push

# Create a test file
echo "# Test Repository" > README.md

# Commit and push
git add README.md
git commit -m "Initial commit"
git push origin main

Expected result:

Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 234 bytes | 234.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To ssh://git.jnss.me:2222/your_username/test-repo.git
 * [new branch]      main -> main

Step 5.4: Verify in web interface

1. Refresh Gitea web UI
2. Navigate to test-repo
3. Verify README.md appears

---

Test 6: fail2ban Protection

Purpose: Verify SSH brute force protection is active

Step 6.1: Check fail2ban status

# On the server
ssh root@arch-vps

# Check gitea-ssh jail
fail2ban-client status gitea-ssh

Expected result:

Status for the jail: gitea-ssh
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- File list:        /var/lib/gitea/log/gitea.log
`- Actions
   |- Currently banned: 0
   |- Total banned:     0
   `- Banned IP list:

Step 6.2: Simulate failed authentication (optional)

# From your local machine, try connecting with wrong key multiple times
ssh -T -p 2222 -i /path/to/wrong/key git@git.jnss.me
# Repeat this 5+ times quickly

Step 6.3: Check if IP was banned

# On the server
fail2ban-client status gitea-ssh

Expected result: Your IP should appear in "Currently banned" list after 5 failed attempts.

Step 6.4: Unban yourself (if needed)

# On the server
fail2ban-client set gitea-ssh unbanip YOUR_IP_ADDRESS

---

Test 7: Firewall Rule Persistence

Purpose: Verify firewall rules survive reboot

Step 7.1: Check current rules

# On the server
ssh root@arch-vps
nft list ruleset | grep 2222

Step 7.2: Reboot server

# On the server
reboot

Step 7.3: After reboot, verify rules are still active

# Wait for server to come back up, then SSH in
ssh root@arch-vps

# Check nftables rules again
nft list ruleset | grep 2222

# Verify Gitea SSH still accessible from outside
exit

# From local machine
ssh -T -p 2222 git@git.jnss.me

Expected result: Port 2222 rule persists after reboot, SSH access still works.

---

Troubleshooting

Issue: Connection timeout on port 2222

Symptoms: ssh: connect to host git.jnss.me port 2222: Connection timed out

Diagnosis:

# On server
systemctl status gitea       # Check if Gitea is running
ss -tlnp | grep 2222         # Check if SSH is listening
nft list ruleset | grep 2222 # Check firewall rule
systemctl status nftables    # Check firewall service

Solutions:

1. Gitea not running: systemctl start gitea
2. Firewall rule missing: Re-run Ansible playbook with Gitea role
3. nftables not running: systemctl start nftables
4. Rule file missing: Check /etc/nftables.d/gitea.nft exists

---

Issue: Permission denied (publickey)

Symptoms: SSH connection succeeds but authentication fails

Diagnosis:

# Verbose SSH connection
ssh -vvv -T -p 2222 git@git.jnss.me

Solutions:

1. SSH key not added to Gitea: Add your public key in Gitea web UI
2. Wrong SSH key used: Specify correct key: ssh -i ~/.ssh/id_ed25519 -T -p 2222 git@git.jnss.me
3. Key permissions wrong: chmod 600 ~/.ssh/id_ed25519

---

Issue: fail2ban not protecting Gitea

Symptoms: fail2ban-client status gitea-ssh shows jail doesn't exist

Diagnosis:

# Check if filter exists
ls -la /etc/fail2ban/filter.d/gitea-ssh.conf

# Check if jail is configured
grep -A 10 "gitea-ssh" /etc/fail2ban/jail.local

# Check fail2ban logs
journalctl -u fail2ban | grep gitea

Solutions:

1. Jail not configured: Re-run Ansible playbook with Gitea role
2. fail2ban not running: systemctl start fail2ban
3. Log file not found: Check Gitea is logging to /var/lib/gitea/log/gitea.log

---

Issue: Git clone works but push fails

Symptoms: Can clone but git push gives permission error

Diagnosis:

• Check repository permissions in Gitea web UI
• Verify you have write access to the repository

Solutions:

1. Not repository owner: Ask owner to give you write access
2. Repository is archived: Unarchive in settings
3. Branch protected: Check branch protection rules

---

Verification Checklist

Use this checklist to verify your Gitea deployment:

• Gitea web interface accessible at https://git.jnss.me
• Port 2222 accessible from external network (nc -zv git.jnss.me 2222)
• SSH connection succeeds (ssh -T -p 2222 git@git.jnss.me)
• SSH key added to Gitea account
• SSH authentication works (shows username in response)
• Can clone repository via SSH
• Can push commits to repository
• nftables rule for port 2222 exists and is active
• fail2ban jail gitea-ssh is running
• Gitea service auto-starts on boot
• nftables rules persist after reboot

---

Post-Deployment Configuration

Disable Public Registration (Recommended)

If you don't want anyone to create accounts:

1. Edit host_vars/arch-vps/main.yml or group_vars/production/main.yml
2. Add:

   gitea_disable_registration: true
   

3. Re-run playbook:

   ansible-playbook -i inventory/hosts.yml rick-infra.yml --tags gitea --limit arch-vps
   

Configure Email (Optional)

For password resets and notifications, configure SMTP:

1. Edit Gitea configuration directly:

   ssh root@arch-vps
   nano /etc/gitea/app.ini
   

2. Add mailer section:

   [mailer]
   ENABLED = true
   FROM = gitea@jnss.me
   PROTOCOL = smtps
   SMTP_ADDR = smtp.example.com
   SMTP_PORT = 465
   USER = gitea@jnss.me
   PASSWD = your_smtp_password
   

3. Restart Gitea:

   systemctl restart gitea
   

Enable Actions/CI (Optional)

Gitea Actions provides GitHub Actions-compatible CI/CD:

1. Edit roles/gitea/templates/app.ini.j2
2. Add Actions section
3. Re-run playbook

---

nftables Architecture

Modular Firewall Design

Rick-infra uses a modular nftables architecture that allows services to self-manage their firewall rules:

Load Order:

1. Base rules (/etc/nftables.conf) - Infrastructure essentials (SSH, HTTP, HTTPS, ICMP)
2. Service rules (/etc/nftables.d/[0-8]*.nft) - Service-specific ports (e.g., 50-gitea.nft)
3. Drop rule (/etc/nftables.d/99-drop.nft) - Final catch-all drop

Key Files:

• /etc/nftables.conf - Base infrastructure firewall rules
• /etc/nftables-load.conf - Loader script that orchestrates rule loading
• /etc/nftables.d/50-gitea.nft - Gitea SSH port (2222) rule
• /etc/nftables.d/99-drop.nft - Final drop rule (loaded last)

How It Works:

┌─────────────────────────────────────────────────┐
│ /etc/nftables-load.conf                         │
│                                                 │
│  1. include "/etc/nftables.conf"               │
│     └─> Allow: SSH(22), HTTP(80), HTTPS(443)  │
│                                                 │
│  2. include "/etc/nftables.d/[0-8]*.nft"      │
│     └─> 50-gitea.nft: Allow SSH(2222)         │
│                                                 │
│  3. include "/etc/nftables.d/99-drop.nft"     │
│     └─> Drop all other traffic                │
└─────────────────────────────────────────────────┘

This ensures service rules are evaluated before the drop rule, allowing each service role to be self-contained.

Security Best Practices

1. Use strong database password: Ensure vault_gitea_db_password is strong
2. Enable 2FA: Enable two-factor authentication in Gitea settings
3. Monitor fail2ban: Regularly check banned IPs: fail2ban-client status gitea-ssh
4. Keep updated: Run security playbook regularly for system updates
5. Review SSH keys: Periodically audit SSH keys in Gitea user accounts
6. Backup repositories: Regular backups of /var/lib/gitea/repositories
7. Monitor logs: Check Gitea logs for suspicious activity: journalctl -u gitea

---

Quick Reference Commands

# Service management
systemctl status gitea
systemctl restart gitea
journalctl -u gitea -f

# Firewall
nft list ruleset | grep 2222
systemctl restart nftables
cat /etc/nftables.d/50-gitea.nft

# fail2ban
fail2ban-client status gitea-ssh
fail2ban-client get gitea-ssh banned
fail2ban-client set gitea-ssh unbanip IP_ADDRESS

# Network
ss -tlnp | grep 2222
nc -zv git.jnss.me 2222

# SSH testing
ssh -T -p 2222 git@git.jnss.me
ssh -vvv -T -p 2222 git@git.jnss.me  # Verbose mode

# Git operations
git clone ssh://git@git.jnss.me:2222/user/repo.git
git remote add origin ssh://git@git.jnss.me:2222/user/repo.git

---

Rick-Infra Gitea Deployment Guide
Self-contained Git service with automatic firewall and security management.