rick-infra/docs/gitea-deployment-guide.md

# Gitea Deployment and Testing Guide

Comprehensive guide for deploying and testing Gitea Git service with SSH access on rick-infra.

## Deployment

### 1. Prerequisites

Ensure you have the required vault variables set in your host_vars:

```yaml
# host_vars/arch-vps/vault.yml (encrypted)
vault_gitea_db_password: "your_secure_password_here"
```

### 2. Deploy Gitea Role

Run the rick-infra playbook with Gitea role:

```bash
# Deploy Gitea to arch-vps
ansible-playbook -i inventory/hosts.yml rick-infra.yml --limit arch-vps

# Or deploy only Gitea role
ansible-playbook -i inventory/hosts.yml rick-infra.yml --tags gitea --limit arch-vps
```

### 3. Verify Deployment

Check that all services are running:

```bash
# SSH into the server
ssh root@arch-vps

# Check Gitea service status
systemctl status gitea

# Check if Gitea is listening on HTTP port
ss -tlnp | grep 3000

# Check if Gitea SSH is listening
ss -tlnp | grep 2222

# Verify firewall rules
nft list ruleset | grep 2222

# Check fail2ban status
fail2ban-client status gitea-ssh
```

Expected output:
- Gitea service: `active (running)`
- HTTP port 3000: listening on `127.0.0.1:3000`
- SSH port 2222: listening on `0.0.0.0:2222`
- nftables: Rule allowing `tcp dport 2222`
- fail2ban: `gitea-ssh` jail active

## Testing Guide

### Test 1: Web Interface Access

**Purpose**: Verify HTTPS access through Caddy reverse proxy

```bash
# From your local machine
curl -I https://git.jnss.me
```

**Expected result**:
- HTTP/2 200 OK
- Redirects to login page
- Valid TLS certificate

**Action**: Open browser to `https://git.jnss.me` and verify web interface loads

---

### Test 2: Firewall Port Verification

**Purpose**: Confirm port 2222 is accessible from external networks

```bash
# From your local machine (not from the server)
nc -zv git.jnss.me 2222
```

**Expected result**:
```
Connection to git.jnss.me 2222 port [tcp/*] succeeded!
```

**If this fails**: The firewall rule is not active or nftables service is not running.

**Troubleshooting**:
```bash
# On the server
ssh root@arch-vps

# Check if nftables service is running
systemctl status nftables

# List all firewall rules
nft list ruleset

# Verify Gitea rule file exists
cat /etc/nftables.d/gitea.nft

# Manually reload nftables
systemctl reload nftables
```

---

### Test 3: SSH Connection Test

**Purpose**: Verify Gitea SSH server accepts connections

```bash
# From your local machine
ssh -T -p 2222 git@git.jnss.me
```

**Expected result** (before adding SSH key):
```
Hi there, You've successfully authenticated, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.
```

**OR** (if authentication fails):
```
git@git.jnss.me: Permission denied (publickey).
```

This is normal - it means Gitea SSH server is responding, you just need to add your SSH key.

**If connection times out**: Port 2222 is blocked or Gitea SSH is not running.

---

### Test 4: SSH Key Setup and Authentication

**Purpose**: Add SSH key to Gitea and test authentication

**Step 4.1**: Create Gitea admin account
1. Visit `https://git.jnss.me`
2. Click "Register" (if registration is enabled) or use initial admin setup
3. Create your user account

**Step 4.2**: Generate SSH key (if needed)
```bash
# On your local machine
ssh-keygen -t ed25519 -C "your_email@example.com"

# View your public key
cat ~/.ssh/id_ed25519.pub
```

**Step 4.3**: Add SSH key to Gitea
1. Log into Gitea web interface
2. Click your profile → Settings
3. Click "SSH / GPG Keys" tab
4. Click "Add Key"
5. Paste your public key (`id_ed25519.pub` contents)
6. Give it a name and click "Add Key"

**Step 4.4**: Test SSH authentication
```bash
# From your local machine
ssh -T -p 2222 git@git.jnss.me
```

**Expected result**:
```
Hi there, <your_username>! You've successfully authenticated with the key named <key_name>
```

---

### Test 5: Repository Operations

**Purpose**: Test actual Git operations over SSH

**Step 5.1**: Create a test repository in Gitea web interface
1. Click "+" → "New Repository"
2. Name: `test-repo`
3. Click "Create Repository"

**Step 5.2**: Clone the repository
```bash
# From your local machine
git clone ssh://git@git.jnss.me:2222/your_username/test-repo.git
cd test-repo
```

**Expected result**: Repository clones successfully

**Step 5.3**: Make a commit and push
```bash
# Create a test file
echo "# Test Repository" > README.md

# Commit and push
git add README.md
git commit -m "Initial commit"
git push origin main
```

**Expected result**: 
```
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Writing objects: 100% (3/3), 234 bytes | 234.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
To ssh://git.jnss.me:2222/your_username/test-repo.git
 * [new branch]      main -> main
```

**Step 5.4**: Verify in web interface
1. Refresh Gitea web UI
2. Navigate to `test-repo`
3. Verify `README.md` appears

---

### Test 6: fail2ban Protection

**Purpose**: Verify SSH brute force protection is active

**Step 6.1**: Check fail2ban status
```bash
# On the server
ssh root@arch-vps

# Check gitea-ssh jail
fail2ban-client status gitea-ssh
```

**Expected result**:
```
Status for the jail: gitea-ssh
|- Filter
|  |- Currently failed: 0
|  |- Total failed:     0
|  `- File list:        /var/lib/gitea/log/gitea.log
`- Actions
   |- Currently banned: 0
   |- Total banned:     0
   `- Banned IP list:
```

**Step 6.2**: Simulate failed authentication (optional)
```bash
# From your local machine, try connecting with wrong key multiple times
ssh -T -p 2222 -i /path/to/wrong/key git@git.jnss.me
# Repeat this 5+ times quickly
```

**Step 6.3**: Check if IP was banned
```bash
# On the server
fail2ban-client status gitea-ssh
```

**Expected result**: Your IP should appear in "Currently banned" list after 5 failed attempts.

**Step 6.4**: Unban yourself (if needed)
```bash
# On the server
fail2ban-client set gitea-ssh unbanip YOUR_IP_ADDRESS
```

---

### Test 7: Firewall Rule Persistence

**Purpose**: Verify firewall rules survive reboot

**Step 7.1**: Check current rules
```bash
# On the server
ssh root@arch-vps
nft list ruleset | grep 2222
```

**Step 7.2**: Reboot server
```bash
# On the server
reboot
```

**Step 7.3**: After reboot, verify rules are still active
```bash
# Wait for server to come back up, then SSH in
ssh root@arch-vps

# Check nftables rules again
nft list ruleset | grep 2222

# Verify Gitea SSH still accessible from outside
exit

# From local machine
ssh -T -p 2222 git@git.jnss.me
```

**Expected result**: Port 2222 rule persists after reboot, SSH access still works.

---

## Troubleshooting

### Issue: Connection timeout on port 2222

**Symptoms**: `ssh: connect to host git.jnss.me port 2222: Connection timed out`

**Diagnosis**:
```bash
# On server
systemctl status gitea       # Check if Gitea is running
ss -tlnp | grep 2222         # Check if SSH is listening
nft list ruleset | grep 2222 # Check firewall rule
systemctl status nftables    # Check firewall service
```

**Solutions**:
1. **Gitea not running**: `systemctl start gitea`
2. **Firewall rule missing**: Re-run Ansible playbook with Gitea role
3. **nftables not running**: `systemctl start nftables`
4. **Rule file missing**: Check `/etc/nftables.d/gitea.nft` exists

---

### Issue: Permission denied (publickey)

**Symptoms**: SSH connection succeeds but authentication fails

**Diagnosis**:
```bash
# Verbose SSH connection
ssh -vvv -T -p 2222 git@git.jnss.me
```

**Solutions**:
1. **SSH key not added to Gitea**: Add your public key in Gitea web UI
2. **Wrong SSH key used**: Specify correct key: `ssh -i ~/.ssh/id_ed25519 -T -p 2222 git@git.jnss.me`
3. **Key permissions wrong**: `chmod 600 ~/.ssh/id_ed25519`

---

### Issue: fail2ban not protecting Gitea

**Symptoms**: `fail2ban-client status gitea-ssh` shows jail doesn't exist

**Diagnosis**:
```bash
# Check if filter exists
ls -la /etc/fail2ban/filter.d/gitea-ssh.conf

# Check if jail is configured
grep -A 10 "gitea-ssh" /etc/fail2ban/jail.local

# Check fail2ban logs
journalctl -u fail2ban | grep gitea
```

**Solutions**:
1. **Jail not configured**: Re-run Ansible playbook with Gitea role
2. **fail2ban not running**: `systemctl start fail2ban`
3. **Log file not found**: Check Gitea is logging to `/var/lib/gitea/log/gitea.log`

---

### Issue: Git clone works but push fails

**Symptoms**: Can clone but `git push` gives permission error

**Diagnosis**:
- Check repository permissions in Gitea web UI
- Verify you have write access to the repository

**Solutions**:
1. **Not repository owner**: Ask owner to give you write access
2. **Repository is archived**: Unarchive in settings
3. **Branch protected**: Check branch protection rules

---

## Verification Checklist

Use this checklist to verify your Gitea deployment:

- [ ] Gitea web interface accessible at `https://git.jnss.me`
- [ ] Port 2222 accessible from external network (`nc -zv git.jnss.me 2222`)
- [ ] SSH connection succeeds (`ssh -T -p 2222 git@git.jnss.me`)
- [ ] SSH key added to Gitea account
- [ ] SSH authentication works (shows username in response)
- [ ] Can clone repository via SSH
- [ ] Can push commits to repository
- [ ] nftables rule for port 2222 exists and is active
- [ ] fail2ban jail `gitea-ssh` is running
- [ ] Gitea service auto-starts on boot
- [ ] nftables rules persist after reboot

---

## Post-Deployment Configuration

### Disable Public Registration (Recommended)

If you don't want anyone to create accounts:

1. Edit `host_vars/arch-vps/main.yml` or `group_vars/production/main.yml`
2. Add:
   ```yaml
   gitea_disable_registration: true
   ```
3. Re-run playbook:
   ```bash
   ansible-playbook -i inventory/hosts.yml rick-infra.yml --tags gitea --limit arch-vps
   ```

### Configure Email (Optional)

For password resets and notifications, configure SMTP:

1. Edit Gitea configuration directly:
   ```bash
   ssh root@arch-vps
   nano /etc/gitea/app.ini
   ```

2. Add mailer section:
   ```ini
   [mailer]
   ENABLED = true
   FROM = gitea@jnss.me
   PROTOCOL = smtps
   SMTP_ADDR = smtp.example.com
   SMTP_PORT = 465
   USER = gitea@jnss.me
   PASSWD = your_smtp_password
   ```

3. Restart Gitea:
   ```bash
   systemctl restart gitea
   ```

### Enable Actions/CI (Optional)

Gitea Actions provides GitHub Actions-compatible CI/CD:

1. Edit `roles/gitea/templates/app.ini.j2`
2. Add Actions section
3. Re-run playbook

---

## nftables Architecture

### Modular Firewall Design

Rick-infra uses a modular nftables architecture that allows services to self-manage their firewall rules:

**Load Order:**
1. **Base rules** (`/etc/nftables.conf`) - Infrastructure essentials (SSH, HTTP, HTTPS, ICMP)
2. **Service rules** (`/etc/nftables.d/[0-8]*.nft`) - Service-specific ports (e.g., `50-gitea.nft`)
3. **Drop rule** (`/etc/nftables.d/99-drop.nft`) - Final catch-all drop

**Key Files:**
- `/etc/nftables.conf` - Base infrastructure firewall rules
- `/etc/nftables-load.conf` - Loader script that orchestrates rule loading
- `/etc/nftables.d/50-gitea.nft` - Gitea SSH port (2222) rule
- `/etc/nftables.d/99-drop.nft` - Final drop rule (loaded last)

**How It Works:**
```
┌─────────────────────────────────────────────────┐
│ /etc/nftables-load.conf                         │
│                                                 │
│  1. include "/etc/nftables.conf"               │
│     └─> Allow: SSH(22), HTTP(80), HTTPS(443)  │
│                                                 │
│  2. include "/etc/nftables.d/[0-8]*.nft"      │
│     └─> 50-gitea.nft: Allow SSH(2222)         │
│                                                 │
│  3. include "/etc/nftables.d/99-drop.nft"     │
│     └─> Drop all other traffic                │
└─────────────────────────────────────────────────┘
```

This ensures service rules are evaluated **before** the drop rule, allowing each service role to be self-contained.

## Security Best Practices

1. **Use strong database password**: Ensure `vault_gitea_db_password` is strong
2. **Enable 2FA**: Enable two-factor authentication in Gitea settings
3. **Monitor fail2ban**: Regularly check banned IPs: `fail2ban-client status gitea-ssh`
4. **Keep updated**: Run security playbook regularly for system updates
5. **Review SSH keys**: Periodically audit SSH keys in Gitea user accounts
6. **Backup repositories**: Regular backups of `/var/lib/gitea/repositories`
7. **Monitor logs**: Check Gitea logs for suspicious activity: `journalctl -u gitea`

---

## Quick Reference Commands

```bash
# Service management
systemctl status gitea
systemctl restart gitea
journalctl -u gitea -f

# Firewall
nft list ruleset | grep 2222
systemctl restart nftables
cat /etc/nftables.d/50-gitea.nft

# fail2ban
fail2ban-client status gitea-ssh
fail2ban-client get gitea-ssh banned
fail2ban-client set gitea-ssh unbanip IP_ADDRESS

# Network
ss -tlnp | grep 2222
nc -zv git.jnss.me 2222

# SSH testing
ssh -T -p 2222 git@git.jnss.me
ssh -vvv -T -p 2222 git@git.jnss.me  # Verbose mode

# Git operations
git clone ssh://git@git.jnss.me:2222/user/repo.git
git remote add origin ssh://git@git.jnss.me:2222/user/repo.git
```

---

**Rick-Infra Gitea Deployment Guide**  
Self-contained Git service with automatic firewall and security management.