joakim/gems
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: implement configurable directory structure with XDG support

Browse Source 
Separate configuration from data storage and make paths configurable
via environment variables and command-line flags. This improves
Unix/Linux compliance and supports both development and production
deployments.

Key changes:
- Separate config dir (opal.yml) from data dir (database, logs)
- Support XDG Base Directory specification
- Add --config-dir and --data-dir flags
- Environment variables: OPAL_CONFIG_DIR, OPAL_DATA_DIR, OPAL_DB_PATH
- Smart fallback: /etc/opal, /var/lib/opal -> ~/.config/opal, ~/.local/share/opal
- Server mode validates required OAuth/JWT environment variables
- Update naming from 'jade' to 'opal' throughout
- Update systemd service name to 'opal.service'
- Add migration guide in README

Default paths:
- Config: /etc/opal (fallback: ~/.config/opal)
- Data: /var/lib/opal (fallback: ~/.local/share/opal)

Files modified:
- internal/engine/config.go: New directory resolution logic
- internal/engine/database.go: Auto-create data directory
- cmd/root.go: Add global flags for directory overrides
- cmd/server.go: Add configuration validation
- cmd/sync.go, internal/sync/*: Use new path helper functions
- tests: Update to use directory overrides
- docs: Update deployment guide and README
This commit is contained in:
Joakim
2026-01-06 20:46:29 +01:00
parent 7ea78d3b54
commit 5d01c9f564
12 changed files with 333 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment.md
+20 -14
View File
@@ -70,8 +70,9 @@ Add:
# Server
SERVER_ADDR=:8080
# Database
OPAL_DB_PATH=/var/lib/opal/opal.db
# Directory Configuration
OPAL_CONFIG_DIR=/etc/opal
OPAL_DATA_DIR=/var/lib/opal
# OAuth (from Authentik setup)
OAUTH_ENABLED=true
@@ -88,9 +89,11 @@ JWT_EXPIRY=3600
REFRESH_TOKEN_EXPIRY=604800
```
**Note:** The config directory (`/etc/opal`) contains read-only settings, while the data directory (`/var/lib/opal`) contains the database and mutable state.
### Create SystemD Service
```bash
sudo nano /etc/systemd/system/opal-api.service
sudo nano /etc/systemd/system/opal.service
```
```ini
@@ -104,7 +107,7 @@ User=opal
Group=opal
WorkingDirectory=/var/lib/opal
EnvironmentFile=/etc/opal/opal.env
ExecStart=/usr/local/bin/opal server start --addr :8080 --db /var/lib/opal/opal.db
ExecStart=/usr/local/bin/opal server start --addr :8080
Restart=always
RestartSec=5
@@ -114,11 +117,14 @@ PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/opal
ReadOnlyPaths=/etc/opal
[Install]
WantedBy=multi-user.target
```
**Note:** The `--db` flag is no longer needed since the database path is configured via `OPAL_DATA_DIR` environment variable.
### Setup Database and User
```bash
# Create user
@@ -136,14 +142,14 @@ sudo cp opal /usr/local/bin/
sudo chmod 755 /usr/local/bin/opal
# Generate first API key (optional - for CLI access)
sudo -u opal opal server keygen --name "Admin CLI" --db /var/lib/opal/opal.db
sudo -u opal OPAL_DATA_DIR=/var/lib/opal opal server keygen --name "Admin CLI"
# Save the generated key!
# Start service
sudo systemctl daemon-reload
sudo systemctl enable opal-api
sudo systemctl start opal-api
sudo systemctl status opal-api
sudo systemctl enable opal
sudo systemctl start opal
sudo systemctl status opal
```
## Step 4: Configure Caddy
@@ -214,8 +220,8 @@ sudo systemctl status caddy
### Check Services
```bash
# API server
sudo systemctl status opal-api
sudo journalctl -u opal-api -n 50
sudo systemctl status opal
sudo journalctl -u opal -n 50
# Caddy
sudo systemctl status caddy
@@ -258,7 +264,7 @@ cd opal-task
git pull
go build -o opal main.go
scp opal server:/tmp/
ssh server "sudo systemctl stop opal-api && sudo cp /tmp/opal /usr/local/bin/ && sudo systemctl start opal-api"
ssh server "sudo systemctl stop opal && sudo cp /tmp/opal /usr/local/bin/ && sudo systemctl start opal"
```
## Troubleshooting
@@ -266,10 +272,10 @@ ssh server "sudo systemctl stop opal-api && sudo cp /tmp/opal /usr/local/bin/ &&
### API Not Responding
```bash
# Check if running
sudo systemctl status opal-api
sudo systemctl status opal
# Check logs
sudo journalctl -u opal-api -f
sudo journalctl -u opal -f
# Test locally
curl http://localhost:8080/health
@@ -297,7 +303,7 @@ curl http://localhost:8080/health
### Logs
```bash
# API logs
sudo journalctl -u opal-api -f
sudo journalctl -u opal -f
# Caddy logs
sudo tail -f /var/log/caddy/opal.log
opal-task/.env.example
-19
View File
@@ -1,19 +0,0 @@
# Server Configuration
SERVER_ADDR=:8080
# Database
OPAL_DB_PATH=/var/lib/opal/opal.db
# OAuth2 / Authentik
OAUTH_ENABLED=true
OAUTH_ISSUER=https://auth.example.com/application/o/opal/
OAUTH_CLIENT_ID=your_client_id_here
OAUTH_CLIENT_SECRET=your_client_secret_here
OAUTH_REDIRECT_URI=https://opal.example.com/auth/callback
# JWT Configuration
JWT_SECRET=generate_random_secret_with_openssl_rand_hex_32
JWT_EXPIRY=3600
# Refresh Token Configuration
REFRESH_TOKEN_EXPIRY=604800
opal-task/README.md
+50 -1
View File
@@ -32,7 +32,56 @@ A task can be recurring. Then we have a template task and instances of that task
A recurring task is given a status of recurring which hides it from view. The recurring task you create is called the template task, from which recurring tasks instances are created. So the template remains hidden, and the recurring instances that spawn from it are the tasks that you will see and complete.
## Storage
SQLite database stored in `~/.config/jade/opal.db`
**Configuration:** `~/.config/opal/opal.yml` (or `$XDG_CONFIG_HOME/opal/opal.yml`)
**Database:** `~/.local/share/opal/opal.db` (or `$XDG_DATA_HOME/opal/opal.db`)
### Customizing Storage Locations
Override default locations using environment variables or command-line flags:
**Environment Variables:**
- `OPAL_CONFIG_DIR` - Override config directory location
- `OPAL_DATA_DIR` - Override data directory location
- `OPAL_DB_PATH` - Override database file path specifically
- `XDG_CONFIG_HOME` - Standard XDG config directory (defaults to `~/.config`)
- `XDG_DATA_HOME` - Standard XDG data directory (defaults to `~/.local/share`)
**Command-Line Flags:**
```bash
opal --config-dir /custom/config --data-dir /custom/data list
```
**Production Server Setup:**
```bash
# Use system-wide paths for production
OPAL_CONFIG_DIR=/etc/opal OPAL_DATA_DIR=/var/lib/opal opal server start
```
### Migrating from Old Paths
If you previously used `~/.config/jade/`, you can migrate your data:
```bash
# Backup first (recommended)
cp -r ~/.config/jade ~/.config/jade.backup
# Create new directories
mkdir -p ~/.config/opal ~/.local/share/opal
# Copy config
cp ~/.config/jade/opal.yml ~/.config/opal/
# Move database and sync data
mv ~/.config/jade/opal.db ~/.local/share/opal/
mv ~/.config/jade/sync_*.* ~/.local/share/opal/ 2>/dev/null || true
# Test that everything works
opal list
# Remove old directory after confirming it works
rm -rf ~/.config/jade
```
## Server & Sync
opal-task/cmd/reports.go
+1 -1
View File
@@ -86,7 +86,7 @@ var reportsCmd = &cobra.Command{
}
sort.Strings(names)
fmt.Println("Available reports:\n")
fmt.Println("Available reports:")
for _, name := range names {
report := reports[name]
fmt.Printf(" %-12s %s\n", name, report.Description)
opal-task/cmd/root.go
+20
View File
@@ -21,6 +21,12 @@ type contextKey string
const parsedArgsKey contextKey = "parsedArgs"
// Global flags
var (
configDirFlag string
dataDirFlag string
)
// Command classification
var commandNames = []string{
"add", "done", "modify", "delete",
@@ -188,6 +194,12 @@ func preprocessArgs(args []string) *ParsedArgs {
}
func init() {
// Add persistent flags for directory overrides
rootCmd.PersistentFlags().StringVar(&configDirFlag, "config-dir", "",
"Config directory (default: $XDG_CONFIG_HOME/opal or ~/.config/opal)")
rootCmd.PersistentFlags().StringVar(&dataDirFlag, "data-dir", "",
"Data directory (default: $XDG_DATA_HOME/opal or ~/.local/share/opal)")
cobra.OnInitialize(initializeApp)
// Add regular subcommands
@@ -212,6 +224,14 @@ func init() {
}
func initializeApp() {
// Set directory overrides from flags if provided
if configDirFlag != "" {
engine.SetConfigDirOverride(configDirFlag)
}
if dataDirFlag != "" {
engine.SetDataDirOverride(dataDirFlag)
}
// Initialize database
if err := engine.InitDB(); err != nil {
fmt.Fprintf(os.Stderr, "Error initializing database: %v\n", err)
opal-task/cmd/server.go
+70
View File
@@ -3,12 +3,76 @@ package cmd
import (
"fmt"
"os"
"strings"
"git.jnss.me/joakim/opal/internal/api"
"git.jnss.me/joakim/opal/internal/engine"
"github.com/spf13/cobra"
)
// validateServerConfig checks that all required environment variables are set for server mode
func validateServerConfig() error {
// Check required OAuth/JWT environment variables
required := map[string]string{
"OAUTH_CLIENT_ID": os.Getenv("OAUTH_CLIENT_ID"),
"OAUTH_CLIENT_SECRET": os.Getenv("OAUTH_CLIENT_SECRET"),
"OAUTH_ISSUER": os.Getenv("OAUTH_ISSUER"),
"OAUTH_REDIRECT_URI": os.Getenv("OAUTH_REDIRECT_URI"),
"JWT_SECRET": os.Getenv("JWT_SECRET"),
}
missing := []string{}
for key, value := range required {
if value == "" {
missing = append(missing, key)
}
}
if len(missing) > 0 {
return fmt.Errorf("missing required environment variables for server mode:\n %s\n\nPlease set these variables before starting the server.", strings.Join(missing, "\n "))
}
// Validate data directory is writable
dataDir, err := engine.GetDataDir()
if err != nil {
return fmt.Errorf("cannot resolve data directory: %w", err)
}
// Check if directory exists and is writable
info, err := os.Stat(dataDir)
if err != nil {
// Directory doesn't exist yet, check parent
parent := dataDir
for parent != "/" && parent != "." {
parent = strings.TrimSuffix(parent, "/")
idx := strings.LastIndex(parent, "/")
if idx <= 0 {
parent = "/"
break
}
parent = parent[:idx]
if parent == "" {
parent = "/"
}
if pInfo, pErr := os.Stat(parent); pErr == nil {
if !pInfo.IsDir() {
return fmt.Errorf("parent path is not a directory: %s", parent)
}
// Check write permission by trying to create data dir
if err := os.MkdirAll(dataDir, 0755); err != nil {
return fmt.Errorf("data directory not writable: %s (error: %v)", dataDir, err)
}
break
}
}
} else if !info.IsDir() {
return fmt.Errorf("data directory path exists but is not a directory: %s", dataDir)
}
return nil
}
var serverCmd = &cobra.Command{
Use: "server",
Short: "Server management commands",
@@ -33,6 +97,12 @@ Examples:
os.Setenv("OPAL_DB_PATH", dbPath)
}
// Validate server configuration
if err := validateServerConfig(); err != nil {
fmt.Fprintf(os.Stderr, "Server configuration validation failed:\n%v\n", err)
os.Exit(1)
}
// Initialize database
if err := engine.InitDB(); err != nil {
fmt.Fprintf(os.Stderr, "Error initializing database: %v\n", err)
opal-task/cmd/sync.go
+1 -3
View File
@@ -286,13 +286,11 @@ var syncLogCmd = &cobra.Command{
Short: "Show conflict resolution log",
Long: `Display the log of sync conflicts and how they were resolved`,
Run: func(cmd *cobra.Command, args []string) {
configDir, err := engine.GetConfigDir()
logPath, err := engine.GetSyncConflictLogPath()
if err != nil {
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
os.Exit(1)
}
logPath := fmt.Sprintf("%s/sync_conflicts.log", configDir)
data, err := os.ReadFile(logPath)
if err != nil {
if os.IsNotExist(err) {
opal-task/internal/engine/config.go
+150 -3
View File
@@ -45,22 +45,151 @@ type Config struct {
var globalConfig *Config
// Global variables for flag/programmatic overrides
var (
configDirOverride string
dataDirOverride string
)
// SetConfigDirOverride sets the config directory override (typically from --config-dir flag)
func SetConfigDirOverride(dir string) {
configDirOverride = dir
}
// SetDataDirOverride sets the data directory override (typically from --data-dir flag)
func SetDataDirOverride(dir string) {
dataDirOverride = dir
}
// GetConfigDir returns the configuration directory path
// Resolution priority:
// 1. Flag override (via SetConfigDirOverride)
// 2. OPAL_CONFIG_DIR environment variable
// 3. XDG_CONFIG_HOME/opal
// 4. /etc/opal (with fallback to ~/.config/opal if not writable)
func GetConfigDir() (string, error) {
// Priority 1: Flag override
if configDirOverride != "" {
return configDirOverride, nil
}
// Priority 2: Environment variable
if dir := os.Getenv("OPAL_CONFIG_DIR"); dir != "" {
return dir, nil
}
// Priority 3: XDG_CONFIG_HOME
if xdgConfig := os.Getenv("XDG_CONFIG_HOME"); xdgConfig != "" {
return filepath.Join(xdgConfig, "opal"), nil
}
// Priority 4: Try /etc/opal, fallback to ~/.config/opal if not writable
etcOpal := "/etc/opal"
if isWritable(etcOpal) {
return etcOpal, nil
}
// Fallback to user config directory
home, err := os.UserHomeDir()
if err != nil {
return "", fmt.Errorf("failed to get home directory: %w", err)
}
return filepath.Join(home, ".config", "jade"), nil
return filepath.Join(home, ".config", "opal"), nil
}
// GetDataDir returns the data directory path
// Resolution priority:
// 1. Flag override (via SetDataDirOverride)
// 2. OPAL_DATA_DIR environment variable
// 3. XDG_DATA_HOME/opal
// 4. /var/lib/opal (with fallback to ~/.local/share/opal if not writable)
func GetDataDir() (string, error) {
// Priority 1: Flag override
if dataDirOverride != "" {
return dataDirOverride, nil
}
// Priority 2: Environment variable
if dir := os.Getenv("OPAL_DATA_DIR"); dir != "" {
return dir, nil
}
// Priority 3: XDG_DATA_HOME
if xdgData := os.Getenv("XDG_DATA_HOME"); xdgData != "" {
return filepath.Join(xdgData, "opal"), nil
}
// Priority 4: Try /var/lib/opal, fallback to ~/.local/share/opal if not writable
varLibOpal := "/var/lib/opal"
if isWritable(varLibOpal) {
return varLibOpal, nil
}
// Fallback to user data directory
home, err := os.UserHomeDir()
if err != nil {
return "", fmt.Errorf("failed to get home directory: %w", err)
}
return filepath.Join(home, ".local", "share", "opal"), nil
}
// isWritable checks if a directory exists and is writable, or if parent exists and is writable
func isWritable(path string) bool {
// Check if path exists
info, err := os.Stat(path)
if err == nil {
// Path exists, check if it's a directory and writable
if !info.IsDir() {
return false
}
// Test write permission by trying to create a temp file
testFile := filepath.Join(path, ".write-test")
f, err := os.Create(testFile)
if err != nil {
return false
}
f.Close()
os.Remove(testFile)
return true
}
// Path doesn't exist, check if parent is writable
parent := filepath.Dir(path)
parentInfo, err := os.Stat(parent)
if err != nil {
return false
}
if !parentInfo.IsDir() {
return false
}
// Test if we can create in parent
testFile := filepath.Join(parent, ".write-test")
f, err := os.Create(testFile)
if err != nil {
return false
}
f.Close()
os.Remove(testFile)
return true
}
// GetDBPath returns the path to the SQLite database
// Resolution priority:
// 1. OPAL_DB_PATH environment variable
// 2. {data-dir}/opal.db
func GetDBPath() (string, error) {
configDir, err := GetConfigDir()
// Priority 1: OPAL_DB_PATH override
if dbPath := os.Getenv("OPAL_DB_PATH"); dbPath != "" {
return dbPath, nil
}
// Priority 2: Data directory + opal.db
dataDir, err := GetDataDir()
if err != nil {
return "", err
}
return filepath.Join(configDir, "opal.db"), nil
return filepath.Join(dataDir, "opal.db"), nil
}
// GetConfigPath returns the path to the config file
@@ -72,6 +201,24 @@ func GetConfigPath() (string, error) {
return filepath.Join(configDir, "opal.yml"), nil
}
// GetSyncQueuePath returns the path to the sync queue file
func GetSyncQueuePath() (string, error) {
dataDir, err := GetDataDir()
if err != nil {
return "", err
}
return filepath.Join(dataDir, "sync_queue.json"), nil
}
// GetSyncConflictLogPath returns the path to the sync conflict log
func GetSyncConflictLogPath() (string, error) {
dataDir, err := GetDataDir()
if err != nil {
return "", err
}
return filepath.Join(dataDir, "sync_conflicts.log"), nil
}
// LoadConfig loads the configuration from file or creates default
func LoadConfig() (*Config, error) {
if globalConfig != nil {
opal-task/internal/engine/database.go
+10
View File
@@ -3,6 +3,7 @@ package engine
import (
"database/sql"
"fmt"
"os"
_ "github.com/mattn/go-sqlite3"
)
@@ -20,6 +21,15 @@ func InitDB() error {
return fmt.Errorf("failed to get database path: %w", err)
}
// Ensure data directory exists
dataDir, err := GetDataDir()
if err != nil {
return fmt.Errorf("failed to get data directory: %w", err)
}
if err := os.MkdirAll(dataDir, 0755); err != nil {
return fmt.Errorf("failed to create data directory: %w", err)
}
// Open database connection
database, err := sql.Open("sqlite3", dbPath)
if err != nil {
opal-task/internal/engine/task_test.go
+9 -6
View File
@@ -9,12 +9,15 @@ import (
)
func TestMain(m *testing.M) {
// Setup test database
os.Setenv("HOME", "/tmp/opal-test")
// Setup test database with explicit directory overrides
testDir := "/tmp/opal-test"
// Ensure config directory exists
configDir := "/tmp/opal-test/.config/jade"
if err := os.MkdirAll(configDir, 0755); err != nil {
// Set directory overrides to use test directory
SetConfigDirOverride(testDir)
SetDataDirOverride(testDir)
// Ensure test directory exists
if err := os.MkdirAll(testDir, 0755); err != nil {
panic(err)
}
@@ -27,7 +30,7 @@ func TestMain(m *testing.M) {
code := m.Run()
// Cleanup
os.RemoveAll("/tmp/opal-test/.config")
os.RemoveAll(testDir)
os.Exit(code)
}
opal-task/internal/sync/queue.go
+1 -3
View File
@@ -4,7 +4,6 @@ import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"git.jnss.me/joakim/opal/internal/engine"
"github.com/google/uuid"
@@ -27,12 +26,11 @@ type Queue struct {
// NewQueue creates a new queue instance
func NewQueue() (*Queue, error) {
configDir, err := engine.GetConfigDir()
queuePath, err := engine.GetSyncQueuePath()
if err != nil {
return nil, err
}
queuePath := filepath.Join(configDir, "sync_queue.json")
q := &Queue{filepath: queuePath}
if err := q.load(); err != nil {
opal-task/internal/sync/strategy.go
+1 -4
View File
@@ -3,7 +3,6 @@ package sync
import (
"fmt"
"os"
"path/filepath"
"time"
"git.jnss.me/joakim/opal/internal/engine"
@@ -112,13 +111,11 @@ func resolveConflict(local, remote *engine.Task, strategy ConflictResolution) *e
// logConflict writes conflict information to log file
func logConflict(local, remote *engine.Task, winner *engine.Task) {
configDir, err := engine.GetConfigDir()
logPath, err := engine.GetSyncConflictLogPath()
if err != nil {
return
}
logPath := filepath.Join(configDir, "sync_conflicts.log")
winnerLabel := "local"
if winner.UUID == remote.UUID && winner.Modified.Equal(remote.Modified) {
winnerLabel = "remote"