insertr/DRAFT_PUBLISH_FEATURE.md

# Feature Document: Draft/Publish System for Insertr CMS

## Overview

This document outlines the design and implementation plan for adding draft/publish functionality to Insertr CMS. Currently, all content changes are immediately reflected when enhancement is triggered manually. This feature will introduce a proper editorial workflow with draft states and controlled publishing.

## Problem Statement

### Current State
- All content stored in database is immediately available for enhancement
- Manual "Enhance" button triggers immediate file updates with latest content
- No separation between working drafts and production-ready content
- Not suitable for production environments where editorial approval is needed

### User Pain Points
- Editors cannot safely make changes without affecting live site
- No preview workflow for reviewing changes before going live
- All-or-nothing manual enhancement process
- No rollback mechanism for published content

## Requirements

### Functional Requirements
- **FR1**: Editors can save draft content without affecting published site
- **FR2**: Editors can preview changes before publishing
- **FR3**: Authorized users can publish draft content to live site
- **FR4**: System supports rollback to previous published versions
- **FR5**: Clear visual indication of draft vs published state
- **FR6**: Batch publishing of multiple content changes

### Non-Functional Requirements
- **NFR1**: Backward compatibility with existing content
- **NFR2**: Minimal performance impact on content editing
- **NFR3**: Support for concurrent editing workflows
- **NFR4**: Audit trail for all publishing actions

## Architecture Approaches

### Option A: Minimal Schema Impact (Published Version Pointer)

**Core Concept**: Use existing version system with a pointer to published versions.

**Schema Changes**:
```sql
CREATE TABLE published_versions (
    content_id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    published_version_id INTEGER NOT NULL,
    published_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    published_by TEXT NOT NULL,
    PRIMARY KEY (content_id, site_id)
);
```

**Pros**:
- Minimal database changes
- Leverages existing version history
- Backward compatible
- Simple migration path

**Cons**:
- Less intuitive data model
- Enhancement modes add complexity
- Auto-save creates unnecessary versions
- Limited workflow capabilities

### Option B: Full Schema Redesign (Recommended)

**Core Concept**: Explicit draft and published content tables with rich workflow support.

**Schema Changes**:
```sql
-- Draft content (working state)
CREATE TABLE content_drafts (
    id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,
    original_template TEXT,
    created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    updated_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    last_edited_by TEXT DEFAULT 'system' NOT NULL,
    is_dirty BOOLEAN DEFAULT TRUE NOT NULL,
    auto_save_at BIGINT,
    PRIMARY KEY (id, site_id)
);

-- Published content (live state)
CREATE TABLE content_published (
    id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,
    original_template TEXT,
    published_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    published_by TEXT DEFAULT 'system' NOT NULL,
    draft_version_at_publish BIGINT NOT NULL,
    PRIMARY KEY (id, site_id)
);

-- Enhanced version tracking
CREATE TABLE content_versions (
    version_id SERIAL PRIMARY KEY,
    content_id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,
    original_template TEXT,
    created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    created_by TEXT DEFAULT 'system' NOT NULL,
    version_type TEXT NOT NULL CHECK (version_type IN ('draft_save', 'publish', 'auto_save')),
    is_published BOOLEAN DEFAULT FALSE NOT NULL
);

-- Future: Scheduled publishing support
CREATE TABLE publish_queue (
    queue_id SERIAL PRIMARY KEY,
    site_id TEXT NOT NULL,
    content_ids TEXT[] NOT NULL,
    scheduled_at BIGINT,
    created_by TEXT NOT NULL,
    status TEXT DEFAULT 'pending' CHECK (status IN ('pending', 'processing', 'completed', 'failed'))
);
```

**Pros**:
- Clear semantic separation
- Rich workflow capabilities
- Optimized for different access patterns
- Supports advanced features (auto-save, batch publish, scheduling)
- Intuitive mental model

**Cons**:
- Significant schema changes
- Complex migration required
- More storage overhead
- Increased development complexity

## Recommended Solution: Option B (Full Schema Redesign)

### Content States

| State | Draft Table | Published Table | Description |
|-------|-------------|-----------------|-------------|
| `draft_only` | ✅ Exists | ❌ None | New content, never published |
| `published` | ✅ Exists | ✅ Exists, Same | Content published, no pending changes |
| `modified` | ✅ Exists | ✅ Exists, Different | Published content with unpublished changes |
| `scheduled` | ✅ Exists | ✅ Exists | Content queued for future publishing |

### API Design

**New Endpoints**:
```
GET    /api/content/{id}?mode=draft|published     # Get content in specific state
POST   /api/content/{id}/save-draft               # Save without publishing
POST   /api/content/{id}/publish                  # Publish draft content
POST   /api/content/bulk-publish                  # Publish multiple items
GET    /api/content/diff/{id}                     # Show draft vs published diff
POST   /api/enhancement/preview                   # Preview with draft content
POST   /api/enhancement/publish                   # Enhance with published content
GET    /api/status/publishing                     # Get site publishing status
```

**Enhanced Endpoints**:
```
PUT    /api/content/{id}                          # Now saves as draft by default
```

### UI/UX Changes

**Control Panel Updates**:
- Replace "🔄 Enhance" with publishing workflow buttons
- Add "💾 Save Draft" (auto-saves every 30s)
- Add "🚀 Publish Changes" with confirmation dialog
- Add "👁️ Preview Changes" for draft enhancement
- Add "📊 Publishing Status" indicator

**Visual States**:
- 🟡 **Draft Pending**: Yellow indicator for unpublished changes
- 🟢 **Published**: Green indicator when draft matches published  
- 🔵 **Scheduled**: Blue indicator for queued publishing
- 🔴 **Error**: Red indicator for publishing failures

**New UI Components**:
- Diff viewer showing draft vs published changes
- Bulk publishing interface for multiple content items
- Publishing history and rollback interface

## Implementation Plan

### Phase 1: Core Infrastructure (Week 1-2)
- [ ] Create new database schema
- [ ] Implement migration scripts
- [ ] Update repository interfaces
- [ ] Add basic draft/publish operations
- [ ] Update content versioning system

### Phase 2: API Development (Week 3-4)
- [ ] Implement new API endpoints
- [ ] Update existing endpoints for draft mode
- [ ] Add enhancement mode switching
- [ ] Implement publishing workflow APIs
- [ ] Add content state management

### Phase 3: UI Integration (Week 5-6)
- [ ] Update control panel with new buttons
- [ ] Add visual state indicators
- [ ] Implement draft auto-save
- [ ] Add preview functionality
- [ ] Create publishing confirmation dialogs

### Phase 4: Advanced Features (Week 7-8)
- [ ] Implement diff viewer
- [ ] Add bulk publishing interface
- [ ] Create publishing history view
- [ ] Add rollback functionality
- [ ] Implement scheduled publishing foundation

### Phase 5: Testing & Polish (Week 9-10)
- [ ] Comprehensive testing across all demo sites
- [ ] Performance optimization
- [ ] Error handling and edge cases
- [ ] Documentation updates
- [ ] Migration testing

## Potential Challenges & Mitigation

### Challenge 1: Data Migration Complexity
**Risk**: Migrating existing content to new schema without data loss
**Mitigation**: 
- Create comprehensive migration scripts with rollback capability
- Test migration on demo sites first
- Implement gradual migration with dual-write period
- Provide data validation and integrity checks

### Challenge 2: Concurrent Editing Conflicts
**Risk**: Multiple editors working on same content simultaneously
**Mitigation**:
- Implement optimistic locking with version checking
- Add conflict detection and resolution UI
- Consider edit session management
- Provide clear error messages for conflicts

### Challenge 3: Performance Impact
**Risk**: Additional database queries and storage overhead
**Mitigation**:
- Optimize database indexes for new access patterns
- Implement efficient content state queries
- Consider caching strategies for published content
- Monitor and profile database performance

### Challenge 4: Backward Compatibility
**Risk**: Breaking existing workflows and integrations
**Mitigation**:
- Maintain existing API compatibility during transition
- Provide clear migration path for existing users
- Implement feature flags for gradual rollout
- Extensive testing with current demo sites

### Challenge 5: Auto-save vs Version History Noise
**Risk**: Too many auto-save versions cluttering history
**Mitigation**:
- Separate auto-save from manual save operations
- Implement version consolidation strategies
- Use different version types (auto_save vs draft_save)
- Provide version cleanup mechanisms

## Testing Strategy

### Unit Tests
- Repository layer draft/publish operations
- Content state transition logic
- API endpoint functionality
- Migration script validation

### Integration Tests
- End-to-end publishing workflow
- Enhancement with different content modes
- Concurrent editing scenarios
- Database migration processes

### User Acceptance Tests
- Editorial workflow testing with demo sites
- Performance testing under load
- Cross-browser UI compatibility
- Mobile device testing

## Success Metrics

### Functional Metrics
- ✅ All existing demo sites work without modification
- ✅ Editors can save drafts without affecting live sites
- ✅ Publishing workflow completes in <5 seconds
- ✅ Zero data loss during migration

### User Experience Metrics
- ✅ Clear visual indication of content states
- ✅ Intuitive publishing workflow
- ✅ Auto-save prevents content loss
- ✅ Preview functionality works accurately

### Technical Metrics
- ✅ API response times remain under 200ms
- ✅ Database migration completes in <30 minutes
- ✅ Memory usage increase <20%
- ✅ 100% test coverage for new functionality

## Future Considerations

### Potential Enhancements
- **Scheduled Publishing**: Full calendar-based publishing system
- **Approval Workflows**: Multi-stage content approval process
- **Content Staging**: Multiple environment support (dev/staging/prod)
- **Collaborative Editing**: Real-time collaborative editing features
- **Content Templates**: Draft templates for consistent content structure

### Technical Debt
- Consider eventual consolidation of version tables
- Evaluate long-term storage strategies for large sites
- Plan for horizontal scaling of publishing operations
- Review and optimize database schema after real-world usage

---

*This document will be updated as the feature evolves through design reviews and implementation feedback.*