insertr/DRAFT_PUBLISH_FEATURE.md

# Feature Document: Draft/Publish System for Insertr CMS

## Overview

This document outlines the design and implementation plan for adding draft/publish functionality to Insertr CMS. Currently, all content changes are immediately reflected when enhancement is triggered manually. This feature will introduce a proper editorial workflow with draft states and controlled publishing.

## Problem Statement

### Current State
- All content stored in database is immediately available for enhancement
- Manual "Enhance" button triggers immediate file updates with latest content
- No separation between working drafts and production-ready content
- Not suitable for production environments where editorial approval is needed

### User Pain Points
- Editors cannot safely make changes without affecting live site
- No preview workflow for reviewing changes before going live
- All-or-nothing manual enhancement process
- No rollback mechanism for published content

## Industry Research: How Other CMS Handle Drafts

### WordPress
**Approach**: Single table (`wp_posts`) with state field (`post_status`)  
**States**: draft, pending, publish, future, private, trash, auto-draft  
**Storage**: All content in one table, differentiated by status field  
**Pros**: Simple schema, easy queries, unified storage  
**Cons**: No separation between draft and live data, potential performance issues

### Drupal
**Approach**: Content moderation module with workflow states  
**States**: Configurable (draft, needs_review, published, archived, etc.)  
**Storage**: Moderation state entities linked to content revisions  
**Pros**: Flexible workflows, proper revision tracking, role-based transitions  
**Cons**: Complex architecture, steep learning curve

### Contentful (Headless)
**Approach**: Separate published/draft versions with sync API  
**States**: draft, published, changed, archived  
**Storage**: Maintains both draft and published versions simultaneously  
**Pros**: Performance optimized, global CDN delivery, precise change tracking  
**Cons**: Complex API, higher storage overhead, sync complexity

### Ghost
**Approach**: Single table with status field plus scheduled publishing  
**States**: draft, published, scheduled, sent  
**Storage**: Uses `status` field + `published_at` timestamp  
**Pros**: Simple but effective, good scheduling support  
**Cons**: Limited editorial workflow, no approval processes

### Strapi
**Approach**: Draft & Publish feature with timestamp-based differentiation  
**States**: draft, published  
**Storage**: Single table with `published_at` field (null = draft)  
**Pros**: Clean API separation, optional feature, good performance  
**Cons**: Limited workflow states, manual schema management

## Requirements

### Functional Requirements
- **FR1**: Editors can save draft content without affecting published site
- **FR2**: Editors can preview changes before publishing
- **FR3**: Authorized users can publish draft content to live site
- **FR4**: System supports rollback to previous published versions
- **FR5**: Clear visual indication of draft vs published state
- **FR6**: Auto-save functionality to prevent content loss

### Non-Functional Requirements
- **NFR1**: Backward compatibility with existing content
- **NFR2**: Minimal performance impact on content editing
- **NFR3**: Support for concurrent editing workflows
- **NFR4**: Audit trail for all publishing actions

## Recommended Solution: State-Based Approach

Based on industry research and our existing architecture, we recommend following the **WordPress/Ghost pattern** with a state field approach. This provides the best balance of simplicity, performance, and functionality.

### Schema Changes

**Core Change**: Add state tracking to existing `content_versions` table:

```sql
-- Add state column to existing content_versions table
ALTER TABLE content_versions ADD COLUMN state TEXT DEFAULT 'history' NOT NULL 
CHECK (state IN ('history', 'draft', 'live'));

-- Create index for efficient state-based queries
CREATE INDEX idx_content_versions_state ON content_versions(content_id, site_id, state);

-- Ensure only one draft and one live version per content item
CREATE UNIQUE INDEX idx_content_versions_unique_draft 
ON content_versions(content_id, site_id) WHERE state = 'draft';

CREATE UNIQUE INDEX idx_content_versions_unique_live 
ON content_versions(content_id, site_id) WHERE state = 'live';
```

**Migration Strategy**:
1. All existing `content_versions` entries become `state='history'`
2. Current `content` table entries migrate to `content_versions` with `state='live'`
3. Drop `content` table after migration (everything now in `content_versions`)

### Content States

| State | Description | Query Pattern |
|-------|-------------|---------------|
| `history` | Previous versions, for rollback | `WHERE state = 'history' ORDER BY created_at DESC` |
| `draft` | Current working version, not published | `WHERE state = 'draft'` |
| `live` | Currently published version | `WHERE state = 'live'` |

### Workflow Logic

**Auto-save Process**:
1. User edits content → Auto-save creates/updates `state='draft'` version
2. Only one draft version exists per content item (upsert pattern)
3. Previous draft becomes `state='history'`

**Publishing Process**:
1. User clicks "Publish" → Current draft version updated to `state='live'`
2. Previous live version becomes `state='history'`
3. Enhancement triggered with all `state='live'` content

**Rollback Process**:
1. User selects historical version → Copy to new `state='live'` version
2. Previous live version becomes `state='history'`
3. Enhancement triggered

### API Design

**New Endpoints**:
```
GET    /api/content/{id}?state=draft|live|history    # Get content in specific state
POST   /api/content/{id}/save-draft                  # Save as draft (auto-save)
POST   /api/content/{id}/publish                     # Publish draft to live
POST   /api/content/{id}/rollback/{version_id}       # Rollback to specific version
GET    /api/content/{id}/diff                        # Compare draft vs live
POST   /api/enhancement/preview                      # Preview site with draft content
GET    /api/status/changes                           # List all unpublished changes
POST   /api/content/bulk-publish                     # Publish multiple items
```

**Enhanced Endpoints**:
```
PUT    /api/content/{id}                             # Now saves as draft by default
POST   /api/enhancement                              # Only processes 'live' content
```

### Repository Layer Changes

**Core Queries**:
```go
// Get current live content for enhancement
func (r *Repository) GetLiveContent(siteID, contentID string) (*Content, error) {
    return r.queryContent(siteID, contentID, "live")
}

// Get current draft for editing
func (r *Repository) GetDraftContent(siteID, contentID string) (*Content, error) {
    return r.queryContent(siteID, contentID, "draft")
}

// Save as draft (upsert pattern)
func (r *Repository) SaveDraft(content *Content) error {
    // Mark existing draft as history
    r.updateState(content.ID, content.SiteID, "draft", "history")
    // Insert new draft
    return r.insertContentVersion(content, "draft")
}

// Publish draft to live
func (r *Repository) PublishDraft(siteID, contentID, publishedBy string) error {
    // Mark existing live as history
    r.updateState(contentID, siteID, "live", "history")
    // Update draft to live
    return r.updateState(contentID, siteID, "draft", "live")
}
```

## Strengths of This Approach

### 1. **Simplicity** 
- Single table with state field (WordPress/Ghost pattern)
- Minimal schema changes to existing system
- Easy to understand and maintain

### 2. **Performance**
- Efficient state-based queries with proper indexing
- No complex joins between draft/live tables
- Leverages existing version history system

### 3. **Backward Compatibility**
- Existing content migrates cleanly to 'live' state
- Current APIs work with minimal changes
- Gradual rollout possible

### 4. **Storage Efficiency**
- No duplicate content storage (unlike Contentful approach)
- Reuses existing version infrastructure
- History naturally maintained

### 5. **Query Simplicity**
```sql
-- Get all draft content for a site
SELECT * FROM content_versions WHERE site_id = ? AND state = 'draft';

-- Get all live content for enhancement
SELECT * FROM content_versions WHERE site_id = ? AND state = 'live';

-- Check if content has unpublished changes
SELECT COUNT(*) FROM content_versions 
WHERE content_id = ? AND site_id = ? AND state = 'draft';
```

## Weaknesses and Potential Roadblocks

### 1. **State Management Complexity**
**Risk**: Ensuring state transitions are atomic and consistent  
**Mitigation**: 
- Use database transactions for state changes
- Implement state validation triggers
- Add comprehensive error handling

### 2. **Concurrent Editing Conflicts**
**Risk**: Multiple editors creating conflicting draft versions  
**Mitigation**:
- Unique constraints prevent multiple drafts
- Last-writer-wins with conflict detection
- Consider optimistic locking for future enhancement

### 3. **Auto-save Performance**
**Risk**: Frequent auto-save creating too many history versions  
**Mitigation**:
- Implement debounced auto-save (30-second intervals)
- Consider version consolidation for excessive history
- Monitor database growth patterns

### 4. **Migration Risk**
**Risk**: Data loss or corruption during content table migration  
**Mitigation**:
- Comprehensive backup before migration
- Gradual migration with validation steps
- Rollback plan if migration fails

### 5. **Limited Workflow States**
**Risk**: Only 3 states may be insufficient for complex editorial workflows  
**Mitigation**:
- Start simple, extend states later if needed
- Most CMS start with basic draft/live model
- Consider "scheduled" state for future enhancement

## UI/UX Changes

### Control Panel Updates
- Replace "🔄 Enhance" with "💾 Save Draft" / "🚀 Publish"
- Add state indicators: 🟡 Draft Pending, 🟢 Published, 🔴 Error
- Add "👁️ Preview Changes" button for draft enhancement
- Show "📊 Publishing Status" with count of unpublished changes

### New UI Components
- Diff viewer showing draft vs published changes
- Publishing confirmation dialog with change summary
- Bulk publishing interface for multiple content items
- Version history with rollback capability

## Implementation Plan

### Phase 1: Database Foundation (Week 1)
- [ ] Add `state` column to `content_versions` table
- [ ] Create state-based indexes and constraints
- [ ] Write migration script for existing content
- [ ] Test migration on demo sites

### Phase 2: Repository Layer (Week 2)
- [ ] Update repository interfaces for state-based queries
- [ ] Implement draft save/publish/rollback operations
- [ ] Add state transition validation
- [ ] Update existing content operations

### Phase 3: API Integration (Week 3)
- [ ] Implement new draft/publish endpoints
- [ ] Update existing endpoints for state handling
- [ ] Add preview enhancement functionality
- [ ] Implement bulk publishing API

### Phase 4: UI Implementation (Week 4)
- [ ] Update control panel with new buttons and states
- [ ] Implement auto-save functionality
- [ ] Add diff viewer and publishing dialogs
- [ ] Create publishing status dashboard

### Phase 5: Testing & Polish (Week 5)
- [ ] Comprehensive testing across demo sites
- [ ] Performance optimization and monitoring
- [ ] Error handling and edge cases
- [ ] Documentation and migration guides

## Testing Strategy

### Migration Testing
- Test content migration with various demo site configurations
- Validate data integrity before/after migration
- Test rollback procedures if migration fails

### Workflow Testing
- Draft save/publish cycles with various content types
- Concurrent editing scenarios
- Auto-save reliability under different conditions
- Enhancement preview vs live comparison

### Performance Testing
- State-based query performance with large datasets
- Auto-save frequency impact on database
- Enhancement speed with draft vs live content

## Success Metrics

### Functional Success
- ✅ Zero data loss during migration
- ✅ All demo sites work without modification post-migration
- ✅ Draft/publish workflow completes in <5 seconds
- ✅ Auto-save prevents content loss in all scenarios

### User Experience Success
- ✅ Clear visual distinction between draft and published states
- ✅ Intuitive publishing workflow requiring minimal training
- ✅ Preview functionality accurately reflects published output

### Technical Success
- ✅ State-based queries perform within 100ms
- ✅ Database size increase <10% due to state optimization
- ✅ 100% test coverage for new draft/publish functionality

## Future Enhancements

### Near-term (Next 6 months)
- **Scheduled Publishing**: Add `scheduled` state with `publish_at` timestamp
- **Bulk Operations**: Enhanced multi-content publishing interface
- **Content Conflicts**: Optimistic locking for concurrent editing

### Long-term (6+ months)
- **Approval Workflows**: Multi-step editorial approval process
- **Content Branching**: Multiple draft versions per content item
- **Real-time Collaboration**: Live editing with conflict resolution

---

*This approach follows industry best practices from WordPress and Ghost while leveraging Insertr's existing version infrastructure for maximum simplicity and reliability.*