insertr/DRAFT_PUBLISH_FEATURE.md

Feature Document: Draft/Publish System for Insertr CMS

Overview

This document outlines the design and implementation plan for adding draft/publish functionality to Insertr CMS. Currently, all content changes are immediately reflected when enhancement is triggered manually. This feature will introduce a proper editorial workflow with draft states and controlled publishing.

Problem Statement

Current State

• All content stored in database is immediately available for enhancement
• Manual "Enhance" button triggers immediate file updates with latest content
• No separation between working drafts and production-ready content
• Not suitable for production environments where editorial approval is needed

User Pain Points

• Editors cannot safely make changes without affecting live site
• No preview workflow for reviewing changes before going live
• All-or-nothing manual enhancement process
• No rollback mechanism for published content

Requirements

Functional Requirements

• FR1: Editors can save draft content without affecting published site
• FR2: Editors can preview changes before publishing
• FR3: Authorized users can publish draft content to live site
• FR4: System supports rollback to previous published versions
• FR5: Clear visual indication of draft vs published state
• FR6: Batch publishing of multiple content changes

Non-Functional Requirements

• NFR1: Backward compatibility with existing content
• NFR2: Minimal performance impact on content editing
• NFR3: Support for concurrent editing workflows
• NFR4: Audit trail for all publishing actions

Architecture Approaches

Option A: Minimal Schema Impact (Published Version Pointer)

Core Concept: Use existing version system with a pointer to published versions.

Schema Changes:

CREATE TABLE published_versions (
    content_id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    published_version_id INTEGER NOT NULL,
    published_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    published_by TEXT NOT NULL,
    PRIMARY KEY (content_id, site_id)
);

Pros:

• Minimal database changes
• Leverages existing version history
• Backward compatible
• Simple migration path

Cons:

• Less intuitive data model
• Enhancement modes add complexity
• Auto-save creates unnecessary versions
• Limited workflow capabilities

Option B: Full Schema Redesign (Recommended)

Core Concept: Explicit draft and published content tables with rich workflow support.

Schema Changes:

-- Draft content (working state)
CREATE TABLE content_drafts (
    id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,
    original_template TEXT,
    created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    updated_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    last_edited_by TEXT DEFAULT 'system' NOT NULL,
    is_dirty BOOLEAN DEFAULT TRUE NOT NULL,
    auto_save_at BIGINT,
    PRIMARY KEY (id, site_id)
);

-- Published content (live state)
CREATE TABLE content_published (
    id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,
    original_template TEXT,
    published_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    published_by TEXT DEFAULT 'system' NOT NULL,
    draft_version_at_publish BIGINT NOT NULL,
    PRIMARY KEY (id, site_id)
);

-- Enhanced version tracking
CREATE TABLE content_versions (
    version_id SERIAL PRIMARY KEY,
    content_id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,
    original_template TEXT,
    created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    created_by TEXT DEFAULT 'system' NOT NULL,
    version_type TEXT NOT NULL CHECK (version_type IN ('draft_save', 'publish', 'auto_save')),
    is_published BOOLEAN DEFAULT FALSE NOT NULL
);

-- Future: Scheduled publishing support
CREATE TABLE publish_queue (
    queue_id SERIAL PRIMARY KEY,
    site_id TEXT NOT NULL,
    content_ids TEXT[] NOT NULL,
    scheduled_at BIGINT,
    created_by TEXT NOT NULL,
    status TEXT DEFAULT 'pending' CHECK (status IN ('pending', 'processing', 'completed', 'failed'))
);

Pros:

• Clear semantic separation
• Rich workflow capabilities
• Optimized for different access patterns
• Supports advanced features (auto-save, batch publish, scheduling)
• Intuitive mental model

Cons:

• Significant schema changes
• Complex migration required
• More storage overhead
• Increased development complexity

Recommended Solution: Option B (Full Schema Redesign)

Content States

State	Draft Table	Published Table	Description
draft_only	✅ Exists	❌ None	New content, never published
published	✅ Exists	✅ Exists, Same	Content published, no pending changes
modified	✅ Exists	✅ Exists, Different	Published content with unpublished changes
scheduled	✅ Exists	✅ Exists	Content queued for future publishing

API Design

New Endpoints:

GET    /api/content/{id}?mode=draft|published     # Get content in specific state
POST   /api/content/{id}/save-draft               # Save without publishing
POST   /api/content/{id}/publish                  # Publish draft content
POST   /api/content/bulk-publish                  # Publish multiple items
GET    /api/content/diff/{id}                     # Show draft vs published diff
POST   /api/enhancement/preview                   # Preview with draft content
POST   /api/enhancement/publish                   # Enhance with published content
GET    /api/status/publishing                     # Get site publishing status

Enhanced Endpoints:

PUT    /api/content/{id}                          # Now saves as draft by default

UI/UX Changes

Control Panel Updates:

• Replace "🔄 Enhance" with publishing workflow buttons
• Add "💾 Save Draft" (auto-saves every 30s)
• Add "🚀 Publish Changes" with confirmation dialog
• Add "👁️ Preview Changes" for draft enhancement
• Add "📊 Publishing Status" indicator

Visual States:

• 🟡 Draft Pending: Yellow indicator for unpublished changes
• 🟢 Published: Green indicator when draft matches published
• 🔵 Scheduled: Blue indicator for queued publishing
• 🔴 Error: Red indicator for publishing failures

New UI Components:

• Diff viewer showing draft vs published changes
• Bulk publishing interface for multiple content items
• Publishing history and rollback interface

Implementation Plan

Phase 1: Core Infrastructure (Week 1-2)

• Create new database schema
• Implement migration scripts
• Update repository interfaces
• Add basic draft/publish operations
• Update content versioning system

Phase 2: API Development (Week 3-4)

• Implement new API endpoints
• Update existing endpoints for draft mode
• Add enhancement mode switching
• Implement publishing workflow APIs
• Add content state management

Phase 3: UI Integration (Week 5-6)

• Update control panel with new buttons
• Add visual state indicators
• Implement draft auto-save
• Add preview functionality
• Create publishing confirmation dialogs

Phase 4: Advanced Features (Week 7-8)

• Implement diff viewer
• Add bulk publishing interface
• Create publishing history view
• Add rollback functionality
• Implement scheduled publishing foundation

Phase 5: Testing & Polish (Week 9-10)

• Comprehensive testing across all demo sites
• Performance optimization
• Error handling and edge cases
• Documentation updates
• Migration testing

Potential Challenges & Mitigation

Challenge 1: Data Migration Complexity

Risk: Migrating existing content to new schema without data loss Mitigation:

• Create comprehensive migration scripts with rollback capability
• Test migration on demo sites first
• Implement gradual migration with dual-write period
• Provide data validation and integrity checks

Challenge 2: Concurrent Editing Conflicts

Risk: Multiple editors working on same content simultaneously Mitigation:

• Implement optimistic locking with version checking
• Add conflict detection and resolution UI
• Consider edit session management
• Provide clear error messages for conflicts

Challenge 3: Performance Impact

Risk: Additional database queries and storage overhead Mitigation:

• Optimize database indexes for new access patterns
• Implement efficient content state queries
• Consider caching strategies for published content
• Monitor and profile database performance

Challenge 4: Backward Compatibility

Risk: Breaking existing workflows and integrations Mitigation:

• Maintain existing API compatibility during transition
• Provide clear migration path for existing users
• Implement feature flags for gradual rollout
• Extensive testing with current demo sites

Challenge 5: Auto-save vs Version History Noise

Risk: Too many auto-save versions cluttering history Mitigation:

• Separate auto-save from manual save operations
• Implement version consolidation strategies
• Use different version types (auto_save vs draft_save)
• Provide version cleanup mechanisms

Testing Strategy

Unit Tests

• Repository layer draft/publish operations
• Content state transition logic
• API endpoint functionality
• Migration script validation

Integration Tests

• End-to-end publishing workflow
• Enhancement with different content modes
• Concurrent editing scenarios
• Database migration processes

User Acceptance Tests

• Editorial workflow testing with demo sites
• Performance testing under load
• Cross-browser UI compatibility
• Mobile device testing

Success Metrics

Functional Metrics

• ✅ All existing demo sites work without modification
• ✅ Editors can save drafts without affecting live sites
• ✅ Publishing workflow completes in <5 seconds
• ✅ Zero data loss during migration

User Experience Metrics

• ✅ Clear visual indication of content states
• ✅ Intuitive publishing workflow
• ✅ Auto-save prevents content loss
• ✅ Preview functionality works accurately

Technical Metrics

• ✅ API response times remain under 200ms
• ✅ Database migration completes in <30 minutes
• ✅ Memory usage increase <20%
• ✅ 100% test coverage for new functionality

Future Considerations

Potential Enhancements

• Scheduled Publishing: Full calendar-based publishing system
• Approval Workflows: Multi-stage content approval process
• Content Staging: Multiple environment support (dev/staging/prod)
• Collaborative Editing: Real-time collaborative editing features
• Content Templates: Draft templates for consistent content structure

Technical Debt

• Consider eventual consolidation of version tables
• Evaluate long-term storage strategies for large sites
• Plan for horizontal scaling of publishing operations
• Review and optimize database schema after real-world usage

---

This document will be updated as the feature evolves through design reviews and implementation feedback.