insertr/SERVER_UPDATE.md

Server Update Plan - Phase 3a: HTML-First API Architecture

🎯 Current Status & Vision

What We Discovered

Our frontend has evolved to a sophisticated HTML-first approach with:

• StyleAware editor with automatic style detection from nested elements
• HTML preservation with perfect attribute fidelity
• Rich content editing capabilities with formatting toolbar
• Template-based style preservation using CLASSES.md methodology

However, our server API is still text-focused, creating a fundamental mismatch between frontend capabilities and backend storage.

Core Requirements Identified

1. HTML-First Storage: Replace value with html_content field for direct HTML storage
2. Template Preservation: Store original_template for consistent style detection
3. Enhancer-First Workflow: Enhancer stores content on first pass, ignores processed elements
4. No Markdown Processing: Remove all markdown logic from injector - HTML only
5. StyleAware Editor Compatibility: API must match library expectations
6. Dev Convenience: Option to clean DB for fresh development iterations

🏗️ Implementation Strategy

1. HTML-First Database Schema (Direct Replacement)

Updated Schema:

-- SQLite schema
CREATE TABLE content (
    id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,        -- Rich HTML content (innerHTML)
    original_template TEXT,            -- Original element markup for style detection (outerHTML)
    type TEXT NOT NULL,
    created_at INTEGER DEFAULT (strftime('%s', 'now')) NOT NULL,
    updated_at INTEGER DEFAULT (strftime('%s', 'now')) NOT NULL,
    last_edited_by TEXT DEFAULT 'system' NOT NULL,
    PRIMARY KEY (id, site_id)
);

-- PostgreSQL schema  
CREATE TABLE content (
    id TEXT NOT NULL,
    site_id TEXT NOT NULL,
    html_content TEXT NOT NULL,        -- Rich HTML content (innerHTML)
    original_template TEXT,            -- Original element markup for style detection (outerHTML)
    type TEXT NOT NULL,
    created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    updated_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
    last_edited_by TEXT DEFAULT 'system' NOT NULL,
    PRIMARY KEY (id, site_id)
);

Key Changes:

• ✅ Removed value field - HTML serves both editing and injection needs
• ✅ Added html_content - Direct HTML storage for content editing and injection
• ✅ Added original_template - Preserves developer templates for StyleAware editor style detection
• ✅ Simplified approach - No complex template locking, focus on core functionality

2. Enhancer-First Workflow (First-Pass Processing)

Unprocessed Element Detection:

• Elements without data-content-id attribute are unprocessed
• Enhancer processes these elements and assigns IDs
• Subsequent enhancer runs skip elements that already have data-content-id

Content Storage on First Pass:

func processElement(node *html.Node, siteID string) {
    // Check if already processed
    existingID := getAttribute(node, "data-content-id")
    if existingID != "" {
        return // Skip - already processed
    }
    
    // Extract content and template
    contentID := generateContentID(node, filePath)
    htmlContent := extractInnerHTML(node)         // For editing/injection
    originalTemplate := extractOuterHTML(node)    // For style detection
    contentType := determineContentType(node)
    
    // Store in database
    createContent(siteID, contentID, htmlContent, originalTemplate, contentType)
    
    // Mark as processed
    setAttribute(node, "data-content-id", contentID)
    setAttribute(node, "data-content-type", contentType)
}

Development Convenience:

• Optional DB cleanup flag: insertr enhance --clean-db for fresh development iterations
• Allows developers to start with clean slate when refining site structure

3. Injector Redesign (HTML-Only)

Remove Markdown Processing:

• Delete MarkdownProcessor and all markdown-related logic
• Direct HTML injection using existing injectHTMLContent() method
• Simplified injection flow focused on HTML fidelity

Updated Injection Process:

func (i *Injector) InjectContent(element *Element, contentID string) error {
    // Fetch content from database
    contentItem, err := i.client.GetContent(i.siteID, contentID)
    if err != nil || contentItem == nil {
        // No content found - add attributes but keep original content
        i.AddContentAttributes(element.Node, contentID, element.Type)
        return nil
    }
    
    // Direct HTML injection - no markdown processing
    i.injectHTMLContent(element.Node, contentItem.HTMLContent)
    i.AddContentAttributes(element.Node, contentID, element.Type)
    
    return nil
}

Content Type Handling:

• All content types use HTML storage and injection
• Remove type-specific processing (text, markdown, link)
• StyleAware editor handles rich editing based on element context

4. StyleAware Editor Integration

API Response Format (Matches Editor Expectations):

{
    "id": "hero-title-abc123",
    "site_id": "mysite",
    "html_content": "<h1>Welcome to <em>Our</em> Company</h1>",
    "original_template": "<h1 class=\"insertr brand-heading\">Welcome to <span class=\"brand\">Our Company</span></h1>",
    "type": "text",
    "created_at": "2024-01-15T10:30:00Z",
    "updated_at": "2024-01-15T10:30:00Z",
    "last_edited_by": "user@example.com"
}

Editor Integration:

• Style Detection: Uses original_template for consistent formatting options
• Content Editing: Uses html_content for rich text editing
• Perfect Alignment: Response format matches StyleAware editor analysis requirements
• Multi-Property Support: Complex elements (links) work seamlessly with preserved templates

Updated API Models:

type ContentItem struct {
    ID               string    `json:"id"`
    SiteID           string    `json:"site_id"`
    HTMLContent      string    `json:"html_content"`      // For editor content
    OriginalTemplate string    `json:"original_template"` // For style detection  
    Type             string    `json:"type"`
    CreatedAt        time.Time `json:"created_at"`
    UpdatedAt        time.Time `json:"updated_at"`
    LastEditedBy     string    `json:"last_edited_by"`
}

🔄 Development Workflows

Enhanced Development Workflow:

# Start fresh development iteration
insertr enhance ./mysite --clean-db --site-id mysite

# Files are processed, content stored, elements marked as processed
# Subsequent enhancement runs skip already processed elements
# Developer can iterate on unprocessed parts without affecting existing content

# Start development server
insertr serve --dev-mode

# Editor changes -> only html_content updated, templates preserved
# Style detection uses stored original_template for consistency

Production Workflow:

# Production enhancement (no DB cleanup)
insertr enhance ./mysite --site-id mysite

# Production server
insertr serve

# All editing preserves original developer templates
# StyleAware editor gets consistent style detection from stored templates
# Content updates only affect html_content field

🎯 Key Benefits

For Developers:

✅ Efficient Processing: Only process unprocessed elements, skip already handled ones
✅ Development Convenience: Optional DB cleanup for fresh iterations
✅ HTML-first approach: Direct alignment with StyleAware editor capabilities
✅ Zero Configuration: Automatic detection and processing of viable elements

For Content Editors:

✅ Style Preservation: Developer styles always available via original_template
✅ Rich Editing: Full HTML capabilities with formatting toolbar
✅ Perfect Fidelity: No lossy conversions, complete attribute preservation
✅ Design Safety: Cannot accidentally break developer styling constraints

For System Architecture:

✅ Simplified Flow: No markdown conversion complexity
✅ Direct Injection: HTML content injects directly into static files
✅ Clean Separation: Enhancement stores content, API serves editing
✅ Performance: Skip already-processed elements for faster builds

📋 Implementation Tasks

Week 1: Database Foundation

1. Schema Updates

   • Update SQLite schema: replace value with html_content, add original_template
   • Update PostgreSQL schema: replace value with html_content, add original_template
   • Update content.sql queries to use new fields
   • Regenerate SQLC models

2. API Models

   • Update ContentItem struct to use html_content and original_template
   • Update request/response structs for new field names
   • Update API handlers to work with new field structure

Week 2: Enhancer Logic

3. First-Pass Processing
   • Update enhancer to detect processed elements via data-content-id
   • Update enhancer to store html_content and original_template on first pass
   • Add development DB cleanup option (--clean-db flag)

Week 3: Injector Redesign

4. HTML-Only Injection
   • Remove MarkdownProcessor and all markdown-related code from injector
   • Update injector to use html_content directly via injectHTMLContent()
   • Remove type-specific content processing (text, markdown, link)

Week 4: Integration Testing

5. StyleAware Editor Compatibility
   • Test API responses work correctly with StyleAware editor
   • Verify original_template enables proper style detection
   • Test rich HTML editing and injection end-to-end

🚀 Implementation Strategy

Priority Order:

1. Database Changes First: Schema, queries, models - foundation for everything else
2. Enhancer Updates: First-pass processing logic and content storage
3. Injector Simplification: Remove markdown, use HTML directly
4. Integration Testing: Verify StyleAware editor compatibility

Key Implementation Notes:

• No Migration Required: Fresh schema replacement, no backward compatibility needed
• Enhancer-Driven: Content storage happens during enhancement, not via API
• HTML-Only: Eliminate all markdown processing complexity
• StyleAware Alignment: API response format matches editor expectations exactly

This represents a fundamental shift to HTML-first content management with enhanced developer workflow efficiency while maintaining the zero-configuration philosophy that makes Insertr unique.

---

Status: Ready for Implementation
Estimated Effort: 1 week for core implementation
Breaking Changes: Yes (fresh schema, enhancer workflow changes)