joakim/insertr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add comprehensive blog demo showcasing advanced content management features

Browse Source 
- Implement complete mushroom foraging blog with chanterelles article
- Add rich demonstration of .insertr and .insertr-add functionality
- Include comprehensive documentation for future .insertr-content vision
- Update project styling and configuration to support blog demo
- Enhance engine and API handlers for improved content management
This commit is contained in:
Joakim
2025-10-23 21:47:51 +02:00
parent 74de64c66b
commit 4874849f80
13 changed files with 1411 additions and 340 deletions
Download Patch File Download Diff File Expand all files Collapse all files

497
DRAFT_PUBLISH_FEATURE.md
View File

@@ -18,6 +18,43 @@ This document outlines the design and implementation plan for adding draft/publi
- All-or-nothing manual enhancement process
- No rollback mechanism for published content
## Industry Research: How Other CMS Handle Drafts
### WordPress
**Approach**: Single table (`wp_posts`) with state field (`post_status`)
**States**: draft, pending, publish, future, private, trash, auto-draft
**Storage**: All content in one table, differentiated by status field
**Pros**: Simple schema, easy queries, unified storage
**Cons**: No separation between draft and live data, potential performance issues
### Drupal
**Approach**: Content moderation module with workflow states
**States**: Configurable (draft, needs_review, published, archived, etc.)
**Storage**: Moderation state entities linked to content revisions
**Pros**: Flexible workflows, proper revision tracking, role-based transitions
**Cons**: Complex architecture, steep learning curve
### Contentful (Headless)
**Approach**: Separate published/draft versions with sync API
**States**: draft, published, changed, archived
**Storage**: Maintains both draft and published versions simultaneously
**Pros**: Performance optimized, global CDN delivery, precise change tracking
**Cons**: Complex API, higher storage overhead, sync complexity
### Ghost
**Approach**: Single table with status field plus scheduled publishing
**States**: draft, published, scheduled, sent
**Storage**: Uses `status` field + `published_at` timestamp
**Pros**: Simple but effective, good scheduling support
**Cons**: Limited editorial workflow, no approval processes
### Strapi
**Approach**: Draft & Publish feature with timestamp-based differentiation
**States**: draft, published
**Storage**: Single table with `published_at` field (null = draft)
**Pros**: Clean API separation, optional feature, good performance
**Cons**: Limited workflow states, manual schema management
## Requirements
### Functional Requirements
@@ -26,7 +63,7 @@ This document outlines the design and implementation plan for adding draft/publi
- **FR3**: Authorized users can publish draft content to live site
- **FR4**: System supports rollback to previous published versions
- **FR5**: Clear visual indication of draft vs published state
- **FR6**: Batch publishing of multiple content changes
- **FR6**: Auto-save functionality to prevent content loss
### Non-Functional Requirements
- **NFR1**: Backward compatibility with existing content
@@ -34,289 +71,277 @@ This document outlines the design and implementation plan for adding draft/publi
- **NFR3**: Support for concurrent editing workflows
- **NFR4**: Audit trail for all publishing actions
## Architecture Approaches
## Recommended Solution: State-Based Approach
### Option A: Minimal Schema Impact (Published Version Pointer)
Based on industry research and our existing architecture, we recommend following the **WordPress/Ghost pattern** with a state field approach. This provides the best balance of simplicity, performance, and functionality.
**Core Concept**: Use existing version system with a pointer to published versions.
### Schema Changes
**Core Change**: Add state tracking to existing `content_versions` table:
**Schema Changes**:
```sql
CREATE TABLE published_versions (
content_id TEXT NOT NULL,
site_id TEXT NOT NULL,
published_version_id INTEGER NOT NULL,
published_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
published_by TEXT NOT NULL,
PRIMARY KEY (content_id, site_id)
);
-- Add state column to existing content_versions table
ALTER TABLE content_versions ADD COLUMN state TEXT DEFAULT 'history' NOT NULL
CHECK (state IN ('history', 'draft', 'live'));
-- Create index for efficient state-based queries
CREATE INDEX idx_content_versions_state ON content_versions(content_id, site_id, state);
-- Ensure only one draft and one live version per content item
CREATE UNIQUE INDEX idx_content_versions_unique_draft
ON content_versions(content_id, site_id) WHERE state = 'draft';
CREATE UNIQUE INDEX idx_content_versions_unique_live
ON content_versions(content_id, site_id) WHERE state = 'live';
```
**Pros**:
- Minimal database changes
- Leverages existing version history
- Backward compatible
- Simple migration path
**Cons**:
- Less intuitive data model
- Enhancement modes add complexity
- Auto-save creates unnecessary versions
- Limited workflow capabilities
### Option B: Full Schema Redesign (Recommended)
**Core Concept**: Explicit draft and published content tables with rich workflow support.
**Schema Changes**:
```sql
-- Draft content (working state)
CREATE TABLE content_drafts (
id TEXT NOT NULL,
site_id TEXT NOT NULL,
html_content TEXT NOT NULL,
original_template TEXT,
created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
updated_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
last_edited_by TEXT DEFAULT 'system' NOT NULL,
is_dirty BOOLEAN DEFAULT TRUE NOT NULL,
auto_save_at BIGINT,
PRIMARY KEY (id, site_id)
);
-- Published content (live state)
CREATE TABLE content_published (
id TEXT NOT NULL,
site_id TEXT NOT NULL,
html_content TEXT NOT NULL,
original_template TEXT,
published_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
published_by TEXT DEFAULT 'system' NOT NULL,
draft_version_at_publish BIGINT NOT NULL,
PRIMARY KEY (id, site_id)
);
-- Enhanced version tracking
CREATE TABLE content_versions (
version_id SERIAL PRIMARY KEY,
content_id TEXT NOT NULL,
site_id TEXT NOT NULL,
html_content TEXT NOT NULL,
original_template TEXT,
created_at BIGINT DEFAULT EXTRACT(EPOCH FROM NOW()) NOT NULL,
created_by TEXT DEFAULT 'system' NOT NULL,
version_type TEXT NOT NULL CHECK (version_type IN ('draft_save', 'publish', 'auto_save')),
is_published BOOLEAN DEFAULT FALSE NOT NULL
);
-- Future: Scheduled publishing support
CREATE TABLE publish_queue (
queue_id SERIAL PRIMARY KEY,
site_id TEXT NOT NULL,
content_ids TEXT[] NOT NULL,
scheduled_at BIGINT,
created_by TEXT NOT NULL,
status TEXT DEFAULT 'pending' CHECK (status IN ('pending', 'processing', 'completed', 'failed'))
);
```
**Pros**:
- Clear semantic separation
- Rich workflow capabilities
- Optimized for different access patterns
- Supports advanced features (auto-save, batch publish, scheduling)
- Intuitive mental model
**Cons**:
- Significant schema changes
- Complex migration required
- More storage overhead
- Increased development complexity
## Recommended Solution: Option B (Full Schema Redesign)
**Migration Strategy**:
1. All existing `content_versions` entries become `state='history'`
2. Current `content` table entries migrate to `content_versions` with `state='live'`
3. Drop `content` table after migration (everything now in `content_versions`)
### Content States
| State | Draft Table | Published Table | Description |
|-------|-------------|-----------------|-------------|
| `draft_only` | ✅ Exists | ❌ None | New content, never published |
| `published` | ✅ Exists | ✅ Exists, Same | Content published, no pending changes |
| `modified` | ✅ Exists | ✅ Exists, Different | Published content with unpublished changes |
| `scheduled` | ✅ Exists | ✅ Exists | Content queued for future publishing |
| State | Description | Query Pattern |
|-------|-------------|---------------|
| `history` | Previous versions, for rollback | `WHERE state = 'history' ORDER BY created_at DESC` |
| `draft` | Current working version, not published | `WHERE state = 'draft'` |
| `live` | Currently published version | `WHERE state = 'live'` |
### Workflow Logic
**Auto-save Process**:
1. User edits content → Auto-save creates/updates `state='draft'` version
2. Only one draft version exists per content item (upsert pattern)
3. Previous draft becomes `state='history'`
**Publishing Process**:
1. User clicks "Publish" → Current draft version updated to `state='live'`
2. Previous live version becomes `state='history'`
3. Enhancement triggered with all `state='live'` content
**Rollback Process**:
1. User selects historical version → Copy to new `state='live'` version
2. Previous live version becomes `state='history'`
3. Enhancement triggered
### API Design
**New Endpoints**:
```
GET /api/content/{id}?mode=draft|published # Get content in specific state
POST /api/content/{id}/save-draft # Save without publishing
POST /api/content/{id}/publish # Publish draft content
POST /api/content/bulk-publish # Publish multiple items
GET /api/content/diff/{id} # Show draft vs published diff
POST /api/enhancement/preview # Preview with draft content
POST /api/enhancement/publish # Enhance with published content
GET /api/status/publishing # Get site publishing status
GET /api/content/{id}?state=draft|live|history # Get content in specific state
POST /api/content/{id}/save-draft # Save as draft (auto-save)
POST /api/content/{id}/publish # Publish draft to live
POST /api/content/{id}/rollback/{version_id} # Rollback to specific version
GET /api/content/{id}/diff # Compare draft vs live
POST /api/enhancement/preview # Preview site with draft content
GET /api/status/changes # List all unpublished changes
POST /api/content/bulk-publish # Publish multiple items
```
**Enhanced Endpoints**:
```
PUT /api/content/{id} # Now saves as draft by default
PUT /api/content/{id} # Now saves as draft by default
POST /api/enhancement # Only processes 'live' content
```
### UI/UX Changes
### Repository Layer Changes
**Control Panel Updates**:
- Replace "🔄 Enhance" with publishing workflow buttons
- Add "💾 Save Draft" (auto-saves every 30s)
- Add "🚀 Publish Changes" with confirmation dialog
- Add "👁️ Preview Changes" for draft enhancement
- Add "📊 Publishing Status" indicator
**Core Queries**:
```go
// Get current live content for enhancement
func (r *Repository) GetLiveContent(siteID, contentID string) (*Content, error) {
return r.queryContent(siteID, contentID, "live")
}
**Visual States**:
- 🟡 **Draft Pending**: Yellow indicator for unpublished changes
- 🟢 **Published**: Green indicator when draft matches published
- 🔵 **Scheduled**: Blue indicator for queued publishing
- 🔴 **Error**: Red indicator for publishing failures
// Get current draft for editing
func (r *Repository) GetDraftContent(siteID, contentID string) (*Content, error) {
return r.queryContent(siteID, contentID, "draft")
}
**New UI Components**:
// Save as draft (upsert pattern)
func (r *Repository) SaveDraft(content *Content) error {
// Mark existing draft as history
r.updateState(content.ID, content.SiteID, "draft", "history")
// Insert new draft
return r.insertContentVersion(content, "draft")
}
// Publish draft to live
func (r *Repository) PublishDraft(siteID, contentID, publishedBy string) error {
// Mark existing live as history
r.updateState(contentID, siteID, "live", "history")
// Update draft to live
return r.updateState(contentID, siteID, "draft", "live")
}
```
## Strengths of This Approach
### 1. **Simplicity**
- Single table with state field (WordPress/Ghost pattern)
- Minimal schema changes to existing system
- Easy to understand and maintain
### 2. **Performance**
- Efficient state-based queries with proper indexing
- No complex joins between draft/live tables
- Leverages existing version history system
### 3. **Backward Compatibility**
- Existing content migrates cleanly to 'live' state
- Current APIs work with minimal changes
- Gradual rollout possible
### 4. **Storage Efficiency**
- No duplicate content storage (unlike Contentful approach)
- Reuses existing version infrastructure
- History naturally maintained
### 5. **Query Simplicity**
```sql
-- Get all draft content for a site
SELECT * FROM content_versions WHERE site_id = ? AND state = 'draft';
-- Get all live content for enhancement
SELECT * FROM content_versions WHERE site_id = ? AND state = 'live';
-- Check if content has unpublished changes
SELECT COUNT(*) FROM content_versions
WHERE content_id = ? AND site_id = ? AND state = 'draft';
```
## Weaknesses and Potential Roadblocks
### 1. **State Management Complexity**
**Risk**: Ensuring state transitions are atomic and consistent
**Mitigation**:
- Use database transactions for state changes
- Implement state validation triggers
- Add comprehensive error handling
### 2. **Concurrent Editing Conflicts**
**Risk**: Multiple editors creating conflicting draft versions
**Mitigation**:
- Unique constraints prevent multiple drafts
- Last-writer-wins with conflict detection
- Consider optimistic locking for future enhancement
### 3. **Auto-save Performance**
**Risk**: Frequent auto-save creating too many history versions
**Mitigation**:
- Implement debounced auto-save (30-second intervals)
- Consider version consolidation for excessive history
- Monitor database growth patterns
### 4. **Migration Risk**
**Risk**: Data loss or corruption during content table migration
**Mitigation**:
- Comprehensive backup before migration
- Gradual migration with validation steps
- Rollback plan if migration fails
### 5. **Limited Workflow States**
**Risk**: Only 3 states may be insufficient for complex editorial workflows
**Mitigation**:
- Start simple, extend states later if needed
- Most CMS start with basic draft/live model
- Consider "scheduled" state for future enhancement
## UI/UX Changes
### Control Panel Updates
- Replace "🔄 Enhance" with "💾 Save Draft" / "🚀 Publish"
- Add state indicators: 🟡 Draft Pending, 🟢 Published, 🔴 Error
- Add "👁️ Preview Changes" button for draft enhancement
- Show "📊 Publishing Status" with count of unpublished changes
### New UI Components
- Diff viewer showing draft vs published changes
- Publishing confirmation dialog with change summary
- Bulk publishing interface for multiple content items
- Publishing history and rollback interface
- Version history with rollback capability
## Implementation Plan
### Phase 1: Core Infrastructure (Week 1-2)
- [ ] Create new database schema
- [ ] Implement migration scripts
- [ ] Update repository interfaces
- [ ] Add basic draft/publish operations
- [ ] Update content versioning system
### Phase 1: Database Foundation (Week 1)
- [ ] Add `state` column to `content_versions` table
- [ ] Create state-based indexes and constraints
- [ ] Write migration script for existing content
- [ ] Test migration on demo sites
### Phase 2: API Development (Week 3-4)
- [ ] Implement new API endpoints
- [ ] Update existing endpoints for draft mode
- [ ] Add enhancement mode switching
- [ ] Implement publishing workflow APIs
- [ ] Add content state management
### Phase 2: Repository Layer (Week 2)
- [ ] Update repository interfaces for state-based queries
- [ ] Implement draft save/publish/rollback operations
- [ ] Add state transition validation
- [ ] Update existing content operations
### Phase 3: UI Integration (Week 5-6)
- [ ] Update control panel with new buttons
- [ ] Add visual state indicators
- [ ] Implement draft auto-save
- [ ] Add preview functionality
- [ ] Create publishing confirmation dialogs
### Phase 3: API Integration (Week 3)
- [ ] Implement new draft/publish endpoints
- [ ] Update existing endpoints for state handling
- [ ] Add preview enhancement functionality
- [ ] Implement bulk publishing API
### Phase 4: Advanced Features (Week 7-8)
- [ ] Implement diff viewer
- [ ] Add bulk publishing interface
- [ ] Create publishing history view
- [ ] Add rollback functionality
- [ ] Implement scheduled publishing foundation
### Phase 4: UI Implementation (Week 4)
- [ ] Update control panel with new buttons and states
- [ ] Implement auto-save functionality
- [ ] Add diff viewer and publishing dialogs
- [ ] Create publishing status dashboard
### Phase 5: Testing & Polish (Week 9-10)
- [ ] Comprehensive testing across all demo sites
- [ ] Performance optimization
### Phase 5: Testing & Polish (Week 5)
- [ ] Comprehensive testing across demo sites
- [ ] Performance optimization and monitoring
- [ ] Error handling and edge cases
- [ ] Documentation updates
- [ ] Migration testing
## Potential Challenges & Mitigation
### Challenge 1: Data Migration Complexity
**Risk**: Migrating existing content to new schema without data loss
**Mitigation**:
- Create comprehensive migration scripts with rollback capability
- Test migration on demo sites first
- Implement gradual migration with dual-write period
- Provide data validation and integrity checks
### Challenge 2: Concurrent Editing Conflicts
**Risk**: Multiple editors working on same content simultaneously
**Mitigation**:
- Implement optimistic locking with version checking
- Add conflict detection and resolution UI
- Consider edit session management
- Provide clear error messages for conflicts
### Challenge 3: Performance Impact
**Risk**: Additional database queries and storage overhead
**Mitigation**:
- Optimize database indexes for new access patterns
- Implement efficient content state queries
- Consider caching strategies for published content
- Monitor and profile database performance
### Challenge 4: Backward Compatibility
**Risk**: Breaking existing workflows and integrations
**Mitigation**:
- Maintain existing API compatibility during transition
- Provide clear migration path for existing users
- Implement feature flags for gradual rollout
- Extensive testing with current demo sites
### Challenge 5: Auto-save vs Version History Noise
**Risk**: Too many auto-save versions cluttering history
**Mitigation**:
- Separate auto-save from manual save operations
- Implement version consolidation strategies
- Use different version types (auto_save vs draft_save)
- Provide version cleanup mechanisms
- [ ] Documentation and migration guides
## Testing Strategy
### Unit Tests
- Repository layer draft/publish operations
- Content state transition logic
- API endpoint functionality
- Migration script validation
### Migration Testing
- Test content migration with various demo site configurations
- Validate data integrity before/after migration
- Test rollback procedures if migration fails
### Integration Tests
- End-to-end publishing workflow
- Enhancement with different content modes
### Workflow Testing
- Draft save/publish cycles with various content types
- Concurrent editing scenarios
- Database migration processes
- Auto-save reliability under different conditions
- Enhancement preview vs live comparison
### User Acceptance Tests
- Editorial workflow testing with demo sites
- Performance testing under load
- Cross-browser UI compatibility
- Mobile device testing
### Performance Testing
- State-based query performance with large datasets
- Auto-save frequency impact on database
- Enhancement speed with draft vs live content
## Success Metrics
### Functional Metrics
- ✅ All existing demo sites work without modification
- ✅ Editors can save drafts without affecting live sites
- ✅ Publishing workflow completes in <5 seconds
### Functional Success
- ✅ Zero data loss during migration
- ✅ All demo sites work without modification post-migration
- ✅ Draft/publish workflow completes in <5 seconds
- ✅ Auto-save prevents content loss in all scenarios
### User Experience Metrics
- ✅ Clear visual indication of content states
- ✅ Intuitive publishing workflow
- ✅ Auto-save prevents content loss
- ✅ Preview functionality works accurately
### User Experience Success
- ✅ Clear visual distinction between draft and published states
- ✅ Intuitive publishing workflow requiring minimal training
- ✅ Preview functionality accurately reflects published output
### Technical Metrics
- ✅ API response times remain under 200ms
- ✅ Database migration completes in <30 minutes
- ✅ Memory usage increase <20%
- ✅ 100% test coverage for new functionality
### Technical Success
- ✅ State-based queries perform within 100ms
- ✅ Database size increase <10% due to state optimization
- ✅ 100% test coverage for new draft/publish functionality
## Future Considerations
## Future Enhancements
### Potential Enhancements
- **Scheduled Publishing**: Full calendar-based publishing system
- **Approval Workflows**: Multi-stage content approval process
- **Content Staging**: Multiple environment support (dev/staging/prod)
- **Collaborative Editing**: Real-time collaborative editing features
- **Content Templates**: Draft templates for consistent content structure
### Near-term (Next 6 months)
- **Scheduled Publishing**: Add `scheduled` state with `publish_at` timestamp
- **Bulk Operations**: Enhanced multi-content publishing interface
- **Content Conflicts**: Optimistic locking for concurrent editing
### Technical Debt
- Consider eventual consolidation of version tables
- Evaluate long-term storage strategies for large sites
- Plan for horizontal scaling of publishing operations
- Review and optimize database schema after real-world usage
### Long-term (6+ months)
- **Approval Workflows**: Multi-step editorial approval process
- **Content Branching**: Multiple draft versions per content item
- **Real-time Collaboration**: Live editing with conflict resolution
---
*This document will be updated as the feature evolves through design reviews and implementation feedback.*
*This approach follows industry best practices from WordPress and Ghost while leveraging Insertr's existing version infrastructure for maximum simplicity and reliability.*