StratIQX Documentation

Appearance

Execution Logging System Documentation ​

Overview ​

The Execution Logging System provides granular, step-by-step monitoring of background job processing with beautiful UI visualization. This system captures detailed execution flow, timing, and error context - offering superior debugging capabilities compared to standard Cloudflare Worker logs.

Architecture ​

High-Level Flow ​

Background Worker (ai-consulting-main-worker)
    ↓ Stores execution logs
KV Namespace (EXECUTION_LOGS)
    ↓ Retrieved by
Pages Functions (strategic-intelligence-onboarding)
    ↓ Displayed in
Admin UI Modal (Process Log Viewer)

Key Components ​

  1. Background Job Processor - Captures execution steps
  2. KV Storage - Stores structured log data
  3. Pages Function API - Retrieves logs with authentication
  4. Admin UI Modal - Beautiful step-by-step visualization

Backend Implementation ​

1. Log Structure ​

Each execution creates a comprehensive log entry:

javascript
const executionLog = {
    tracking_id: "60B3RNTB",
    execution_id: "exec_1703123456789_xyz123abc",
    started_at: "2024-01-15T10:30:00.000Z",
    completed_at: "2024-01-15T10:32:30.000Z",
    duration_ms: 150000,
    tier: "comprehensive",
    company: "Acme Corp",
    steps: [
        {
            step: "initialization",
            timestamp: "2024-01-15T10:30:00.000Z",
            status: "completed",
            message: "Started processing for Acme Corp (comprehensive tier)"
        },
        {
            step: "status_update",
            timestamp: "2024-01-15T10:30:01.000Z",
            status: "completed",
            message: "Updated status to processing"
        },
        {
            step: "orchestrator_execution",
            timestamp: "2024-01-15T10:30:02.000Z",
            status: "in_progress",
            message: "Starting orchestrator execution..."
        },
        {
            step: "error_handling",
            timestamp: "2024-01-15T10:32:30.000Z",
            status: "failed",
            message: "Processing failed: AI generation timeout after 480000ms",
            error_details: "Error: AI generation timeout after 480000ms\n    at AIService.generateContent (worker.js:1234:56)"
        }
    ]
};

2. Background Job Integration ​

BackgroundJobProcessor.js ​

javascript
// Create fresh execution log for each processing attempt
async processProfile(profile) {
    const executionLog = {
        tracking_id: trackingId,
        execution_id: `exec_${Date.now()}_${Math.random().toString(36).substr(2, 9)}`,
        started_at: new Date().toISOString(),
        tier: profile.tier_level,
        company: profile.company_name,
        steps: []
    };

    try {
        // Log initialization
        executionLog.steps.push({
            step: 'initialization',
            timestamp: new Date().toISOString(),
            status: 'completed',
            message: `Started processing for ${profile.company_name} (${profile.tier_level} tier)`
        });

        // Log status updates
        await this.updateProfileStatus(trackingId, 'processing', {
            processed_at: new Date().toISOString(),
            processing_mode: 'background'
        });

        executionLog.steps.push({
            step: 'status_update',
            timestamp: new Date().toISOString(),
            status: 'completed',
            message: 'Updated status to processing'
        });

        // Log orchestrator execution
        executionLog.steps.push({
            step: 'orchestrator_execution',
            timestamp: new Date().toISOString(),
            status: 'in_progress',
            message: 'Starting orchestrator execution...'
        });

        const response = await this.orchestratorService.executeOrchestrator(mockRequest);

        if (response.success) {
            executionLog.steps.push({
                step: 'orchestrator_execution',
                timestamp: new Date().toISOString(),
                status: 'completed',
                message: `Orchestrator completed successfully. Response: ${responseData.message || 'Analysis complete'}`
            });
        }

    } catch (error) {
        // Log errors with full context
        executionLog.steps.push({
            step: 'error_handling',
            timestamp: new Date().toISOString(),
            status: 'failed',
            message: `Processing failed: ${error.message}`,
            error_details: error.stack
        });
    } finally {
        // Always store execution log
        executionLog.completed_at = new Date().toISOString();
        executionLog.duration_ms = Date.now() - startTime;
        await this.storeExecutionLog(executionLog);
    }
}

3. KV Storage Implementation ​

javascript
/**
 * Store detailed execution log for admin monitoring
 */
async storeExecutionLog(executionLog) {
    try {
        // Store in KV for quick admin access
        const key = `execution_log:${executionLog.tracking_id}`;
        await this.env.EXECUTION_LOGS.put(key, JSON.stringify(executionLog), {
            expirationTtl: 7 * 24 * 60 * 60 // 7 days retention
        });

        // Log storage confirmation
        this.logger.info(`Execution Log Stored: ${executionLog.tracking_id} (${executionLog.duration_ms}ms)`, {
            tracking_id: executionLog.tracking_id,
            duration: executionLog.duration_ms,
            steps_count: executionLog.steps.length,
            final_status: executionLog.steps[executionLog.steps.length - 1]?.status || 'unknown'
        });
    } catch (error) {
        this.logger.error('Failed to store execution log:', error);
    }
}

/**
 * Retrieve execution log for a specific tracking ID
 */
async getExecutionLog(trackingId) {
    try {
        const key = `execution_log:${trackingId}`;
        const logData = await this.env.EXECUTION_LOGS.get(key);
        return logData ? JSON.parse(logData) : null;
    } catch (error) {
        this.logger.error(`Failed to retrieve execution log for ${trackingId}:`, error);
        return null;
    }
}

4. KV Namespace Configuration ​

wrangler.jsonc ​

json
{
  "kv_namespaces": [
    {
      "binding": "EXECUTION_LOGS",
      "id": "89bc42b35fa848498eb44de24367029c",
      "preview_id": "89bc42b35fa848498eb44de24367029c"
    }
  ]
}

Frontend Implementation ​

1. Pages Function API ​

functions/api/execution-logs/[trackingId].ts ​

typescript
export const onRequestGet: typeof onRequest = async (context) => {
  const { request, env, params } = context;
  const trackingId = params.trackingId as string;

  try {
    // Access the background worker's KV store
    const typedEnv = env as unknown as Env;
    const key = `execution_log:${trackingId}`;
    const logData = await typedEnv.EXECUTION_LOGS.get(key);

    if (!logData) {
      return new Response(JSON.stringify({
        success: false,
        error: 'No execution log found for this tracking ID',
        tracking_id: trackingId
      }), {
        status: 404,
        headers: { 'Content-Type': 'application/json' }
      });
    }

    const executionLog: ExecutionLog = JSON.parse(logData);

    // Format for admin display
    const formattedLog = {
      tracking_id: executionLog.tracking_id,
      execution_id: executionLog.execution_id,
      company: executionLog.company,
      tier: executionLog.tier,
      started_at: executionLog.started_at,
      completed_at: executionLog.completed_at,
      duration_ms: executionLog.duration_ms,
      duration_readable: formatDuration(executionLog.duration_ms),
      total_steps: executionLog.steps.length,
      final_status: getFinalStatus(executionLog.steps),
      steps: executionLog.steps.map(step => ({
        step: step.step,
        status: step.status,
        timestamp: step.timestamp,
        message: step.message,
        time_from_start: getTimeFromStart(executionLog.started_at, step.timestamp),
        error_details: step.error_details || null
      }))
    };

    return new Response(JSON.stringify({
      success: true,
      data: formattedLog
    }), {
      headers: { 'Content-Type': 'application/json' }
    });

  } catch (error) {
    console.error('Execution log retrieval error:', error);
    return new Response(JSON.stringify({
      success: false,
      error: 'Failed to retrieve execution log',
      details: error instanceof Error ? error.message : 'Unknown error'
    }), {
      status: 500,
      headers: { 'Content-Type': 'application/json' }
    });
  }
};

2. React Component Integration ​

AnalysisQueuedSection.tsx ​

tsx
const fetchExecutionLog = async () => {
  if (!trackingId) return;

  setLoadingLog(true);
  setLogError(null);

  try {
    const response = await fetch(`/api/execution-logs/${trackingId}`);
    const data = await response.json();

    if (data.success) {
      setExecutionLog(data.data);
    } else {
      setLogError(data.error || 'Failed to load execution log');
    }
  } catch (error) {
    console.error('Failed to fetch execution log:', error);
    setLogError('Failed to connect to execution log service');
  } finally {
    setLoadingLog(false);
  }
};

const handleProcessLogClick = () => {
  setShowProcessLog(true);
  fetchExecutionLog();
};

UI Design System ​

Color Palette & Status Mapping ​

Step Status Indicators ​

css
/* Completed Steps */
.step-completed {
  border-left: 4px solid #10b981; /* Green-400 */
  background-color: #f0fdf4;      /* Green-50 */
}

/* Failed Steps */
.step-failed {
  border-left: 4px solid #ef4444; /* Red-400 */
  background-color: #fef2f2;      /* Red-50 */
}

/* In Progress Steps */
.step-in-progress {
  border-left: 4px solid #3b82f6; /* Blue-400 */
  background-color: #eff6ff;      /* Blue-50 */
}

Final Status Badges ​

css
/* Completed Analysis */
.status-completed {
  background-color: #dcfce7;      /* Green-100 */
  color: #166534;                 /* Green-800 */
}

/* Failed Analysis */
.status-failed {
  background-color: #fee2e2;      /* Red-100 */
  color: #991b1b;                 /* Red-800 */
}

/* In Progress Analysis */
.status-in-progress {
  background-color: #fef3c7;      /* Yellow-100 */
  color: #92400e;                 /* Yellow-800 */
}

Typography & Layout ​

Font Specifications ​

css
/* Modal Header */
.log-header {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto';
  font-size: 1.125rem;            /* 18px */
  font-weight: 600;               /* Semibold */
  color: #1f2937;                 /* Gray-800 */
}

/* Metadata Grid */
.log-metadata {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto';
  font-size: 0.75rem;             /* 12px */
  font-weight: 500;               /* Medium */
}

/* Execution Steps */
.log-steps {
  font-family: 'SF Mono', Monaco, Inconsolata, 'Roboto Mono', monospace;
  font-size: 0.875rem;            /* 14px */
  line-height: 1.5;
}

/* Step Messages */
.step-message {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto';
  font-size: 0.75rem;             /* 12px */
  line-height: 1.4;
}

/* Error Details */
.error-details {
  font-family: 'SF Mono', Monaco, Inconsolata, 'Roboto Mono', monospace;
  font-size: 0.625rem;            /* 10px */
  background-color: #fef2f2;      /* Red-50 */
  color: #dc2626;                 /* Red-600 */
  padding: 4px 8px;
  border-radius: 4px;
  white-space: pre-wrap;
}

Usage Patterns ​

1. Fresh Execution Logs ​

Each re-process creates a fresh execution log:

  • Clean slate for each attempt
  • Focused on current execution
  • No historical clutter
  • Unique execution_id per attempt
  • Consistent KV key for easy lookup

2. Error Debugging Workflow ​

1. User clicks "Re-process" on failed analysis
2. Background job starts with fresh execution log
3. Each step is logged with precise timing
4. Error occurs and is captured with full stack trace
5. Admin clicks "Process Log" to see detailed failure context
6. Admin identifies exact failure point and timing
7. Admin can fix issue and re-process again

3. Performance Monitoring ​

1. Admin opens "Process Log" for completed analysis
2. Reviews step-by-step timing breakdown
3. Identifies bottlenecks (e.g., "orchestrator_execution: +2m 30s")
4. Optimizes slow components based on granular timing data

Benefits Over Standard Cloudflare Logs ​

Granular Execution Logging System ​

  • Step-by-step visibility: See exactly where processing succeeds/fails
  • Precise timing: Know how long each step takes
  • Rich context: Company, tier, tracking ID metadata
  • Beautiful UI: Professional modal with color-coded steps
  • Error context: Full stack traces and error details
  • Self-contained: No external log aggregation needed
  • User-friendly: Accessible to non-technical users
  • Real-time: Available immediately after execution

Standard Cloudflare Worker Logs ​

  • Still valuable for: Infrastructure debugging, broader context
  • Best for: Worker deployment issues, environment problems
  • Limited for: Application-level step-by-step debugging

Configuration ​

Environment Variables ​

json
{
  "AI_GENERATION_TIMEOUT": "300000",     // 5 minutes
  "LLAMA4_SCOUT_TIMEOUT": "480000",      // 8 minutes
  "LOG_LEVEL": "info"
}

KV Namespace Settings ​

json
{
  "binding": "EXECUTION_LOGS",
  "id": "89bc42b35fa848498eb44de24367029c",
  "expirationTtl": 604800              // 7 days retention
}

Extensibility ​

Adding New Step Types ​

javascript
// In BackgroundJobProcessor.js
executionLog.steps.push({
    step: 'pdf_generation',              // New step type
    timestamp: new Date().toISOString(),
    status: 'in_progress',
    message: 'Generating PDF report...'
});

// After PDF generation
executionLog.steps.push({
    step: 'pdf_generation',
    timestamp: new Date().toISOString(),
    status: 'completed',
    message: `PDF generated successfully (${pdfSize} bytes)`
});

Custom Error Handling ​

javascript
try {
    await complexOperation();
} catch (error) {
    executionLog.steps.push({
        step: 'complex_operation',
        timestamp: new Date().toISOString(),
        status: 'failed',
        message: `Complex operation failed: ${error.message}`,
        error_details: error.stack,
        retry_attempt: retryCount,        // Custom metadata
        operation_params: operationConfig // Custom context
    });
}

Best Practices ​

1. Meaningful Step Names ​

javascript
// Good
step: 'ai_model_inference'
step: 'pdf_generation'
step: 'data_validation'

// Avoid
step: 'step1'
step: 'process'
step: 'thing'

2. Descriptive Messages ​

javascript
// Good
message: 'AI inference completed for comprehensive tier analysis (1,247 tokens)'
message: 'PDF generation failed: Template not found for tier level'

// Avoid
message: 'Done'
message: 'Error occurred'

3. Consistent Status Values ​

javascript
// Standard status values
'completed'    // Step finished successfully
'failed'       // Step failed with error
'in_progress'  // Step currently executing
'skipped'      // Step skipped due to conditions

4. Error Context ​

javascript
// Always include error details for failed steps
executionLog.steps.push({
    step: 'operation_name',
    status: 'failed',
    message: `Clear description: ${error.message}`,
    error_details: error.stack,           // Full stack trace
    error_code: error.code,               // Error classification
    retry_possible: true                  // Retry indication
});

Future Enhancements ​

1. Log Aggregation ​

  • Collect logs across multiple executions
  • Identify patterns in failures
  • Performance trending over time

2. Alert Integration ​

  • Notify on repeated failures
  • Performance degradation alerts
  • Custom webhook integrations

3. Export Capabilities ​

  • CSV export for analysis
  • Integration with external monitoring
  • Automated reporting

4. Enhanced UI Features ​

  • Log search and filtering
  • Step duration visualizations
  • Error pattern analysis

Conclusion ​

The Execution Logging System provides comprehensive, granular visibility into background job processing with a beautiful, professional UI. This system offers superior debugging capabilities compared to standard infrastructure logs while maintaining clean architecture and excellent user experience.

The fresh execution log approach ensures focused, actionable information for each processing attempt, making it perfect for both development debugging and production monitoring.

Strategic Intelligence Hub Documentation

Copyright © 2024-present StratIQX