Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
1045 lines (848 loc) · 45.9 KB

ARCHITECTURE.md

File metadata and controls

1045 lines (848 loc) · 45.9 KB
Raw

CodeWave Architecture

Complete technical architecture documentation for CodeWave system.

Table of Contents

  1. System Overview
  2. Technology Stack
  3. Core Components
  4. Data Flow
  5. Multi-Agent Orchestration
  6. Developer Overview Generation
  7. Convergence Detection Algorithm
  8. LLM Integration
  9. RAG System
  10. Output Generation
  11. State Management
  12. Error Handling

System Overview

CodeWave is a multi-tier, event-driven system for AI-powered code review that combines:

  • CLI Layer: Interactive command-line interface for user interaction
  • Orchestration Layer: LangGraph-based workflow management
  • Agent Layer: 5 specialized AI agents with distinct expertise
  • LLM Layer: Multi-provider language model integration
  • Storage Layer: Commit data, embeddings, and evaluation results
  • Output Layer: Report generation in multiple formats

Architecture Diagram

┌─────────────────────────────────────────────────────────────�
│                    CLI LAYER                               │
│  ┌──────────────┬──────────────┬──────────────────────�   │
│  │  Evaluate    │  Batch       │  Config              │   │
│  │  Command     │  Evaluate    │  Management          │   │
│  └──────┬───────┴──────┬───────┴────────┬─────────────┘   │
└─────────┼──────────────┼────────────────┼─────────────────┘
          │              │                │
┌─────────▼──────────────▼────────────────▼─────────────────�
│                  ORCHESTRATION LAYER                       │
│              (LangGraph Workflow Engine)                  │
│  ┌────────────────────────────────────────────────────�  │
│  │  Evaluation Orchestrator                           │  │
│  │  - Round 1: Independent Assessment                │  │
│  │  - Round 2: Concerns & Cross-examination          │  │
│  │  - Round 3: Validation & Agreement                │  │
│  │  - Consensus Calculation                          │  │
│  └────────────────────────────────────────────────────┘  │
│  ┌────────────────────────────────────────────────────�  │
│  │  State Machine                                     │  │
│  │  - Conversation History                           │  │
│  │  - Metrics Tracking                               │  │
│  │  - Error Recovery                                 │  │
│  └────────────────────────────────────────────────────┘  │
└────┬──────────────┬──────────────┬───────────────────────┘
     │              │              │
┌────▼──────────────▼──────────────▼────────────────────────�
│                  AGENT LAYER                              │
│  ┌──────� ┌──────� ┌──────� ┌──────� ┌──────�           │
│  │ BA   │ │ DA   │ │ DR   │ │ SA   │ │ QA   │           │
│  │ 🎯   │ │ 👨�💻   │ │ �   │ │ ��   │ │ 🧪   │           │
│  └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘           │
│     │        │        │        │        │                │
│     └────────┼────────┼────────┼────────┘                │
│              │        │        │                         │
└──────────────┼────────┼────────┼──────────────────────────┘
               │        │        │
┌──────────────▼────────▼────────▼──────────────────────────�
│                  LLM LAYER                                │
│  ┌────────────┬────────────┬────────────────────────�   │
│  │ Anthropic  │ OpenAI     │ Google Gemini          │   │
│  │ Claude     │ GPT-4      │ Gemini Pro             │   │
│  └────────────┴────────────┴────────────────────────┘   │
│  ┌────────────────────────────────────────────────�     │
│  │ Token Manager                                  │     │
│  │ - Token Counting                               │     │
│  │ - Cost Estimation                              │     │
│  │ - Rate Limiting                                │     │
│  └────────────────────────────────────────────────┘     │
└──────────────┬────────────────────────────────────────────┘
               │
┌──────────────▼────────────────────────────────────────────�
│              STORAGE & SERVICES LAYER                     │
│  ┌──────────────────� ┌──────────────────────────────�  │
│  │ Git Service      │ │ Vector Store Service (RAG)   │  │
│  │ - Commit Fetch   │ │ - Embeddings                 │  │
│  │ - Diff Parsing   │ │ - Semantic Search            │  │
│  │ - Metadata       │ │ - Chunking Strategy          │  │
│  └──────────────────┘ └──────────────────────────────┘  │
│  ┌──────────────────────────────────────────────────�   │
│  │ Evaluation Service                               │   │
│  │ - Result Persistence                             │   │
│  │ - Batch Management                               │   │
│  └──────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘
               │
               â–¼
┌──────────────────────────────────────────────────────────�
│            OUTPUT GENERATION LAYER                       │
│  ┌──────────────┬──────────────┬──────────────�         │
│  │ HTML Report  │ JSON         │ Markdown     │         │
│  │ Formatter    │ Formatter    │ Formatter    │         │
│  └──────────────┴──────────────┴──────────────┘         │
└──────────────────────────────────────────────────────────┘

Technology Stack

Runtime & Language

  • Node.js: 18.0.0+
  • TypeScript: 5.3.3+
  • Runtime: CommonJS modules

Core Dependencies

AI & LLM Integration

  • LangChain (v0.3.36): Chain and agent orchestration
  • LangGraph (v0.2.74): State machine and workflow management
  • @langchain/anthropic: Claude API integration
  • @langchain/openai: OpenAI API integration
  • @langchain/google-genai: Google Gemini integration
  • js-tiktoken: Token counting for OpenAI models

CLI & UI

  • Commander.js (v14.0.2): Command-line framework
  • Inquirer.js (v8.2.7): Interactive CLI prompts
  • Chalk (v4.1.2): Terminal color output
  • Ora (v5.4.1): Spinner/progress indicators
  • cli-progress (v3.12.0): Progress bars

Utilities

  • dotenv (v17.2.3): Environment variable management

Development Dependencies

  • TypeScript: Type system
  • ESLint: Code linting
  • Prettier: Code formatting
  • @types/node: Node.js type definitions

Core Components

1. CLI Layer (cli/)

Entry point for user interactions. Handles command routing and user-facing logic.

Files:

  • cli/index.ts - Main CLI entry point (Commander setup)
  • cli/commands/evaluate-command.ts - Single commit evaluation
  • cli/commands/batch-evaluate-command.ts - Multiple commits
  • cli/commands/config.command.ts - Configuration management
  • cli/utils/progress-tracker.ts - Progress UI
  • cli/utils/git-utils.ts - Git operations (commit diffs, file extraction)
  • cli/utils/diagnostic-filter.ts - Log filtering for progress display
  • cli/utils/shared.utils.ts - CLI utilities

Responsibilities:

  • Parse command-line arguments
  • Display interactive setup wizard
  • Manage user configuration
  • Display real-time progress
  • Format and display results

2. Agent Layer (src/agents/)

Specialized AI agents with distinct expertise areas.

Architecture:

BaseAgentWorkflow (Abstract)
  ├── BusinessAnalystAgent
  ├── DeveloperAuthorAgent
  ├── DeveloperReviewerAgent
  ├── SeniorArchitectAgent
  └── QAEngineerAgent

Key Classes:

BaseAgentWorkflow

abstract class BaseAgentWorkflow {
  abstract name: string;
  abstract emoji: string;
  abstract role: string;
  abstract metrics: string[];

  async assessCommit(context: AgentContext): Promise<AgentResponse>;
  async raiseConcerns(context: AgentContext): Promise<AgentResponse>;
  async validateAndAgree(context: AgentContext): Promise<AgentResponse>;

  protected buildPrompt(systemRole: string, context: AgentContext): string;
  protected parseResponse(response: string): AgentResponse;
}

Workflow:

  1. Receive commit context and conversation history
  2. Generate specialized prompt based on agent role
  3. Call LLM service with prompt
  4. Parse structured response
  5. Extract metrics and reasoning
  6. Return structured agent response

3. Orchestration Layer (src/orchestrator/)

LangGraph-based workflow coordination for multi-round conversations.

Key Component: Orchestrator

class Orchestrator {
  // Initialize with agents and LLM service
  constructor(agents: Agent[], llmService: LLMService);

  // Execute full evaluation workflow
  async executeEvaluation(
    commitData: CommitData,
    conversationHistory: ConversationTurn[] = [],
  ): Promise<EvaluationResult>;

  // Internal methods for each round
  private round1Independent Assessment(): Promise<AgentResponse[]>;
  private round2CrossExamination(round1Responses: AgentResponse[]): Promise<AgentResponse[]>;
  private round3FinalValidation(
    round1Responses: AgentResponse[],
    round2Responses: AgentResponse[],
  ): Promise<AgentResponse[]>;

  // Calculate consensus from all responses
  private calculateConsensus(allResponses: AgentResponse[][]): ConsensusData;
}

Workflow State Machine:

START
  │
  ├─► ROUND_1_ASSESSMENT
  │   └─► All agents assess independently
  │       State: { round: 1, responses: AgentResponse[] }
  │
  ├─► ROUND_2_CROSS_EXAMINATION
  │   └─► Agents respond to each other's concerns
  │       State: { round: 2, responses: AgentResponse[] }
  │
  ├─► ROUND_3_FINAL_VALIDATION
  │   └─► Agents finalize positions
  │       State: { round: 3, responses: AgentResponse[] }
  │
  ├─► CONSENSUS_CALCULATION
  │   └─► Calculate final metrics and consensus
  │       State: { consensus: ConsensusData, metrics: EvaluationMetrics }
  │
  └─► COMPLETE
      Result: EvaluationResult

4. LLM Service Layer (src/llm/)

Multi-provider LLM abstraction.

Key Classes:

LLMService (Interface)

interface LLMService {
  generateMessage(
    systemPrompt: string,
    userMessage: string,
    options?: GenerateOptions
  ): Promise<string>;

  countTokens(text: string): Promise<number>;
  estimateCost(tokensUsed: number): Promise<number>;
}

Provider Implementations:

  • AnthropicLLMService - Claude API
  • OpenAILLMService - GPT-4, GPT-4 Turbo
  • GoogleLLMService - Gemini

Token Management (TokenManager):

class TokenManager {
  countTokens(text: string, model: string): number;
  estimateCost(tokensUsed: number, model: string): number;
  trackUsage(request: string, response: string): void;
  getSummary(): TokenUsageSummary;
}

5. Storage & Services Layer

Git Service

class CommitService {
  getCommit(hash: string): Promise<CommitData>;
  getCommitRange(since: string, until: string): Promise<CommitData[]>;
  getDiff(hash: string): Promise<string>;
}

Vector Store Service (RAG)

class VectorStoreService {
  addDocuments(texts: string[], metadata: Record<string, any>): Promise<void>;
  similaritySearch(query: string, k?: number): Promise<string[]>;
  clear(): Promise<void>;
}

Evaluation Service

class EvaluationService {
  saveEvaluation(result: EvaluationResult): Promise<string>;
  loadEvaluation(id: string): Promise<EvaluationResult>;
  listEvaluations(): Promise<EvaluationResult[]>;
}

6. Output Generation Layer (src/formatters/)

Formatters:

HTMLReportFormatterEnhanced

  • Generates interactive HTML reports
  • Timeline visualization
  • Agent role identification
  • Metric evolution display
  • Bootstrap-based responsive design

JSONFormatter

  • Structured data export
  • Schema-validated output
  • Complete conversation history

MarkdownFormatter

  • Human-readable transcripts
  • Suitable for documentation
  • Git-compatible format

Data Flow

Evaluation Flow (High-Level)

User Input
  │
  â–¼
Parse Arguments & Config
  │
  â–¼
Fetch Commit from Git
  │
  ├─► Determine if Large Diff
  │   ├─► If < 100KB: Process normally
  │   └─► If > 100KB: Initialize RAG
  │
  â–¼
Orchestrator.executeEvaluation()
  │
  ├─► Round 1: All agents assess independently
  │   ├─► Business Analyst → Functional Impact, Ideal Time
  │   ├─► Developer Author → Actual Time
  │   ├─► Developer Reviewer → Code Quality
  │   ├─► Senior Architect → Complexity, Technical Debt
  │   └─► QA Engineer → Test Coverage
  │
  ├─► Round 2: Cross-examination with concerns
  │   └─► All agents respond to concerns from Round 1
  │
  ├─► Round 3: Final consensus positions
  │   └─► All agents finalize scores and recommendations
  │
  ├─► Calculate Consensus
  │   ├─► Weighted averaging of metrics
  │   ├─► Confidence level calculation
  │   ├─► Top concerns extraction
  │   └─► Recommendations synthesis
  │
  â–¼
Generate Output
  ├─► HTML Report (interactive)
  ├─► JSON Results (structured data)
  ├─► Markdown Transcript (documentation)
  ├─► Text Summary (quick reference)
  └─► Archive Original Diff
  │
  â–¼
Save to Output Directory
  │
  â–¼
Display Results to User

Batch Evaluation Flow

User Input (count, date range, branch)
  │
  â–¼
Fetch Commit List from Git
  │
  â–¼
Initialize Progress Tracker
  │
  ├─► Split into parallel queues (default: 3)
  │
  ├─► For Each Queue:
  │   └─► While Commits Remaining:
  │       ├─► Evaluate commit (same as single flow)
  │       ├─► Update progress bar
  │       ├─► Handle errors (skip or halt)
  │       └─► Track metrics (quality, coverage, cost)
  │
  â–¼
Generate Summary Report
  ├─► Statistics (avg quality, coverage, time)
  ├─► Cost analysis
  ├─► Error log
  └─► Individual results list
  │
  â–¼
Display Summary to User

Multi-Agent Orchestration

Execution Model

CodeWave uses a round-robin discussion model with shared context:

State Representation

interface ConversationState {
  round: 1 | 2 | 3;
  commit: CommitData;
  agentResponses: Map<AgentName, AgentResponse[]>; // Round history
  sharedContext: {
    conversationHistory: ConversationTurn[];
    previousConcerns: string[];
    emergingConsensus: Partial<EvaluationMetrics>;
  };
}

Round Execution

async function executeRound(
  round: 1 | 2 | 3,
  state: ConversationState
): Promise<ConversationState> {
  const roundPromises = agents.map((agent) => {
    const prompt = buildPrompt(agent, round, state);
    return agent.respond(prompt, state);
  });

  const responses = await Promise.all(roundPromises);

  return {
    ...state,
    round: round + 1,
    agentResponses: updateResponses(state.agentResponses, responses),
  };
}

Agent Context Management

Each agent receives context including:

  1. Commit Data: Files, changes, metadata
  2. Previous Responses: What other agents said
  3. Conversation History: Full transcript
  4. Shared Metrics: Emerging consensus
  5. Agent Role: Their specific responsibility

Developer Overview Generation

Architecture

Developer Overview runs as the first node in the LangGraph workflow, before any agent evaluation:

Commit Input
  │
  â–¼
┌──────────────────────────────�
│ Developer Overview Generator │ ◄─── First Node
│  ┌─────────────────────────� │
│  │ - Extract key changes   │ │
│  │ - Identify file purposes│ │
│  │ - Generate summary      │ │
│  │ - Format for readability│ │
│  └─────────────────────────┘ │
└──────────────────────────────┘
  │
  ├─► Stored in State (all agents access)
  │
  â–¼
Agents Begin Evaluation (Round 1)

Implementation

Location: src/orchestrator/commit-evaluation-graph.ts

// First node in graph
const generateDeveloperOverview = async (state: GraphState) => {
  console.log('� Generating developer overview from commit diff...');

  const overview = await developerOverviewGenerator.generate(
    state.commitDiff,
    state.commitMetadata
  );

  console.log(`✅ Developer overview generated (${overview.length} chars)`);

  return {
    ...state,
    developerOverview: overview,
  };
};

// Add to graph
graph.addNode('generateDeveloperOverview', generateDeveloperOverview);
graph.setEntryPoint('generateDeveloperOverview');
graph.addEdge('generateDeveloperOverview', 'round1Agents');

Benefits

  1. Shared Context: All agents see same overview
  2. Consistency: Same summary regardless of agent disagreement
  3. Token Efficiency: Overview reduces needed context
  4. Debugging: Clear record of what was analyzed
  5. Documentation: Auto-generated change summary

Error Handling

If generation fails:

  • Empty overview provided to agents
  • Agents evaluate using raw diff
  • Report shows placeholder message
  • Evaluation continues without blocking

Convergence Detection Algorithm

Purpose

Measure how much agents agree and optimize evaluation rounds.

Implementation

Location: src/orchestrator/convergence-calculator.ts

interface ConvergenceMetrics {
  perMetric: {
    [metric: string]: number; // 0-1, lower variance = higher consensus
  };
  weighted: number; // Final convergence score
  targetMet: boolean; // Did we reach target?
  shouldContinue: boolean; // Continue to next round?
}

class ConvergenceCalculator {
  calculate(
    round1Responses: AgentResponse[],
    round2Responses: AgentResponse[],
    round3Responses?: AgentResponse[]
  ): ConvergenceMetrics {
    // 1. Calculate metric variance for each pillar
    // 2. Apply weights (quality and coverage are 2x weight)
    // 3. Compare to target threshold (0.75)
    // 4. Determine if consensus reached or should continue
  }
}

Algorithm Steps

Step 1: Extract Final Scores for Each Metric

Code Quality scores: [7, 7, 8, 6, 7]
Test Coverage scores: [6, 5, 7, 5, 6]
... (for all 7 pillars)

Step 2: Calculate Variance per Metric

StdDev(Code Quality) = 0.6 → Normalized = 0.15 (low variance = high agreement)
StdDev(Test Coverage) = 0.8 → Normalized = 0.20
... (aggregate remaining metrics)

Step 3: Apply Weights

Weighted Convergence = 1.0 - (
  0.15 * 2.0 (quality weight) +
  0.20 * 2.0 (coverage weight) +
  ... (other metrics with 1x weight)
) / total_weight

Step 4: Compare to Target & Decide

if (convergenceScore >= 0.75) {
  return { shouldContinue: false, targetMet: true };
} else if (currentRound < 3) {
  return { shouldContinue: true, targetMet: false };
} else {
  return { shouldContinue: false, targetMet: false };
}

Convergence Thresholds

0.9+:  Excellent consensus, very reliable evaluation
0.7-0.8: Good consensus, minor disagreements acceptable
0.5-0.6: Moderate agreement, review disagreements
<0.5:   Low consensus, significant debate ongoing

Round Continuation Logic

Round 1 Output → Calculate Convergence
  │
  ├─ If >= 0.75: STOP ✓ (High confidence)
  └─ If < 0.75:  Continue to Round 2

Round 2 Output → Calculate Convergence
  │
  ├─ If >= 0.75: STOP ✓ (Good agreement reached)
  ├─ If improved but < 0.75: Continue to Round 3
  └─ If no improvement: STOP (Max rounds reached)

Round 3 Output → Final Convergence
  │
  └─► STOP (Always stop after Round 3)

Storage in History

Each evaluation stores convergence:

{
  "evaluationNumber": 7,
  "timestamp": "2025-11-08T22:58:27.689Z",
  "convergenceScore": 0.51,
  "metrics": { ... },
  "rounds": {
    "round1": {
      "convergence": 0.45,
      "shouldContinue": true
    },
    "round2": {
      "convergence": 0.62,
      "shouldContinue": true
    },
    "round3": {
      "convergence": 0.51,
      "shouldContinue": false
    }
  }
}

LLM Integration

Multi-Provider Pattern

interface LLMProvider {
  name: string;
  models: ModelConfig[];
  generateMessage(prompt: string): Promise<string>;
  countTokens(text: string): Promise<number>;
}

class LLMFactory {
  static createProvider(config: LLMConfig): LLMProvider {
    switch (config.provider) {
      case 'anthropic':
        return new AnthropicProvider(config);
      case 'openai':
        return new OpenAIProvider(config);
      case 'google':
        return new GoogleProvider(config);
    }
  }
}

Token Counting Strategy

Different models have different token counting:

  • Claude: Uses js-tiktoken or Claude API token count
  • GPT-4: Uses js-tiktoken
  • Gemini: Uses Gemini API token count

Each call is tracked:

interface TokenTrack {
  requestTokens: number;
  responseTokens: number;
  totalTokens: number;
  model: string;
  timestamp: Date;
}

Cost Estimation

const costModel = {
  'claude-3-5-sonnet-20241022': {
    input: 0.003 / 1000, // $0.003 per 1K tokens
    output: 0.015 / 1000, // $0.015 per 1K tokens
  },
  'gpt-4o': {
    input: 0.015 / 1000,
    output: 0.03 / 1000,
  },
  'gemini-2.0-flash': {
    input: 0.075 / 1000,
    output: 0.3 / 1000,
  },
};

cost = inputTokens * costModel.input + outputTokens * costModel.output;

RAG System

Activation Strategy

if (diffSize > ragThreshold) {
  // Use RAG for large diffs
  initialize RagEvaluationWorkflow;
} else {
  // Use standard evaluation
  initialize StandardEvaluationWorkflow;
}

RAG Process

  1. Chunking

    • Split large diff into semantic chunks
    • Default chunk size: 2000 characters
    • Preserve file and hunk boundaries

  2. Embedding

    • Generate vector embeddings using local model
    • Store in memory vector store

  3. Retrieval

    • Agents query most relevant chunks
    • Retrieves top-k chunks per query
    • Provides context window of relevant changes

  4. Processing

    • Agents work with subset of diff
    • Reduced token count
    • Faster evaluation

Output Generation

HTML Report Structure

<!DOCTYPE html>
<html>
  <head>
    <!-- Bootstrap CSS -->
    <!-- Custom styles -->
  </head>
  <body>
    <header>
      <!-- Commit metadata -->
      <!-- Overall quality score -->
    </header>

    <main>
      <!-- Metrics cards -->
      <!-- Agent profiles -->
      <!-- Round-by-round timeline -->
      <!-- Concerns and recommendations -->
    </main>

    <aside>
      <!-- Conversation transcript -->
      <!-- Full metrics table -->
    </aside>
  </body>
</html>

JSON Structure

interface EvaluationResultJSON {
  metadata: {
    evaluationId: string;
    timestamp: string;
    version: string;
  };
  commit: CommitMetadata;
  metrics: EvaluationMetrics;
  rounds: EvaluationRound[];
  consensus: ConsensusData;
  conversation: ConversationTurn[];
}

State Management

Orchestrator State Machine (LangGraph)

┌─────────────────�
│    START        │
└────────┬────────┘
         │
         â–¼
┌─────────────────────────────────────�
│  INITIALIZE_STATE                   │
│  - Load commit data                 │
│  - Setup RAG if needed              │
│  - Initialize conversation history  │
└────────┬────────────────────────────┘
         │
         â–¼
┌─────────────────────────────────────�
│  ROUND_1_INDEPENDENT_ASSESSMENT     │
│  - Run all agents in parallel       │
│  - Collect initial responses        │
└────────┬────────────────────────────┘
         │
         â–¼
┌─────────────────────────────────────�
│  ROUND_2_CONCERNS                   │
│  - Pass Round 1 to all agents       │
│  - Collect concerns and updates     │
└────────┬────────────────────────────┘
         │
         â–¼
┌─────────────────────────────────────�
│  ROUND_3_FINAL_VALIDATION           │
│  - Pass all history to agents       │
│  - Collect final positions          │
└────────┬────────────────────────────┘
         │
         â–¼
┌─────────────────────────────────────�
│  CALCULATE_CONSENSUS                │
│  - Weight all responses             │
│  - Calculate final metrics          │
│  - Extract themes and concerns      │
└────────┬────────────────────────────┘
         │
         â–¼
┌─────────────────────────────────────�
│  GENERATE_OUTPUT                    │
│  - Format all output types          │
│  - Save to filesystem               │
└────────┬────────────────────────────┘
         │
         â–¼
┌─────────────────�
│     COMPLETE    │
└─────────────────┘

Error Recovery

try {
  result = await evaluateCommit(commit);
} catch (error) {
  if (isRetryable(error)) {
    if (retryCount < maxRetries) {
      retryCount++;
      await delay(exponentialBackoff(retryCount));
      return evaluateCommit(commit); // Retry
    }
  }

  if (skipErrors) {
    return { error: error.message, commit };
  } else {
    throw error; // Halt execution
  }
}

Error Handling

Error Categories

Recoverable Errors (retry or skip):

  • Network timeouts
  • Rate limiting (retry with backoff)
  • Temporary API failures

Non-Recoverable Errors (halt or skip):

  • Invalid commit hash
  • Authentication failures
  • Corrupted diff data

Warnings (continue with caution):

  • Large diffs requiring RAG
  • Unusual complexity patterns
  • High divergence between agents

Error Workflow

Error Occurs
  │
  ├─► Classify error type
  │   ├─► Recoverable
  │   ├─► Non-recoverable
  │   └─► Warning
  │
  ├─► If Recoverable:
  │   ├─► Retry with backoff
  │   └─► Increment retry counter
  │
  ├─► If Non-Recoverable:
  │   ├─► Log error
  │   ├─► If batch: skip commit (if skipErrors=true)
  │   └─► If single: throw error
  │
  ├─► If Warning:
  │   ├─► Log warning
  │   ├─► Continue evaluation
  │   └─► Flag in results
  │
  â–¼
Continue Processing

Performance Considerations

Optimization Strategies

  1. Parallel Agent Execution

    • Round 1: All agents assess simultaneously
    • Reduces total evaluation time

  2. Token Optimization

    • RAG reduces tokens for large commits
    • Summarization of agent responses

  3. Caching

    • Cache agent responses for identical commits
    • Reuse embeddings for similar diffs

  4. Batch Efficiency

    • Process multiple commits in parallel (default: 3)
    • Configurable parallelization

Performance Metrics

  • Single Commit: 2-5 seconds average
  • 100 Commits: 3-8 minutes (with parallelization)
  • Token Usage: 3,000-5,000 per commit
  • Cost: ~$0.015-0.030 per commit

For more information: