[Future] Implement custom SDK hooks for tool-specific metrics

[frankbria]

Overview

SDK hooks have a TODO for recording tool-specific metrics (e.g., how often agents use Write vs Bash).

Current State

File: codeframe/lib/sdk_hooks.py lines 237-239

if tool_name in ["Write", "Bash"]:
    # TODO: Extend this to record tool-specific metrics in database
    # For now, just log the usage

Expected Behavior

• Track tool usage per agent (Write, Bash, Read, etc.)
• Store metrics in database
• Enable analysis of agent behavior patterns
• Display tool usage in dashboard

Implementation Requirements

1. Create tool_usage table in database
2. Record tool name, agent_id, timestamp on each tool use
3. Aggregate metrics for dashboard
4. Support querying by agent, tool, date range

Database Schema

CREATE TABLE IF NOT EXISTS tool_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_id TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    project_id INTEGER,
    task_id INTEGER,
    timestamp TEXT NOT NULL,
    duration_ms INTEGER,
    success BOOLEAN NOT NULL,
    FOREIGN KEY (project_id) REFERENCES projects(id),
    FOREIGN KEY (task_id) REFERENCES tasks(id)
);

Example Metrics

# Get tool usage for agent
usage = db.get_tool_usage(agent_id="backend-001")

# Result:
# {
#     "Write": 45,
#     "Read": 120,
#     "Bash": 30,
#     "Edit": 25
# }

Acceptance Criteria

• tool_usage table created
• Tool usage recorded on each tool invocation
• Metrics queryable by agent, tool, date
• Dashboard displays tool usage charts
• Unit tests verify metric recording

Priority

P3 - Advanced: Interesting analytics, but not critical for core functionality.

References

• File: codeframe/lib/sdk_hooks.py line 238

[traycerai]

Plan

Observations

The codebase has a well-established metrics tracking system for token usage with a repository pattern. The TODO at file:codeframe/lib/sdk_hooks.py lines 237-239 indicates where tool usage should be recorded. The existing token_usage table and TokenRepository provide a clear template. The database uses SQLite with a schema manager pattern, and metrics are exposed via FastAPI endpoints and displayed in React components. The SDK hooks system captures tool invocations with tool_name, tool_use_id, and context information.

This feature is marked as "Future" priority—an advanced extensibility feature that should be deferred. Core metrics tracking is already functional, and custom SDK hooks add complexity for edge cases that are not critical to current operations. The implementation should be structured as a deferred feature with clear prerequisites and integration points for future development.

Approach

This is a future-state feature that extends the existing metrics tracking pattern. When prioritized for implementation, create a tool_usage table following the schema in the issue, implement a ToolUsageRepository following the TokenRepository pattern, extend MetricsTracker to record tool usage, update the SDK hooks to call the tracker, add API endpoints for querying tool usage, and create dashboard components to visualize tool usage patterns. This approach maintains consistency with existing code patterns and integrates seamlessly with the current architecture. The implementation should be deferred until core metrics functionality is fully stabilized and there is clear demand for tool-specific analytics.

Implementation Steps (Future)

This feature should be implemented in a future sprint when the following prerequisites are met:

Prerequisites for Implementation

1. Core metrics tracking (token usage) is fully stabilized and performing well in production
2. Dashboard infrastructure is mature and can accommodate new metric visualizations
3. There is clear product demand for tool-specific analytics from users
4. Context propagation (agent_id, project_id, task_id) is reliably available in SDK hooks

Phase 1: Database & Data Layer (Foundation)

File: file:codeframe/persistence/schema_manager.py

In the _create_metrics_audit_tables method (around line 492), add the tool_usage table creation after the token_usage table:

CREATE TABLE IF NOT EXISTS tool_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_id TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    project_id INTEGER NOT NULL,
    task_id INTEGER,
    timestamp TEXT NOT NULL,
    duration_ms INTEGER,
    success BOOLEAN NOT NULL,
    tool_use_id TEXT,
    error_message TEXT,
    FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
    FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE SET NULL
)

In the _create_indexes method (around line 530), add indexes for tool_usage:

CREATE INDEX IF NOT EXISTS idx_tool_usage_agent ON tool_usage(agent_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_project ON tool_usage(project_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_task ON tool_usage(task_id)
CREATE INDEX IF NOT EXISTS idx_tool_usage_tool_name ON tool_usage(tool_name, timestamp DESC)

File: file:codeframe/core/models.py

Add a new ToolUsage dataclass following the pattern of TokenUsage:

@dataclass
class ToolUsage:
    agent_id: str
    tool_name: str
    project_id: int
    timestamp: datetime
    success: bool
    task_id: Optional[int] = None
    duration_ms: Optional[int] = None
    tool_use_id: Optional[str] = None
    error_message: Optional[str] = None
    id: Optional[int] = None

File: file:codeframe/persistence/repositories/tool_usage_repository.py (new file)

Create a new repository following the pattern of TokenRepository:

• Extend BaseRepository
• Implement save_tool_usage(tool_usage: ToolUsage) -> int method
• Implement get_tool_usage(project_id, agent_id, tool_name, start_date, end_date) -> List[Dict] method
• Implement get_tool_usage_aggregate(project_id, agent_id) -> Dict method for aggregated statistics
• Use thread-safe locking with self._sync_lock for synchronous operations

File: file:codeframe/persistence/repositories/__init__.py

Add import and export for ToolUsageRepository

File: file:codeframe/persistence/database.py

• Add tool_usage: Optional[ToolUsageRepository] = None to __init__ (around line 108)
• Initialize repository in _initialize_repositories method (around line 149)
• Add delegation methods: save_tool_usage, get_tool_usage, get_tool_usage_aggregate

Phase 2: Metrics Recording (Integration)

File: file:codeframe/lib/metrics_tracker.py

Add new methods to MetricsTracker class:

• async def record_tool_usage(agent_id, tool_name, project_id, task_id, success, duration_ms, tool_use_id, error_message) -> int - Records tool usage to database
• async def get_tool_usage_stats(project_id, agent_id, start_date, end_date) -> Dict - Returns aggregated tool usage statistics
• async def get_agent_tool_usage(agent_id, project_id) -> Dict - Returns tool usage breakdown for an agent

Follow the same pattern as token usage methods with proper error handling and logging.

File: file:codeframe/lib/sdk_hooks.py

Update the create_metrics_post_hook function (lines 186-253):

In the metrics_post_hook async function, replace the TODO section (lines 237-240) with:

if tool_name in ["Write", "Bash", "Read", "Edit", "Grep"]:
    # Extract context information
    agent_id = context.get("agent_id", "unknown") if context else "unknown"
    project_id = context.get("project_id") if context else None
    task_id = context.get("task_id") if context else None
    
    # Determine success based on tool_response
    success = "error" not in str(tool_response).lower()
    error_message = str(tool_response) if not success else None
    
    # Record tool usage
    if project_id:
        try:
            await metrics_tracker.record_tool_usage(
                agent_id=agent_id,
                tool_name=tool_name,
                project_id=project_id,
                task_id=task_id,
                success=success,
                duration_ms=None,  # SDK doesn't provide duration
                tool_use_id=tool_use_id,
                error_message=error_message
            )
        except Exception as e:
            logger.error(f"Failed to record tool usage: {e}")

Note: Ensure context object is populated with agent_id, project_id, and task_id when creating the SDK agent.

Phase 3: API & Frontend (Exposure)

File: file:codeframe/ui/routers/metrics.py

Add new endpoints following the pattern of existing metrics endpoints:

• GET /api/projects/{project_id}/metrics/tools - Get tool usage for a project with optional filters (tool_name, start_date, end_date)
• GET /api/agents/{agent_id}/metrics/tools - Get tool usage breakdown for an agent
• GET /api/projects/{project_id}/metrics/tools/summary - Get aggregated tool usage summary

File: file:web-ui/src/types/metrics.ts

Add TypeScript interfaces for ToolUsage, ToolUsageSummary, and AgentToolMetrics

File: file:web-ui/src/api/metrics.ts

Add API client functions: getProjectToolUsage, getAgentToolMetrics, getToolUsageSummary

File: file:web-ui/src/components/metrics/ToolUsageMetrics.tsx (new file)

Create a new React component to display tool usage metrics with filtering and visualization

Phase 4: Testing (Validation)

File: file:tests/persistence/test_tool_usage_repository.py (new file)

Unit tests for ToolUsageRepository operations

File: file:tests/lib/test_sdk_hooks.py

Add test cases for tool usage recording in SDK hooks

File: file:tests/api/test_api_tool_metrics.py (new file)

Integration tests for tool metrics API endpoints

File: file:tests/e2e/test_tool_metrics_ui.spec.ts (new file)

E2E tests for tool metrics dashboard functionality

Architecture Diagram (Future State)

sequenceDiagram
    participant Agent as Agent (SDK)
    participant Hook as SDK Post-Hook
    participant Tracker as MetricsTracker
    participant Repo as ToolUsageRepository
    participant DB as SQLite Database
    participant API as FastAPI Router
    participant UI as React Dashboard

    Agent->>Hook: Tool execution complete
    Hook->>Hook: Extract tool_name, context
    Hook->>Tracker: record_tool_usage()
    Tracker->>Repo: save_tool_usage()
    Repo->>DB: INSERT INTO tool_usage
    DB-->>Repo: Return ID
    Repo-->>Tracker: Return ID
    Tracker-->>Hook: Success
    
    UI->>API: GET /api/projects/{id}/metrics/tools
    API->>Tracker: get_tool_usage_stats()
    Tracker->>Repo: get_tool_usage()
    Repo->>DB: SELECT FROM tool_usage
    DB-->>Repo: Return rows
    Repo-->>Tracker: Return data
    Tracker-->>API: Return aggregated stats
    API-->>UI: JSON response
    UI->>UI: Display tool usage charts

Loading

Implementation Roadmap

Phase	Focus	Estimated Effort	Dependencies
Phase 1	Database schema, data models, repository	2-3 days	None
Phase 2	Metrics recording, SDK hook integration	2-3 days	Phase 1 complete
Phase 3	API endpoints, frontend components	3-4 days	Phase 2 complete
Phase 4	Unit, integration, and E2E tests	2-3 days	Phases 1-3 complete
Total	Full implementation	9-13 days	Prerequisites met

Key Considerations for Future Implementation

1. Context Propagation: Ensure agent_id, project_id, and task_id are reliably available in the SDK hook context. This is a critical prerequisite—verify context is properly threaded through agent initialization before starting implementation.

2. Performance & Scalability: The tool_usage table will grow quickly with high-frequency tool invocations. Plan for:

   • Efficient indexing strategy (already included in schema)
   • Periodic archival of old records (similar to session cleanup in file:codeframe/persistence/database.py line 694)
   • Potential partitioning by date if table grows beyond 10M+ rows

3. Tool Name Consistency: Establish a canonical list of tool names early. Consider creating an enum in file:codeframe/core/models.py to prevent inconsistencies.

4. Duration Tracking: The SDK hooks don't currently provide duration information. If needed, implement timing by storing start time in pre-hook and calculating duration in post-hook.

5. Error Categorization: Consider adding an error_type field to categorize different failure modes (permission denied, file not found, command failed, etc.) for better analytics.

6. Dashboard Integration: Plan where to place tool usage metrics—as a new tab, a section within existing metrics, or a separate analytics page. Coordinate with dashboard redesign efforts.

7. Data Privacy: Ensure tool usage data doesn't expose sensitive information (file paths, command contents). Consider what metadata to store and what to redact.

Deferred Scope

The following enhancements are explicitly deferred and should be considered in future iterations:

• Real-time dashboards: WebSocket-based live metrics updates
• Advanced analytics: Machine learning-based anomaly detection in tool usage patterns
• Tool-specific configurations: Custom metrics per tool type
• Comparative analysis: Cross-agent tool usage benchmarking
• Alerts: Notifications for unusual tool usage patterns

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a well-established metrics tracking system for token usage with a repository pattern. The TODO at `file:codeframe/lib/sdk_hooks.py` lines 237-239 indicates where tool usage should be recorded. The existing `token_usage` table and `TokenRepository` provide a clear template. The database uses SQLite with a schema manager pattern, and metrics are exposed via FastAPI endpoints and displayed in React components. The SDK hooks system captures tool invocations with tool_name, tool_use_id, and context information.

This feature is marked as "Future" priority—an advanced extensibility feature that should be deferred. Core metrics tracking is already functional, and custom SDK hooks add complexity for edge cases that are not critical to current operations. The implementation should be structured as a deferred feature with clear prerequisites and integration points for future development.

## Approach

This is a future-state feature that extends the existing metrics tracking pattern. When prioritized for implementation, create a `tool_usage` table following the schema in the issue, implement a `ToolUsageRepository` following the `TokenRepository` pattern, extend `MetricsTracker` to record tool usage, update the SDK hooks to call the tracker, add API endpoints for querying tool usage, and create dashboard components to visualize tool usage patterns. This approach maintains consistency with existing code patterns and integrates seamlessly with the current architecture. The implementation should be deferred until core metrics functionality is fully stabilized and there is clear demand for tool-specific analytics.

## Implementation Steps (Future)

This feature should be implemented in a future sprint when the following prerequisites are met:

### Prerequisites for Implementation
1. Core metrics tracking (token usage) is fully stabilized and performing well in production
2. Dashboard infrastructure is mature and can accommodate new metric visualizations
3. There is clear product demand for tool-specific analytics from users
4. Context propagation (agent_id, project_id, task_id) is reliably available in SDK hooks

### Phase 1: Database & Data Layer (Foundation)

**File**: `file:codeframe/persistence/schema_manager.py`

In the `_create_metrics_audit_tables` method (around line 492), add the tool_usage table creation after the token_usage table:

```sql
CREATE TABLE IF NOT EXISTS tool_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_id TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    project_id INTEGER NOT NULL,
    task_id INTEGER,
    timestamp TEXT NOT NULL,
    duration_ms INTEGER,
    success BOOLEAN NOT NULL,
    tool_use_id TEXT,
    error_message TEXT,
    FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
    FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE SET NULL
)
```

In the `_create_indexes` method (around line 530), add indexes for tool_usage:

```sql
CREATE INDEX IF NOT EXISTS idx_tool_usage_agent ON tool_usage(agent_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_project ON tool_usage(project_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_task ON tool_usage(task_id)
CREATE INDEX IF NOT EXISTS idx_tool_usage_tool_name ON tool_usage(tool_name, timestamp DESC)
```

**File**: `file:codeframe/core/models.py`

Add a new `ToolUsage` dataclass following the pattern of `TokenUsage`:

```python
@dataclass
class ToolUsage:
    agent_id: str
    tool_name: str
    project_id: int
    timestamp: datetime
    success: bool
    task_id: Optional[int] = None
    duration_ms: Optional[int] = None
    tool_use_id: Optional[str] = None
    error_message: Optional[str] = None
    id: Optional[int] = None
```

**File**: `file:codeframe/persistence/repositories/tool_usage_repository.py` (new file)

Create a new repository following the pattern of `TokenRepository`:

- Extend `BaseRepository`
- Implement `save_tool_usage(tool_usage: ToolUsage) -> int` method
- Implement `get_tool_usage(project_id, agent_id, tool_name, start_date, end_date) -> List[Dict]` method
- Implement `get_tool_usage_aggregate(project_id, agent_id) -> Dict` method for aggregated statistics
- Use thread-safe locking with `self._sync_lock` for synchronous operations

**File**: `file:codeframe/persistence/repositories/__init__.py`

Add import and export for `ToolUsageRepository`

**File**: `file:codeframe/persistence/database.py`

- Add `tool_usage: Optional[ToolUsageRepository] = None` to `__init__` (around line 108)
- Initialize repository in `_initialize_repositories` method (around line 149)
- Add delegation methods: `save_tool_usage`, `get_tool_usage`, `get_tool_usage_aggregate`

### Phase 2: Metrics Recording (Integration)

**File**: `file:codeframe/lib/metrics_tracker.py`

Add new methods to `MetricsTracker` class:

- `async def record_tool_usage(agent_id, tool_name, project_id, task_id, success, duration_ms, tool_use_id, error_message) -> int` - Records tool usage to database
- `async def get_tool_usage_stats(project_id, agent_id, start_date, end_date) -> Dict` - Returns aggregated tool usage statistics
- `async def get_agent_tool_usage(agent_id, project_id) -> Dict` - Returns tool usage breakdown for an agent

Follow the same pattern as token usage methods with proper error handling and logging.

**File**: `file:codeframe/lib/sdk_hooks.py`

Update the `create_metrics_post_hook` function (lines 186-253):

In the `metrics_post_hook` async function, replace the TODO section (lines 237-240) with:

```python
if tool_name in ["Write", "Bash", "Read", "Edit", "Grep"]:
    # Extract context information
    agent_id = context.get("agent_id", "unknown") if context else "unknown"
    project_id = context.get("project_id") if context else None
    task_id = context.get("task_id") if context else None
    
    # Determine success based on tool_response
    success = "error" not in str(tool_response).lower()
    error_message = str(tool_response) if not success else None
    
    # Record tool usage
    if project_id:
        try:
            await metrics_tracker.record_tool_usage(
                agent_id=agent_id,
                tool_name=tool_name,
                project_id=project_id,
                task_id=task_id,
                success=success,
                duration_ms=None,  # SDK doesn't provide duration
                tool_use_id=tool_use_id,
                error_message=error_message
            )
        except Exception as e:
            logger.error(f"Failed to record tool usage: {e}")
```

Note: Ensure context object is populated with agent_id, project_id, and task_id when creating the SDK agent.

### Phase 3: API & Frontend (Exposure)

**File**: `file:codeframe/ui/routers/metrics.py`

Add new endpoints following the pattern of existing metrics endpoints:

- `GET /api/projects/{project_id}/metrics/tools` - Get tool usage for a project with optional filters (tool_name, start_date, end_date)
- `GET /api/agents/{agent_id}/metrics/tools` - Get tool usage breakdown for an agent
- `GET /api/projects/{project_id}/metrics/tools/summary` - Get aggregated tool usage summary

**File**: `file:web-ui/src/types/metrics.ts`

Add TypeScript interfaces for ToolUsage, ToolUsageSummary, and AgentToolMetrics

**File**: `file:web-ui/src/api/metrics.ts`

Add API client functions: `getProjectToolUsage`, `getAgentToolMetrics`, `getToolUsageSummary`

**File**: `file:web-ui/src/components/metrics/ToolUsageMetrics.tsx` (new file)

Create a new React component to display tool usage metrics with filtering and visualization

### Phase 4: Testing (Validation)

**File**: `file:tests/persistence/test_tool_usage_repository.py` (new file)

Unit tests for ToolUsageRepository operations

**File**: `file:tests/lib/test_sdk_hooks.py`

Add test cases for tool usage recording in SDK hooks

**File**: `file:tests/api/test_api_tool_metrics.py` (new file)

Integration tests for tool metrics API endpoints

**File**: `file:tests/e2e/test_tool_metrics_ui.spec.ts` (new file)

E2E tests for tool metrics dashboard functionality

## Architecture Diagram (Future State)

```mermaid
sequenceDiagram
    participant Agent as Agent (SDK)
    participant Hook as SDK Post-Hook
    participant Tracker as MetricsTracker
    participant Repo as ToolUsageRepository
    participant DB as SQLite Database
    participant API as FastAPI Router
    participant UI as React Dashboard

    Agent->>Hook: Tool execution complete
    Hook->>Hook: Extract tool_name, context
    Hook->>Tracker: record_tool_usage()
    Tracker->>Repo: save_tool_usage()
    Repo->>DB: INSERT INTO tool_usage
    DB-->>Repo: Return ID
    Repo-->>Tracker: Return ID
    Tracker-->>Hook: Success
    
    UI->>API: GET /api/projects/{id}/metrics/tools
    API->>Tracker: get_tool_usage_stats()
    Tracker->>Repo: get_tool_usage()
    Repo->>DB: SELECT FROM tool_usage
    DB-->>Repo: Return rows
    Repo-->>Tracker: Return data
    Tracker-->>API: Return aggregated stats
    API-->>UI: JSON response
    UI->>UI: Display tool usage charts
```

## Implementation Roadmap

| Phase | Focus | Estimated Effort | Dependencies |
|-------|-------|------------------|--------------|
| Phase 1 | Database schema, data models, repository | 2-3 days | None |
| Phase 2 | Metrics recording, SDK hook integration | 2-3 days | Phase 1 complete |
| Phase 3 | API endpoints, frontend components | 3-4 days | Phase 2 complete |
| Phase 4 | Unit, integration, and E2E tests | 2-3 days | Phases 1-3 complete |
| **Total** | **Full implementation** | **9-13 days** | **Prerequisites met** |

## Key Considerations for Future Implementation

1. **Context Propagation**: Ensure agent_id, project_id, and task_id are reliably available in the SDK hook context. This is a critical prerequisite—verify context is properly threaded through agent initialization before starting implementation.

2. **Performance & Scalability**: The tool_usage table will grow quickly with high-frequency tool invocations. Plan for:
   - Efficient indexing strategy (already included in schema)
   - Periodic archival of old records (similar to session cleanup in `file:codeframe/persistence/database.py` line 694)
   - Potential partitioning by date if table grows beyond 10M+ rows

3. **Tool Name Consistency**: Establish a canonical list of tool names early. Consider creating an enum in `file:codeframe/core/models.py` to prevent inconsistencies.

4. **Duration Tracking**: The SDK hooks don't currently provide duration information. If needed, implement timing by storing start time in pre-hook and calculating duration in post-hook.

5. **Error Categorization**: Consider adding an error_type field to categorize different failure modes (permission denied, file not found, command failed, etc.) for better analytics.

6. **Dashboard Integration**: Plan where to place tool usage metrics—as a new tab, a section within existing metrics, or a separate analytics page. Coordinate with dashboard redesign efforts.

7. **Data Privacy**: Ensure tool usage data doesn't expose sensitive information (file paths, command contents). Consider what metadata to store and what to redact.

## Deferred Scope

The following enhancements are explicitly deferred and should be considered in future iterations:

- **Real-time dashboards**: WebSocket-based live metrics updates
- **Advanced analytics**: Machine learning-based anomaly detection in tool usage patterns
- **Tool-specific configurations**: Custom metrics per tool type
- **Comparative analysis**: Cross-agent tool usage benchmarking
- **Alerts**: Notifications for unusual tool usage patterns

---

Execution Information

Branch: main
Commit: 2e66ed3

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

_{You can configure auto-planning by selecting labels in the issue_enrichment configuration.}

🔗 Similar Issues

Related Issues

• [P1] Add token tracking after LLM calls in WorkerAgent #99
• [Phase 5] Implement by_day token usage aggregation in MetricsTracker #120
• [Future] Audit and Enforce Tool Definition Stability #69
• [P0][Reliability] Implement hook retry mechanism for failed event delivery claude-code-orchestrator#4
• [Phase 5] Add performance and load testing benchmarks #115

👤 Suggested Assignees

• frankbria

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!

[traycerai]

I have updated the Implementation Plan to reflect the recent changes in the ticket.

New Plan

Observations

The codebase has a well-established metrics tracking system for token usage with a repository pattern. The TODO at file:codeframe/lib/sdk_hooks.py lines 237-239 indicates where tool usage should be recorded. The existing token_usage table and TokenRepository provide a clear template. The database uses SQLite with a schema manager pattern, and metrics are exposed via FastAPI endpoints and displayed in React components. The SDK hooks system captures tool invocations with tool_name, tool_use_id, and context information.

Approach

Follow the existing metrics tracking pattern by creating a parallel tool usage tracking system. Create a tool_usage table following the schema in the issue, implement a ToolUsageRepository following the TokenRepository pattern, extend MetricsTracker to record tool usage, update the SDK hooks to call the tracker, add API endpoints for querying tool usage, and create dashboard components to visualize tool usage patterns. This approach maintains consistency with existing code patterns and integrates seamlessly with the current architecture.

Implementation Steps

1. Database Schema - Create tool_usage Table

File: file:codeframe/persistence/schema_manager.py

In the _create_metrics_audit_tables method (around line 492), add the tool_usage table creation after the token_usage table:

CREATE TABLE IF NOT EXISTS tool_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_id TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    project_id INTEGER NOT NULL,
    task_id INTEGER,
    timestamp TEXT NOT NULL,
    duration_ms INTEGER,
    success BOOLEAN NOT NULL,
    tool_use_id TEXT,
    error_message TEXT,
    FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
    FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE SET NULL
)

In the _create_indexes method (around line 530), add indexes for tool_usage:

CREATE INDEX IF NOT EXISTS idx_tool_usage_agent ON tool_usage(agent_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_project ON tool_usage(project_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_task ON tool_usage(task_id)
CREATE INDEX IF NOT EXISTS idx_tool_usage_tool_name ON tool_usage(tool_name, timestamp DESC)

2. Data Model - Create ToolUsage Model

File: file:codeframe/core/models.py

Add a new ToolUsage dataclass following the pattern of TokenUsage:

@dataclass
class ToolUsage:
    agent_id: str
    tool_name: str
    project_id: int
    timestamp: datetime
    success: bool
    task_id: Optional[int] = None
    duration_ms: Optional[int] = None
    tool_use_id: Optional[str] = None
    error_message: Optional[str] = None
    id: Optional[int] = None

3. Repository - Create ToolUsageRepository

File: file:codeframe/persistence/repositories/tool_usage_repository.py (new file)

Create a new repository following the pattern of TokenRepository:

• Extend BaseRepository
• Implement save_tool_usage(tool_usage: ToolUsage) -> int method
• Implement get_tool_usage(project_id, agent_id, tool_name, start_date, end_date) -> List[Dict] method
• Implement get_tool_usage_aggregate(project_id, agent_id) -> Dict method for aggregated statistics
• Use thread-safe locking with self._sync_lock for synchronous operations

File: file:codeframe/persistence/repositories/__init__.py

Add import and export for ToolUsageRepository

File: file:codeframe/persistence/database.py

• Add tool_usage: Optional[ToolUsageRepository] = None to __init__ (around line 108)
• Initialize repository in _initialize_repositories method (around line 149)
• Add delegation methods: save_tool_usage, get_tool_usage, get_tool_usage_aggregate

4. Metrics Tracker - Extend for Tool Usage

File: file:codeframe/lib/metrics_tracker.py

Add new methods to MetricsTracker class:

• async def record_tool_usage(agent_id, tool_name, project_id, task_id, success, duration_ms, tool_use_id, error_message) -> int - Records tool usage to database
• async def get_tool_usage_stats(project_id, agent_id, start_date, end_date) -> Dict - Returns aggregated tool usage statistics
• async def get_agent_tool_usage(agent_id, project_id) -> Dict - Returns tool usage breakdown for an agent

Follow the same pattern as token usage methods with proper error handling and logging.

5. SDK Hooks - Implement Tool Usage Recording

File: file:codeframe/lib/sdk_hooks.py

Update the create_metrics_post_hook function (lines 186-253):

In the metrics_post_hook async function, replace the TODO section (lines 237-240) with:

if tool_name in ["Write", "Bash", "Read", "Edit", "Grep"]:
    # Extract context information
    agent_id = context.get("agent_id", "unknown") if context else "unknown"
    project_id = context.get("project_id") if context else None
    task_id = context.get("task_id") if context else None
    
    # Determine success based on tool_response
    success = "error" not in str(tool_response).lower()
    error_message = str(tool_response) if not success else None
    
    # Record tool usage
    if project_id:
        try:
            await metrics_tracker.record_tool_usage(
                agent_id=agent_id,
                tool_name=tool_name,
                project_id=project_id,
                task_id=task_id,
                success=success,
                duration_ms=None,  # SDK doesn't provide duration
                tool_use_id=tool_use_id,
                error_message=error_message
            )
        except Exception as e:
            logger.error(f"Failed to record tool usage: {e}")

Note: The context object needs to be populated with agent_id, project_id, and task_id when creating the SDK agent. This may require updates to agent initialization code.

6. API Endpoints - Tool Usage Metrics

File: file:codeframe/ui/routers/metrics.py

Add new endpoints following the pattern of existing metrics endpoints:

• GET /api/projects/{project_id}/metrics/tools - Get tool usage for a project with optional filters (tool_name, start_date, end_date)
• GET /api/agents/{agent_id}/metrics/tools - Get tool usage breakdown for an agent
• GET /api/projects/{project_id}/metrics/tools/summary - Get aggregated tool usage summary

Each endpoint should:

• Validate project/agent access using get_current_user and user_has_project_access
• Use MetricsTracker to fetch data
• Return properly formatted JSON responses
• Include error handling with appropriate HTTP status codes

7. TypeScript Types - Tool Usage Types

File: file:web-ui/src/types/metrics.ts

Add new TypeScript interfaces:

export interface ToolUsage {
  id: number;
  agent_id: string;
  tool_name: string;
  project_id: number;
  task_id?: number;
  timestamp: string;
  duration_ms?: number;
  success: boolean;
  tool_use_id?: string;
  error_message?: string;
}

export interface ToolUsageSummary {
  tool_name: string;
  total_uses: number;
  success_count: number;
  failure_count: number;
  success_rate: number;
}

export interface AgentToolMetrics {
  agent_id: string;
  project_id?: number;
  total_tool_uses: number;
  by_tool: ToolUsageSummary[];
  by_success: {
    successful: number;
    failed: number;
  };
}

8. API Client - Tool Usage Functions

File: file:web-ui/src/api/metrics.ts

Add new API client functions:

export async function getProjectToolUsage(
  projectId: number,
  toolName?: string,
  startDate?: string,
  endDate?: string
): Promise<ToolUsage[]>

export async function getAgentToolMetrics(
  agentId: string,
  projectId?: number
): Promise<AgentToolMetrics>

export async function getToolUsageSummary(
  projectId: number
): Promise<ToolUsageSummary[]>

Follow the same pattern as existing metrics functions with proper error handling.

9. Dashboard Component - Tool Usage Display

File: file:web-ui/src/components/metrics/ToolUsageMetrics.tsx (new file)

Create a new React component following the pattern of AgentMetrics.tsx:

• Display tool usage summary with cards for each tool (Write, Bash, Read, Edit, Grep)
• Show success/failure rates with visual indicators
• Include a table with detailed tool usage records
• Add filtering by tool name and date range
• Implement auto-refresh functionality
• Use the same styling patterns as existing metrics components

File: file:web-ui/src/components/Dashboard.tsx

Add the ToolUsageMetrics component to the dashboard, likely in a new tab or section alongside existing metrics displays.

10. Unit Tests - Tool Usage Repository

File: file:tests/persistence/test_tool_usage_repository.py (new file)

Create comprehensive tests:

• Test save_tool_usage with valid data
• Test get_tool_usage with various filters
• Test get_tool_usage_aggregate for statistics
• Test error handling for invalid data
• Test thread-safety with concurrent operations

Follow the pattern of existing repository tests.

11. Unit Tests - SDK Hooks Tool Recording

File: file:tests/lib/test_sdk_hooks.py

Add new test cases to the existing test file:

• test_post_hook_records_tool_usage_write - Verify Write tool usage is recorded
• test_post_hook_records_tool_usage_bash - Verify Bash tool usage is recorded
• test_post_hook_records_tool_usage_read - Verify Read tool usage is recorded
• test_post_hook_records_tool_usage_with_error - Verify error tracking
• test_post_hook_records_tool_usage_with_context - Verify context extraction

Mock the metrics_tracker.record_tool_usage method and assert it's called with correct parameters.

12. Integration Tests - API Endpoints

File: file:tests/api/test_api_tool_metrics.py (new file)

Create integration tests for the new API endpoints:

• Test GET /api/projects/{project_id}/metrics/tools with various filters
• Test GET /api/agents/{agent_id}/metrics/tools
• Test authorization checks (403 for unauthorized access)
• Test error cases (404 for non-existent projects)
• Test data accuracy by creating tool usage records and verifying API responses

13. E2E Tests - Dashboard Tool Metrics

File: file:tests/e2e/test_tool_metrics_ui.spec.ts (new file)

Create Playwright tests:

• Test tool usage metrics display on dashboard
• Test filtering by tool name
• Test date range filtering
• Test auto-refresh functionality
• Test navigation between different tool metrics views

Architecture Diagram

sequenceDiagram
    participant Agent as Agent (SDK)
    participant Hook as SDK Post-Hook
    participant Tracker as MetricsTracker
    participant Repo as ToolUsageRepository
    participant DB as SQLite Database
    participant API as FastAPI Router
    participant UI as React Dashboard

    Agent->>Hook: Tool execution complete
    Hook->>Hook: Extract tool_name, context
    Hook->>Tracker: record_tool_usage()
    Tracker->>Repo: save_tool_usage()
    Repo->>DB: INSERT INTO tool_usage
    DB-->>Repo: Return ID
    Repo-->>Tracker: Return ID
    Tracker-->>Hook: Success
    
    UI->>API: GET /api/projects/{id}/metrics/tools
    API->>Tracker: get_tool_usage_stats()
    Tracker->>Repo: get_tool_usage()
    Repo->>DB: SELECT FROM tool_usage
    DB-->>Repo: Return rows
    Repo-->>Tracker: Return data
    Tracker-->>API: Return aggregated stats
    API-->>UI: JSON response
    UI->>UI: Display tool usage charts

Loading

Data Flow Summary

Component	Responsibility
SDK Hook	Capture tool invocations and extract metadata
MetricsTracker	Business logic for recording and aggregating tool usage
ToolUsageRepository	Database operations for tool_usage table
Database	Persist tool usage records with indexes for efficient queries
API Router	Expose tool usage data via REST endpoints
React Components	Visualize tool usage patterns and statistics

Key Considerations

1. Context Propagation: Ensure agent_id, project_id, and task_id are available in the SDK hook context. This may require updates to agent initialization code in file:codeframe/agents/ to pass context when creating SDK agents.

2. Performance: The tool_usage table will grow quickly. Consider adding a cleanup job to archive old records (similar to session cleanup in file:codeframe/persistence/database.py line 694).

3. Tool Name Consistency: Ensure tool names are consistent across the codebase. Consider creating an enum for tool names in file:codeframe/core/models.py.

4. Duration Tracking: The SDK hooks don't provide duration information. If needed, implement timing in the hook by storing start time in pre-hook and calculating duration in post-hook.

5. Error Categorization: Consider adding an error_type field to categorize different types of failures (permission denied, file not found, command failed, etc.).

6. Dashboard Integration: Decide where to place the tool usage metrics in the dashboard - as a new tab, a section within existing metrics, or a separate page.

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a well-established metrics tracking system for token usage with a repository pattern. The TODO at `file:codeframe/lib/sdk_hooks.py` lines 237-239 indicates where tool usage should be recorded. The existing `token_usage` table and `TokenRepository` provide a clear template. The database uses SQLite with a schema manager pattern, and metrics are exposed via FastAPI endpoints and displayed in React components. The SDK hooks system captures tool invocations with tool_name, tool_use_id, and context information.

## Approach

Follow the existing metrics tracking pattern by creating a parallel tool usage tracking system. Create a `tool_usage` table following the schema in the issue, implement a `ToolUsageRepository` following the `TokenRepository` pattern, extend `MetricsTracker` to record tool usage, update the SDK hooks to call the tracker, add API endpoints for querying tool usage, and create dashboard components to visualize tool usage patterns. This approach maintains consistency with existing code patterns and integrates seamlessly with the current architecture.

## Implementation Steps

### 1. Database Schema - Create `tool_usage` Table

**File**: `file:codeframe/persistence/schema_manager.py`

In the `_create_metrics_audit_tables` method (around line 492), add the tool_usage table creation after the token_usage table:

```sql
CREATE TABLE IF NOT EXISTS tool_usage (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    agent_id TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    project_id INTEGER NOT NULL,
    task_id INTEGER,
    timestamp TEXT NOT NULL,
    duration_ms INTEGER,
    success BOOLEAN NOT NULL,
    tool_use_id TEXT,
    error_message TEXT,
    FOREIGN KEY (project_id) REFERENCES projects(id) ON DELETE CASCADE,
    FOREIGN KEY (task_id) REFERENCES tasks(id) ON DELETE SET NULL
)
```

In the `_create_indexes` method (around line 530), add indexes for tool_usage:

```sql
CREATE INDEX IF NOT EXISTS idx_tool_usage_agent ON tool_usage(agent_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_project ON tool_usage(project_id, timestamp DESC)
CREATE INDEX IF NOT EXISTS idx_tool_usage_task ON tool_usage(task_id)
CREATE INDEX IF NOT EXISTS idx_tool_usage_tool_name ON tool_usage(tool_name, timestamp DESC)
```

### 2. Data Model - Create `ToolUsage` Model

**File**: `file:codeframe/core/models.py`

Add a new `ToolUsage` dataclass following the pattern of `TokenUsage`:

```python
@dataclass
class ToolUsage:
    agent_id: str
    tool_name: str
    project_id: int
    timestamp: datetime
    success: bool
    task_id: Optional[int] = None
    duration_ms: Optional[int] = None
    tool_use_id: Optional[str] = None
    error_message: Optional[str] = None
    id: Optional[int] = None
```

### 3. Repository - Create `ToolUsageRepository`

**File**: `file:codeframe/persistence/repositories/tool_usage_repository.py` (new file)

Create a new repository following the pattern of `TokenRepository`:

- Extend `BaseRepository`
- Implement `save_tool_usage(tool_usage: ToolUsage) -> int` method
- Implement `get_tool_usage(project_id, agent_id, tool_name, start_date, end_date) -> List[Dict]` method
- Implement `get_tool_usage_aggregate(project_id, agent_id) -> Dict` method for aggregated statistics
- Use thread-safe locking with `self._sync_lock` for synchronous operations

**File**: `file:codeframe/persistence/repositories/__init__.py`

Add import and export for `ToolUsageRepository`

**File**: `file:codeframe/persistence/database.py`

- Add `tool_usage: Optional[ToolUsageRepository] = None` to `__init__` (around line 108)
- Initialize repository in `_initialize_repositories` method (around line 149)
- Add delegation methods: `save_tool_usage`, `get_tool_usage`, `get_tool_usage_aggregate`

### 4. Metrics Tracker - Extend for Tool Usage

**File**: `file:codeframe/lib/metrics_tracker.py`

Add new methods to `MetricsTracker` class:

- `async def record_tool_usage(agent_id, tool_name, project_id, task_id, success, duration_ms, tool_use_id, error_message) -> int` - Records tool usage to database
- `async def get_tool_usage_stats(project_id, agent_id, start_date, end_date) -> Dict` - Returns aggregated tool usage statistics
- `async def get_agent_tool_usage(agent_id, project_id) -> Dict` - Returns tool usage breakdown for an agent

Follow the same pattern as token usage methods with proper error handling and logging.

### 5. SDK Hooks - Implement Tool Usage Recording

**File**: `file:codeframe/lib/sdk_hooks.py`

Update the `create_metrics_post_hook` function (lines 186-253):

In the `metrics_post_hook` async function, replace the TODO section (lines 237-240) with:

```python
if tool_name in ["Write", "Bash", "Read", "Edit", "Grep"]:
    # Extract context information
    agent_id = context.get("agent_id", "unknown") if context else "unknown"
    project_id = context.get("project_id") if context else None
    task_id = context.get("task_id") if context else None
    
    # Determine success based on tool_response
    success = "error" not in str(tool_response).lower()
    error_message = str(tool_response) if not success else None
    
    # Record tool usage
    if project_id:
        try:
            await metrics_tracker.record_tool_usage(
                agent_id=agent_id,
                tool_name=tool_name,
                project_id=project_id,
                task_id=task_id,
                success=success,
                duration_ms=None,  # SDK doesn't provide duration
                tool_use_id=tool_use_id,
                error_message=error_message
            )
        except Exception as e:
            logger.error(f"Failed to record tool usage: {e}")
```

Note: The context object needs to be populated with agent_id, project_id, and task_id when creating the SDK agent. This may require updates to agent initialization code.

### 6. API Endpoints - Tool Usage Metrics

**File**: `file:codeframe/ui/routers/metrics.py`

Add new endpoints following the pattern of existing metrics endpoints:

- `GET /api/projects/{project_id}/metrics/tools` - Get tool usage for a project with optional filters (tool_name, start_date, end_date)
- `GET /api/agents/{agent_id}/metrics/tools` - Get tool usage breakdown for an agent
- `GET /api/projects/{project_id}/metrics/tools/summary` - Get aggregated tool usage summary

Each endpoint should:
- Validate project/agent access using `get_current_user` and `user_has_project_access`
- Use `MetricsTracker` to fetch data
- Return properly formatted JSON responses
- Include error handling with appropriate HTTP status codes

### 7. TypeScript Types - Tool Usage Types

**File**: `file:web-ui/src/types/metrics.ts`

Add new TypeScript interfaces:

```typescript
export interface ToolUsage {
  id: number;
  agent_id: string;
  tool_name: string;
  project_id: number;
  task_id?: number;
  timestamp: string;
  duration_ms?: number;
  success: boolean;
  tool_use_id?: string;
  error_message?: string;
}

export interface ToolUsageSummary {
  tool_name: string;
  total_uses: number;
  success_count: number;
  failure_count: number;
  success_rate: number;
}

export interface AgentToolMetrics {
  agent_id: string;
  project_id?: number;
  total_tool_uses: number;
  by_tool: ToolUsageSummary[];
  by_success: {
    successful: number;
    failed: number;
  };
}
```

### 8. API Client - Tool Usage Functions

**File**: `file:web-ui/src/api/metrics.ts`

Add new API client functions:

```typescript
export async function getProjectToolUsage(
  projectId: number,
  toolName?: string,
  startDate?: string,
  endDate?: string
): Promise<ToolUsage[]>

export async function getAgentToolMetrics(
  agentId: string,
  projectId?: number
): Promise<AgentToolMetrics>

export async function getToolUsageSummary(
  projectId: number
): Promise<ToolUsageSummary[]>
```

Follow the same pattern as existing metrics functions with proper error handling.

### 9. Dashboard Component - Tool Usage Display

**File**: `file:web-ui/src/components/metrics/ToolUsageMetrics.tsx` (new file)

Create a new React component following the pattern of `AgentMetrics.tsx`:

- Display tool usage summary with cards for each tool (Write, Bash, Read, Edit, Grep)
- Show success/failure rates with visual indicators
- Include a table with detailed tool usage records
- Add filtering by tool name and date range
- Implement auto-refresh functionality
- Use the same styling patterns as existing metrics components

**File**: `file:web-ui/src/components/Dashboard.tsx`

Add the `ToolUsageMetrics` component to the dashboard, likely in a new tab or section alongside existing metrics displays.

### 10. Unit Tests - Tool Usage Repository

**File**: `file:tests/persistence/test_tool_usage_repository.py` (new file)

Create comprehensive tests:

- Test `save_tool_usage` with valid data
- Test `get_tool_usage` with various filters
- Test `get_tool_usage_aggregate` for statistics
- Test error handling for invalid data
- Test thread-safety with concurrent operations

Follow the pattern of existing repository tests.

### 11. Unit Tests - SDK Hooks Tool Recording

**File**: `file:tests/lib/test_sdk_hooks.py`

Add new test cases to the existing test file:

- `test_post_hook_records_tool_usage_write` - Verify Write tool usage is recorded
- `test_post_hook_records_tool_usage_bash` - Verify Bash tool usage is recorded
- `test_post_hook_records_tool_usage_read` - Verify Read tool usage is recorded
- `test_post_hook_records_tool_usage_with_error` - Verify error tracking
- `test_post_hook_records_tool_usage_with_context` - Verify context extraction

Mock the `metrics_tracker.record_tool_usage` method and assert it's called with correct parameters.

### 12. Integration Tests - API Endpoints

**File**: `file:tests/api/test_api_tool_metrics.py` (new file)

Create integration tests for the new API endpoints:

- Test GET `/api/projects/{project_id}/metrics/tools` with various filters
- Test GET `/api/agents/{agent_id}/metrics/tools` 
- Test authorization checks (403 for unauthorized access)
- Test error cases (404 for non-existent projects)
- Test data accuracy by creating tool usage records and verifying API responses

### 13. E2E Tests - Dashboard Tool Metrics

**File**: `file:tests/e2e/test_tool_metrics_ui.spec.ts` (new file)

Create Playwright tests:

- Test tool usage metrics display on dashboard
- Test filtering by tool name
- Test date range filtering
- Test auto-refresh functionality
- Test navigation between different tool metrics views

## Architecture Diagram

```mermaid
sequenceDiagram
    participant Agent as Agent (SDK)
    participant Hook as SDK Post-Hook
    participant Tracker as MetricsTracker
    participant Repo as ToolUsageRepository
    participant DB as SQLite Database
    participant API as FastAPI Router
    participant UI as React Dashboard

    Agent->>Hook: Tool execution complete
    Hook->>Hook: Extract tool_name, context
    Hook->>Tracker: record_tool_usage()
    Tracker->>Repo: save_tool_usage()
    Repo->>DB: INSERT INTO tool_usage
    DB-->>Repo: Return ID
    Repo-->>Tracker: Return ID
    Tracker-->>Hook: Success
    
    UI->>API: GET /api/projects/{id}/metrics/tools
    API->>Tracker: get_tool_usage_stats()
    Tracker->>Repo: get_tool_usage()
    Repo->>DB: SELECT FROM tool_usage
    DB-->>Repo: Return rows
    Repo-->>Tracker: Return data
    Tracker-->>API: Return aggregated stats
    API-->>UI: JSON response
    UI->>UI: Display tool usage charts
```

## Data Flow Summary

| Component | Responsibility |
|-----------|---------------|
| SDK Hook | Capture tool invocations and extract metadata |
| MetricsTracker | Business logic for recording and aggregating tool usage |
| ToolUsageRepository | Database operations for tool_usage table |
| Database | Persist tool usage records with indexes for efficient queries |
| API Router | Expose tool usage data via REST endpoints |
| React Components | Visualize tool usage patterns and statistics |

## Key Considerations

1. **Context Propagation**: Ensure agent_id, project_id, and task_id are available in the SDK hook context. This may require updates to agent initialization code in `file:codeframe/agents/` to pass context when creating SDK agents.

2. **Performance**: The tool_usage table will grow quickly. Consider adding a cleanup job to archive old records (similar to session cleanup in `file:codeframe/persistence/database.py` line 694).

3. **Tool Name Consistency**: Ensure tool names are consistent across the codebase. Consider creating an enum for tool names in `file:codeframe/core/models.py`.

4. **Duration Tracking**: The SDK hooks don't provide duration information. If needed, implement timing in the hook by storing start time in pre-hook and calculating duration in post-hook.

5. **Error Categorization**: Consider adding an error_type field to categorize different types of failures (permission denied, file not found, command failed, etc.).

6. **Dashboard Integration**: Decide where to place the tool usage metrics in the dashboard - as a new tab, a section within existing metrics, or a separate page.

---

Execution Information

Branch: main
Commit: 52e78e0

[frankbria]

Re-labeled from P3 to Future. Custom SDK hooks are an advanced extensibility feature. Core metrics tracking is already functional - custom hooks add complexity for edge cases.