-
Notifications
You must be signed in to change notification settings - Fork 27
Description
Use Case
The Claude Code SDK currently filters out important metadata from the CLI response, limiting observability and analytics capabilities. Applications need access to:
- Performance Metrics:
duration_ms,duration_api_ms,num_turns - Model Usage Breakdown: Per-model token counts and costs via
modelUsage - Permission Tracking:
permission_denialsfor permission auditing - System Capabilities: Full system init data (tools, MCP servers, agents, skills, slash commands)
- Request Tracking:
uuidfor request correlation - Cache Analytics: Ephemeral cache tier breakdowns (5m vs 1h)
- Web Search Usage: Track web search requests per query and per model
Current Limitation & Architectural Bug
🔴 Critical Issue: Type-Implementation Mismatch + Missing Fields
After analyzing the actual CLI output with claude --verbose --output-format stream-json, we discovered THREE categories of problems:
- Fields defined in types but never populated (Type Safety Bug)
- Fields provided by CLI but not defined in types (Type Completeness Bug)
- System init data completely discarded (Data Loss Bug)
The Bug Revealed
Problem 1: Types Promise Fields That Are Never Populated
The ResultMessage interface includes fields in src/types.ts (lines 89-120), but parseMessage() in src/_internal/client.ts (lines 61-74) never populates them!
This creates a TypeScript footgun where users import the SDK, see these fields in the interface, but get undefined at runtime.
Evidence Table: Defined vs. Populated
| Field | Defined in ResultMessage |
CLI Provides | SDK Extracts | Status |
|---|---|---|---|---|
duration_ms |
✅ Yes (line 95) | ✅ Yes (2125ms) | ❌ No | 🔴 Type lies |
duration_api_ms |
✅ Yes (line 96) | ✅ Yes (2108ms) | ❌ No | 🔴 Type lies |
num_turns |
✅ Yes (line 97) | ✅ Yes (1) | ❌ No | 🔴 Type lies |
modelUsage |
✅ Yes (line 113) | ✅ Yes (full object) | ❌ No | 🔴 Type lies |
permission_denials |
✅ Yes (line 115) | ✅ Yes ([]) | ❌ No | 🔴 Type lies |
uuid |
✅ Yes (line 117) | ✅ Yes | ❌ No | 🔴 Type lies |
total_cost_usd |
✅ Yes (line 119) | ✅ Yes (0.029) | ❌ No | 🔴 Type lies |
usage (tokens) |
✅ Yes (lines 99-104) | ✅ Yes | ✅ Yes | ✅ Works |
cost (breakdown) |
✅ Yes (lines 105-111) | ✅ Yes | total_cost |
|
session_id |
✅ Yes (line 93) | ✅ Yes | ✅ Yes | ✅ Works |
subtype |
✅ Yes (line 91) | ✅ Yes | ✅ Yes | ✅ Works |
content |
✅ Yes (line 92) | ✅ Yes | ✅ Yes | ✅ Works |
Problem 2: CLI Provides Fields Not Defined in Types
The CLI outputs additional fields that aren't even defined in the TypeScript types!
Actual CLI Result Message (from claude --verbose --output-format stream-json):
New Fields Missing from Types:
| Field | CLI Provides | In TypeScript Types | Impact |
|---|---|---|---|
is_error |
✅ Yes | ❌ No | Can't distinguish error/success results |
result |
✅ Yes | ❌ No | Simplified result text access |
usage.server_tool_use |
✅ Yes | ❌ No | Web search tracking unavailable |
usage.cache_creation (breakdown) |
✅ Yes | ❌ No | Can't distinguish 5m vs 1h cache tiers |
modelUsage[].webSearchRequests |
✅ Yes | ❌ No | Per-model web search tracking lost |
modelUsage[].contextWindow |
✅ Yes | ❌ No | Model context window size unknown |
Problem 3: System Init Data Completely Discarded
The CLI emits a system init message with ALL capabilities, but SDK returns null (lines 57-59 in client.ts)
Actual CLI System Init Message:
{
"type": "system",
"subtype": "init",
"cwd": "/home/stevenp/projects/sandbox/claude-code-sdk-ts",
"session_id": "fdcba3aa-...",
"tools": ["Task", "Bash", "Glob", "Grep", "Read", "Edit", "Write", ...],
"mcp_servers": [
{"name": "plugin:superpowers-chrome:chrome", "status": "connected"},
{"name": "context7", "status": "connected"}
],
"model": "claude-haiku-4-5-20251001",
"permissionMode": "default",
"slash_commands": ["hotfix-mfe", "jira-issue-context", "ship-changes", ...],
"apiKeySource": "none",
"claude_code_version": "2.0.26",
"output_style": "default",
"agents": ["general-purpose", "statusline-setup", ...],
"skills": ["superpowers-chrome:browsing"],
"uuid": "63e82cd1-..."
}System Init Fields (ALL LOST):
| Field | CLI Provides | SDK Returns | Impact |
|---|---|---|---|
cwd |
✅ Yes | ❌ null | Current working directory unknown |
tools |
✅ Yes (19 tools) | ❌ null | Available tools list lost |
mcp_servers |
✅ Yes (with status) | ❌ null | MCP server status unknown |
model |
✅ Yes | ❌ null | Active model name unknown |
permissionMode |
✅ Yes | ❌ null | Permission mode unknown |
slash_commands |
✅ Yes (23 commands) | ❌ null | Available slash commands lost |
apiKeySource |
✅ Yes | ❌ null | API key source unknown |
claude_code_version |
✅ Yes | ❌ null | CLI version unknown |
output_style |
✅ Yes | ❌ null | Output style unknown |
agents |
✅ Yes (19 agents) | ❌ null | Available agents list lost |
skills |
✅ Yes | ❌ null | Available skills lost |
uuid |
✅ Yes | ❌ null | Session UUID lost |
Code Evidence
Current Implementation (src/_internal/client.ts:57-74):
case 'system':
// System messages (like init) - skip these
return null; // ❌ ALL capabilities data lost!
case 'result': {
const resultMsg = output as any;
return {
type: 'result',
subtype: resultMsg.subtype,
content: resultMsg.content || '',
session_id: resultMsg.session_id,
usage: resultMsg.usage, // ✅ Extracted
cost: {
total_cost: resultMsg.cost?.total_cost_usd // ⚠️ Only total_cost
}
} as Message;
// ❌ Missing: duration_ms, duration_api_ms, num_turns, modelUsage,
// permission_denials, uuid, total_cost_usd, is_error, result
// usage.server_tool_use, usage.cache_creation breakdown
}Why This is a Bug
This is worse than "missing a feature" - it's THREE architectural bugs:
- TypeScript lies to users: Type system promises fields that will never be populated
- Silent data loss: CLI provides rich data, but SDK throws it away
- No runtime warning: Users discover missing data only at runtime, not compile time
- Incomplete type definitions: CLI provides more fields than types define
- Capability blindness: Applications can't discover what tools/servers/agents are available
Missing CLI Options
The SDK also doesn't expose several CLI flags from claude --help:
| CLI Option | SDK Support | Use Case |
|---|---|---|
--fallback-model |
❌ Missing | Automatic fallback when model overloaded |
--settings <file-or-json> |
❌ Missing | Load settings from file/JSON |
--strict-mcp-config |
❌ Missing | Restrict to only specified MCP servers |
--agents <json> |
❌ Missing | Define custom agent configurations |
--setting-sources |
❌ Missing | Control which setting sources to load |
--plugin-dir |
❌ Missing | Load plugins from custom directories |
--fork-session |
❌ Missing | Fork sessions for experimentation |
--include-partial-messages |
❌ Missing | Stream partial message chunks |
--replay-user-messages |
❌ Missing | Echo user messages for acknowledgment |
Proposed Solution
Fix all three problems while maintaining backwards compatibility:
1. Fix parseMessage() - Extract ALL Fields (src/_internal/client.ts)
Lines 61-74 should extract everything the CLI provides:
case 'result': {
const resultMsg = output as any;
return {
type: 'result',
subtype: resultMsg.subtype,
content: resultMsg.content || '',
session_id: resultMsg.session_id,
// Performance metrics (FIX: extract from CLI)
duration_ms: resultMsg.duration_ms,
duration_api_ms: resultMsg.duration_api_ms,
num_turns: resultMsg.num_turns,
// NEW: Fields not currently in types
is_error: resultMsg.is_error,
result: resultMsg.result,
// Token usage (existing, but enhance)
usage: {
...resultMsg.usage,
// NEW: Add missing nested fields
server_tool_use: resultMsg.usage?.server_tool_use,
cache_creation: resultMsg.usage?.cache_creation
},
// Cost breakdown (FIX: extract full object, not just total)
cost: resultMsg.cost,
// Model usage breakdown (FIX: extract from CLI)
modelUsage: resultMsg.modelUsage,
// Permission tracking (FIX: extract from CLI)
permission_denials: resultMsg.permission_denials,
// Request tracking (FIX: extract from CLI)
uuid: resultMsg.uuid,
total_cost_usd: resultMsg.total_cost_usd
} as Message;
}Lines 57-59 should return system init data:
case 'system': {
// System messages (like init) - return full capabilities
const systemMsg = output as any;
return {
type: 'system',
subtype: systemMsg.subtype,
data: systemMsg.data,
session_id: systemMsg.session_id,
// System capabilities (NEW)
cwd: systemMsg.cwd,
model: systemMsg.model,
claude_code_version: systemMsg.claude_code_version,
permissionMode: systemMsg.permissionMode,
apiKeySource: systemMsg.apiKeySource,
output_style: systemMsg.output_style,
uuid: systemMsg.uuid,
tools: systemMsg.tools,
mcp_servers: systemMsg.mcp_servers,
slash_commands: systemMsg.slash_commands,
agents: systemMsg.agents,
skills: systemMsg.skills
} as SystemMessage;
}2. Extend TypeScript Types (src/types.ts)
Add missing fields to ResultMessage:
export interface ResultMessage {
type: 'result';
subtype?: string;
content: string;
session_id?: string;
// Performance metrics (already defined)
duration_ms?: number;
duration_api_ms?: number;
num_turns?: number;
// NEW: Error status
is_error?: boolean;
result?: string;
// Token and cost usage (enhanced)
usage?: {
input_tokens?: number;
output_tokens?: number;
cache_creation_input_tokens?: number;
cache_read_input_tokens?: number;
service_tier?: string;
// NEW: Server tool usage
server_tool_use?: {
web_search_requests?: number;
};
// NEW: Cache tier breakdown
cache_creation?: {
ephemeral_5m_input_tokens?: number;
ephemeral_1h_input_tokens?: number;
};
};
cost?: {
input_cost?: number;
output_cost?: number;
cache_creation_cost?: number;
cache_read_cost?: number;
total_cost?: number;
};
// Per-model usage (already defined, but enhance ModelUsageInfo)
modelUsage?: Record<string, ModelUsageInfo>;
// Permission tracking (already defined)
permission_denials?: string[];
// Request tracking (already defined)
uuid?: string;
total_cost_usd?: number;
}Enhance ModelUsageInfo type:
export interface ModelUsageInfo {
inputTokens: number;
outputTokens: number;
cacheReadInputTokens: number;
cacheCreationInputTokens: number;
costUSD: number;
// NEW: Additional fields from CLI
webSearchRequests?: number;
contextWindow?: number;
}3. Add Convenience Methods to ResponseParser
// Get performance metrics
async getPerformanceMetrics(): Promise<{
durationMs?: number;
apiDurationMs?: number;
numTurns?: number;
} | null>
// Get permission denials
async getPermissionDenials(): Promise<string[] | null>
// Get model usage breakdown (with web search counts)
async getModelUsage(): Promise<Record<string, ModelUsageInfo> | null>
// Get system capabilities
async getSystemCapabilities(): Promise<{
cwd?: string;
model?: string;
claudeCodeVersion?: string;
permissionMode?: string;
apiKeySource?: string;
outputStyle?: string;
tools?: string[];
mcpServers?: Array<{ name: string; status: string }>;
slashCommands?: string[];
agents?: string[];
skills?: string[];
uuid?: string;
} | null>
// Get web search usage
async getWebSearchUsage(): Promise<{
total: number;
byModel: Record<string, number>;
} | null>
// Get cache breakdown
async getCacheBreakdown(): Promise<{
ephemeral5m: number;
ephemeral1h: number;
} | null>4. Add Missing CLI Options to ClaudeCodeOptions
export interface ClaudeCodeOptions {
// ... existing options ...
// NEW: Missing CLI options
fallbackModel?: string; // --fallback-model
settings?: string | object; // --settings
strictMcpConfig?: boolean; // --strict-mcp-config
agents?: Record<string, AgentConfig>; // --agents
settingSources?: string[]; // --setting-sources
pluginDir?: string[]; // --plugin-dir
forkSession?: boolean; // --fork-session
includePartialMessages?: boolean; // --include-partial-messages
replayUserMessages?: boolean; // --replay-user-messages
}Implementation Strategy
- ✅ Backwards Compatible: All changes are additive
- ✅ Type-Safe: Types now match runtime behavior
- ✅ Opt-In: New fields/methods available but not required
- ✅ Bug Fix: Aligns types with CLI output
Real-World Examples
const parser = claude().query('Analyze this codebase');
const result = await parser.asText();
// ✅ Performance tracking
const perf = await parser.getPerformanceMetrics();
console.log(`API took ${perf.apiDurationMs}ms, ${perf.numTurns} turns`);
// ✅ Permission auditing
const denied = await parser.getPermissionDenials();
if (denied?.length) {
console.log(`Denied tools: ${denied.join(', ')}`);
}
// ✅ Cost analytics per model (with web search costs)
const modelUsage = await parser.getModelUsage();
for (const [model, stats] of Object.entries(modelUsage)) {
console.log(`${model}:`);
console.log(` Tokens: ${stats.inputTokens + stats.outputTokens}`);
console.log(` Web searches: ${stats.webSearchRequests}`);
console.log(` Context window: ${stats.contextWindow}`);
console.log(` Cost: $${stats.costUSD}`);
}
// ✅ System capabilities
const caps = await parser.getSystemCapabilities();
console.log(`Running CLI v${caps.claudeCodeVersion}`);
console.log(`Model: ${caps.model}`);
console.log(`Available tools (${caps.tools?.length}): ${caps.tools?.join(', ')}`);
console.log(`MCP servers: ${caps.mcpServers?.map(s => `${s.name} (${s.status})`).join(', ')}`);
console.log(`Agents: ${caps.agents?.join(', ')}`);
// ✅ Web search usage
const webSearch = await parser.getWebSearchUsage();
console.log(`Total web searches: ${webSearch.total}`);
// ✅ Cache tier breakdown
const cache = await parser.getCacheBreakdown();
console.log(`Cache: ${cache.ephemeral5m} tokens (5m), ${cache.ephemeral1h} tokens (1h)`);Summary: Complete Gap Analysis
Critical Bugs to Fix:
| Category | Count | Description |
|---|---|---|
| 🔴 Fields in types but not extracted | 7 | duration_ms, duration_api_ms, num_turns, modelUsage, permission_denials, uuid, total_cost_usd |
| 🔴 Fields in CLI but not in types | 7 | is_error, result, usage.server_tool_use, usage.cache_creation, ModelUsageInfo.webSearchRequests, ModelUsageInfo.contextWindow |
| 🔴 System init fields discarded | 12 | cwd, tools, mcp_servers, model, permissionMode, slash_commands, apiKeySource, claude_code_version, output_style, agents, skills, uuid |
| 🟡 CLI options not in SDK | 9 | fallback-model, settings, strict-mcp-config, agents, setting-sources, plugin-dir, fork-session, include-partial-messages, replay-user-messages |
Total: 35 missing features/fixes
Request
Would you be open to a PR that:
- ✅ Fixes the type-implementation bugs by extracting all fields the CLI provides
- ✅ Extends types to include fields CLI outputs but types don't define
- ✅ Returns system init data instead of discarding it
- ✅ Adds convenience methods to
ResponseParserfor easier metadata access - ✅ Adds missing CLI options to
ClaudeCodeOptions - ✅ Maintains 100% backwards compatibility (all changes are additive)
- ✅ Includes comprehensive tests and documentation
This would solve the type safety issues AND unlock powerful observability, analytics, and capability discovery features that the CLI already provides.
{ "type": "result", "subtype": "success", "is_error": false, // ❌ NOT in ResultMessage type "duration_ms": 2125, // ✅ In type, not extracted "duration_api_ms": 2108, // ✅ In type, not extracted "num_turns": 1, // ✅ In type, not extracted "result": "Hello, how are you?", // ❌ NOT in ResultMessage type "session_id": "fdcba3aa-...", "total_cost_usd": 0.02948675, // ✅ In type, not extracted "usage": { "input_tokens": 3, "cache_creation_input_tokens": 23551, "cache_read_input_tokens": 0, "output_tokens": 9, "server_tool_use": { // ❌ NOT in usage type "web_search_requests": 0 }, "service_tier": "standard", "cache_creation": { // ❌ NOT in usage type "ephemeral_1h_input_tokens": 0, "ephemeral_5m_input_tokens": 23551 } }, "modelUsage": { // ✅ In type, not extracted "claude-haiku-4-5-20251001": { "inputTokens": 3, "outputTokens": 9, "cacheReadInputTokens": 0, "cacheCreationInputTokens": 23551, "webSearchRequests": 0, // ❌ NOT in ModelUsageInfo type "costUSD": 0.02948675, "contextWindow": 200000 // ❌ NOT in ModelUsageInfo type } }, "permission_denials": [], // ✅ In type, not extracted "uuid": "4b734e1d-..." // ✅ In type, not extracted }