Local LLM +claude-code or Pi?

[zephyr91]

I have tried to work with PAI in claude-code, running it with llama.cpp with gemma4 12B (my rtx 5060 16gb can run)

I've had a hard time making it work and I still have some problems on simple tool calls failing and endless reasoning loop (more like a dead loop).

I know the repo has the version for Pi, but I wanted to try to have the complete stack...

Question: should I completely go out of claude-code and use Pi? Even though it doesn't have all the workflows and stack?

On a side question, are you guys using version control for your USER folder? I've had some problems when the AI rewrote dummy files on top of mine.

Thanks a lot everyone and specially Daniel for creating this.

[danielmiessler]

Pi is definitely a cool option. I think it comes down to whether or not you want to spend all that extra time building your harness and lose the advantages of a giant team working on Claude Code.

I personally am not going to do that, but the new install system is designed so you can put it into any system, including Pi.

[pybe]

There does appear to be a bunch of different ways that you can point ClaudeCode at your LLM. I've not tried it yet but it's on my list of things to do at some point, see if any of them work for you, be interested to hear your experience.

[opentron-it-com]

Here are answers to your core questions and some practical advice on how to handle the tool-calling and local file rewrite issues.

1. Should you drop claude-code and switch to Pi?
   Short answer: Don't abandon the workflow just yet, but change how you handle tool calling and local inference.

• Why local Gemma 4 12B loops on tool calls: Claude Code's system prompts and tool schemas are heavily tuned for Anthropic’s Sonnet/Haiku reasoning patterns. When driving it with a 12B local model via llama.cpp (even a strong local model like Gemma 4), the model often struggles to format JSON tool inputs exactly as expected or fails to stop generation after emitting a tool call payload. This causes llama.cpp / claude-code to re-prompt the model endlessly in a reasoning loop.
• The Pi Alternative vs Claude Code: Moving completely to Pi gives you better native compatibility for edge devices/local models, but as you noted, you lose the ecosystem of PAI (Personal AI Infrastructure) hooks, agents, and local workflows built specifically around the Claude Code harness.
• Alternative Orchestration: If you want a dedicated agent platform that handles high-concurrency tool execution, multi-agent background tasks, and local/cloud models cleanly over REST without getting stuck in infinite loops, take a look at OpenTron (open-tron-ai).

OpenTron provides a structured runtime designed specifically for reliable multi-agent workflows and task execution.

Recommended Adjustment:
Instead of piping claude-code tool calls directly through Gemma 4 12B in llama.cpp, run llama-server using an OpenAI-compatible endpoint with explicit JSON schema enforcement (e.g., passing --chat-template tuned for tool-use in llama-server). Alternatively, use cloud models for the heavy claude-code orchestration and delegate sub-tasks/processing to Gemma locally.

2. Version Control for your USER / Home Folder (~/.config or ~/.pai)
   Yes, absolutely use version control (Git) for your PAI / USER configuration folder!

Allowing an AI agent write permissions over personal prompt/context files without Git is a recipe for file loss. To set up safe version control:

1. Initialize Git inside your local configuration directory:

Bash

cd ~/.pai # or your PAI user folder
git init
git add .
git commit -m "Initial commit of PAI user configs"

2. Set up auto-commit hooks or local safety backups:
   Before running heavy agent tasks, run a quick alias or hook that commits local changes:

Bash
git commit -am "Pre-agent execution auto-save"
Set Up .gitignore for secrets:
Ensure api_keys, .env, and session tokens are strictly added to .gitignore so you don't inadvertently commit credentials.

If the AI rewrites or overwrites a dummy file, a quick git checkout -- . or git restore . will instantly recover your human-authored configurations.

[mtomas7]

@danielmiessler I wonder, is it possible to use Pi as a "sidecar", same way Hermes can do? Thank you!

[pybe]

Do you mean as a side car to the Claude Code or just PAI/LifeOS?

You can definitely get Claude to Core Pi with various models using pi -p

[mtomas7]

Pi and Claude Code are both harnesses, so my question is what special do I need to use PAI directly via Pi? Thank you!

[pybe]

You can have pi use Claude -p as and agent or similar

[pybe]

I asked pi on deepseek-v4-flash and this is the response:

How pi could embed claude -p as an internal sub-agent

You have several routes, each with a different tradeoff. The core idea in all of them: treat claude -p as a black-box single-shot agent that pi can
invoke, capture, and fold back into its own turn. Here's the full design.

1. The plumbing — how a prompt gets in and a result gets out

claude -p is headless. You hand it one prompt and it returns one answer. Two ways to get the prompt in:

  # As an argv argument (single-line, can get shell-quoting nasty)
  claude -p --output-format json "refactor parse.go and explain"

  # As stdin (multi-line prompts, no quoting hell)
  printf '%s' "$PROMPT" | claude -p --output-format json

For parsing, you want --output-format json, which emits { "type": "result", "result": "...", "is_error": false } — a single field you can JSON.parse
reliably, with is_error for the failure case. --verbose adds progress to stderr so you can stream status without polluting stdout.

2. Route A — pi's own bash tool (zero code, the naive version)

Because pi already has a bash tool, pi can just run claude -p today:

  Run: bash(claude -p --output-format json "summarize the CHANGELOG")

This is genuinely useful as a quick cross-check, but it's fragile as a sub-agent pattern: output parsing is manual, you can't stream, permissions can
hang the process, and pi has no structured feedback loop from the child.

3. Route B — a custom extension tool (the clean way)

This is where pi is designed to do exactly this. An extension registers a tool whose execute spawns claude -p as a child process and returns the
result as tool output. From docs/extensions.md:

  // ~/.pi/agent/extensions/claude-p.ts
  import { execFile } from "node:child_process";
  import { promisify } from "node:util";
  import type { Type } from "typebox"; // pi bundles typebox

  const execFileAsync = promisify(execFile);

  export default (pi) => {
    pi.registerTool({
      name: "claude_p",
      label: "Claude -p",
      description: "Run a one-shot Claude Code agent headlessly and return its answer",
      parameters: Type.Object({
        prompt: Type.String({ description: "Instruction for the Claude sub-agent" }),
        cwd: Type.Optional(Type.String({ description: "Project dir (picks up its CLAUDE.md)" })),
      }),
      async execute(toolCallId, params, signal) {
        // --permission-mode bypassPermissions: the child must not block on prompts.
        const { stdout, stderr } = await execFileAsync(
          "claude", ["-p", "--output-format", "json",
                     "--permission-mode", "bypassPermissions",
                     params.prompt],
          { cwd: params.cwd ?? process.cwd(), timeout: 120_000, signal }
        );
        const parsed = JSON.parse(stdout);
        return {
          content: [{ type: "text", text: parsed.result }],
          details: { isError: parsed.is_error, stderr },
        };
      },
    });
  };

Now claude_p is a first-class tool pi can call in any turn. The extension can even onUpdate to stream the child's output via tool_execution_update
events so the operator sees it live. The signal argument (an AbortSignal) lets pi abort the child mid-run.

4. Route C — the SDK (in-process, type-safe)

Instead of shelling out, pi can drive a second pi session in-process via @earendil-works/pi-coding-agent. This is the same layer the built-in
RPC/print modes use. But note: the SDK spawns pi agents, not Claude Code. To specifically get claude -p you still shell out — unless you don't
actually need Claude Code's machinery and just want "another agent in-process," in which case Route C is cleaner than shelling out at all:

  import { createAgentSession, SessionManager } from "@earendil-works/pi-coding-agent";
  const { session } = await createAgentSession({
    sessionManager: SessionManager.inMemory(),
    tools: ["read", "bash"],   // constrain the sub-agent
  });
  await session.prompt("Review the diff in cwd and return verdicts");

If the goal is "a sub-agent," Route C is usually the better answer than embedding claude -p, because you get typed events, direct agent-state access,
and no subprocess. Embed claude -p only when you specifically want Claude Code's toolchain (its hooks, its CLAUDE.md conventions, its permissions
model) inside pi.

The gotchas that will bite you

• Permission prompts hang headless mode. In Route B you must pre-authorize the child. --permission-mode bypassPermissions (or
  --dangerously-skip-permissions) is required or claude -p blocks forever on a Y/n prompt. Only use it in a sandbox.
• Recursion guard (most important). If your claude -p sub-agent's own system prompt allows it to call tools, and its project has pi installed, it
  could spawn pi, which spawns claude, which spawns pi… Guard by setting an env var the child inherits and the wrapper checks — e.g.
  CLAUDE_P_EMBEDDED=1, and refuse to re-enter if already set.
• Context duplication. claude -p re-reads its own CLAUDE.md and may re-load the whole repo into its context, independent of pi's. Set a dedicated
  CLAUDE_CONFIG_DIR/--add-dir scope so the sub-agent has bounded, predictable context rather than the full tree.
• Timeouts & aborts. Always set a timeout on the child (Route B above uses 120_000) and wire signal. A runaway claude -p otherwise pins your turn.
• Auth collision. Both use ANTHROPIC_API_KEY / keychain. Usually fine, but the sub-agent can trigger re-auth flows; pin the auth explicitly.
• Streaming is stderr. With --output-format json, progress goes to stderr; stdout is pure JSON. Don't interleave them or JSON.parse breaks.

When you'd actually want this

• Model/strategy fan-out — pi orchestrates N claude -p reviewers with --agent reviewer on different parts of a codebase, then synthesizes their
  verdicts. That's a real, valuable pattern (and parallels what the LifeOS Delegation skill does with parallel agents).
• Cross-harness collaboration — pi (a different harness) intentionally delegates a subtask to Claude Code to leverage its distinct hooks/permissions,
  rather than reimplementing them.

But my honest recommendation: prefer Route C (the pi SDK's own createAgentSession) for generic sub-agents. Reaching for claude -p specifically is only
justified when you want Claude Code's own machinery — and in that case Route B's extension tool is the idiomatic pi-native way to expose it.

[danielmiessler]

The sidecar pattern is engine-agnostic by design—Hermes supplies the engine and the channels while LifeOS supplies identity, rules, and skills, so the mount can move. A Pi mount is feasible in principle, but nothing ships for it today: the guard layer (policy deny-globs, the secrets-used-never-seen contract) would need porting to Pi's own guard surface before it'd be safe. If you build one, the Hermes doc's architecture section is the map.