Skip to content

internal/llm: loopback-enforced OpenAI-compatible client #24

Description

@Saber5656

Title

internal/llm: loopback-enforced OpenAI-compatible client

Summary

Implement the HTTP client for local LLM endpoints with the loopback-only
dialer (invariant I1), proxy/redirect hardening, the chat-completions and
models calls, timeout semantics, and a typed error taxonomy — per DESIGN.md
§11.1 and ADR-002.

Context

This is the ONLY package allowed to import net/net/http (CI boundary,
issue 02). The dialer is the technical heart of the "no external
communication" promise: it must verify loopback per resolved IP at dial
time, because URL string checks can be defeated by DNS.

Scope

  • internal/llm/client.go, transport.go, errors.go, tests
    (httptest-based; no real LLM needed).

Detailed Requirements

  1. Transport (NewTransport(resolver Resolver) *http.Transport):

    • Proxy: nil (never ProxyFromEnvironment).
    • DialContext: split host/port → resolve via injected Resolver
      (interface over net.Resolver.LookupIPAddr; real + fake for tests) →
      if ANY resolved IP fails ip.IsLoopback() → return
      ErrNonLoopbackBlocked (include host and the offending IP in the
      message) → else dial the first loopback IP with net.Dialer
      (timeout 5s).
    • IP literals: same path (resolver short-circuits literals).
    • ForceAttemptHTTP2: false (local servers; keep it boring),
      MaxIdleConns: 2, TLSClientConfig default (https allowed —
      loopback TLS is unusual but legal).
  2. Client:

    type Client struct { /* base *url.URL, apiKey, http *http.Client, model string, temperature float64 */ }
    func New(cfg config.LLM) (*Client, error)   // validates scheme http/https, host non-empty
    func (c *Client) Model() string             // configured model id (used by 24 for attribution)
    func (c *Client) Chat(ctx context.Context, system, user string, maxOutputChars int) (string, error)
    func (c *Client) Models(ctx context.Context) ([]string, error)
    • http.Client{ Transport: NewTransport(...), Timeout: cfg.Timeout, CheckRedirect: func(...) error { return ErrRedirectBlocked } }
      ALL redirects denied (DESIGN §11.1).
    • Chat: POST {base}/chat/completions, body
      {"model","messages":[{role:system},{role:user}],"temperature", "stream":false}; Authorization: Bearer only when apiKey != "";
      Content-Type: application/json.
    • Response: decode choices[0].message.content; empty/whitespace →
      ErrEmptyResponse. Response body read capped at 8 MiB.
    • HTTP ≥ 400: read ≤ 64 KiB of body, first line into
      ErrHTTPStatus{Code, Line} (research doc: bodies may be non-JSON).
    • Models: GET {base}/modelsdata[].id strings (tolerant decode).
    • Exactly ONE attempt per call — no retries (DESIGN/research: local
      endpoints fail deterministically; retrying delays fallback).
  3. Error taxonomy (all errors.Is-able): ErrNonLoopbackBlocked,
    ErrRedirectBlocked, ErrEmptyResponse, ErrHTTPStatus,
    plus transparent wrapping of ctx/timeout errors. Callers (24, 28) map
    them to warnings/diagnostics; this package never logs.

  4. URL join must tolerate base with/without trailing slash and with /v1
    suffix (join, don't concatenate).

Acceptance Criteria

  • httptest.Server (loopback) happy path: Chat returns content;
    request body JSON asserted (model, both messages, stream:false,
    temperature).
  • Dialer attack table (fake resolver): example.com→93.184.216.34
    blocked; evil→10.0.0.5 blocked (private ≠ loopback);
    evil→[127.0.0.1, 8.8.8.8] blocked (ANY non-loopback fails);
    localhost→127.0.0.1,::1 allowed; literal 127.0.0.1, [::1]
    allowed; literal 192.168.1.10 blocked.
  • HTTP_PROXY/HTTPS_PROXY/ALL_PROXY env set to a canary server
    during a test request → canary receives zero connections.
  • Redirect (301 to loopback AND to external) → ErrRedirectBlocked,
    no second request performed (canary asserts).
  • Bearer header present iff apiKey set.
  • 404 with HTML body → ErrHTTPStatus carrying first line, no panic.
  • Timeout: server sleeping past timeout_seconds → ctx deadline error
    within timeout+1s.
  • Models parses Ollama-shaped and LM-Studio-shaped fixtures.
  • Boundary script: still only this package imports net/http.

Validation

go test -race -cover ./internal/llm/ in PR. Manual (optional, if Ollama
present): go test -tags manual -run TestManualOllama gated helper — NOT
required for CI.

Dependencies

01, 05.

Non-goals

Prompt building/fallback policy (24), doctor UX (28), streaming, retries,
non-loopback support (ADR-002 hard scope), model management.

Design References

  • docs/DESIGN.md §11.1, §12.1 (B3), §12.2 (DNS row)
  • docs/decisions/ADR-002-network-boundary.md
  • docs/research/local-llm-endpoints.md

Source of truth: docs/issues/23-llm-client.md (PR #1, branch docs/v1-design). If this issue and the repo docs disagree, the docs win. Execution order and dependencies: docs/ISSUE_PLAN.md (this is issue 23 of 33).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions