feat: add optional scope path-prefix filter to list_files and list CLI (#165)

[shinpr]

Summary

Adds an optional scope path-prefix filter to the list_files MCP tool and the list CLI subcommand, resolving the large-volume timeout reported in #165. scope is one absolute path prefix or a list (unioned), matched exact-or-descendant on each file's on-disk scan path — reusing the semantics locked for query_documents in #146.

The origin is a real timeout: on a volume with several thousand files, list_files enumerates the whole tree. Rather than filter the result after the fact (which would leave the walk cost intact), the scope predicate is pushed into the BFS traversal — a directory outside every prefix is never readdir'd — so a scoped listing costs work proportional to the scoped subtree instead of the whole volume.

Minor release: 0.15.4 → 0.16.0. Fully backward compatible — omitting scope (and calling list_files with no arguments) behaves exactly as before.

Behavior

• files[] is restricted to entries whose scan path is equal to or under a prefix; boundary-safe (/docs/api matches /docs/api/x.md, not /docs/apiv2).
• sources: ingest_data (raw-data) sources are always listed regardless of scope; real-file source entries respect scope. This split is applied by a shared classifyIngestedSources helper used by both surfaces.
• Non-absolute prefix: matches nothing (documented), and now surfaces a non-fatal warning (MCP: a warning content block; CLI: a stderr Warning, exit 0) so an empty result is explained rather than silent.
• Empty/whitespace scope: rejected at the boundary (MCP McpError(InvalidParams); CLI non-zero exit).
• No-arg list_files (arguments omitted / {}) still returns the full listing — no regression.

Path basis

scope is guaranteed over the on-disk reachable (scan) path, which makes the traversal pushdown exact. Under a symlinked/aliased root the returned filePath keeps its stored spelling while scope matches the reachable path; this edge is defined and fixture-covered.

Implementation

• src/utils/scope-match.ts (new) — pure boundary-safe matcher (isUnderOrEqual, matchesAnyScope, shouldVisitDir, isInScope, nonAbsolutePrefixes), the JS counterpart of the SQL buildPrefixPredicate.
• src/utils/list-sources.ts (new) — pure classifyIngestedSources shared by the MCP handler and the CLI (removes a prior byte-for-byte duplication).
• Scope pushdown threaded into both BFS walkers (scanBaseDir, bfsCollectSupportedFiles) with a root gate before the first readdir.
• MCP boundary: ListFilesInput + parseListFilesInput (accepts undefined/{} as no-scope), scope on the list_files schema with scan-path-basis wording, warning content block.
• CLI: repeatable --scope, threading, and HELP_TEXT.
• Tool-schema description clarified so an MCP client learns from the in-protocol description alone that sources are ingest_data items and that scope filters files (not ingest_data sources).

Tests

• Unit: scope-match, list-sources, parseListFilesInput, schema.
• Integration (real LanceDB + real-FS fixtures):
  • INT-2 — walker pushdown proven directly on both walkers via a readdir EACCES mock/spy (a scope-outside path is never visited), discriminating real pruning from a post-scan filter.
  • INT-1 / INT-3 — MCP handler / CLI scoped listing, sources split, and a scanned-file realpath spy.
  • Non-absolute-prefix warning tests on both surfaces.
• E2E: a service-integration-e2e spawning the real CLI list --scope against a real fixture.
• pnpm run check:all green (1047 tests); cross-platform-sensitive fixtures (symlink alias) skip on unsupported platforms rather than fail.

Closes #165.

🤖 Generated with Claude Code