feat: optional scope path-prefix filter for query_documents (#146) - #156
Merged
Conversation
…ject)
Change search(queryVector, queryText?, limit?) to
search(queryVector, options?: { queryText?; limit?; scope? }). Add a
private scope-predicate helper that, per prefix, normalizes trailing
separators (preserving posix root), derives the boundary separator from
the prefix (else node:path.sep), and builds an exact-or-descendant
predicate (`filePath = P OR filePath LIKE D% ESCAPE '\'`) with ' % _ \
escaped, OR'd across prefixes and applied via a single .where() on
vectorSearch only when scope is present. Scope-absent behavior is
unchanged. Add SearchOptions type and 10 real-LanceDB scope tests.
Consumer call sites (server handler, CLI) remain positional and migrate
in later tasks. Part of #146.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The FTS/keyword-boost branch already inherits scope because its `filePath IN (...)` set derives from the already-scoped vector hits. Add a `results.length > 0` condition to the FTS guard so the branch is skipped when the scoped vector step returns zero hits, avoiding a malformed `filePath IN ()` predicate. Add an 8-test real-LanceDB FTS/ hybrid integration suite (in-scope-only, skip-on-empty via spy, scope-absent regression, backslash, exact-file, root, multi-prefix, injection). Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add scope?: string | string[] to QueryDocumentsInput and a normalizeScope helper in parseQueryDocumentsInput, mirroring the limit defense-in-depth pattern: accept a non-empty string or a non-empty array of non-empty strings, reject all other shapes with McpError(InvalidParams), and normalize to string[]. Add 14 unit tests. Handler wiring follows in a later task. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migrate the handler call site to the options-object search() and pass scope through (array-wrapped, key omitted when absent to preserve the scope-absent path under exactOptionalPropertyTypes). Add a boundary test asserting the scope string[] reaches search() unchanged when present, the key is omitted when absent, and limit defaulting is preserved. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an optional scope property (oneOf string | string[]) to the query_documents inputSchema with a description covering exact-or-descendant path-prefix semantics. query remains the only required field. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a repeatable --scope <prefix> flag that accumulates path prefixes into a string[], reject empty values, and thread scope into the options-object search() call (only when present). Document --scope in HELP_TEXT. This migrates the last positional search() call site, so the project type-checks clean again. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add user-facing scope documentation to README (Searching section + CLI example) alongside the peer limit parameter. Final QA for #146: AC1-AC9 verified, full check:all green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Align the CLI --scope validation with the MCP boundary, which rejects whitespace-only scope via trim(). A whitespace-only prefix is harmless (fail-closed) but the asymmetry was inconsistent. Add a parseArgs test. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trim accepted scope values at the MCP and CLI boundaries so a whitespace-padded prefix no longer silently matches nothing (the whitespace-only case was already rejected). Correct the stale buildScopePredicate comment: the FTS branch does not consume the predicate, it inherits scope via its own filePath IN (...). Note the posix backslash caveat in separator derivation. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trim the scope-feature comments that restated the code (buildScopePredicate step narration, stripTrailingSeparators Windows paragraph, escapeLike, SearchOptions/QueryDocumentsInput scope fields, normalizeScope, handler array-wrap), keeping only the non-obvious rationale. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the query_documents/ingest_file/ingest_data/list_files/status return shapes explicit, correct the status description to the actual getStatus fields (was "database size"/"configuration information", neither returned), note that scope must be an absolute prefix (relative matches nothing), drop vague query-writing coaching and low-signal visualQuality detail, and use positive "exactly one" phrasing. Necessary-and-sufficient, no behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add the scope parameter to the installable mcp-local-rag skill: --scope in the CLI query reference, a Scope usage section and workflow/tool-table mentions in SKILL.md, and an absolute-path check under result-refinement's No Results. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MCP tool descriptions must stand alone (the skill is supplementary and may not be installed). Restore compactly: query construction guidance (preserve specific terms / add context when vague), how to obtain an absolute scope prefix (derive from an earlier result's filePath, or omit), and the ingest_data format mapping (text/html/markdown). Align the shipped skill's Scope section with the same relative-path pre-action. Part of #146. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an optional
scopeparameter toquery_documents(MCP) and thequeryCLI subcommand that limits a search to documents under one or more absolute path prefixes. This lets a single shared database hold multiple corpora while letting a query focus on one when precision matters. Closes #146.scopeaccepts one prefix or a list (results are unioned).filePathequal to it or under it (/docs/apimatches/docs/api/auth.mdbut not/docs/apiv2)..where()prefilter on the vector candidate prefetch; the FTS/keyword-boost branch inherits scope via its existingfilePath IN (...).scopeis byte-for-byte unchanged.Design notes
filePath = P OR filePath LIKE 'D%'whereDcarries a separator boundary. Reuses the existing single-quote escaping and additionally escapes%,_,\withESCAPE '\'.filePaths are absolute, so a relative prefix matches nothing by design. The MCP tool description and shipped skill document how to obtain an absolute prefix (or omitscope).VectorStore.searchnow takes an options objectsearch(queryVector, { queryText?, limit?, scope? }).filePath IN ()).McpError) and CLI (process.exit) boundaries.Surface
query_documentsgainsscope(string | string[]).querygains a repeatable--scope <prefix>flag.list_files/read_chunk_neighborsare intentionally out of scope for this PR.Docs
statusdescription fixed to actual fields).scopein the shippedskills/mcp-local-rag(SKILL.md, CLI reference, result-refinement).Testing
pnpm run check:allgreen (947 tests)./- and\-style paths, multi-prefix union, injection escaping, FTS skip-on-empty, and scope-absent regression. Plus MCP/CLI validation and handler-threading tests.🤖 Generated with Claude Code