feat(sync): --src-subpath + --exclude for monorepo subdir-source support#774
feat(sync): --src-subpath + --exclude for monorepo subdir-source support#774jeremyknows wants to merge 2 commits intogarrytan:masterfrom
Conversation
…support PR-A: discoverGitRoot() walks up from any path to the git root so `gbrain sync wiki/` works without requiring the git root as cwd. Splits the existing repoPath into gitContextRoot (git ops) and syncScopeRoot (file walking / import), keeping slugs relative to gitContextRoot so wiki/page1.md lands as slug "wiki/page1" not "page1". Scope guard: realpathSync() scope-entry check + per-file TOCTOU check (NAV-1) reject --src-subpath values that traverse outside the repo via `../` or symlinks. NAV-4: warn when --exclude filters everything out. PR-B: expose the existing internal `exclude` SyncOpts field as a repeatable --exclude <glob> CLI flag. Also threads exclude support into runImport() for full-sync paths. runImport() gains a `slugRoot` option so callers can walk from a subdirectory while keeping slugs relative to the repo root. matchesAnyGlob promoted to export from core/sync.ts. manageGitignore() always resolves to git root via discoverGitRoot() even when repoPath is a subdirectory. Tests: 10 tests in test/sync-monorepo.test.ts covering back-compat, auto-discovery, --src-subpath scoping, 2-source isolation, path-traversal rejection (NAV-1/NAV-2), and --exclude single/glob/NAV-4.
…slugs for direct-dir sources Sources registered pointing at a subdirectory (e.g. wiki → ~/atlas/shared/wiki) expect slugs relative to their own path, not the git context root. The slugRoot opt was introduced for the --src-subpath monorepo case where git-root-relative slugs are intentional. Applying it to all subdirectory sources breaks frontmatter slug matching (e.g. agent-roster.md slug:'agent-roster' vs path-derived 'shared/wiki/agent-roster'). Fix: `slugRoot = (opts.srcSubpath && scopeRel) ? gitContextRoot : undefined` 10/10 tests pass. Discovered during 4th-wipe re-import (wiki: 4 → 64 pages). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Follow-up commit: Discovered during live testing: the original PR applied Fix: Git-root-relative slugs are only activated when All 10 tests still pass. |
Summary
Adds two flags to
gbrain syncthat together make Atlas-style monorepos (1 git repo, N logical sources at subdirs) work without any workarounds.PR-A —
--src-subpath <subdir>discoverGitRoot()helper:git -C <path> rev-parse --show-toplevelwalks up from any path to the git root — handles worktrees + submodules natively.repoPathinto two axes:gitContextRoot— all git operations (rev-parse, diff, pull)syncScopeRoot— file walking and importsgitContextRootsowiki/page1.mdlands as slugwiki/page1, notpage1, regardless of which subpath is synced.runImport()gains aslugRootoption to support this: walk fromsyncScopeRoot, compute slugs relative togitContextRoot.manageGitignore()always resolves to git root even whenrepoPathis a subdirectory.repoPath(no--src-subpath) also works — same code path.PR-B —
--exclude <glob>(repeatable)SyncOpts.excludefield as a repeatable CLI flag.runImport()opts for full-sync paths.matchesAnyGlobpromoted toexportfromcore/sync.tsfor reuse inimport.ts.Security guards
--src-subpath ../escape—realpathSyncscope-entry check before any git oprealpathSynccheck during walkgitContextRootitself passed throughrealpathSync--excludepatterns filter out all files.git/hooks/) run duringgit pull— unchanged from current behavior; callers trusting remote repos already accept this riskTests
test/sync-monorepo.test.ts— 10 tests, all pass:What was NOT tested
--src-subpath(requires 2 commits in a real git repo; incremental path usesinScope()filter ongit diffoutput which is git-root-relative — tested manually by inspection but not automated here)--src-subpath(PGLite forces serial; Postgres worker path is unchanged)Motivation
Closes the gap for monorepo deployments where a single git repo contains multiple logical knowledge bases (e.g.,
wiki/,memory/,projects/). Previously,gbrain syncrequired the git root and imported everything; there was no way to scope a sync to a subdirectory. With--src-subpath, each logical source can be synced independently with its own source ID, enabling per-source search and cross-contamination prevention.Relates to #753.
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmithwith what you need.