Skip to content

feat: add explicit search backend selection#58

Open
na-navi wants to merge 7 commits into
mainfrom
feat/backend-selector
Open

feat: add explicit search backend selection#58
na-navi wants to merge 7 commits into
mainfrom
feat/backend-selector

Conversation

@na-navi
Copy link
Copy Markdown
Owner

@na-navi na-navi commented May 21, 2026

Refs #55

Depends on #57 (repo hygiene docs/templates already merged).

Summary

Adds the explicit --backend selector layer for the #55 MVP:

  • everything — existing Windows / Everything / es.exe backend
  • fd — filename/path backend via fd
  • rg-filesrg --files plus Python regex filtering

This PR intentionally does not add plocate, arch-linux, automatic backend detection, or fallback behavior. Those remain for #56 follow-up PRs.

Compatibility Notes

  • Windows default behavior remains se query → Everything / es.exe.
  • Non-Windows with no --backend now fails closed with exit 2 and a hint to use --backend fd or --backend rg-files.
  • --stats now includes backend=<name>.
  • Search JSON output now includes "backend": "...".
  • --log entries now include "backend": "...".
  • Timeout behavior remains exit 124.
  • -f/--fzf remains disabled in non-interactive mode.

Implementation split

  1. feat: add explicit backend dispatch
    • argparse + backend resolver + dispatch layer
    • _backend_invoke(argv, timeout) seam only
    • no _backend_stream yet
  2. test: cover explicit backend selection
    • new tests/test_backend.py
  3. docs: document backend selector compatibility
    • README / CONTRIBUTING / PR template compatibility notes

Verification

  • python -m py_compile src/se.py
  • python -m pytest tests/test_cli.py tests/test_backend.py -q → 36 passed
  • powershell.exe ... .\src\se.cmd --check --jsonok: true
  • Windows default smoke: python src/se.py se.py -n 3 --statsbackend=everything, rc 0
  • fd smoke: python src/se.py --backend fd --literal -n 3 se.py --stats → rc 0
  • rg-files smoke: python src/se.py --backend rg-files --literal -n 3 se.py --json → JSON includes backend: rg-files
  • NDA banned-term scan on changed files → clean
  • public-fork guard pattern scan → clean

AnoKno added 3 commits May 22, 2026 00:55
Refs #55

Adds --backend choices for everything, fd, and rg-files, resolves the default backend with Windows-first behavior, and routes search execution through a backend dispatch layer.

Windows default remains Everything/es.exe. Non-Windows requires explicit --backend fd or --backend rg-files. Stats, JSON output, and search logs now include the selected backend name.
Refs #55

Adds focused backend tests for default resolution, missing backend errors, dispatch to existing Everything functions, fd/rg-files command construction, timeout handling, and backend fields in stats/JSON/log output.
Refs #55

Documents explicit backend selection, clarifies search JSON output and backend stats, and adds Compatibility Notes guidance for backend-related PRs.
Lyra and others added 2 commits May 22, 2026 03:21
- validate_everything_backend() raises BackendConfigError instead of
  surfacing a raw FileNotFoundError from subprocess.run
- call it from backend_search() before invoking es_search / es_search_multi_path
- README: list fd and ripgrep as optional tools, clarify that the
  Everything/es.exe restriction is on the backend (not on se as a whole),
  and note that --backend fd / rg-files can be explicitly opted into on Windows
- test: cover BackendConfigError when get_es_path() points at a missing file
@na-navi
Copy link
Copy Markdown
Owner Author

na-navi commented May 22, 2026

Review feedback addressed (force-pushed da505e0)

External review on this PR raised three actionable items. All three are addressed in da505e0 without expanding scope beyond #55.

1. everything backend now fails fast when es.exe is missing

Before: a missing es.exe surfaced as a raw FileNotFoundError from subprocess.run.

After: new validate_everything_backend() raises BackendConfigError with a doctor hint, called from backend_search() before any es_search / es_search_multi_path invocation.

def validate_everything_backend() -> None:
    es = Path(get_es_path())
    if not es.exists():
        raise BackendConfigError(
            f"se: es.exe not found at {es}. "
            "Run `se --doctor` or set es_path in ~/.serc."
        )

Test coverage: test_everything_missing_es_exe_raises_backend_config_error.

2. README Optional tools table now lists fd and ripgrep

Optional Note
fd --backend fd でファイル名/パス検索(POSIX 系の主用途。Windows でも明示指定可)
ripgrep --backend rg-filesrg --files + Python 正規表現フィルタ

3. Wording: Everything restriction is on the backend, not on se

Before:

Everything は Windows 専用です。他の OS では動作しません。

After:

Everything / es.exe backend は Windows 専用です。Linux/POSIX 系では --backend fd または --backend rg-files を明示指定してください。

Backend selection section now also explicitly states that --backend fd / --backend rg-files can be opted into on Windows (PATH の fd.exe / rg.exe) for smoke / portable fallback, while the default backend is unchanged.

Verification (re-run after da505e0)

  • python -m pytest -q37 passed (+1 new test, 0 regressions)
  • python src/se.py --check --jsonok: true on Windows
  • python src/se.py --literal -n 3 pyeverything default unchanged
  • python src/se.py --backend fd --literal -n 5 py → rc 0, results returned
  • python src/se.py --backend rg-files --literal -n 5 py → rc 0, results returned
  • python src/se.py --backend fd --max-seconds 0.0001 --literal -p "C:/Users" py --stats[se] backend=fd ... timed_out=True, rc 0 (partial-results convention preserved)
  • python -c "...patched get_es_path -> missing path..."BackendConfigError: se: es.exe not found at C:\does-not-exist\es.exe. Run \se --doctor` or set es_path in ~/.serc.`
  • NDA banned-term scan on the repo → clean
  • git diff --check → clean

Out of scope (deferred to #56)

  • --backend plocate / --backend arch-linux
  • plocate → fd automatic fallback
  • --install-hint flag
  • backend-aware --check (Linux probing, install-hint surfacing)

These were drafted locally, reviewed, and intentionally not included in this PR per the review recommendation to keep #55 a clean backend-selector layer.

- move backend resolution after --expand-only output so Linux/POSIX can run se -e without --backend
- allow validate_everything_backend() to accept PATH-resolved es.exe values via shutil.which
- add regression tests for both review findings
@na-navi
Copy link
Copy Markdown
Owner Author

na-navi commented May 22, 2026

Follow-up review fixes addressed (ab54080)

Thanks — both remaining regressions were real and are now fixed in ab54080.

1. --expand-only no longer resolves a search backend

cmd_search() now builds the query/regex first and returns immediately for --expand-only before calling resolve_backend().

This preserves the expected non-search behavior on Linux/POSIX:

python src/se.py -e test

No --backend is required for expansion-only mode.

Regression test added:

def test_expand_only_does_not_require_backend_on_linux(...)

2. validate_everything_backend() now preserves PATH-resolved es_path

The validator now accepts either:

  • an existing configured path, or
  • a non-absolute command resolvable through shutil.which()

So configurations like es_path: es.exe continue to work if es.exe is on PATH.

Regression test added:

def test_everything_accepts_path_resolved_es_exe(...)

Verification after ab54080

Windows/Git Bash environment:

  • python -m py_compile src/se.py → pass
  • python -m pytest -q39 passed
  • python src/se.py -e test → pass, prints expansion
  • python src/se.py --backend fd --literal -n 5 py --stats → pass, backend=fd
  • python src/se.py --backend rg-files --literal -n 5 py --json → pass, JSON includes backend: rg-files
  • python src/se.py --literal -n 3 py --stats → pass, Windows default backend=everything

WSL/Linux smoke:

  • python3 src/se.py -e test → pass without --backend (migemo fallback message in this WSL env, then prints raw query)
  • python3 src/se.py test → expected rc 2 with non-Windows platforms require --backend fd or --backend rg-files
  • python3 src/se.py --backend fd --literal -n 5 py --stats → pass, backend=fd
  • python3 src/se.py --backend rg-files --literal -n 5 py --json → pass, JSON includes backend: rg-files

Guards:

  • NDA banned-term scan → clean
  • git diff --check → clean
  • public-fork staged path guard → clean

Scope remains unchanged: #58 is only the #55 backend-selector layer. plocate / arch-linux / fallback remain deferred to #56.

- _search_fd / _search_rg_files: raise BackendConfigError instead of
  sys.exit(1) when the underlying tool exits non-zero with stderr, so
  cmd_search owns exit code, stats, and log emission consistently
- backend_search everything single-path branch: use _remaining_timeout
  instead of inline (deadline - time.monotonic()), so an exhausted
  budget fails before spawning es.exe (matches fd / rg-files paths)
- tests: cover both helper failure paths and the early-exhausted budget
  case for the everything single-path branch
@na-navi
Copy link
Copy Markdown
Owner Author

na-navi commented May 22, 2026

Layer-responsibility fixes (7bf47c7)

Two small follow-ups that came out of another review pass, both kept inside the existing exception contract used by backend_search().

1. fd / rg-files helpers no longer call sys.exit()

Before: _search_fd() and _search_rg_files() would print the underlying error and sys.exit(1) when the tool exited non-zero with a stderr message. That bypassed cmd_search()'s exit code, --stats, and --log handling, and conflicted with exit 1 already used for scope / forbidden-path failures.

After: both helpers raise BackendConfigError(f"se: {backend} backend failed: ..."), which cmd_search() already catches and maps to exit 2. Stats, JSON, and log paths are now reachable in this branch too.

Observed before vs after:

# before — exit 1, no stats, no log
$ python src/se.py --backend fd -p Z:/does-not-exist --literal py
fd error: [fd error]: Search path 'Z:/does-not-exist' is not a directory.

# after — exit 2, prefixed with the backend, cmd_search owns the path
$ python src/se.py --backend fd -p Z:/does-not-exist --literal py
se: fd backend failed: [fd error]: Search path 'Z:/does-not-exist' is not a directory.

2. everything single-path branch routes through _remaining_timeout()

backend_search() was computing (deadline - time.monotonic()) inline for the everything single-path call, while every other backend path goes through _remaining_timeout(deadline). Consolidating means an already-exhausted global budget raises SearchTimeout before es.exe is spawned, matching the fd / rg-files behavior. No user-visible change on the happy path (deadline=NoneNone).

Tests added

  • test_fd_nonzero_rc_with_stderr_raises_backend_config_error
  • test_rg_files_nonzero_rc_with_stderr_raises_backend_config_error
  • test_everything_single_path_uses_remaining_timeout — asserts es_search is not called when the budget is already exhausted

Verification

Windows:

  • python -m py_compile src/se.py → pass
  • python -m pytest -q42 passed
  • python src/se.py --backend fd -p Z:/does-not-exist --literal py → exit 2, prefixed message
  • python src/se.py --backend rg-files -p Z:/does-not-exist --literal py → exit 2, prefixed message
  • python src/se.py --literal -n 3 py --stats → unchanged (backend=everything, exit 0)
  • python src/se.py --backend fd --literal -n 3 py --stats → unchanged (exit 0)

WSL / Linux:

  • python3 -m pytest -q42 passed
  • python3 src/se.py --backend fd -p /does-not-exist --literal py → exit 2, prefixed message
  • python3 src/se.py --backend rg-files -p /does-not-exist --literal py → exit 2, prefixed message
  • python3 src/se.py -e test → exit 0, no --backend required (regression from earlier fix preserved)

Guards:

  • NDA banned-term scan → clean
  • git diff --check → clean
  • public-fork staged-path guard → clean

Scope

Diff size: src/se.py +6/-9, tests/test_backend.py +26. Still strictly the #55 backend-selector layer. plocate / arch-linux / fallback remain deferred to #56.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants