Skip to content

No tests: add a smoke test that runs without downloading a model #2

Description

@tonydzi

What

677 lines of Python across index_notes.py, brain_ask.py, turnstate_hook.py,
turnstate_show.py — and no tests at all. The README is honest about it ("No incremental indexing,
no eval suite, no packaging. Pilot."), but it means any contributor's first PR has no way to show it
did not break retrieval, which is the single biggest reason a stranger's PR stalls here.

The interesting constraint

The obvious blocker: retrieval needs sentence-transformers, and nobody should download a few
hundred MB of model weights to run a unit test in CI.

So the useful shape is a seam: let the embedder be injected, and have the tests pass in a
deterministic fake (for example, a hash-based vector, or a bag-of-words vector over a tiny fixed
vocabulary). That is a small refactor with a real payoff — it also makes the A/B mode (--ab)
testable, which is currently only observable by eyeballing the SQLite table.

Suggested scope for a first PR

Don't try to cover everything. One end-to-end smoke test is worth more than ten unit tests here:

  1. Build a synthetic corpus of ~6 tiny markdown notes in a temp dir.
  2. Index it with a fake embedder.
  3. Assert: the right note is in the top-K for an obvious query, the schema matches schema.sql,
    and re-running the indexer does not duplicate rows.

Then, if you want a second PR: assert --graph actually promotes a linked note that vector search
alone misses — that is the claim the repo is built on, and nothing currently checks it.

Definition of done

  • pytest runs green with no model download and no network.
  • The command you ran and its output pasted in the PR body.
  • The fake-embedder seam does not change the default behaviour: python index_notes.py <dir> with
    no configuration still uses the real model.

This one is a design conversation as much as code — say how you want to cut the seam in the thread
before you build it, and I will answer within 48 hours.

Metadata

Metadata

Assignees

No one assigned

    Labels

    acceptedWe want this and it is free to take - comment 'claiming this'help wantedWe want it but havent scoped it - comment and we shape it with you

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions