Skip to content

feat: add a queue reset so a repeated push cannot enqueue work twice - #28

Merged
beagleknight merged 1 commit into
mainfrom
feat/reset-queue
Aug 12, 2026
Merged

feat: add a queue reset so a repeated push cannot enqueue work twice#28
beagleknight merged 1 commit into
mainfrom
feat/reset-queue

Conversation

@beagleknight

Copy link
Copy Markdown
Collaborator

Mirrors specbandit#27 (Ruby gem). Both are needed: factorial pushes backend specs through the gem and everything else through this package.

Problem

A backend spec failed in factorial CI with a T::Struct prop redefinition error. It looked like a Sorbet/Tapioca problem. It was not: the same spec file was loaded twice in one process.

The queue key is scoped by CI run, not by CI attempt. That is deliberate. Re-running a single failed runner does not re-run the job that pushed, so the runner must still find the queue and the published marker that the first attempt created.

The cost is that the producer is not idempotent:

  • A producer that pushes and then fails leaves the work list orphaned, because its runners never start.
  • A producer that runs again (whole-workflow re-run, or a failed-jobs re-run that includes it) appends a second copy on top.

Every file is then enqueued twice. The suite runs twice, and any two copies that reach the same worker are loaded twice in one process.

Change

RedisQueue.clear removes the queue and its <key>:published marker in a single DEL. It is exposed two ways:

specbandit push --key KEY --reset    # empty the key, then push
specbandit reset --key KEY           # standalone

Reset is opt-in. push keeps its current meaning and appending to a queue stays possible.

reset is registered as a boolean flag in the argument parser. A flag missing from that list consumes the next token as its value, so push --reset a.test.ts would otherwise lose the file.

What reset does not touch

Per-runner rerun keys and failed keys. A runner's replay memory has to survive a re-run of that runner. A runner that finds data in both the shared queue and its rerun key is the full-rerun case, already handled in v1.3.0: it resets its own memory and steals.

Nothing is cleared when there is nothing to push in its place. Dropping the marker on its own would make every worker on that key crash as "never published".

Version drift, fixed here too

VERSION in src/configuration.ts said 0.10.0 while package.json was at 1.3.0, so specbandit --version and the specbandit_version field in --report JSON both reported a version nobody shipped. The constant cannot import package.json, because tsconfig rootDir is src, so the release workflow now rewrites it alongside the package.json bump and fails loudly if the constant moves.

Tests

  • RedisQueue.clear issues one DEL with both key names.
  • Publisher: clear happens before the push, only with reset: true, never on an empty push, and the leftover count is reported.
  • New E2E scenario 8 against the real Redis service: two plain pushes stack 14 entries, a third push with --reset leaves 7, the marker survives, a worker sees each file exactly once, and a standalone reset removes the marker. Verified locally against a real Redis before pushing.

Two pre-existing test failures (jestAdapter / cypressAdapter "throws when not installed") reproduce on unmodified main in a local checkout where those optional peers are present. They pass in CI.

Follow-up, not in this PR

The flaky-retry pass consumes <key>-failed-<n>, which is written by work, not push, so a re-run can still leave stale entries there. The blast radius is re-running an already-failed spec, not a double load.

Next

Needs a Release dispatch (bump: minor → v1.4.0) before factorial can adopt it.

Mirrors specbandit (Ruby) #27.

The queue key is scoped by CI run, not by CI attempt. That is deliberate:
re-running a single failed runner does not re-run the job that pushed, so
the runner must still find the queue and the published marker the first
attempt created.

The cost is that the producer is not idempotent. A producer that pushes
and then fails, or that is re-run with the whole workflow, appends a
second copy of the work list. Every file is then enqueued twice, the
suite runs twice, and any two copies that reach the same worker are
loaded twice in one process.

Add RedisQueue#clear, which removes the queue and its published marker in
one DEL, and expose it two ways:

- `specbandit push --reset` empties the key immediately before the RPUSH.
- `specbandit reset --key KEY` does the same as a standalone command.

Reset is opt-in, so `push` keeps its current meaning and appending to a
queue stays possible.

`reset` is registered as a boolean flag in the argument parser. A flag
missing from that list consumes the next token as its value, so
`push --reset a.test.ts` would otherwise lose the file.

Per-runner rerun and failed keys are left alone, so a single-runner
re-run can still replay its own files. Nothing is cleared when there is
nothing to push in its place: dropping the marker on its own would make
every worker on that key crash as "never published".

Also closes a version drift. The VERSION constant said 0.10.0 while the
package was at 1.3.0, so `--version` and the report JSON both reported a
version nobody shipped. It cannot import package.json, because tsconfig
rootDir is src, so the release workflow now rewrites it alongside the
package.json bump.
@beagleknight
beagleknight marked this pull request as ready for review August 12, 2026 08:25

@jacobobq jacobobq left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!

@beagleknight
beagleknight merged commit 3277996 into main Aug 12, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants