Skip to content

fix(worker): hash-partition by ordering_key for multi-worker fifo#5

Merged
zeybek merged 2 commits into
mainfrom
fix/worker-ordering-key-partition
May 12, 2026
Merged

fix(worker): hash-partition by ordering_key for multi-worker fifo#5
zeybek merged 2 commits into
mainfrom
fix/worker-ordering-key-partition

Conversation

@zeybek
Copy link
Copy Markdown
Owner

@zeybek zeybek commented May 12, 2026

Summary

Fix multi-worker partitioning so that messages sharing an ordering_key route to a single worker. Switch from id % total_workers to abs(hashtext(ordering_key)) % total_workers when the key is set; id-based fallback preserves balance for unkeyed messages.

Why

In src/worker/batch_processor.c:460, id-based modulo spread siblings of the same ordering_key across worker partitions. Per-key FIFO was only enforced by a NOT EXISTS subquery, which produced starvation and unnecessary cross-worker coordination under N>1 workers. Hash-partitioning the key restores per-key FIFO at the partition level.

Validation

  • make installcheck-regress — 27/27 regress + 13/13 isolation (PG18, all protocols)
  • make installcheck-tap — 3/3 (worker lifecycle intact)
  • New isolation spec ordering_key_partition passes — with hash('partition-X') % 2 = 0, w0 locks 5 rows, w1 locks 0
  • Spec registered in Makefile ISOLATION list
  • make format / make lint / make tidy (verified by CI)

Checklist

  • Isolation test added (tests/isolation/specs/ordering_key_partition.spec)
  • Migration file added (sql/ulak--0.0.2--0.0.3.sql — marker, no schema change)
  • Unkeyed messages keep id-based fallback for load balance
  • Not a user-facing behavior change; no docs update required
  • Commit message follows Conventional Commits

@zeybek zeybek marked this pull request as ready for review May 12, 2026 21:28
@zeybek zeybek merged commit 13129ec into main May 12, 2026
11 of 13 checks passed
@zeybek zeybek deleted the fix/worker-ordering-key-partition branch May 12, 2026 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant