Add concurrency-limited batching and circuit-breaker awareness to bulk role assignment

[Lakes41]

Difficulty: Expert
Type: performance (scalability)

Background

app/admin/members/page.tsx's executeBulkAssign() handles the bulk role-assignment flow (triggered from components/ui/bulk-action-toolbar.tsx after selecting multiple members). It calls Promise.allSettled(items.map(async (item) => { await api.assignRole(item.address, item.role); ... })) — every selected member's assignRole() call is fired concurrently, with no limit on how many are in flight at once. Separately, lib/api/live.ts already implements a per-path circuit breaker (CircuitEntry, assertCircuitAllowsRequest, recordCircuitFailure) that opens after CIRCUIT_FAILURE_THRESHOLD (default 3) failures within CIRCUIT_FAILURE_WINDOW_MS (default 30s) for a given request path, and REST retry logic (backoffDelayMs) — but both of these only apply per-request; shouldRetryRequest() also only retries GET requests by design, so assignRole (a POST) is never retried automatically.

Problem

For a moderately large community, an admin could plausibly select dozens or hundreds of members and trigger a bulk role change (the UI has a "select all on page" affordance per pageAddresses.forEach). Firing that many concurrent POST /v1/members/:address/roles requests at once:

1. Provides no client-side throttling, so it can behave like a self-inflicted burst against the backend, independent of whatever server-side capacity exists.
2. Interacts poorly with the existing circuit breaker: once 3 of these concurrent requests fail within the 30-second window (e.g., the backend is genuinely struggling under the burst), the circuit for that path opens and every other in-flight request in the same batch immediately fails with a 503 serviceUnavailableError, even though those requests might have succeeded individually at a lower concurrency — so a single problematic backend response can cascade into a much larger batch failure than necessary, and the admin sees a confusing wall of both real errors and circuit-breaker-induced 503s in the same "results" list with no way to distinguish them.
3. Gives no backpressure signal back to the UI to slow down or pause mid-batch — it's all-or-nothing per the initial Promise.allSettled call.

Expected Outcome

Bulk role assignment processes selected members in bounded-concurrency chunks (not fully sequential — that would be too slow for the "select all" case — and not fully parallel), pauses/backs off automatically when the shared circuit breaker for the assignRole path opens (rather than continuing to fire requests that are guaranteed to fail), and clearly distinguishes in the results UI between "failed at the backend" and "skipped because the circuit breaker was open" so admins get an accurate, actionable picture and can safely retry just the right subset.

Suggested Implementation

1. Add a small, dependency-free concurrency-limited batch runner (e.g., lib/api/batch-runner.ts, exporting something like runWithConcurrency<T, R>(items: T[], limit: number, worker: (item: T) => Promise<R>)), since the project has no existing utility for this (lodash is available per the frontend/React artifact tooling docs, but check whether it's actually a listed dependency of this app in package.json before reaching for _.chunk/a manual queue — prefer a small hand-rolled implementation if not already a dependency, to avoid adding one for this alone).
2. Expose a way to observe the circuit breaker's current state for a given path from lib/api/live.ts without changing its existing per-request behavior (e.g., a new exported isCircuitOpen(path: string): boolean, reading from the existing circuitBreakers map/getCircuit() helper) — this is additive and shouldn't change any existing REST call semantics.
3. In executeBulkAssign() (app/admin/members/page.tsx), replace the single Promise.allSettled(items.map(...)) call with the new bounded-concurrency runner, using a sensible default limit (e.g., 5–10 concurrent requests — make it a named constant, not a magic number).
4. Before/between chunks, check isCircuitOpen() for the role-assignment path; if it's open, stop dispatching new requests in this batch and mark all remaining, not-yet-attempted items in the results as a distinct status (e.g., "skipped_circuit_open", separate from "error"), rather than firing them into guaranteed 503s.
5. Update the bulk-results UI (and the "retry failed items" flow already present, given setBulkFailedItems) to visually and semantically distinguish ok / error / skipped_circuit_open, and make "retry failed/skipped items" work correctly for both categories once the circuit closes again.
6. Add unit tests for the batch runner in isolation (concurrency is actually bounded; all items eventually settle; ordering/results integrity) and integration-style tests for executeBulkAssign()'s new circuit-breaker-aware behavior (mock assignRole to fail enough times to open the circuit mid-batch, assert remaining items are marked skipped rather than attempted).

Acceptance Criteria

• Bulk role assignment never has more than the configured concurrency limit of assignRole requests in flight at once.
• If the circuit breaker for the role-assignment path opens mid-batch, no further new requests are dispatched for the remainder of that batch, and the affected items are marked distinctly (not conflated with genuine per-item failures).
• The bulk-results UI clearly differentiates backend failures from circuit-breaker-skipped items, and the existing retry flow correctly re-attempts both kinds once retried.
• A new isCircuitOpen() (or equivalently named) export from lib/api/live.ts does not alter any existing REST call's retry/circuit-breaker behavior — all existing tests in test/live-api-resilience.test.ts continue to pass unmodified.
• New unit tests cover the batch runner's concurrency bound and the bulk-assign flow's circuit-open handling.
• npm run typecheck, npm run lint, npm test pass.

Likely Affected Files/Directories

• app/admin/members/page.tsx
• lib/api/live.ts
• lib/api/batch-runner.ts (new)
• components/ui/bulk-action-toolbar.tsx
• test/live-api-resilience.test.ts
• test/ (new test file for the batch runner and bulk-assign behavior)

---

[Fale1278]

Hi! I'd love to work on this issue.

I have experience managing API traffic and preventing app slowdowns during large bulk updates. Adding smooth batching and smart error handling to this bulk role assignment feature is clear, and I can implement it cleanly.

I'll start by building a simple helper to process member updates in small batches instead of all at once. Before starting each batch, the app will check if the server is struggling. If it is, the app will pause sending requests and clearly mark remaining members as "skipped" instead of showing confusing server errors. Finally, I'll update the results UI so admins can easily see who was updated, who failed, and retry anyone who was skipped.

ETA: less than 24 hours

[grantfox-oss]

🦊 GrantFox — @Fale1278 has been assigned to this issue as part of the Official Campaign | FWC26 campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #308)
2. Your PR will be reviewed by the Adamantine-guild maintainers

Good luck! Track your progress on GrantFox.