fix(masspay): survive CoinPay's rate limit instead of writing off invoices

[ralyodio]

The bug

A bulk pay run reported 30 accepted invoices as un-payable:

67af1bc1… — CoinPay create failed 429: Too many requests
c9c788e9… — CoinPay create failed 429: Too many requests
…

None of them were actually un-payable.

Why

CoinPayPortal's edge proxy limits every /api/ route to 60 requests per rolling minute per client IP (coinpayportal/src/proxy.ts). ugig.net calls it from a single server IP, so that budget is shared across every payment request, status poll and webhook the box makes — for all users at once.

/api/invoices/bulk-payment-request fires one payments/create per invoice at concurrency 5 with no pacing and no retry. A 62-invoice run exhausts the minute in seconds, createPayment throws on the resulting 429, and the bulk payer records each one as a permanent skip — rendered in the same list as "Already paid" and "Not accepted yet".

A 429 is transient by definition. Treating it as terminal is what turned a payroll run into 30 write-offs.

The fix

src/lib/coinpay-throttle.ts (new) — coinpayFetch, a paced + retrying fetch that every CoinPay call now goes through:

• Rolling-window pacer (45/min by default, under the provider's 60 so single-invoice flows and status polls still fit alongside a bulk run). Slot reservation is serialized so concurrent callers can't both claim the last slot.
• Retries 429 and 5xx, honouring Retry-After / X-RateLimit-Reset, with jittered exponential backoff as fallback.
• On a 429 it syncs to the provider's stated reset and holds every caller off — our local count is optimistic because it can't see the app's other CoinPay traffic.
• Retries dropped connections too, which fail a batch the same way.
• Only replays a positively-identified 429/5xx. Anything it can't classify is returned untouched — replaying a payment creation on a guess would mint a second live address for one debt.

Reporting — a rate-limited invoice comes back retryable: true with a reason that says so, instead of a raw provider string. The confirm step surfaces those separately and offers a one-click "prepare the N that were limited". Re-preparing reuses unexpired requests, so it costs one provider call per invoice actually still missing a quote.

Bounding — the batch shares one deadline (4 min) rather than a per-invoice budget. Quotes are only good for 15 minutes, so the last invoice must not still be waiting out a rate-limit window long after the first quote expired. Anything unprepared when it elapses returns as a retryable skip rather than holding the request open.

Behaviour change

A large batch now takes up to ~2 minutes to prepare instead of failing fast — it waits out the provider's window rather than sprinting into it. Well inside the 15-minute quote validity.

COINPAY_MAX_REQUESTS_PER_MINUTE and COINPAY_MAX_ATTEMPTS are env-tunable. The counters are per-process; railway.json pins numReplicas: 1, so one process is the whole outbound budget. If that ever scales out, divide the setting by the replica count — the provider counts replicas together and we can't see across them.

Verification

• 1810 tests pass across 198 files, including 10 new throttle tests and 3 new bulk-route regression tests covering the exact reported failure
• tsc --noEmit clean, eslint clean on changed files, production build succeeds
• Full precommit (lint + type-check + tests + build) passed on commit

Not done here

CoinPayPortal's 60/min-per-IP cap is low for a server-to-server integration, and this PR works around it rather than changing it. Raising it for API-key-authenticated callers would be the other half of the fix, but it weakens a rate limiter in a payments product, so it's left as a deliberate call rather than folded in silently.

🤖 Generated with Claude Code

[github-actions]

vu1nz Security Review

0 finding(s) in PR #?

No security issues found.