-
-
Notifications
You must be signed in to change notification settings - Fork 97
Expand file tree
/
Copy pathwrangler.data.jsonc
More file actions
687 lines (686 loc) · 38.3 KB
/
Copy pathwrangler.data.jsonc
File metadata and controls
687 lines (686 loc) · 38.3 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
{
// Dedicated Postgres-serving Worker. Reached only via the main Worker's
// DATA_API service binding (no public routes of its own).
// Deploy: wrangler deploy -c wrangler.data.jsonc
//
// WHY THIS IS STILL A SEPARATE WORKER — and it is no longer the reason first
// written here (#10619).
//
// The original claim was ADR 0013 plus "so the postgres.js driver + growing
// read surface don't eat the main Worker's bundle budget". Both have expired:
// ADR 0013 is SUPERSEDED (by ADR 0014, Postgres cutover), and the budget
// argument is measurably not binding. Measured 2026-08-11:
//
// metagraphed 944.8 KiB gzipped 9.2% of the 10 MB paid limit
// metagraphed-data-api 384.3 KiB gzipped
// both, merged ~1.3 MB gzipped ~13%
//
// Nor is placement the differentiator — both Workers are `mode: "smart"`.
// On size alone these could be one Worker today.
//
// What keeps them apart is BLAST RADIUS, not bytes. This Worker holds every
// read AND every write route into the one Postgres instance, on its own cron
// schedule and its own queue consumer. Behind a service binding, a bad deploy
// or a runaway query here cannot take api.metagraph.sh down with it, and it
// redeploys without touching the public API. Merging would also put
// DB-latency-bound work in the same isolate as the edge API and add this
// Worker's crons to a main Worker already dispatching 37 of them.
//
// So: keep the split, but keep it for the reason that is actually true. If
// the day comes that the isolation stops earning its keep, the merge is
// mechanically easy -- which was NOT true when the bundle argument was
// written, and is the part worth re-checking rather than assuming.
//
// Also owns every write route into this same Postgres instance (#4771's
// neurons-sync, subnet-hyperparams-sync, account-identity-sync,
// subnet-identity-sync, health-checks-sync, health-uptime-rollup-sync,
// subnet-snapshot-sync, rpc-usage-sync, subnet-locks-sync (#6638),
// chain-detail-sync (#9208), subnet-ownership-sync (#10836) -- see
// workers/data-api.ts's handle*Sync
// functions) -- each requires its own secret, set via:
// wrangler secret put <NAME>_SYNC_SECRET -c wrangler.data.jsonc
//
// CHAIN_DETAIL_SYNC_SECRET (#9208) is the one that must be set on BOTH
// Workers: the main Worker proxies the route (so it never checks the value)
// but the producer POSTs through it, and data-api is where the check
// happens. It also gates the head GET the producer resumes from.
// (and, for the ones called from the main Worker's own cron rather than an
// external GitHub Actions workflow, the SAME secret value on the main
// Worker too -- see wrangler.jsonc.)
//
// #4984 Part 1's chain_alert_triggers CRUD (workers/data-api.ts's
// handleAlertTrigger* functions) needs three MORE secrets, none yet
// provisioned on any deployment:
// wrangler secret put ALERT_TRIGGER_CREATE_TOKEN -c wrangler.data.jsonc
// wrangler secret put ALERT_TRIGGERS_INTERNAL_TOKEN -c wrangler.data.jsonc
// (ALERT_TRIGGER_CREATE_TOKEN gates public trigger creation, anti-abuse,
// like METAGRAPH_WEBHOOK_SUBSCRIPTION_TOKEN; ALERT_TRIGGERS_INTERNAL_TOKEN
// gates the #4984 Part 2 evaluator's active-trigger list scan. There is no
// third secret to provision here -- per-trigger owner tokens are minted
// per row by the CREATE route itself, not a deployment secret.) Until
// ALERT_TRIGGER_CREATE_TOKEN is set, POST /api/v1/alerts/triggers 503s;
// the table/routes are otherwise fully live once this file's schema.sql
// is applied.
//
// Wallet-signature login + self-serve fullnode/freemium API keys
// (src/wallet-auth.ts + workers/data-api.ts's handleWallet*/
// handleAccountKeys*/handleApiKeyVerify/handleAccountTierPromote
// functions; Unkey is the actual key store since the 2026-07-19 rework --
// src/unkey-client.ts) need secrets + one plain var:
// wrangler secret put WALLET_SESSION_SECRET -c wrangler.data.jsonc
// wrangler secret put UNKEY_ROOT_KEY -c wrangler.data.jsonc
// wrangler secret put API_KEY_LOOKUP_INTERNAL_TOKEN -c wrangler.data.jsonc
// wrangler secret put ACCOUNT_TIER_PROMOTE_INTERNAL_TOKEN -c wrangler.data.jsonc
// (WALLET_SESSION_SECRET signs the key-management session token;
// UNKEY_ROOT_KEY must be scoped to ONLY this deployment's one Unkey API/
// keyspace (UNKEY_API_ID below) with exactly api.<id>.{create,verify,
// update,delete,read}_key -- never the account-wide api.*.* wildcard, and
// never the account's default/master root key;
// API_KEY_LOOKUP_INTERNAL_TOKEN gates the internal key-verify route the
// fullnode RPC gate's validator calls -- the SAME value must also be set
// on the main Worker, wrangler.jsonc, see that file's own comment;
// ACCOUNT_TIER_PROMOTE_INTERNAL_TOKEN gates the ops-only tier-promotion
// route, called manually after confirming out of band that an account
// should move off the 'free' default -- no self-serve path to a higher
// tier exists by design.) There is no invite-code gate anymore: every
// wallet-connected account can self-serve a key immediately.
// Until WALLET_SESSION_SECRET is set, POST /api/v1/auth/wallet/verify and
// every /api/v1/keys route 503; the rpc_accounts/api_keys.account_id
// schema is otherwise live once this file's schema.sql is applied.
//
// Chain-wide account balance snapshot (#6741/#6742, the balance-based
// top-holder leaderboard epic's data tier) needs:
// wrangler secret put ACCOUNT_BALANCES_SYNC_SECRET -c wrangler.data.jsonc
// Set on metagraphed-infra's account-balances data-refresh job too
// (roles/data-refresh-cron's vault, same secret_var convention every
// other sync job uses). Until set, POST /api/v1/internal/account-balances-sync
// 503s; the account_balances table is otherwise live once this file's
// schema.sql is applied.
"$schema": "node_modules/wrangler/config-schema.json",
"name": "metagraphed-data-api",
// metagraphed#7766: Sentry's deploy-entry wrapper removed (Sentry fully
// decommissioned) -- "main" now points straight at the raw handler.
"main": "workers/data-api.ts",
// Reached only through the main Worker's DATA_API service binding. Wrangler
// defaults workers.dev to enabled when a Worker has no public routes, so keep
// both public and preview URLs disabled for this private data tier.
"workers_dev": false,
"preview_urls": false,
"compatibility_date": "2026-06-06",
"compatibility_flags": ["nodejs_compat"],
// Cloudflare's own per-deploy version UUID (metagraphed#7766: no longer
// read for Sentry release tagging). upload_source_maps makes Wrangler
// generate + upload sourcemaps for this Worker's own deploy, feeding
// PostHog's error-tracking source-map resolution.
"version_metadata": {
"binding": "CF_VERSION_METADATA",
},
"upload_source_maps": true,
// Public identifier for this deployment's one Unkey API/keyspace -- not a
// secret (see UNKEY_ROOT_KEY above for the actual credential), just an
// opaque id every src/unkey-client.ts call needs to scope its request to.
"vars": {
// #10659: lanes whose Neon writes are deferred through the write-behind
// buffer and flushed every ten minutes, so the compute does fewer, larger
// writes and autoscaling sits nearer the 0.25 CU floor.
//
// THE SAME VALUE IN EVERY GATED CONFIG (the dual-write flag that sat beside
// it followed D1 out in #10051). Each Worker ignores the lanes it does
// not run, and
// tests/neon-flag-config-parity.test.ts enforces the agreement -- a per-
// config subset would be exactly the drift that made two Neon gates fail
// silently (#10152, #10164), because a flag absent from the Worker actually
// running the code reads as "no" rather than as a mistake.
//
// WHAT IS ABSENT IS THE POINT. `blocks-head` and `chain-detail` are the
// block explorer's live read path -- blocks_head for headers above the
// decode seam, chain_detail_* for a recent block's detail -- so deferring
// either would lag the explorer. NEVER_BUFFER_LANES in
// src/neon-write-buffer.ts makes naming them a no-op regardless; they are
// left out so this list reads as what it is.
//
// Every lane below is held to `2 * HOUR` by src/table-freshness-watchdog.ts,
// so a ten-minute flush is twelve ticks inside a bound that already exists.
"NEON_WRITE_BUFFER_LANES": "raw-capture-state,neurons,subnet-hyperparams,subnet-identity,account-identity,nominator-positions,account-balances,hotkey-alpha,validator-nominator-counts,tao-usd-index",
"UNKEY_API_ID": "api_2Dv8cVjrTSiD",
// Tier-source flags. These are READ by this Worker -- neuronsServedFromD1()
// and its two siblings in workers/data-api.ts gate on `!== "postgres"` --
// but they were declared only in wrangler.jsonc, which configures the MAIN
// Worker. `vars` is replaced wholesale on deploy, so over here they were
// always `undefined`: the comparison happened to yield the intended "serve
// from D1", and the documented 503 fall-through was unreachable.
//
// Declared here at the SAME values wrangler.jsonc carries, so the two
// Workers cannot disagree about which tier answers. Change them together:
// flipping one side alone is the split-brain this pins shut -- the main
// Worker would stop forwarding while this one kept serving D1.
"METAGRAPH_NEURONS_SOURCE": "data-api",
// metagraphed-infra#336, step one of the order #9711 set out.
//
// THE MIRROR IS ON; NOTHING READS FROM IT. `neurons` now writes D1 first
// and Neon after, for itself plus the two tables handleNeuronsSync derives
// (neuron_daily, account_position_daily). Every route still reads D1, so a
// Neon that refuses writes costs a mirror and a `neon:` lane verdict -- and
// nothing a caller can see.
//
// A companion NEON_READ_LANES flag held the reads back until those
// verdicts had been green across many 15-minute producer ticks AND the two
// stores had been reconciled row for row. Reading before that is exactly
// #9704, and "the mirror is deployed" is not evidence that it wrote
// anything. It is gone (#10051): with D1 deleted it named the only store
// there is, so it could refuse a read but no longer redirect one.
//
// Schema note: `neuron_daily` was created in Neon BY HAND on 2026-08-07,
// which is the practice migrations/neon/ + the `Neon migrate` workflow now
// end (#9814). This comment is why: it claimed SMALLINT for the 0/1 CHECK
// columns and information_schema reports BOOLEAN for every one of them, so
// the only written record of the schema had drifted from the schema. Read
// migrations/neon/*.sql instead. Postgres types otherwise as described --
// BIGINT for the epoch-millisecond columns -- and the
// same PRIMARY KEY (netuid, uid, snapshot_date) the mirror's ON CONFLICT
// names. An ON CONFLICT with no unique index behind it is a runtime error,
// not a slower query.
// Tables whose ONLY home is Neon -- D1 is not behind them any more.
//
// A FOURTH flag, and the one the other three converge on. All three above
// describe a table that LIVES IN D1 and is being shadowed: the dual-write
// list says new writes also reach Neon, the backfill list says older ones
// do too, the read list says reads are served from the copy. Each of them
// presumes a D1 original still exists, so none of them can express the end
// state, and the end state is the goal (#9787).
//
// It also fits a class of table the other three do not. The user-state
// tier -- accounts, keys, quotas, alerts, push subscriptions -- is written
// by REQUEST HANDLERS, not lanes. There is no producer tick to mirror on
// and no second writer to reconcile against, and the whole tier is ~1,200
// rows, which one statement copies. Those tables move by copying once,
// repointing the runner, and never writing D1 again; a mirror for them
// would mean operating two stores to keep a table honest that has exactly
// one writer.
//
// EMPTY UNTIL A COPY IS VERIFIED, and per table. workers/data-api.ts's
// userStateRunner requires EVERY table in a group to be named here before
// it hands out a Postgres runner -- the same all-or-nothing rule the
// per-route read gate used to apply before #10051 deleted it, because the
// helper picks one runner for the whole callback and a half-listed group
// would send a statement to a store where its table does not exist.
//
// Rollback is deleting a name. When this holds every table D1 has, the D1
// binding comes off.
//
// THE FIRST TEN (#9987). The whole user-state tier, moved together because
// its two runner-acquiring helpers are the unit that moves -- see
// userStateRunner. Copied and verified before this line changed, and
// verified structurally rather than by row count alone: github_accounts
// 3/3, rpc_accounts 2/2, api_usage_rollup 1162/1162, the other seven empty
// on both sides. The two identity sequences were setval-ed past D1's max,
// because an explicit insert does not advance them and the next mint would
// otherwise collide on id 1.
//
// ORDERED AFTER #9992, and that ordering is load bearing. node-postgres
// returns int8 as a STRING where D1 returns a number, and this tier's
// schema is BIGINT-heavy by necessity -- epoch-ms does not fit in int4.
// Setting this flag before that parser existed would have turned
// created_at, units_spent, request_count, github_user_id and every other
// *_at column into strings on the wire, with nothing failing anywhere.
//
// THE FIRST PRODUCER LANE (#10037). Everything above is written by a
// request handler; these three are written by a producer, and they are the
// first lane whose D1 write stops entirely.
//
// Earned on parity held ACROSS TICKS, not on one snapshot. Sampled twice
// ~20 minutes apart on 2026-08-08, spanning a producer pass: neurons
// 30,118/30,118 both times, neuron_daily 877,039/877,039 both times, and
// account_position_daily moving 864,953 -> 864,983 on BOTH sides together.
// A mirror that keeps pace THROUGH a pass is the evidence that matters; a
// single equal count only says the last backfill finished.
//
// What changes: handleNeuronsSync skips the D1 write outright rather than
// writing and ignoring it, and the Neon write becomes authoritative -- any
// table's failure is now the request's failure. A pass that did not reach
// the store did not happen.
"METAGRAPH_SUBNET_HYPERPARAMS_SOURCE": "data-api",
"METAGRAPH_ACCOUNT_IDENTITY_SOURCE": "data-api",
// THE QUEUE CUTOVER (metagraphed-infra#348). Comma-separated lanes whose D1
// write goes through sync-batches instead of landing inline on the request.
// One lane at a time; adding a name routes it, removing a name routes it
// back, and neither is a code change.
//
// NEVER BOTH PATHS AT ONCE. The flag selects, it does not fan out --
// dual-writing would double the D1 load that caused the saturation, and
// duplicate arrivals would corrupt the completeness tally.
//
// Declared INSIDE this block deliberately: a second top-level "vars" key
// would silently replace this one wholesale, which is the trap the tier-flag
// comment above already records.
//
// THE REMAINING SIMPLE LANES, cut over 2026-08-06 on the strength of
// account-balances' first queued pass rather than on hope:
//
// 364,644/364,644 in 518s with ZERO producer errors, against 993s and
// 2,668 errors on the inline pass six hours earlier.
//
// Those 2,668 were the producer retrying through a saturated D1. The
// queue's max_concurrency: 2 removed them, which is the backpressure claim
// this migration was making and could not previously test. The 364k-row
// lane is the largest of the four, so the three joining it here are being
// cut over on a STRONGER result than any of them could produce alone.
//
// ATTRIBUTION SURVIVES THE BATCH because the timers separate them:
// hotkey-alpha and validator-nominators are both 86400 and account-balances
// is 21600, so their passes land hours apart and an incident points at one
// lane on its own. That is why this does not violate the epic's
// one-lane-at-a-time rule so much as satisfy its reason.
//
// nominator-positions REJOINS after #9684. It was rolled back on 2026-08-06
// because its largest coldkey holds 722 positions -- 141 KB at a MEASURED
// 200 bytes a row -- and packSyncBatchMessages correctly refused to split a
// coldkey, so the route 502'd on the first tick.
//
// Two things were wrong and both are fixed. The byte budget was doing two
// jobs: 96 KB governs COMBINING groups, while a single indivisible group
// only has to fit the 128 KB transport. And even alone that group did not
// fit -- so its coldkey is now HOISTED off the rows (it is identical on
// every row by construction), which removes ~62 bytes a row and brings the
// message to ~97 KB. syncBatchRows re-injects before the writer sees it.
//
// THE RESIDUAL LIMIT IS REAL: hoisting buys ~31%, not immunity. A coldkey
// past roughly 950 positions overflows again, with the same loud 502.
//
// Still the one to watch, because it PRUNES
// per coldkey, so a
// message missing rows for a coldkey it names deletes rows it never
// carried, and no retry undoes a delete. Three things have to hold at once:
// pack_coldkey_chunks never splits a coldkey across POSTs (producer,
// tested), the packer never splits one across messages (route, tested), and
// the consumer refuses any message that does not assert key_complete. The
// first queued pass is the first time all three run together.
//
// WHAT "NO PASS YET" MEANS HERE. These lanes are timers, not streams. An
// absence of rows is not a failure until it exceeds the interval -- reading
// one as the other is what turned a preventive rollback into a reported
// incident earlier today.
//
// ROLLBACK IS DROPPING A WORD FROM THIS STRING. These are latest-only
// upsert tables refreshed on a tick, so the next pass simply writes the old
// way. No backfill, no reconciliation. Watch the *_passes tables: an
// INCOMPLETE pass is the signal -- received_rows short of expected_rows with
// a null completed_at -- and completeness is a fact the queue does not
// itself provide.
// chain-detail JOINS, and it is the one the whole migration was for.
//
// It is the largest D1 writer here and the only CONTINUOUS one: ~1,245 rows
// every 12 seconds is ~9M rows/day, against account-balances' ~1.5M. The
// bulk lanes are bursty; this one never stops. #346's argument for one
// queue was global backpressure -- "a lane left out is a lane that can
// still overwhelm the database the others are being polite about" -- and
// this was that lane.
//
// IT COULD NOT TRAVEL RAW, at any producer setting. Its four families are
// posted together so a block and its extrinsics cannot land separately, so
// the batch is already ONE BLOCK -- and one block measured 476.6 KiB of
// JSON (#8790494, the busiest captured) against a 128 KiB cap. Not a
// batching problem, a units problem: the same class as #360's rows-vs-bytes.
//
// Three things had to land first, and all three are deployed:
//
// metagraphed#9765 compress the message. 476.6 KiB -> 40.5 KiB,
// 11.8x, and the budget now measures the
// COMPRESSED size, which is the only number
// the transport sees.
// metagraphed#9718 the consumer's batch log read `rows` on a
// message that has none -- above the
// per-message try/catch, so one families
// message would have failed the whole batch,
// five retries, into the DLQ, taking these
// four lanes with it.
// metagraphed-infra#386 the producer posts ONE block per POST, not
// two. Two compress to 64.9 KiB against a
// 96 KiB budget -- and that pairs the busiest
// block with a QUIET neighbour. Two genuinely
// busy blocks land near 81 KiB, ~16% margin,
// on a lane where overflow does not degrade:
// the producer advances its cursor only on a
// POST that succeeded, so it WEDGES.
//
// WHAT TO WATCH, and it is not the row count. This lane declares no
// pass_total, so there is no completeness tally to read -- the signal is
// `chain-detail` in lane_health staying `ok` at its ~5-minute cadence with
// `1 scanned, 1 written, 0 error(s)`, and `chain_detail_blocks` continuing
// to advance. A wedge shows up as the SAME block retried, which the
// staleness watchdog and lane-alarm both reach.
//
// ROLLBACK IS DROPPING THE WORD. Unlike account-balances, this lane still
// has its inline D1 write, so removing it here restores the old path on the
// next tick with no backfill.
"SYNC_QUEUE_LANES": "account-balances,hotkey-alpha,validator-nominator-counts,nominator-positions,chain-detail",
// #9430's $exception storm guard reads this var per-Worker
// (src/usage-telemetry.ts), and `vars` are per-config -- declaring it
// only in wrangler.jsonc left THIS Worker's captures entirely unguarded,
// and this is the Worker where the 871,649-event `wallet-auth-keys`
// storm actually happened (see wrangler.jsonc's narration). Same value
// as the main Worker: one capture per fingerprint per 5 minutes per
// isolate, suppressed count carried on the next admitted event.
"POSTHOG_EXCEPTION_STORM_WINDOW_MS": "300000",
// #9466: REST trace sampling, finally non-zero on this Worker.
//
// #9000 set POSTHOG_TRACES_SAMPLE_RATE_MCP in wrangler.jsonc and left the
// general rate unset, which reads as "REST stays dark" -- but `vars` are
// per-config, so what it actually left unset was the rate on THIS config
// too. src/tracing.ts defaults to 0, so the sampled branch in this
// Worker's fetch (workers/data-api.ts) has been dead code since #7768.
//
// THE ARITHMETIC, measured 2026-08-04 from Cloudflare's own
// workersInvocationsAdaptive (30-day window, per scriptName). NOT from
// usage_event: this Worker emits none -- recordUsageEvent is called only
// from api.ts and the five hub Workers -- which is also why these spans
// are worth having at all.
//
// peak day 1,032,820 requests (2026-07-31)
// after the wipe 49,670 requests (2026-08-04)
//
// That 20x spread inside four days is structural, not a lull, and both
// steps are ours: #9186 (08-02) unbound HYPERDRIVE, #9411 (08-04) stopped
// the main Worker forwarding chain-events here. Confirmed by the hourly
// data-api:main ratio collapsing 0.85 -> 0.20 -> 0.08 while the main
// Worker's OWN hourly volume rose -- a quiet window would have moved both.
//
// So the rate is sized against the PEAK, not the current floor: this
// Worker's volume has already moved 20x once, and a backfill or a
// re-tiered read path moves it back with no config change to catch it.
// Against ~600K spans/month of headroom (1M free tier less the ~400K/month
// wrangler.jsonc measured on 2026-08-02):
//
// 0.02 -> 620K/month at peak -- over the headroom, rejected
// 0.01 -> 310K/month at peak, ~15K/month at current volume
//
// 0.01 is simply the largest round rate that still fits on the worst day
// observed. It buys per-route p50/p95 for the tier the main Worker's
// usage_event can only measure from the outside (that timing includes the
// DATA_API subrequest, so tier latency is currently unattributable).
// Re-derive from measurement -- not estimate -- if volume shape moves.
//
// 0.01 -> 0.002, because the headroom line above is off by 10x. Spans are
// NOT product-analytics events: PostHog bills generations, spans and
// traces against AI Observability, 100K events/month free
// (posthog.com/ai-observability/pricing). There is no "~600K spans/month
// of headroom" -- the entire allocation is 100K, so "0.01 -> 310K/month at
// peak" was never 'fits', it was 3.1x the whole tier from this Worker
// alone. 0.02 was rejected for being 620K; both numbers were compared
// against a budget that does not exist.
//
// MEASURED 2026-08-10: 2,657 of this Worker's ~2,900 spans over 2.6 days
// were /api/v1/internal/usage-rollup -- 92% of its trace volume spent on
// machine-to-machine plumbing. src/tracing.ts now excludes internal routes
// from SUCCESS spans (#9005's line, finally drawn for the trace lane), so
// most of that is gone before this rate is even consulted.
//
// 0.002 re-derived against the REAL tier and the SAME peak this comment
// already refuses to size below:
//
// peak 1,032,820 req/day x 0.002 -> ~2.1K spans/day, ~64K/month
// current 49,670 req/day x 0.002 -> ~10 spans/day
//
// The peak case now fits inside 100K alongside MCP's ~14K; the current
// case is admittedly thin. That trade is only acceptable because the
// sample no longer carries the incident signal: shouldRecordTraceSpan
// keeps every 5xx and every uncaught fault at rate 1, storm-guarded. This
// rate governs latency percentiles on healthy traffic and nothing else.
"POSTHOG_TRACES_SAMPLE_RATE": "0.002",
},
// Smart placement: run the Worker close to the Hyperdrive origin (self-hosted indexer
// box Postgres, reached via Cloudflare Tunnel) to minimize the per-request DB round-trip,
// not at the nearest edge to the client.
"placement": { "mode": "smart" },
// Full observability — logs AND traces (matches the main Worker).
"observability": {
"enabled": true,
"head_sampling_rate": 1,
"logs": {
"enabled": true,
"head_sampling_rate": 1,
"invocation_logs": true,
},
"traces": {
"enabled": true,
"head_sampling_rate": 1,
},
},
// #8600: the TAO/USD index tick (ADR 0025 decision 5's 60-second cadence,
// which is also Cloudflare's finest cron granularity). This Worker's only
// cron. (Historically here because HYPERDRIVE was bound here; the index
// itself lives in D1, so the placement outlived the binding.) See
// TAO_USD_INDEX_CRON in workers/config.ts.
//
// ETH_RPC_URL is a Worker SECRET, not a var, and not in this file:
//
// npx wrangler secret put ETH_RPC_URL --config wrangler.data.jsonc
//
// Every provider whose terms actually permit programmatic access
// authenticates by embedding the key IN THE URL
// (https://eth-mainnet.g.alchemy.com/v2/<key>), so the endpoint and the
// credential are the same string. A "vars" entry here would commit it.
//
// No committed default either: unset, the tick is a recorded no-op rather
// than a silent fallback to somebody's public node. See
// workers/env-extra.d.ts for why that is a choice and not an oversight.
// EVERY EXPRESSION HERE RESOLVES TO A BRANCH in this Worker's `scheduled`,
// and tests/data-api-crons-have-handlers.test.ts asserts it. Three did not
// until #10814: "*/3 * * * *" (NEON_BACKFILL_CRON), "26 * * * *" (the
// mirror-lag watchdog) and "38 * * * *" (D1<->Neon row-count parity). All
// three producers were deleted with D1 -- `src/neon-backfill.ts`,
// `src/neon-mirror-lag.ts`, `src/neon-parity.ts` are gone and their lanes are
// in RETIRED_LANES -- so the expressions fell through to
// `{ skipped: true, reason: "unknown cron" }` roughly 23 times an hour.
//
// Nothing caught that because every cron gate in tests/ ran one direction:
// constant -> declared here. A constant that no longer exists cannot fail
// that test, which is precisely the shape of a check that passes hardest
// when there is nothing to check.
//
// Cloudflare delivers one event per matching expression with its own
// `controller.cron`, so the handler dispatches on the string rather than on
// what minute it happens to be -- which is why a subset expression like
// "*/3 * * * *" was a separate entry rather than folded into "* * * * *".
"triggers": {
"crons": ["* * * * *", "52 * * * *", "46 * * * *"],
},
// HYPERDRIVE is gone with the box (2026-08-03): the Postgres it fronted was
// wiped, and every handler already gates on `env.HYPERDRIVE?.connectionString`
// -- but with the binding present, each sync PAID for the dead leg: the
// neurons sync wrote D1 successfully and then 502'd "write failed" against a
// connection string pointing at destroyed hardware, which made every
// refresh-metagraph run read as a failure while the data was in fact
// landing. Unbound, the writes are D1-complete 200s and the reads fall
// through to the D1/lakehouse tiers, which is the post-wipe design.
// The SAME bounded D1 database the main Worker binds under the same name
// (wrangler.jsonc): registry + observations + user-state -- accounts, API
// keys, usage accounting, alert triggers, push subscriptions, and the
// TAO/USD index (tests/fixtures/sqlite-schema/0004_user_state.sql). Chain-scale data
// (blocks/extrinsics/chain_events/...) never goes to D1; those reads live
// in the R2 lakehouse cold tier.
// The bulk sync path (metagraphed-infra#346/#347). Producers enqueue chunks
// instead of POSTing them at a sync route; this Worker consumes and writes.
//
// WHY. Each bulk lane grew its own substitute for what a queue provides --
// retry loops, a 1-second inter-chunk sleep standing in for backpressure, a
// pass tally standing in for "did the batch set drain". Every piece was a
// correct local fix for the D1 saturation on 2026-08-05
// (`D1_ERROR: D1 DB is overloaded`, which also failed wallet-auth and
// tao-usd-index), and together they are an orchestration layer nobody
// designed.
//
// NO DUAL WRITE, ever. A lane enqueues OR posts, never both -- writing
// everything twice during a migration doubles exactly the load that caused
// the incident, and duplicate arrivals corrupt the completeness tally that
// stopped a partial ledger publishing a leaderboard missing its #2.
//
// max_batch_size / max_concurrency ARE the backpressure. They replace the
// hand-rolled sleep, and are deliberately conservative until measured against
// real traffic: the failure this exists to prevent is overwhelming D1, so
// starting slow and raising on evidence is the whole point.
"queues": {
"producers": [{ "binding": "SYNC_BATCHES", "queue": "sync-batches" }],
"consumers": [
{
"queue": "sync-batches",
"max_batch_size": 10,
"max_batch_timeout": 5,
"max_retries": 5,
"max_concurrency": 2,
"dead_letter_queue": "sync-batches-dlq",
},
// THE DEAD LETTER, finally read (metagraphed-infra#354/#363). It was
// declared above from the first deploy and consumed by nothing, so the
// one failure the migration to a queue was meant to make visible -- a
// chunk that exhausted every retry -- expired silently on the queue's
// own retention.
//
// SAME WORKER, and the handler branches on `batch.queue` BEFORE anything
// else. Without that branch this binding would hand a five-times-failed
// message back to the writer, which is a sixth attempt at the write that
// killed it.
//
// "max_retries": 0 because there is nothing to retry: the reader acks and
// records, and a failure to record must not put the message back. No
// dead_letter_queue of its own for the same reason -- a DLQ for the DLQ
// is a second place nobody looks.
{
"queue": "sync-batches-dlq",
// Bigger batch and a longer timeout than the live queue: this path is
// not latency-sensitive, and a burst of dead letters is exactly when
// reading them one small batch at a time is least useful.
"max_batch_size": 100,
"max_batch_timeout": 30,
"max_retries": 0,
"max_concurrency": 1,
},
],
},
// Neon Postgres, via Hyperdrive (metagraphed-infra#336). NOT the return of the
// box: HYPERDRIVE was unbound in #9186 because the Postgres it fronted was
// wiped, and the two Hyperdrive configs that pointed at it have since been
// deleted. This is a NEW config against a managed database, bound for one
// table.
//
// WHY ANY OF THIS. Bulk sync passes saturate D1 -- `D1_ERROR: D1 DB is
// overloaded. Requests queued for too long.`, which on 2026-08-05 aborted the
// account-balances passes AND failed unrelated writers (wallet-auth,
// tao-usd-index) on the database everything shares. The neurons family is the
// dominant write load: one 15-minute tick writes ~30k rows to EACH of
// `neurons`, `neuron_daily` and `account_position_daily`, so ~8.6M row-writes
// a day.
//
// `account_position_daily` goes first because it carries a third of that load
// behind the smallest read surface in the family -- two query sites in one
// file, against `neurons`' forty across eighteen. Same relief, a twentieth of
// the blast radius. The D1 table is deliberately LEFT IN PLACE: rollback is a
// revert, not a restore.
//
// The id is a Hyperdrive config, not a connection string -- the credentials
// live in Cloudflare, exactly as `database_id` above is not a D1 credential.
// Never add `localConnectionString` here: Neon's own guide suggests it and it
// would commit a live password to a public repository. Use `.dev.vars`, which
// IS gitignored, for local work.
// REBOUND 2026-08-07, with the two questions finally separated.
//
// It was unbound hours earlier (#9704) because the Neon pilot moved a READ
// without ever building a WRITE: `account_position_daily` got ONE load on
// 2026-08-05 at 19:35, nothing wrote to it after, and
// GET /api/v1/accounts/{ss58}/subnets/{netuid}/history served a snapshot two
// days old and aging -- 26 points to 08-05 where D1 had 27 rows to 08-07.
//
// THE BINDING WAS NEVER THE PROBLEM. The read gate was
// `env.HYPERDRIVE && ...`, so a connection provisioned for a WRITE pilot
// silently moved a public route's read. "This Worker can reach Neon" and
// "this route should read Neon" were different questions, and a separate
// NEON_READ_LANES flag answered the second one: a lane was named in the
// dual-write flag first (also gone, #10051), its `neon:` verdict in
// lane_health watched green across several producer ticks, and only then
// named for reading.
//
// That ordering mattered while D1 held the other copy. It does not now
// (#10051). With one store the two questions have collapsed back into one,
// and the read flag has been deleted rather than left to answer a question
// it can no longer get wrong in the interesting direction -- it could refuse
// a route, never send it somewhere better.
"hyperdrive": [
{
"binding": "HYPERDRIVE",
"id": "8a09dc673733497f987434f59689287e",
},
],
// #10659: the write-behind buffer, whose class is OWNED BY THE metagraphed
// Worker (wrangler.jsonc declares the class and its migration). `script_name`
// is what makes this a reference rather than a second, independent instance
// -- and a second instance would be the whole bug: two buffers mean two
// alarms drifting apart, waking the compute twice per window and halving the
// gap that was the point. No "migrations" entry belongs here; the owning
// Worker already has it.
"durable_objects": {
"bindings": [
{
"name": "NEON_WRITE_BUFFER",
"class_name": "NeonWriteBufferHub",
"script_name": "metagraphed",
},
],
},
// Wallet-login challenge nonces (ADR 0021, #6835, src/wallet-auth.ts) --
// the SAME namespace id as the main Worker's own METAGRAPH_CONTROL binding
// (wrangler.jsonc), bound under the same name here so both Workers read/
// write the identical KV namespace (Cloudflare namespaces can be bound to
// multiple Worker scripts; this is not a second namespace).
"kv_namespaces": [
{
"binding": "METAGRAPH_CONTROL",
"id": "9c5577a3454d4830b2ffd2f83775efb1",
},
],
// Found by adversarial review (#4984 Part 3): ALERT_TRIGGER_CREATE_TOKEN
// alone doesn't bound REQUEST VOLUME from a legitimate token holder --
// every created row becomes a permanent per-event cost in AlerterHub's
// O(active triggers) match scan. Namespace id is local to THIS Worker's
// own bindings (Cloudflare's Workers-native rate-limiting API, not the
// older account-wide Rate Limiting Rules product) -- reusing "1001" here
// does not collide with wrangler.jsonc's own RPC_RATE_LIMITER using the
// same id, since ratelimit namespaces are scoped per Worker script.
// Skipped when unbound (local dev/CI), matching every other optional
// rate-limiter's convention in this codebase.
"ratelimits": [
{
"name": "ALERT_TRIGGER_CREATE_RATE_LIMITER",
"namespace_id": "1001",
"simple": {
"limit": 10,
"period": 60,
},
},
// Wallet-login abuse control (ADR 0021, #6835) -- an unauthenticated
// caller can hit challenge/verify before any session exists, so this is
// keyed by client IP, same posture/figure as ALERT_TRIGGER_CREATE_RATE_LIMITER
// above. Skipped when unbound (local dev/CI).
{
"name": "WALLET_AUTH_RATE_LIMITER",
"namespace_id": "1002",
"simple": {
"limit": 10,
"period": 60,
},
},
// Per-account key-mint abuse control -- with no invite-code gate at all,
// "you can sign in as this wallet" is the only bound on WHO can mint;
// this is what stops one signed-in account from minting rows in a tight
// loop (the same gap adversarial review found in
// ALERT_TRIGGER_CREATE_TOKEN alone), keyed by account id rather than IP.
// Skipped when unbound (local dev/CI).
{
"name": "ACCOUNT_KEYS_MINT_RATE_LIMITER",
"namespace_id": "1003",
"simple": {
"limit": 10,
"period": 60,
},
},
],
}