[Holesky] Blazar (v0.9.0-rc.0)

Summary

This release is required for users and operators to upgrade.

We are excited to release EigenDA v0.9.0, code name "Blazar" (aka. EigenDA V2), a major protocol and architecture upgrade for the entire system.

Key Features and Enhancements

• Control Plane & Data Plane Separation
  Separates the flow of blob data from its metadata/control as they move through disperser, encoders and validators. This improves end-to-end data movement efficiency and boosts validator node performance and robustness by increasing network utilization through concurrent download connections.

• Payments Support
  EigenDA now supports on-demand payments and reservations, enabling applications to access the network permissionlessly with flexible payment options.

• Faster and Cheaper Dispersal with DA Certificate
  Reduces end-to-end dispersal latency and cost by eliminating on-chain confirmation and returning a DA certificate representing DA nodes' attestations.

• New Chunk Allocation Model
  Introduces a fixed-chunk model, making encoding a stateless operation and reducing system complexity.

• Revamped Blob State Management
  Enhances robustness by enforcing immutable records and introducing constraints into the data model.

• Improved Blob Addressing
  Introduces Blobkey, a universally unique string identifier for each blob, replacing the previous composite key (batch header hash + blob index). This simplifies interactions and improves UX when working with blobs.

Further Details

• Executive Summary
• Payments Summary
• Validator Migration Guide

What's Changed

• Add latency metric for dispersal/retrieval with blob size breakdown by @jianoaix in #939
• [v2][node] Construct relay client from relay address from chain by @ian-shim in #931
• [v2] disperser server payments api by @hopeyen in #902
• GPU accelerated encoder by @dmanc in #895
• Remove duplicate flags from relay by @ian-shim in #944
• Add controller to Dockerfile by @yujiezhu0 in #952
• fix: skip socket registry in contract bindings by @hopeyen in #951
• Remove the unnecessary module by @jianoaix in #956
• Adds grpcurl/reflection support for v1 apis via disperser-v2 reflection service by @pschork in #954
• Add a new dataapi server by @jianoaix in #955
• Add controller to docker bake by @yujiezhu0 in #959
• fix: no local flags for skipping socket registry, only warn by @hopeyen in #964
• Automatically generate protobuf documentation by @cody-littley in #946
• Update RelayKey type to uint32 by @ian-shim in #963
• Add timeout to download chunks from node by @ian-shim in #941
• Add relay to docker build by @pschork in #957
• Relay metrics by @cody-littley in #938
• [v2] disperser client payments api by @hopeyen in #928
• v2 Retrieval Client by @ian-shim in #953
• Added metrics for the encoding manager. by @cody-littley in #967
• Add metrics to the dispatcher by @cody-littley in #968
• Revert changes to churner metrics. by @cody-littley in #970
• Support blob and batch fetch APIs by @jianoaix in #966
• [payment] transactor ping payment vault contract by @hopeyen in #827
• [ci] Disable graph indexer integration test by @ian-shim in #978
• [v2] Remove InsufficientSignatures blob state by @ian-shim in #936
• Register dispatcher metrics. by @cody-littley in #971
• Remove wrapper from relay by @cody-littley in #975
• Add metrics to v2 api server. by @cody-littley in #969
• [node] Don't store certs in store v2 by @ian-shim in #977
• V2 Blob Verification by @0x0aa0 in #781
• [v2] Offchain integrations with v2 contracts by @ian-shim in #960
• Write TestRandom util by @litt3 in #976
• payment struct and names ReservationPeriod and Salt by @hopeyen in #965
• [v2] Inabox test by @ian-shim in #958
• Pull out the operator handling logic that's shareable by @jianoaix in #981
• Payments storage packing by @0x0aa0 in #942
• fix: cumulative payment dynamo db unit conversion by @hopeyen in #979
• [v2] Update blob header hasher by @ian-shim in #962
• fix: typos in documentation files by @leopardracer in #989
• Update proto docs by @ian-shim in #993
• payments - global reservation uses interval config by @hopeyen in #980
• update blob verifier by @0x0aa0 in #992
• reservation timestamp check by @hopeyen in #990
• Update docker-bake.hcl by @yujiezhu0 in #996
• Support operator APIs by @jianoaix in #987
• fix: V2 merkle check by @0x0aa0 in #998
• chore: change from pk to random bytes by @shrimalmadhur in #1001
• Node metrics by @cody-littley in #948
• Separate out client v2 package by @ian-shim in #986
• [v2] inabox test retrieval from DA nodes by @ian-shim in #988
• Use hex representation of operator ID in dispatcher logs by @dmanc in #1002
• Create codeQL-scanning.yaml by @anupsv in #997
• [v2] integrate with new payments contract by @hopeyen in #995
• chore: move kzgpad main.go: kzgpad/cmd/main.go -> kzgpad/main.go by @samlaf in #1004
• Fix docker build for controller by @dmanc in #1009
• bump middleware to rewardsV2 by @0x0aa0 in #1008
• fix: make the api path naming plural by @jianoaix in #1013
• Handle nil operator ID in GetChunks() by @cody-littley in #1006
• Fix: return attestation err result to dispatcher by @ian-shim in #1011
• Disperser Registry by @0x0aa0 in #1019
• update contract bindings by @hopeyen in #1010
• refactor: refreshOnchainPaymentState arg by @hopeyen in #1012
• Assign relay proto its own package by @ian-shim in #1025
• fix: properly set fields without onchain correspondance by @hopeyen in #1014
• Nil checks in disperser v2 server constructor by @ian-shim in #1024
• Remove hardcoded localhost auth from relay authenticator by @ian-shim in #1007
• Add AlreadyExists error from disperser by @ian-shim in #1026
• Improvements to the test random library. by @cody-littley in #1029
• Regenerate bindings by @cody-littley in #1030
• init once accountant by @hopeyen in #1015
• [v2] Add quorum results to attestation by @ian-shim in #1018
• Sign get chunks by @cody-littley in #1022
• fix: global symbols per period and interval by @hopeyen in #1033
• Update dependabot.yml by @anupsv in #1027
• Refactor the metrics computing at dataapi by @jianoaix in #1039
• Fail v2 blobs for no attestations by @ian-shim in #1043
• Handle signature aggregation failure in dispatcher by @ian-shim in #1045
• symbols charged and payment metadata check by @hopeyen in #1034
• [v2] payments inabox by @hopeyen in #991
• Create util to verify KZG commitment by generation and comparison by @litt3 in #1042
• fix: v2 integration test flakiness by @ian-shim in #1041
• Add support for metrics APIs by @jianoaix in #1044
• Add support for blob certification and verification info lookup APIs by @jianoaix in #1056
• Simplify commitment generation, and choose better name for util file by @litt3 in #1063
• Dataapi v2 swagger minimal refactor by @pschork in #1065
• [v2] eth address as account id by @hopeyen in #1053
• Add missing objectives to SummaryVec in dispatcher by @dmanc in #1067
• Ignore .DS_Store files, which are generated by the OSX Finder app. by @cody-littley in #1070
• Update codeQL-scanning.yaml by @anupsv in #1020
• fix: payment metadata invalid if both payment fields empty by @hopeyen in #1073
• [v2] Retriever server by @ian-shim in #1048
• Compress relay auth signature by @ian-shim in #1068
• feat: add basic logging in disperser meterer by @hopeyen in #1069
• Simplify docker build by @cody-littley in #1071
• Enable relay authentication for GetChunks() requests in inabox. by @cody-littley in #1072
• dispersal dynamo db payment parsing fix by @hopeyen in #1074
• Add support for operator responses APIs by @jianoaix in #1075
• Update codeQL-scanning.yaml by @anupsv in #1028
• Disperser auth by @cody-littley in #984
• start on-chain refresh by @hopeyen in #1083
• Update and rename codeQL-scanning.yaml to codeql-scanning.yaml by @anupsv in #1081
• Enable disperser-v2 prometheus metrics registry by @pschork in #1079
• Onchain verification v2 inabox test by @ian-shim in #1077
• Add metrics to encoder v2 by @dmanc in #1080
• Don't log relay config. by @cody-littley in #1085
• protobuf rename bin record -> period record by @hopeyen in #1084
• Inabox should produce logs in text format by @cody-littley in #1090
• Add a new relay metric to more accurately describe cache behavior. by @cody-littley in #1092
• fix: inabox flaky test by @ian-shim in #1086
• Add e2e latency metrics in controller by @ian-shim in #1095
• Fix: add blobkey in blob response by @jianoaix in #1087
• Add throughput metrics to controller by @dmanc in #1094
• Make choosing relay IDs thread safe. by @cody-littley in #1091
• Disperser auth innabox by @cody-littley in #1052
• doc: remove duplicate word by @kilavvy in #1089
• Fix v2 doc collisions with v1. Make protoc multithreaded for doc generation. by @cody-littley in #1096
• Add support for fetch all operators' responses for a batch by @jianoaix in #1078
• Gracefully shut down relay GRPC server. by @cody-littley in #1099
• Create a blob verification utility by @litt3 in #1066
• KMS client from AWS config by @pschork in #1103
• Better printing during inabox failure mode by @cody-littley in #1101
• Add support for range query on dyanmo for blob feed by @jianoaix in #1102
• Refactor the unit test of dataapi serverv2 by @jianoaix in #1106
• Traffic generator V2 by @dmanc in #1055
• Add trivial PR warning to pull request template by @pschork in #1108
• add bls signer abstraction from eigensdk by @shrimalmadhur in #985
• Improve blob feed cursor key and add unit test by @jianoaix in #1113
• chore: bump eigensdk to dev branch by @shrimalmadhur in #1114
• Remove NK bot reputation farming doc fix warning by @pschork in #1115
• Add utility function to call verifyBlobV2 by @litt3 in #1107
• Add suppport for blob feed API at dataapi by @jianoaix in #1116
• Add attestation metrics by @dmanc in #1122
• Added missing v2 protobuf documentation. by @cody-littley in #1082
• add api key bls remote signer by @shrimalmadhur in #1128
• V2 contract testing by @0x0aa0 in #1032
• [v2] fix: Ignore chunks outside operator quorums by @ian-shim in #1123
• Wrap errors in relay by @dmanc in #1130
• refactor: protobuf payment header in v2 and salt by @hopeyen in #1126
• Add service verification to reachability check by @pschork in #1127
• update middleware commit by @0x0aa0 in #1134
• Add support for range query for batches/attestations feed on dynamo by @jianoaix in #1129
• Use upload artifact v4 by @cody-littley in #1141
• Update unit-tests.yml by @anupsv in #1144
• Fallback ip by @cody-littley in #1147
• Rename v2 structs by @ian-shim in #1136
• Refactor the nonsigner signing rate computing by @jianoaix in #1150
• Add suppport for batch feed API at dataapi by @jianoaix in #1138
• Update node socket registration format by @ian-shim in #1132
• Check type assertion for dynamo types by @ian-shim in #1146
• refactor: required quorum number contract read by @hopeyen in #1148
• Reachability check against v2 dispersal port by @pschork in #1135
• fix(clients/v2): Update blob verifier to not reference client interface via pointer by @ethenotethan in #1153
• Implement v2 client GET functionality by @litt3 in #972
• Downgrade error logs by @ian-shim in #1151
• Rename existing client to PayloadRetriever by @litt3 in #1155
• Add better error reporting for relay rate limits by @cody-littley in #1156
• Adopt CertVerifier naming scheme by @litt3 in #1157
• Add support for batch operator addresses to IDs translation by @jianoaix in #1160
• [ejector] Don't create txn when there's no one to eject by @ian-shim in #1164
• Return blobkey in the blob feed response by @jianoaix in #1166
• Check blob key from disperser against actual key by @litt3 in #1109
• Refactor the operator quorum interval computing by @jianoaix in #1165
• onchain renaming by @0x0aa0 in #1154
• Update download-artifact version by @ian-shim in #1168
• Quorum operator stake report cli by @pschork in #1162
• fix blobs/batches path by @jianoaix in #1171
• fix(node): corev2 dup import with diff alias by @hopeyen in #1169
• Functional tests by @cody-littley in #1167
• Fix how chunk size is computed for throttling. by @cody-littley in #1158
• Rename "node" to "validator" for v2 protobufs. by @cody-littley in #1125
• Disperser api audit by @jianoaix in #1170
• Allow relay protos to be larger. by @cody-littley in #1174
• fix: Add salt to blob header conversion util by @litt3 in #1177
• Upgrade icicle library to v3.4.0 by @dmanc in #1172
• Update v2 swagger by @pschork in #1176
• Fix CORs support in dataAPI by @pschork in #1180
• Fix relay cache sizes. by @cody-littley in #1182
• Extract blob key computation logic into static function by @litt3 in #1179
• Create payload disperser by @litt3 in #1159
• docs: proto on paymentHeader by @hopeyen in #1178
• swap g2 A0, A1 for X and Y by @bxue-l2 in #1184
• Misc documentation fix by @jianoaix in #1139
• chore(api/clients/v2): Add build function for PayloadDisperser by @ethenotethan in #1188
• fix(api/clients): Update v1 client to take in eigen-sdk logger by @ethenotethan in #1183
• Extract a payload retriever interface by @litt3 in #1187
• V2 Retrieval Isolation by @supriya-premkumar in #1181
• Refactor the quorum batches computing by @jianoaix in #1191
• Update v2 blob statuses by @ian-shim in #1185
• Make the signer constructor return an error if it fails by @litt3 in #1190
• Ignore socket parsing error from dispatcher by @ian-shim in #1198
• chore(api/clients/v2): Update payload disperser build function for optional prover usage by @ethenotethan in #1201
• CertVerifier updates by @0x0aa0 in #1200
• Disable auth cache by @cody-littley in #1195
• Moar tests by @cody-littley in #1193
• chore: better logs in txn_manager by @samlaf in #1202
• Fix swagger doc gen to resolve v2.BlobHeader by @pschork in #1199
• Adds isolated v2 reachability check to operator node by @pschork in #1152
• Added check in API server for commitment length by @cody-littley in #1189
• Operator signing computing by @jianoaix in #1206
• Wait for internal eth client to reach block num by @litt3 in #1204
• docs: add minimum symbols to proto api explanation by @hopeyen in #1211
• Disable v1 blobstore dynamo collector in v2 by @pschork in #1213
• Fix CI test failure by @jianoaix in #1216
• Additional correctness tests by @cody-littley in #1208
• Load generator by @cody-littley in #1218
• docs: quorum number requirement for payment method by @hopeyen in #1196
• Remove on chain cert verification from relay payload retriever by @litt3 in #1205
• Improve test orginazation by @cody-littley in #1219
• Robustness fix for relay by @jianoaix in #1220
• feat: metrics on meterer usage by @hopeyen in #1212
• Add BlobAttestaionInfo data type and fetch API support by @jianoaix in #1214
• Don't create V2 server instance in v1-only runtime mode by @pschork in #1222
• Support blob attestation info API by @jianoaix in #1227
• Enable onchain verification in correctness tests. by @cody-littley in #1221
• Implement validator payload retrieval by @litt3 in #1194
• Fix max proto size for retrieval client by @cody-littley in #1231
• Handle unsorted quorum numbers by @ian-shim in #1234
• Fix git actions by @anupsv in #1210
• Use payload clients in preprod tests by @litt3 in #1232
• Add v2 dynamodb collector by @dmanc in #1226
• Encoder API robustness audit/fix by @jianoaix in #1236
• chore(api/clients/v2): Reference prover via interface vs raw struct by @ethenotethan in #1215
• Do not panic when a quorum has zero stake. by @cody-littley in #1233
• More granular node StoreChunks latency metric by @ian-shim in #1240
• Remove unused NumConnectionsToNodesFlag by @ian-shim in #1239
• A round of misc improvements to dataapi by @jianoaix in #1242
• Only override private BLS key in test mode if TestPrivateBlsFlag is defined by @pschork in #1244
• Second round of misc improvements to dataapi by @jianoaix in #1245
• chore(contracts): Add binding check by @ethenotethan in #1217
• feat(contracts): Refactor dependency routing to ensure seamless usability from importer contract repos by @ethenotethan in #1163
• Blob feed iteration with dynamo pagination by @jianoaix in #1252
• chore: attempt to fix test-contracts ci by @hopeyen in #1256
• Refactor the dataapi server v2 file into modular files by @jianoaix in #1255
• Test improvements by @cody-littley in #1251
• CI: run test-contracts when a new commit pushed to the PR by @jianoaix in #1259
• Onchain payment state refresh robustness fix by @ian-shim in #1250
• chore(contracts): v0.0.1-rc-2 release + package README by @ethenotethan in #1258
• fix(api/clients/v2): Add BlobStatus_GATHERING_SIGNATURES to retry loop by @ethenotethan in #1264
• Update codeql-scanning.yaml by @anupsv in #1267
• Report hashing error from v1 batcher dispatcher by @ian-shim in #1247
• Add context to GetCurrentBlockNumber() by @dmanc in #1268
• Comapct relay chunks by @cody-littley in #1197
• Reset cursors and deduplicate blobs in controller by @ian-shim in #1261
• Provide signer and nonsigner operator IDs in blob attestation-info API by @jianoaix in #1266
• Fix overflow on node block stale measure & store duration blocks override values when test-mode is enabled by @pschork in #1270
• Handle edge case with negative request size by @cody-littley in #1272
• Provide operator addresses for signers/nonsigners by @jianoaix in #1276
• refactor: api rm Salt, reservationPeriod->timestamp by @hopeyen in #1254
• Disable auto IP discovery if not all ports are configured by @ian-shim in #1275
• Update codeql-scanning.yaml by @anupsv in #1277
• Node api audit by @jianoaix in #1186
• Make operator stake API return operator address by @jianoaix in #1287
• Load generator docker by @cody-littley in #1260
• Reduce relay memory usage by @cody-littley in #1293
• fix: accountBlob reservation no flaky by @hopeyen in #1294
• Add pprof to relay server by @dmanc in #1285
• Add more logging in relay by @dmanc in #1288
• Create payload->blob type system by @litt3 in #1120
• Fix relay build by @dmanc in #1296
• Node API: make chunk encoding explicit by @jianoaix in #1278
• Check that disperser uses correct blob length by @litt3 in #1290
• Support dynamic cert verifier address by @litt3 in #1289
• Support two-way traversal in blob feed API by @jianoaix in #1286

New Contributors

• @yujiezhu0 made their first contribution in #952
• @litt3 made their first contribution in #976
• @leopardracer made their first contribution in #989
• @kilavvy made their first contribution in #1089
• @ethenotethan made their first contribution in #1153
• @supriya-premkumar made their first contribution in #1181

Full Changelog: v0.8.6...v0.9.0-rc.0