Rebase Fulu onto unstable (#7070)

* Bump nim-web3 to c8f36f59cb354196cfe117b6866e81d450c8cfd7 (#6727)

* Bump nim-web3 to c8f36f59cb354196cfe117b6866e81d450c8cfd7
* no need to convert accessList and authorizationList anymore

* fix typo README.md (#6735)

"cliant" - "client"

* Run LC data collection tests (#6737)

Add test runner for future tests / ensure it gets compiled.

* version v24.11.0

* use --platform syntax for Dockerfiles

* use non-EOL Debian image and correct --platform arguments

* `getMetaData_v3` (#6739)

* add getMetaDataV3

* simpler fix, to review

* drop returns

* update Docker images to stable/Bookworm (#6740)

* update Docker images to stable/Bookworm

* add copyright headers

* Bump nim-sqlite3-abi to `v3.47.1.0` (#6744)

- Bump sqlite-amalgamation to `3.47.1`

* Bump nim-json-rpc to `274372132de497e6b7b793c9d5d5474b71bf80a2` (#6743)

- chore: use supported selector operators

* Bump nim-stint to `5c5e01cef089a261474b7abfe246b37447aaa8ed` (#6745)

- fix compile-time hash, port to unittest2

* Bump nim-websock to `v0.2.0` (#6747)

- chore: use supported selector operators
- Attempt to fix broken autobahn CI artifact uploading

* Bump nim-eth to `aa92ad4f42d772c53ce4a2d9c76cf760b4450031` (#6742)

- discv4: Fix assert on invalid RLP list in Neighbours message

* Bump nim-web3 to `2bdffd61128f9142a0d19a9f3b6e6503b974929b` (#6746)

- Add newPayload template for engine_newPayloadV4 and
  restore eth_getBlockReceipts callsig

* Bump nim-libbacktrace to `6da0cda88ab7780bd5fd342327adb91ab84692aa` (#6749)

- Bump libbacktrace to `1db85642e3fca189cf4e076f840a45d6934b2456`

* Update Electra art with text 🦒 (#6748)

Add `Compounding on` text to Electra fork transition art.

Co-authored-by: Beatscribe <[email protected]>

* stint: bump for better endians (#6750)

faster endian conversion et al

* Obtain extra `LightClientFinalityUpdate` after LC sync (#6752)

To avoid having to wait several minutes for LC finality gossip, obtain
extra finality update once LC sync finishes.

* clean up obsolete nimble files

* Support reading `bootstrap_nodes.yaml` (#6751)

* Support reading `bootstrap_nodes.yaml`

`bootstrap_nodes.txt` is retired in lieu of `bootstrap_nodes.yaml`,
start reading `.yaml` format (similar to `.enr`).

* Support Gnosis Chiado format (duplicates of entries in other file)

* taskpools: bump (#6757)

* taskpools: bump

* better raises effects
* get rid of custom naming

* bump

* oops

* bump stable

* fix: response checking from RequestManager for blobs (#6755)

* rewrite: response checking from RequestManager for blobs

* clean whitespace

* use a comparator and std instead

* sort incoming blobs by index

* fix list traversal

---------

Co-authored-by: tersec <[email protected]>

* version v24.12.0

* fix typo in most recent changelog

* undo accidental nim-taskpools change

* wait for fulu fork epoch to pass, then trigger metadata_v3 (#6759)

* wait for fulu fork epoch to pass, then trigger metadata_v3

* tiny fix

* add: column support to ENR, Metadata and Request Manager (#6741)

* add csc to enr and metadata

* add column filtering into RequestManager

* nits

* add comment

* resolved reviews 1

* added local custody column set into RequestManager as a field

* faster lookups with hashsets

* fix regressions, fix other reviews, fix response checking for columns

* simpler fix for hashsets

* use EF consensus spec v1.5.0-alpha.10 test vectors (#6761)

* automated consensus spec URL updating to v1.5.0-alpha.10 (#6762)

* increase default gas limit from 30M to 36M (#6763)

* adds peerdas-supernode flag (#6764)

* adds peerdas-supernode flag instead of launching a peerdas supernode in subscribeAllSubnets

* make the flag default and _hidden_

* fix indentation

* update to current EIP-7685 engine API request format (#6766)

* Typos Documentation in README.md Files (#6771)

* Update README.md

* Update README.md

* Update README.md

* implement EIP-7691 blob sidecar req/resp endpoints (#6769)

* implement EIP-7691 blob sidecar req/resp endpoints

* refactor common code out of the blob root/range request handlers

* use template instead of proc

* custody subnet count decoding during discovery (#6777)

* renamed eip7594_helpers to peerdas_helpers, added csc subnet decoding during discovery

* added link

* make `getTargetGossipState` aware of Fulu + other fixes (#6780)

* make getTargetGossipState aware of Fulu, make it aware of electra as well in LC

* fix indent

* minonr fix

* add tests for Electra and Fulu gossip transitions (#6784)

* implement EIP-7691 gossip validation for increased max blobs per block (#6785)

* restore electra aggregate gossip validation to parity (#6787)

* rm daily (not nightly) GitHub Actions workflow (#6786)

* implement SingleAttestation (#6783)

* implement SingleAttestation

* improve SingleAttestation shortLog

* avoid double ShuffleRef lookups in majority/happy case

* restore full electra check_aggregation_count, separately from phase0 one

* switch to Nim v2.0.14 (#6795)

* introduced custody groups, and renamed csc to cgc (#6789)

* introduced custody groups, and renamed csc to cgc

* accomodate tests

* revert to naming columns

* applied review changes

* updated all tests file

* addressed review 2

* merged in typo fixes by airdop farmers/other spam PRs

* handle lint ci

* shift to iterators, avoid redundant copying

* fix Electra aggregated attestation gossip verification committee index (#6800)

* update ShufflingRef version of get_attesting_indices to Electra v1.5.0-alpha.10 (#6801)

* docker: Restore more liberal permissions /home/user. (#6803)

* rm Nimbus book reference to import --method=single-salt (#6805)

* add support for Electra blob subnets and IDs (#6806)

* add support for Electra blob subnets and IDs

* fix tests

* fix loading Nix Flake shell via direnv

Signed-off-by: Jakub Sokołowski <[email protected]>

* update Nix readme to fix submodules flag usage

Signed-off-by: Jakub Sokołowski <[email protected]>

* exclude gitignore and envrc from copyright requirement

Signed-off-by: Jakub Sokołowski <[email protected]>

* fix: pass `optionalForks` to consensus fork parser (#6811)

* voluntary exit fix

* ci lint fix

* Fix EL manager to wait for non-syncing/accepted response. (#6812)

* Fix EL manager to wait for non-syncing/accepted response.

* Update copyright year.

* rm unused Makefile devnet-supporting scripts (#6813)

* Bump mainnet to `dd91689396a9824ed3c78d0576cf5d854504c25c` (#6815)

- Align config structure with other public testnet repositories

* Bump nim-unicodedb to `v0.13.2` (#6816)

- https://github.com/nitely/nim-unicodedb/blob/v0.13.2/CHANGELOG.md

* Bump holesky to `f6761b531dae01e30ca05658d01853415465d1e0` (#6814)

- Align config scructure with eth-clients/mainnet repository

* Bump nim-web3 to `a3bc5ad48e2b05fa253ba68bbd5b84e4ea234f50` (#6817)

- stint: bump for endians
- Add newPayload template

* Bump sepolia to `da5654742513435bdd6dbc5fd033cf593ce57a0f` (#6818)

- Align config scructure with eth-clients/mainnet repository

* Bump nim-sqlite3-abi to `v3.47.2.0` (#6819)

- Bump sqlite-amalgamation to `3.47.2`

* Avoid `IndexDefect` when Prague EL sends malformed requests (#6822)

* Avoid `IndexDefect` when Prague EL sends malformed requests

When using an outdated EL that does not prefix requests with types,
the `.len < 2` check was performed after already doing `mapIt(it[0])`.
Reorder the checks and also avoid the O(n) `request_types` allocation.

* Somehow this `mapIt` is used by `message_router_mev.nim`

* Update db size estimates for Electra light client data (#6823)

* Update db size estimates for Electra light client data

Requests are no longer part of `ExecutionPayloadHeader` and therefore
no longer included in LC headers. Also fix extra 40 byte in counts.

* Lint

* nimble: better isdir check (#6825)

* nimble: better isdir check

The current one seems to not find the file sometimes - also update
nimble file to something more useful.

* nanny

* clean up eip7594 naming, useful links, test suite naming (#6820)

* clean up eip7594 naming, useful links, test suite naming

* fix copyright year

* updated tests

* Bump jinja2 from 3.1.4 to 3.1.5 in /ncli (#6828)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.5.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.4...3.1.5)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* some manual consensus spec URL updates to v1.5.0-alpha.10 (#6827)

* some manual consensus spec URL updates to v1.5.0-alpha.10

* copyright year linting

* Fix last-seen-p2p-address values in /eth/v1/node/peers API endpoint. (#6595)

* Attempt to fix issue with AnyLocal addresses in last-seen-p2p-address field.

* Address issues with IPv6 and QUIC protocol addresses.

* Attempt to avoid QUIC addresses from site-local networks.

* Fix IPv6 handling.

* Perform normalization of `last_libp2p_address`.

* Switch to LastSeenBook.

* Fix address deserialization.

* Bump nim-libp2p.

* Eliminate unneeded function.

* Eliminate unneeded import.

* Bump jinja2 from 3.1.4 to 3.1.5 in /docs (#6830)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.5.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.4...3.1.5)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix /eth/v1/beacon/rewards/block/{block_id} sync_aggregate value (#6829)

* fix /eth/v1/beacon/rewards/block/{block_id} sync_aggregate value

* adjust tests

* copyright year linting

* kzg: load trusted setup for new constant (#6826)

* kzg: load trusted setup for new constant

* bump

* copyright

* use EF consensus spec v1.5.0-beta.0 test vectors (#6832)

* automated consensus spec URL updating to v1.5.0-beta.0 (#6833)

* validate EL block hash when running consensus block tests (#6406)

* validate EL block hash when running consensus block tests

We currently don't have an easy way to test EL block hash computation.
As the EL block hash in consensus-spec-tests is computed correctly,
update the test runners that load block from test files to also verify
the EL block hash. This increases missing test coverage.

Requires ethereum/consensus-specs#3829

* fix

* resolve merge conflicts

* fix genesis case, and deal with `incorrect_block_hash` test

* add missing export marker

* fix import

* htr mutates underlying data, messing with differ, create copy in test

* Handle payloads with empty tx (unsupported in ordered trie tool)

* Update copyright years

---------

Co-authored-by: tersec <[email protected]>

* Bump nim-ssz-serialization to `6106ceee6da5f710b7ccc8d98c41e790cc918201` (#6834)

- Add `maxSize` helper
- Add `minSize` helper

* Add test to compare SSZ bounds against Teku impl (#6836)

* Add test to compare SSZ bounds against Teku impl

Ensure that our SSZ bounds are computed the same way as in Teku by
comparing the bounds for `deneb.SignedBeaconBlock`.

* Lint

* Make platform agnostic regarding how Git checks out newlines

* Various fixes for light forward syncing algorithm. (#6831)

* Fix light forward syncing from finishing with inconsistent state.

* Update copyright year.

* Add block retention period check

* increase default verbosity of Nix builds to V=1

Signed-off-by: Jakub Sokołowski <[email protected]>

* add: `compute_columns_for_custody_group` tests (#6838)

* add new fixture networking tests

* update all tests

* fix test

* avoid extra variables

* other fixes

* oops

* updated tests file

* assert submodules are available to Nix derivation (#6835)

This should result in a human-readable error when `?submodules=1#` is
not appended to the project URL as the README explains:
```
 > nix build '.'
       error: Unable to build without submodules. Append '?submodules=1#' to the URL.

 > nix build '.?submodules=1#'
```

Signed-off-by: Jakub Sokołowski <[email protected]>

* Bump nim-ssz-serialization to `d7552018fe2fe1ab1b74a68517caa085a2309622` (#6842)

- fix duplicated testing of different MMs

* Bump nim-stint to `v0.8.1` (#6843)

- fixes asm statement in `nimHasAsmSemSymbol` in the devel branch
- enable some disabled tests
- adds `nodecl` to imported types
- v0.8.1

* Bump nim-taskpools to `7b74a716a40249720fd7da428113147942b9642d` (#6844)

- remove empty lock file
- use the common CI workflow

* Bump dependencies: use the common CI workflow (#6841)

- nim-bearssl to `c7683c5a6221605bbab31b53d01feb3a3161bb8b`
- nim-blscurve to `3737c9663d26076ed89f011c2688c1b9ee1eb73a`
- nim-confutils to `e214b3992a31acece6a9aada7d0a1ad37c928f3b`
- nim-faststreams to `2b08c774afaafd600cf4c6f994cf78b8aa090c0c`
- nim-http-utils to `79cbab1460f4c0cdde2084589d017c43a3d7b4f1`
- nim-kzg4844 to `33d2e9a5d4df1bbf4aca7cf00457bf15b1d657a6`
- nim-metrics to `25ffd054fd774f8cf7935e75d6cad542306d7802`
- nim-nat-traversal to `53b5883f410ac7d84683dfbc894dccd660e68f6d`
- nim-secp256k1 to `62e16b4dff513f1eea7148a8cbba8a8c547b9546`
- nim-zlib to `3f7998095264d262a8d99e2be89045e6d9301537`
- nim-zxcvbn to `d827fdc9968a4f06bba587dc38df2b20399f8cf5`

* Bump nim-nat-traversal to `213ac13dfe5c4830474912c48181b86b73f1ec1f` (#6845)

- Bump miniupnp to `miniupnpc_2_3_0`

* bump nim-libp2p (#6847)

* Revert "bump nim-libp2p (#6847)" (#6848)

This reverts commit 4b66a9e.

* Adjust default port numbers for tests. (#6849)

* Adjust default port numbers.

* Update copyright year.

* bump nim-libp2p an even tinier bit (#6853)

* sync manager should allow for electra blob counts (#6855)

* sync manager should allow for electra blob counts

* upload-artifact@v3 deprecation

* fix: keeping `MAX_REQUEST_BLOB_SIDECARS` limit higher upto electra, so that deneb limits can also be sufficed under one version (v1) (#6856)

* keeping limit higher upto electra, so that deneb can also be sufficed under one version (1)

* fixed res limit to electra

* fix copyright year

* rm support for deprecated reqStep in req/resp (#6857)

* rm support for deprecated reqStep in req/resp

* work around / character in platform

* mini libp2p bump (#6859)

* bump nim-libp2p (#6861)

* ramify the new ELECTRA blob settings through more; add single_attestation event queue; add initial vanity logs phrasing (#6865)

* ramify the new ELECTRA blob settings through more; add single_attestation event queue; add initial vanity logs phrasing

* add REST test rule

* make Nimbus Nim 2.2-compatible (#6869)

* use EF consensus spec v1.5.0-beta.1 test vectors (#6870)

* automated consensus spec URL updating to v1.5.0-beta.1 (#6871)

* prepare builder API, modify electra BuilderBid (#6872)

* prepare builder API, modify electra BuilderBid

* rename to blindedBlockRes and include executionRequests

* fix copyright year

* resolved review comments

* more review changes

* revert to Opt[ExecutionRequests]

* reviews

* allow electra blobs per block in req/resp (#6880)

* version v25.1.0

* fix changelog year

* show vanity logs on compounding activation of known validators (#6882)

* use SingleAttestation for submitPoolAttestationsV2 (#6883)

* update VC to use SingleAttestation (#6884)

* Eliminate build warnings in applications. (#6889)

* Initial commit.

* Update copyright years.

* Make syncing performance more steady. (#6722)

* Initial commit.

* Update tests.

* Update AllTests.

* Post-rebase update AllTests.

* Improve log statements which was hidden because of logScope issues.

* Eliminate duplicate `direction` fields.
Debugging blobs map.

* Fix blobs shortmap generation.

* Add fixes to map generation and add test helpers.

* Fix missing dependency.

* Adjust callbacks functionality for untrusted sync.

* Attempt to fix defect state issue with new flag `setLastStateRoot`.

* Make possible fix less intrusive.

* Fix forward declaration missing `=`.

* Revert attempt to improve performance.

* Revert some all the changes.

* Remove spaces.

* Remove LFS fixes from the PR.

* Update AllTests.

* Update copyright year.

* Update AllTests.

* Post-rebase fixes.

* fix upgrade to fulu for future beta2 tests (#6898)

* start tracking Nim 2.2 as possible next Nim version (#6899)

* start metrics server later in sequence, after BN is initialized (#6902)

* Fix capital version name being returned in JSON response `version` field. (#6905)

* Fix REST /eth/v1/config/spec endpoint should provide electra specific values. (#6906)

* Fix add more electra values to /eth/v1/config/spec endpoint.

* Move some constants to public space.
Fix places where constants being used.
Fix output of /eth/v1/config/spec to be consistent with electra specification.

* update Holesky and Sepolia Electra fork epochs (#6908)

* use EF consensus spec v1.5.0-beta.2 test vectors (#6909)

* Fix syncqueue.nim crash in the end of forward syncing process. (#6910)

* add slots sanity and forks fixtures for Fulu (#6911)

* add KZG commitment to first-blob-found check in blobs gossip validation (#6912)

* Bump nim-libbacktrace to `0a438d70312de253694748346e002418bd127829` (#6864)

- Add script to bump `libunwind` dependency
- Bump libunwind to `llvmorg-13.0.1`

* Bump `is_better_update` to match latest spec (#6914)

Add `signature_slot` comparison to `is_better_update` LC data comparator
for robustness in edge cases, and sync style with latest specs.

- ethereum/consensus-specs#4124

* VC: Fix validator registration cache issue. (#6907)

* Fix cache invalidation issue in VC.

* Update copyright source.

* Address review comments.

* Update cache invalidation for gas limit calls.

* Update copyright year.

* add attestation pool checks for aggregating across committees (#6915)

Co-authored-by: Etan Kissling <[email protected]>

* update Electra attestation and aggregation gossip validation (#6917)

* use a consistent committee_index in Electra attestation validation

* update local tesnet Geth version and test Electra by default (#6921)

* Keymanager FR and GL fixes (#6916)

* Fix FR and GL management for dynamic validators.

* Fix importing keystore could be blocked by setting gas limit or feerecipient values.
Add tests.

* Enable setTopicState() values.

* Update copyright years.

* Update AllTests.

* Post-rebase updates oof AllTests.

* Fix percent value goes bigger than 100%. (#6922)

* version v25.2.0

* tweak changelog phrasing

* Bump nim-blscurve to `52ae4332c749d89fa05226f5493decae568f682c` (#6920)

- Bump blst to `v0.3.14`
- bench: add serial MSM benchmark

* Bump nim-libbacktrace to `616e5f4a6e2c751b85487e2e855c6b7f90848503` (#6919)

- Remove patches for `libunwind` and use upstream build system
- Re-root `libunwind` CMake invocation at `<monorepo>/runtimes`
- Bump libunwind to `llvmorg-19.1.7`
- Compile libunwind in release mode

* update to Geth v1.15.1 in local testnet (#6923)

* update block_sim to start at Electra (#6924)

* mostly automated consensus spec URL updating to v1.5.0-beta.2 (#6925)

* mostly automated consensus spec URL updating to v1.5.0-beta.2

* copyright year linting

* Bump nim-eth submodule (#6926)

- Improvements to discv5 sessions
- Adjust uTP benchmarking tests to use multiple nodes
- Discv5 Protocol: Add support for banning nodes
- Reduce declared but not used warnings
- Optimize RLP serialization
- Discv5 Routing Table: Add support for banning nodes
- use the common CI workflow
- add discv5_network_byte metric in discoveryv5 protocol
- ecies: cleanup

* add --finalized-checkpoint-state suggestion to the TNS state download timeout message (#6927)

* add: basic validator custody (#6929)

* add: basic validator custody

* address review comments

* renaming to snake case

* Bump nim-libbacktrace to `624d1f8719d9e34c5d0c1f15652168a1222e1782` (#6930)

- Always use system unwinder

* Remove unused `StatusUpdateInterval` (#6932)

The `StatusUpdateInterval` definition is not used anywhere. Remove it.

* Add log parameters to sync manager debug message (#6933)

There is a single debug message emitted by `sync_manager` that does not
include log parameters. Add them, so that `topics` filtering works.

* Bump nim-testutils to `v0.6.0` (#6931)

- https://github.com/status-im/nim-testutils/releases/tag/v0.6.0

* lower testnet base ports to avoid port clashes (#6934)

Hitting the same ports as `status-desktop` is using during end-to-end tests
is more probably when our rage is higher.

Signed-off-by: Jakub Sokołowski <[email protected]>

* Remove unused functions blob test (#6935)

`createBlobSidecars` and `createBlobRange` are not used in syncman tests
and can be removed.

* Re-sync docs for Gnosis Altair preset (#6940)

* Re-sync docs for Gnosis Altair preset

The comment for `EPOCHS_PER_SYNC_COMMITTEE_PERIOD` of Gnosis Altair
preset was outdated and referring to a wrong number.

* Lint

* rm SyncRequestQueueItem (#6941)

* Disable totals summary in Markdown test reports (#6937)

* Disable totals summary in Markdown test reports

The totals summaries in .md test reports are frequently leading to merge
conflicts and may even disincentivize adding new tests. As we only use
the list of tests anyway (primarily for consensus-spec-tests comparison)
we can drop the total amounts summaries from the Markdown files.

* Re-generate tests

* VC: Fix graffiti settings (#6946)

* Fix VC not using graffiti settings from filesystem file or keymanager API.

* Fix VC unable to set graffiti bytes before validator being added via keymanager api.

* Update copyright year.

* Remove unneeded code.

* Compute `BlobQuarantine` limit from dynamic config (#6948)

`MAX_BLOBS_PER_BLOCK_ELECTRA` is configurable by network definitions,
compute the `BlobQuarantine` limit on initialization at runtime instead
of at compile time.

* Expose runtime `MAX_BLOBS_PER_BLOCK_ELECTRA` in REST config endpoint (#6949)

The REST config endpoint exposes the compile-time constant instead of
the config value for `MAX_BLOBS_PER_BLOCK_ELECTRA`. Fix that, so that
the compile-time constant can be removed eventually.

* Use correct blob subnet when `BLOB_SIDECAR_SUBNET_COUNT_*` is overridden (#6950)

`BLOB_SIDECAR_SUBNET_COUNT_ELECTRA` is configurable separately from
`MAX_BLOBS_PER_BLOCK_ELECTRA` to allow configs with multiple blob IDs
mapped to the same gossip topic. There is no large runtime overhead in
simply using the correct constant as defined by spec. If we want to
restrict to values that map 1 blob to 1 topic, there should be a check.

Also fix max blobs per block computation to be based on the block slot,
rather than the time at which it was received.

* Use consistent merkleization limit for `/eth/v1/beacon/blob_sidecars` (#6951)

Use the `MAX_BLOB_COMMITMENTS_PER_BLOCK` constant as merkleization limit
for the `/eth/v1/beacon/blob_sidecars` response so that `hash_tree_root`
results in same value regardless of configured serialization limits.
At compile-time, we don't have access to the actual serialization limit,
so using a mainnet derived value in the type definition is not flexible.

* bump nim-snappy (#6947)

* Expose `MAX_REQUEST_BLOB_SIDECARS_ELECTRA` config via REST (#6953)

The `MAX_REQUEST_BLOB_SIDECARS_ELECTRA` value is configurable and should
therefore be queriable via REST to see how the BN is configured.

* Remove unused `deadlineVar` from p2p dsl (#6952)

* Remove unused `deadlineVar` from p2p dsl

The `deadlineVar` is not used anywhere and can be removed.

* Lint

* When requesting lists via req/resp, allow runtime config of `maxLen` (#6954)

The maximum number of items that are acceptable for any given req/resp
response can be smaller than the hard maximum. For example, if we only
request blobs for a single block, the limit is `MAX_BLOBS_PER_BLOCK`
so we will eventually reject any response that exceeds that limit even
though it technically fits into the `MAX_REQUEST_BLOB_SIDECARS` based
response type.

This adds an optional `maxResponseItems` parameter to requests that
provide `List` responses, so that the caller can further restrict the
maximum expected number of response chunks. If a peer provides more
chunks than expected, this now fails early with `ResponseChunkOverflow`.

* Add SSZ encoding support to REST API endpoint /eth/v1/validator/register_validator (#6943)

* Add SSZ version of /eth/v1/validator/register_validator

* Add ability to test `application/octet-stream` bodies to `restclient`.
Add tests for SSZ version `/eth/v1/validator/register_validator`.

* Fix test value to be correct hexadecimal string.

* Update copyright years.

* setup logging before any procedure with logging (#6942)

Co-authored-by: pmmiranda <[email protected]>

* Merge `GOSSIP_MAX_SIZE` and `MAX_CHUNK_SIZE` into `MAX_PAYLOAD_SIZE` (#6955)

* Merge `GOSSIP_MAX_SIZE` and `MAX_CHUNK_SIZE` into `MAX_PAYLOAD_SIZE`

In upstream specs the preset values for indicating max payload sizes
have been merged into a single `MAX_PAYLOAD_SIZE` constant.

* Lint

* Update config endpoint

* Use larger serialization limit when parsing fork choice tests (#6956)

`MAX_BLOBS_PER_BLOCK` became configurable with v1.5.0-alpha.4 specs
(moved from preset to config), so use the theoretical max allowed value
when parsing fork choice tests.

* Extend `RuntimeConfig` with constants for blob sidecar limits (#6957)

Since v1.5.0-alpha.4 the blob sidecar limits are no longer configured in
the presets but are now part of the network configuration. This sets up
`RuntimeConfig` to parse those values so that can phase out the compile
time constants later.

* rm spec-removed produceBlockV2 and produceBlindedBlock beacon API endpoints (#6885)

* Use `MAX_BLOBS_PER_BLOCK` from config in gossip validation (#6958)

Start phasing out the compile-time `MAX_BLOBS_PER_BLOCK` and use the
value from the network config in gossip validation instead.

* When requesting blobs by root, limit response size to request size (#6961)

When requesting `idList.len` blobs, the response can't be larger than
`idList.len` or it is invalid.

* Use fork-specific blob subnet count when subscribing to gossip topics (#6962)

* Use fork-specific blob subnet count when subscribing to gossip topics

We were always subscribing based on Electra blob gossip limits, despite
no valid traffic being possible if those topics are used on Deneb.
Use the correct limit for each fork:

- `BLOB_SIDECAR_SUBNET_COUNT` on Deneb
- `BLOB_SIDECAR_SUBNET_COUNT_ELECTRA` on Electra and later

* Lint

* Use `MAX_BLOBS_PER_BLOCK` from config in state transition (#6959)

Start phasing out the compile-time `MAX_BLOBS_PER_BLOCK` and use the
value from the network config in state transition instead.

* Move sync queue assertions to preset loader (#6960)

We already check in `presets.nim`:

- `MAX_BLOBS_PER_BLOCK_ELECTRA` ≥ `MAX_BLOBS_PER_BLOCK`
- Config values match preset constant values until constants phased out

The ≤ 9 check can be moved over, then sync queue no longer depends on
the deprecated preset constant values and it remains a one-time check.

* Use `MAX_BLOBS_PER_BLOCK_ELECTRA` from config in sync manager (#6963)

Since v1.5.0-alpha.4 the `MAX_BLOBS_PER_BLOCK` are part of config and
are not preset constants anymore. This means we have to pass it into
the sync manager so it can use the configured value.

* Allow runtime config of `MAX_REQUEST_BLOB_SIDECARS` (#6964)

With `MAX_REQUEST_BLOB_SIDECARS` becoming dynamic, we can no longer use
the value from network config in the `List` datatype and have to rely on
an upper bound constant for the maximum supported value instead and then
check the actual length at runtime.

As a side effect, this also extends `BlobIdentifierList` to use the
correct electra specific request limit instead of the Deneb constant.

* Bump geth to `v1.15.3` (#6965)

- https://github.com/ethereum/go-ethereum/releases/tag/v1.15.3

* fix ncli slots help (#6966)

* fix ncli slots help

* copyright year

* Limit `BlobId` to a maximum supported range (#6967)

* Limit `BlobId` to a maximum supported range

`BlobId` historically tracked `MAX_BLOBS_PER_BLOCK_ELECTRA` which is no
longer correct because the network configuration may assign multiple
blobs to the same subnet dynamically since v1.5.0-alpha.4. Instead,
restrict it to a specific range, there is no `for x in BlobId` anywhere.

* Lint

* Bump gnosis-chain-configs to `3f44fafa888439f56fc60d2f6173bfe63f31c962` (#6968)

- Bump to v1.5.0-alpha.10
- Bump configs to spec v1.5.0 beta.2
- Schedule pectra on chiado
- Make config comments match the values

* increase trusted node sync state download timeout to 3 minutes (#6969)

* version v25.3.0

* address most remaining debugComments (including single_attestation event support) (#6974)

* Bump geth to `v1.15.4` (#6975)

- https://github.com/ethereum/go-ethereum/releases/tag/v1.15.4

* add signing node Electra block tests (#6981)

* security: fixed sidecar response checking allowing subsets (#6985)

* fixed sidecar response checking allowing subsets

* make subset checking faster

* make it even faster

* Respond to un-synced blocks request with `ResourceUnavailable` (#6977)

* Respond to un-synced blocks request with `ResourceUnavailable`

When serving beacon blocks by range requests, there are three outcomes:

1. We have blocks available: Response is non-empty
2. The entire epoch actually is empty: Response is empty
3. We have not backfilled that far or have pruned the history

To distinguish (2) from (3), send `ResourceUnavailable` if the
requested range is for blocks earlier than our backfill progress.

* Update beacon_chain/sync/sync_protocol.nim

* Bump geth to `v1.15.5` (#6988)

- https://github.com/ethereum/go-ethereum/releases/tag/v1.15.5

* Bump NimYAML to `v2.2.0` (#6989)

* Bump NimYAML to `v2.2.0`

- https://github.com/flyx/NimYAML/blob/v2.2.0/CHANGELOG.md

* Lint

* Bump nim-sqlite3-abi to `v3.49.1.0` (#6990)

- Bump sqlite-amalgamation to `3.49.1`

* Bump holesky to `8caffe0c004402f5cbbd3164a722927900220608` (#6991)

- added electra fields to configuration

* Bump nim-eth to `ac1be3135a7f41d2822544a7c0b7b498c5b824da` (#6994)

- uTP: add invalid packet header version to error result
- Add function for calculating serialised/RLP length without encoding
  data
- Move p2p code to execution client

* Bump nim-eth2-scenarios to `217a1776adb5120bc44f870e407747902089f343` (#6995)

- use --retry-all-errors instead of --retry-connrefused
  now that can rely on curl 7.81.0 or newer

* Bump nim-chronos to `v4.0.4` (#6993)

- Replace `apt-fast` with `apt-get`
- Fix possible issues with byte order in ipnet
- use the common CI workflow
- Fix baseUri should provide correct value for ANY_ADDRESS
- Fix, waitpid() should wait for finished process
- Bump nimble file

* Bump nim-chronicles to `a8fb38a10bcb548df78e9a70bd77b26bb50abd12` (#6992)

- Reduce redefine template warning

* Bump nim-libbacktrace to `65f9ed0a3e8aa4c860ccb659ae20d5795aed8207` (#6996)

- update ci.yml to account for arm64 macos
- Bump libbacktrace to `0034e33946824057b48c5e686a3aefc761b37384`
- Make sure install directory is present

* Bump nim-minilru to `0c4b2bce959591f0a862e9b541ba43c6d0cf3476` (#6997)

- Use the common CI workflow
- Remove duplicate concurrency setting from CI
- Add pop operation

* Bump sepolia to `ca07b8f41ebfe995d9dde0525f3da9cf1d89aa2e` (#6999)

- update config with fields required at Electra

* Bump nim-nat-traversal to `dae59ddfd514260bb8586b700ec20f58c4ea30ff` (#6998)

- fix(build): include directory using `-iquote` instead of `-I`
- Bump miniupnp to `miniupnpc_2_3_2`

* Bump jinja2 from 3.1.5 to 3.1.6 in /ncli (#7002)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump jinja2 from 3.1.5 to 3.1.6 in /docs (#7003)

Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.5 to 3.1.6.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](pallets/jinja@3.1.5...3.1.6)

---
updated-dependencies:
- dependency-name: jinja2
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Retain vanity specific logic back through second latest Mainnet fork (#7000)

To avoid accumulating vanity code, define a policy to retire old logic.

* config.nims template redefinition disabling unnecessary now that nim-chronicles bumped (#7001)

* Change ChainId type to UInt256, bump nim-eth, and nim-web3 (#7005)

* Change ChainId type to UInt256

* Remove TransitionConfigurationV1 usage

* Lint

* Bump nim-web3 to d8a91d040975cd3dd2a10c26456fab2d7523e8dd

* Add SSZ encoding support to MEV client calls. (#6970)

* Add SSZ support to MEV client functions.

* Add tests.

* Update copyright years.

* Update AllTests.

* ensure post-Electra produceAttestationData returns 0 committee_index (#6984)

* Improve quarantine performance on forked chains (#7006)

* Improve quarantine performance on forked chains

A bit silly, but if we downloaded an orphan, we'd keep re-downloading it
if the missing list was full.

Also, we'd keep redownloading unviables :/

* lint

* implement getPendingDeposits and getPendingPartialWithdrawals beacon API endpoints (#7010)

* Fix incompatible uint64 chain ID remenants (#7011)

Corrects issues introduced in #7005 that changed the Nim signatures of
exported API functions without adjusting the corresponding .h import
header. Also use specific `ChainId` data type where it makes sense,
and remove assertions that are no longer required.

* LC support for receipt logs that are not following Ethereum ABI (#7013)

`data` may contain arbitrary data if using EVM assembly directly, the
restriction to a multiple of 32 Bytes is not generally correct.

- ethereum/ethereum-org-website#15096

* use EF consensus spec v1.5.0-beta.3 test vectors (#7015)

* automated consensus spec URL updating to v1.5.0-beta.3 (#7016)

* Add Electra/Fulu support to `ncli dumpBlock` (#7017)

As the code for dumping blocks in `ncli` is not maintained, replace it
with logic that automatically works with new forks. Similar semantics,
except that `ConsensusFork` is now enumerated in backward direction.

* Bump nim-kzg4844 to `2163a77cb66b1b0faf032a735a751d0ea1e83499` (#7018)

- Bump c-kzg-4844 to `v2.1.0`

* Bump nim-libp2p to `v1.9.0` (#7020)

- https://github.com/vacp2p/nim-libp2p/releases/tag/v1.9.0

* Add `hoodi` testnet (#7021)

* Add `hoodi` testnet

Hoodi is the new public testnet. Add support to use it like Holesky.

- https://github.com/eth-clients/hoodi

* Lint

* Exclude `parsedConsensusGenesis.json` from `git lfs` as we don't need it

* Reinstate Sepolia bitrot as it is not in scope of the PR

* Bump hoodi

* Inline wrapper script

* Add *.tpl to Lint exclude list

* Continue recommending the wrapper scripts for mainnet to keep them

* Re-add `SEPLIA_WEB3_URL`

* bump hoodi network configuration (#7022)

* stop dumping attestation data with --dump enabled (#7012)

* Bump sepolia to 562d9938f08675e9ba490a1dfba21fb05843f39f (#7027)

- Update Config Values to Match With the Latest Spec

* Bump holesky to 32a72e21c6e53c262f27d50dd540cb654517d03a (#7024)

- Update config.yaml

* Bump hoodi to 85f30d3f3ac02f901038782f6844b2453d677ee9 (#7025)

- Add Hoodi EL `stateroot`
- Add Blockscout Explorer
- add bootnodes to el genesis files

* version v25.3.1

* use Nim version-2-2 branch as prospective next target (#7028)

* Bump nim-ssz-serialization to 55ac17ca1f42afa35db9a06dd50c4d79a17c5d28 (#7026)

- Extend multiproof tests
- Fix SSZ Union not to allow for extra data after the None selector

* Fix Electra / Fulu .graffiti proven property in remote keystore (#7030)

The gindex for Electra / Fulu when requesting `.graffiti` as proven
property is incorrect:

- Electra: #6177 lacks reasoning and likely introduced copy paste error?
- Fulu: #6677 copied the broken Electra gindex, likely assuming its good

`BeaconBlockBody` has not changed. The correct gindex is still `18`.

- 13 fields total --> next pow2 is 16, so indexing starts at 16
- fields are 0-indexed, `graffiti` is at index 2
- 16 + 2 = 18

* re-enable keymanager API tests on Windows (#7031)

* Bump nim-snappy to 829133735113951b219e3b108a6bd2146209300b (#7034)

- Bump snappycpp to 1.2.2

* Bump geth to v1.15.6 (#7033)

- https://github.com/ethereum/go-ethereum/releases/tag/v1.15.6

* Bump hoodi to 08dd242abdb1f93026453bf8e63e6bba1c7b1bc1 (#7035)

- Add otterscan hosted instance
- Update bootstrap_nodes.yaml with Teku Hoodi bootnodes

* PeerPool refactoring and custom filtering (#7023)

* Add custom filters for PeerPool.

* Update AllTests.

* Eliminate unneeded constructions.

* Address review comments.
Fix clear() does not properly cleanup empties[] array.

* Attempt to fix crash but still use binary search instead of linear.

* Make iterators safe in async environment.

* Add comments about iterators usage.

* Address review comments.

* Use correct phase0 or electra candidateIdx (#7036)

* fix incorrect indexing of attestationdata in attestationpool (#7037)

* Revert "PeerPool refactoring and custom filtering (#7023)" (#7038)

This reverts commit ad52ba3.

* gate sync manager and sync overseer from asking blobs if given fork is post fulu fork epoch (#7039)

* Peer pool custom filtering (attempt 2). (#7040)

* Add custom filters for PeerPool.

* Update AllTests.

* Eliminate unneeded constructions.

* Address review comments.
Fix clear() does not properly cleanup empties[] array.

* Attempt to fix crash but still use binary search instead of linear.

* Make iterators safe in async environment.

* Add comments about iterators usage.

* Address review comments.

* Fix number of PeerPool crashes.
Switch back to distinct PeerIndex type to get ability to use custom cmp function.
Switch back to linear search because binarySearch could not be working correctly.
Make tests work with `ref Peer` instead of `Peer`, which helps reveal crashes.

* Eliminate debugging symbols.

* Address review comments.

* use EF consensus spec v1.5.0-beta.4 test vectors (#7042)

* fix checking for existing covering Electra aggregates in attestation pool (#7043)

* fix checking for existing covering Electra aggregates in attestation pool

* fix gossip validation usage of Electra covers

* automated consensus spec URL updating to v1.5.0-beta.4 (#7047)

* rm last debugComment for Electra migration (#7048)

* some manual consensus spec v1.5.0-beta.4 URL updates (#7049)

* ci: update NIM_COMMIT value for dedicated job (#7050)

Signed-off-by: Jakub Sokołowski <[email protected]>

* Make MEV decoding less strict for Deneb encoded responses. (#7045)

* Disable strict requirement for `eth-consensus-version` header for Deneb fork.

* Making it working with weird specification requirements, with `eth-consensus-version` is not required for JSON responses.

* Address review comments.

* added custody group count in node/identity endpoint (#7046)

* added custody group count in node/identity endpoint

* fix copyright year

* Use ordered table for attestation packing (#7053)

* added chain aggregates cache tests

* in order to pack electra chained attestations, nimbus scores and
aggregates in an ordered fashion to ease the packing of these
attestations.

However, after the scored and aggregate, a nimbus table (not ordered)
was being used, and such, not producing optimal chain aggregates.

Solution arise from the use of an ordered table.

* review fixes

---------

Co-authored-by: pmmiranda <[email protected]>

* Enable SSE block_gossip event (#7054)

* feat: add BlockGossipObject for sse event

* feat: enable block_gossip topic

* feat: create async queue and emit event

* chore: copyright year

* fix: check before onAdded exists and remove decodeString

* add MsgSource as el too (#7057)

* add MsgSource as el too

* lint

* add: compute_cells for getBlobsV2 (#7056)

* add: compute_cells for getBlobsV2

* updated link

* less verbose Electra aggregation bits logging (#7058)

* bump Gnosis configs to schedule mainnet Pectra hardfork (#7062)

* bump mainnet for Pectra hardfork (#7063)

* updated erigon hoodi cmd (#7065)

* reduce light client data store stack usage (#7067)

* reduce light client data store stack usage

* copyright year linting

* use EF consensus spec v1.5.0-beta.5 test vectors (#7068)

* automated consensus spec URL updating to v1.5.0-beta.5 (#7069)

* Rebase: fulu unto unstable

* fix ncli

* attempt to fix ci

* updated tests

* updates copyright year

---------

Signed-off-by: Jakub Sokołowski <[email protected]>
Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: andri lim <[email protected]>
Co-authored-by: Danyll <[email protected]>
Co-authored-by: Etan Kissling <[email protected]>
Co-authored-by: tersec <[email protected]>
Co-authored-by: Agnish Ghosh <[email protected]>
Co-authored-by: Beatscribe <[email protected]>
Co-authored-by: Jacek Sieka <[email protected]>
Co-authored-by: Jacek Sieka <[email protected]>
Co-authored-by: futreall <[email protected]>
Co-authored-by: Félinàun Chapeau <[email protected]>
Co-authored-by: Jakub Sokołowski <[email protected]>
Co-authored-by: Eugene Kabanov <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kdeme <[email protected]>
Co-authored-by: Pedro Miranda <[email protected]>
Co-authored-by: pmmiranda <[email protected]>
Co-authored-by: ahshum <[email protected]>
Co-authored-by: Dmytro Vovk <[email protected]>
Co-authored-by: Agnish Ghosh <[email protected]>

Author: 18 people

.envrc

@@ -1,16 +1,6 @@
 NBS_ONLY_LOAD_ENV_VARS=1 source env.sh
 
-if command -v nix > /dev/null
-then
-  export NIMBUS_NIX_ENV=1
-
-  cd installer/nix
-  # watch_file tells direnv that changes to any of the watched files
-  # should trigger a re-evalution of the environment
-  watch_file flake.nix
-  watch_file flake.lock
-  watch_file shell.nix
-
-  mkdir -p .flake-profiles
-  eval "$(nix print-dev-env --profile ".flake-profiles/profile")"
+if ! has nix_direnv_version || ! nix_direnv_version 3.0.6; then
+  source_url "https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.6/direnvrc" "sha256-RYcUJaRMf8oF5LznDrlCXbkOQrywm0HDv1VjYGaJGdM="
 fi
+use flake

.github/workflows/ci.yml

@@ -1,5 +1,5 @@
 # beacon_chain
-# Copyright (c) 2020-2024 Status Research & Development GmbH
+# Copyright (c) 2020-2025 Status Research & Development GmbH
 # Licensed and distributed under either of
 #   * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
 #   * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
@@ -37,17 +37,17 @@ jobs:
             cpu: arm64
           - os: windows
             cpu: amd64
-        branch: [~, upstream/version-2-0]
+        branch: [~, upstream/version-2-2]
         exclude:
           - target:
               os: macos
-            branch: upstream/version-2-0
+            branch: upstream/version-2-2
           - target:
               os: windows
-            branch: upstream/version-2-0
+            branch: upstream/version-2-2
         include:
-          - branch: upstream/version-2-0
-            branch-short: version-2-0
+          - branch: upstream/version-2-2
+            branch-short: version-2-2
             nimflags-extra: --mm:refc
           - target:
               os: linux
@@ -205,9 +205,14 @@ jobs:
       # The upload creates a combined report that gets posted as a comment on the PR
       # https://github.com/EnricoMi/publish-unit-test-result-action
       - name: Upload combined results
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
-          name: Unit Test Results ${{ matrix.target.os }}-${{ matrix.target.cpu }}
+          # upload-artifact requires avoiding "/", because "To maintain file
+          # system agnostic behavior, these characters are intentionally not
+          # allowed to prevent potential problems with downloads on different
+          # file systems". However, GitHub Actions workflows do not support a
+          # usual assortment of string functions.
+          name: Unit Test Results ${{ matrix.target.os }}-${{ matrix.target.cpu }}-${{ matrix.branch == 'upstream/version-2-2' && 'version-2-2' || matrix.branch }}
           path: build/*.xml
 
   devbuild:
@@ -253,8 +258,8 @@ jobs:
       - name: Check copyright year
         if: ${{ !cancelled() }} && github.event_name == 'pull_request'
         run: |
-          excluded_files="config.yaml"
-          excluded_extensions="ans|bin|cfg|json|json\\.template|md|png|service|ssz|txt|lock|nix"
+          excluded_files="config.yaml|config.nims|beacon_chain.nimble"
+          excluded_extensions="ans|bin|cfg|yml|json|json\\.template|md|png|service|ssz|tpl|txt|lock|nix|gitignore|envrc"
 
           current_year=$(date +"%Y")
           problematic_files=()
@@ -327,7 +332,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Upload
-        uses: actions/upload-artifact@v3
+        uses: actions/upload-artifact@v4
         with:
           name: Event File
           path: ${{ github.event_path }}