feat(l1): process blocks in batches when syncing and importing

[MarcosNicolau]

Motivation
Accelerate syncing!

Description
This PR introduces block batching during full sync:

1. Instead of storing and computing the state root for each block individually, we now maintain a single state tree for the entire batch, committing it only at the end. This results in one state trie per n blocks instead of one per block (we'll need less storage also).
2. The new full sync process:
   • Request 1024 headers
   • Request 1024 block bodies and collect them
   • Once all blocks are received, process them in batches using a single state trie, which is attached to the last block.
3. Blocks are now stored in a single transaction.
4. State root, receipts root, and request root validation are only required for the last block in the batch.
5. The new add_blocks_in_batch function includes a flag, should_commit_intermediate_tries. When set to true, it stores the tries for each block. This functionality is added to make the hive test pass. Currently, this is handled by verifying if the block is within the STATE_TRIES_TO_KEEP range. In a real syncing scenario, my intuition is that it would be better to wait until we are fully synced and then we would start storing the state of the new blocks and pruning when we reach STATE_TRIES_TO_KEEP.
6. Throughput when syncing is now measured per batches.
7. A new command was added to import blocks in batch

Considerations:

1. Optimize account updates: Instead of inserting updates into the state trie after each block execution, batch them at the end, merging repeated accounts to reduce insertions and improve performance (see Optimize account updates in add_blocks_in_batch #2216) Closes Optimize account updates in add_blocks_in_batch #2216.
2. Improve transaction handling: Avoid committing storage tries to the database separately. Instead, create a single transaction for storing receipts, storage tries, and blocks. This would require additional abstractions for transaction management (see Write batch of blocks in a single transaction in add_blocks_in_batch #2217).
3. This isn't working for levm backend we need it to cache the executions state and persist it between them, as we don't store anything until the final of the batch (see Make add_blocks_in_batch work for LEVM #2218).
4. In ci(core): benchmark for batch block import  #2210 a new ci is added to run a bench comparing main and head branch using import-in-batch

Closes None

[github-actions]

Lines of code report

Total lines added: 477
Total lines removed: 5
Total lines changed: 482

Detailed view

+----------------------------------------------+-------+------+
| File                                         | Lines | Diff |
+----------------------------------------------+-------+------+
| ethrex/cmd/ethrex/cli.rs                     | 199   | +3   |
+----------------------------------------------+-------+------+
| ethrex/cmd/ethrex/subcommands/import.rs      | 61    | -4   |
+----------------------------------------------+-------+------+
| ethrex/crates/blockchain/blockchain.rs       | 540   | +154 |
+----------------------------------------------+-------+------+
| ethrex/crates/common/trie/trie.rs            | 813   | +5   |
+----------------------------------------------+-------+------+
| ethrex/crates/networking/p2p/peer_handler.rs | 515   | -1   |
+----------------------------------------------+-------+------+
| ethrex/crates/networking/p2p/sync.rs         | 563   | +82  |
+----------------------------------------------+-------+------+
| ethrex/crates/storage/api.rs                 | 192   | +7   |
+----------------------------------------------+-------+------+
| ethrex/crates/storage/store.rs               | 1159  | +48  |
+----------------------------------------------+-------+------+
| ethrex/crates/storage/store_db/in_memory.rs  | 506   | +38  |
+----------------------------------------------+-------+------+
| ethrex/crates/storage/store_db/libmdbx.rs    | 1145  | +55  |
+----------------------------------------------+-------+------+
| ethrex/crates/storage/store_db/redb.rs       | 936   | +74  |
+----------------------------------------------+-------+------+
| ethrex/crates/vm/backends/mod.rs             | 318   | +11  |
+----------------------------------------------+-------+------+

[mpaulucci]

Ahh, I understand now. I was hoping we could just call: https://github.com/lambdaclass/lambda_ethereum_rust/blob/0acc5e28b861f88c30cebb6cbfe0230970df25ed/crates/vm/backends/revm.rs#L96 get_state_transitions only once.

We would need to add a execute_blocks inside vm.

[mpaulucci]

we can discuss this later.

[MarcosNicolau]

I've tried this approach but it won't work without making larger modifications to the vm backend.

[MarcosNicolau]

Note: mutating the heads is a temporary fix so that we don't have to refactor the syncing implementation too much. When refactoring this module and separating snap sync from full sync, we should be able to make this more elegant.

[mpaulucci]

do we need this? can't we just call execute_block (previously moving the find_parent_header part)?

[MarcosNicolau]

Yes because this one takes a vm in the param and does not clear its state after the block execution since it calls vm.execute_block_without_clearing_state instead of vm.execute_block().