-
Notifications
You must be signed in to change notification settings - Fork 149
feat: Add Gas dimension tracers #459
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced May 13, 2025
c7b583c
to
122c51f
Compare
f8fb608
to
c4f9da3
Compare
amsanghi
previously approved these changes
May 29, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Mostly looks, have tagged @eljobe to get his input on the protobuf parts.
Also the lint is failing due to some minor issue.
Support live tracing and by RPC, support all pure-CPU opcodes. Support the simple state access opcodes. Rest still TODO
1. EXTCODECOPY with no memory expansion, and warm storage access 2. EXTCODECOPY with memory expansion and warm storage access 3. EXTCODECOPY with memory expansion and cold storage access 4. EXTCODECOPY with no memory expansion and cold storage access
Cold access list SLOAD Warm access list SLOAD
all of the LOG opcodes with both indexed and unindexed data
Selfdestruct to: * cold, empty account target, sender has balance * warm, empty account target, sender has balance * warm, code/value at target, sender has balance * warm, code/value at target, no money to send * cold, code/value at target, sender has balance
Tested: * calling to a contract that is cold but has code * calling to a contract that is cold and has no code
Tested: * calling to a contract that is warm and has code * calling to a contract that is cold and has code * calling to a contract that is warm and has no code * calling to a contract that is cold and has no code
Tested by hand: * Delegatecall to warm address, to a contract with code * Delegatecall to cold address, to a contract with code * Delegatecall to cold address, to a contract with no code * Delegatecall to warm address, to a contract with no code
Tested by Hand: * warm access list+ target has code + zero value * warm access list+ target has code + positive value * warm access list+ target has no code + zero value * warm access list+ target has no code + positive value * cold access list+ target has code + zero value * cold access list+ target has code + positive value * cold access list+ target has no code + zero value * cold access list+ target has no code + positive value
Tested By hand: * CREATE * CREATE2 * CREATE sending value * CREATE2 sending value sending value does not affect gas dimensions but I tested it anyways
Tested by hand in variations: * writing to a new slot * writing to the same slot twice with the same value * writing to the same slot twice with a different value * writing to a slot that was uninitialized and setting it back to zero * writing to a slot that was cold but set to non-zero and setting it to nonzero
This PR exposes the GasUsed, L1GasUsed, and L2GasUsed from the transaction reciept directly in the tracer response. It also changes the tracers for CALL, CALLCODE, STATICCALL, DELEGATECALL, CREATE and CREATE2 to explicitly return the gas consumed by the child execution of the call when the call stack depth is increased by those opcodes.
aa024a9
to
f116416
Compare
eljobe
previously approved these changes
Jun 7, 2025
…racing Gas Dimension Tracer modifications to handle Arbitrum precompiles and stylus
eljobe
approved these changes
Jun 17, 2025
joshuacolvin0
added a commit
to OffchainLabs/nitro
that referenced
this pull request
Jun 18, 2025
Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue. Also pulls in OffchainLabs/go-ethereum#470 OffchainLabs/go-ethereum#469 OffchainLabs/go-ethereum#459
Sneh1999
added a commit
to EspressoSystems/nitro-espresso-integration
that referenced
this pull request
Jul 10, 2025
* Add option to enable PreImages * Enable Golang linter misspell (for spell check) * Update to modern golangci-lint version and enable misspell * Add nil check to SendExpressLaneTransaction RPC method * Increase auctioneer gas margin, lower log level for transient falures * WasmModuleRoot only needed when validating * Remove extraneous `--node.feed.output.disable-signing` parameter Means the same thing as `--node.feed.output.signed`, so just use that. * update bold pin * Test Push0 for eth_simulateV1 * Fix eth_simulateV1 and add test for the same * update geth * Tests exact ink usage of HostIOs Separate program gas-usage tests in two sets: * One set of tests compares the HostIO ink usage with the equivalent EVM opcode gas usage. The objective of these tests is to ensure Stylus HostIOs don't diverge too much from EVM opcodes. These tests make a rough comparison between them. The expected values are not exact and, depending on the HostIO, the error margins can be huge. * The other set compares the HostIO ink usage with exact hardcoded values. The objective of these tests is to ensure there isn't a regression when changing the Stylus implementation. * Solve race in TestRequestForwardingToArchiveNodes system_test * add documentation * Fix ink usage tests When using a Rust Stylus program, updating the Rust compiler can produce a different wasm and then change the amount of ink charged for a few HostIOs. * Add more cases to storage ink-usage test * Add ink-usage test case with clear between loads * Add more cases for read-return-data ink-usage test * Use simpler wasm program in call ink test * Add more cases for account_code ink-usage test * feat: Live tracing Implement go-ethereum's live tracing feature in Nitro - Add the live tracer to go-ethereum/eth/tracers/live. Rebuild nitro - To activate a tracer pass --node.trace.tracername and optionally --node.vmtrace.jsonconfig - Support --dev and production mode, in both sequencer and non-sequencer mode - Changes made 1. Import the tracers/live directory in cmd/nitro.go to call init() functions on tracers 2. Add config params to the nitro binary to activate a tracer from bash 3. Call LiveDirectory.New() in func GetBlockchain() in gethexec/blockchain.go to activate the tracer based on config. The tracer is attached to vmconfig 4. Read the tracer in executionengine.go. Pass it to block_processor.go 5. Missing hook calls OnBlockStart and OnBlockEnd in block_processor.go 6. If tracer hooks are provided and isMsgForPrefetch is false then attach hooks to the stateDB, pass them to vm.Config to spawn an EVM instance with tracing enabled. - Credits: OffchainLabs#2934 * Fix OnBlockStart and OnBlockEnd live tracer hooks not firing * ignore cursor rules * gas dimension trace tests: proof of concept integration test live tracer * gas dimension trace tests: simple calculation passing * gas dimension trace tests: rename file * gas dimension trace tests: SLOAD cold, warm * gas dimension trace tests: BALANCE,EXTCODESIZE,EXTCODEHASH warm and cold * gas dimension trace tests: EXTCODECOPY with memory expansion * gas dimension trace tests: use core/params values and one-dimensional check * gas dimension trace tests: enumerate list of all remaining test cases * gas dimension trace tests: log test cases * gas dimension trace tests: write header comments for all the sstore tests, including doing the math for the expected values. * gas dimension trace tests: refactor reduce code duplication on deploy and call function * gas dimension trace tests: refactor get specific opcode log pattern * gas dimension trace tests: single SSTORE test cases * gas dimension trace tests: multiple SSTORE in a row test cases * gas dimension trace tests: SELFDESTRUCT tests * gas dimension trace tests: DELEGATECALL tests * gas dimension trace tests: STATICCALL tests * gas dimension trace tests: tweak comments + start on CALL * gas dimension trace tests: split into separate files it was becoming far too unweildy in one file for navigation * gas dimension trace tests: CALL tests * gas dimension trace tests: refactor CALL tests order, comment headers * gas dimension trace tests: CALLCODE tests * gas dimension trace tests: CREATE + CREATE2 opcodes * gas dimension trace tests: boilerplate for gas_dim_tx_opcode tests The test itself is failing * gas dimension trace tests: computation-only passing, boilerplate BALANCE add boilerplate for the read only opcodes like BALANCE, SLOAD, etc * gas dimension trace tests: use tracer L1/L2GasUsed + ChildExecutionGas Adapt tests to take advantage of the now exposed L1GasUsed and L2GasUsed values in the tracer output. Check against it in the gas_dim_tx_op_state tests. Change all the tests to handle child execution gas separately from one dimensional gas for CALL, CALLCODE STATICCALL, DELEGATECALL, CREATE, and CREATE2. * gas dimension trace tests: tx opcodes by dimension one sstore test is still failing, need to examine further * gas dimension trace tests: use capped adjusted gas refund * gas dimension trace tests: fix linter issues * gas dimension trace tests: Resolve conflicts from OffchainLabs#3066 and OffchainLabs#3138 * gas dimension trace tests: tests and solgen added to contracts subrepo * gas dimension trace tests: add tests for INVALID and REVERT opcodes * gas dimension trace tests: @tsahee says set MaxRevertGasReject = 0 * gas dimension trace tests: use t.Parallel() speeds up gas-dimension tests from 263.030s -> 133.336s * gas dimension trace tests: address reviewer nits * chore: use contract-local * chore: move gas-dimension-test-contracts Co-authored-by: relyt29 <[email protected]> * chore: restore nitro-contracts pin * chore: remove unused hardhat for now * format: fix * fix: include yulFilePathsGasDimensions * refactor: move mocks to contracts-local * fix: missing contract * fix: build contracts-local * fix: try mocksgen for mocks_legacy_gen * Revert "fix: try mocksgen for mocks_legacy_gen" This reverts commit 0ef85f4. * fix: mocks-contracts-legacy * fix: build Simple in paris to avoid PUSH0 * chore: remove hardcode oz lib * chore: use foundry lib for oz * refactor: mocksgen to localgen * refactor: use makefile in contract-local * fix: localgen import * chore: getting rid of boldmocksgen * refactor: use precompile interface submodule * format: fix * fix: use mocksgen in bold solgen * Refactor HostIO Pricing * Refactor the HostIO pricing to gather all base prices in the pricing.rs file. * General code improvements to maximize readability. Close NIT-3434 Co-authored-by: Raul Jordan <[email protected]> * We have been experiencing a lot of CI flakiness The working theory is that the hardware is just overwhelmed while building the tests and this is because of intensive I/O and system resources, so we're moving our three heaviest workloads to a 32 core 128GB machine. * Fix the "runs-on" string It should be one of the standard box sizes. The "label" in the UI for the runner has nothing to do with the profile of the box (even though for the other two hosted runners, we have labels set that match.) * Go back to using arbitrator-ci * update geth * Update or add copyright and license headers * Add copyright and license headers * Fix Makefile and one solidity contract * Fix the path to the gas-dimensions files * Update openzeppelin-contracts pin * feat: add static foundry profile for gas sensitive build * fix: replicate old config * Revert "fix: replicate old config" This reverts commit 19893c3. * fix: use 0.8.17 for Simple.sol * fix: replicate old config * chore: use solc 0.8.30 for gasdim * fix: use paris for default * specify solidity version in naming Co-authored-by: gzeon <[email protected]> * gas dimension trace tests: precompile tests * refactor: copy Precompile.sol from contracts to contracts-local * gas dimension trace tests: precompile tests pass for SLOAD, SSTORE, CALL * gas dimension trace tests: call into stylus, but tracer has errors * gas dimension trace tests: passing stylus and precompile tests * fix: linter formatting * update go-ethereum pin * revert 2674370 - move back openzeppelin-contracts pin to v4.7.3 * revert 2674370 - move back openzeppelin-contracts pin to v4.7.3 * fix allowed wasm module roots check * Add a line for the submodule-pin-check for oppenzeplin-contracts This way, the presubmit check won't fail. * Add a submodule pin check for the openzeppelin-contracts submodule This has to be merged on the `master` branch before the check can pass in a branch. * Limit parallel jobs within make to 8 Currently, we very frequently see the build step in the CI workflow run into: `error: Resource temporarily unavailable (os error 11)` This is typically because there are too many open file descriptors, and that's possible given that `-j` doesn't limit the number of jobs that make will try to run in parallel. 8 might still be too many, but let's try this value first, and then ramp down until we find a consistently healthy CI sweet spot. * Submodule pin check only works with branch names, not tags This is something we should try to remember for the future. * Update the openzeppelin-pin to the release-v4.7 branch This is only a few commits ahead of the v4.7.3 label, so hopefully, it won't break the build, and it should allow the new submodule-pin-check.yml to work with this configuraiton after OffchainLabs#3323 * Add missing account lookups for trace Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue. Also pulls in OffchainLabs/go-ethereum#470 OffchainLabs/go-ethereum#469 OffchainLabs/go-ethereum#459 * Remove stateless_block_validator public function for reader, streamer and tracker * Fix the Docker build for contracts-local In the context of a Docker build, the contracts-local directory is not part of a git repo, which causes forge install to fail. Also, the foundryup line is needed to get the more recent versions of the solc compiler. * fix ci * The foundryup can't run in the Makefile or it will break CI The thing we really need is for folks to be using a modern enough version of foundry to have the newer solc compiler. So, that's being done in the Dockerfile itself now. * Update submodule geth * Handle merge conflicts * Fix go import * go mod tidy * Fix * Fix staker * Fix caff node * Attempt to update version of bold. * Current state of merge In this commit make build is working, but make build-replay env is currently failing. I haven't been able to run the tests yet. * Update contracts submodule commit * Update submodules * Add bindings * Fix tests * Update contracts-legacy commit * Fix merge errors * Result of go mod tidy * Lint * Update CI * add logging to caff node test to determine reason for panic * Fix caff node * Fix caff node config test * Fix golangci.yml * Lint files * Lint files * Lint files * Lint files * Fix data poster.go * Lint files * Lint files * Fix * Fix * Fix TestOversizedMessages * Update CI time for Espresso E2E test * Fix tests * Add time for CI * Fix free disk space * Fix test and remove one from the skip list * Remove escape hatch test * Faster CI * buildjet * Go * WIP * Github path * CI upterm * ubuntu latest * Limit parallel * Skip one recource consuming test * Avoid writing logs to files * Skip tests, use self-hosted runner * Fix * Fix redis tests * Remove espresso-network-go * Address comments --------- Co-authored-by: Aman Sanghi <[email protected]> Co-authored-by: Aman Sanghi <[email protected]> Co-authored-by: Tristan Wilson <[email protected]> Co-authored-by: Tristan-Wilson <[email protected]> Co-authored-by: Pepper Lebeck-Jobe <[email protected]> Co-authored-by: Raul Jordan <[email protected]> Co-authored-by: Joshua Colvin <[email protected]> Co-authored-by: Gabriel de Quadros Ligneul <[email protected]> Co-authored-by: Ganesh Vanahalli <[email protected]> Co-authored-by: Tsahi Zidenberg <[email protected]> Co-authored-by: rollchad <[email protected]> Co-authored-by: relyt29 <[email protected]> Co-authored-by: gzeon <[email protected]> Co-authored-by: Gabriel de Quadros Ligneul <[email protected]> Co-authored-by: Pepper Lebeck-Jobe <[email protected]> Co-authored-by: ǝuᴉu ʎʇuǝʍʇ ʇʎlǝɹ <[email protected]> Co-authored-by: gzeon <[email protected]> Co-authored-by: Maciej Kulawik <[email protected]> Co-authored-by: Jeremy <[email protected]> Co-authored-by: Sneh Koul <[email protected]>
Sneh1999
added a commit
to EspressoSystems/nitro-espresso-integration
that referenced
this pull request
Jul 17, 2025
* Allow validator failures to be able to retry in case of failure * skip l3 tests until we have stronger ci machines * include delayed message lookup * gci * Refactor validator wallet * Add zero-config HA coordination for Timeboost auctioneer Implements automatic primary/secondary failover for multiple auctioneer instances using a single Redis coordination key. This ensures high availability without requiring any additional configuration - just run multiple instances with the same Redis URL. The design generates a unique ID for each auctioneer on startup, with all instances competing for a single "auctioneer.chosen" key in Redis. Only the primary instance that holds this key consumes bids from the Redis stream. The lock value stores both the instance ID and timestamp in the format "id:timestamp" to enable stale lock detection without relying on Redis TTL mechanisms. When acquiring stale locks, the system uses a delete + SetNX pattern to ensure atomic acquisition. For failover protection, the system employs timestamp-based staleness detection that doesn't depend on Redis passive expiration timing. The delete + SetNX pattern prevents race conditions by ensuring only one instance can take over at a time. The stale lock timeout is set to 3x the IdletimeToAutoclaim duration to ensure proper coordination. Failover Timeline Example: T=0s: Primary auctioneer crashes T=1s: Primary's unacked messages become autoclaimable (IdletimeToAutoclaim) T=3s: Secondary detects stale lock and acquires it T=3s+: Secondary starts consuming messages This timing ensures that all messages are autoclaimable before the secondary takes over, preserving message ordering during failover and preventing split-brain scenarios. The result is a clean handoff with approximately 3 second failover time. Regarding auction resolution, it's still possible that an auctioneer could crash when it was supposed to resolve the auction, and the new primary that comes up doesn't resolve it because resolution happens at a fixed time. Further work is needed if we want to handle this case - we'd need to check for resolution events and retry if no resolution occurred within the expected window. The auction contract does prevent double resolution, ensuring correctness even in edge cases. For S3 storage, bids are stored locally before being uploaded to S3 on a default 15 minute interval. During failover, any bids not yet uploaded by a crashed primary may be lost. This is acceptable as the data is primarily for research purposes. The implementation includes comprehensive failover tests covering basic failover scenarios, multiple instance coordination, concurrent start handling, rapid recovery prevention, stale lock detection, invalid lock format recovery, and message reprocessing after failover. Usage (no config changes needed): # Start primary ./nitro --timeboost.auctioneer.redis-url="redis://localhost:6379" # Start secondary (identical command) ./nitro --timeboost.auctioneer.redis-url="redis://localhost:6379" The first instance to start becomes primary. If it fails, the secondary automatically takes over after detecting the stale lock. * Lower log levels for pubsub producer errors Changed consumer error and timeout logs from Error to Debug level in the pubsub producer. These aren't always system errors - in the timeboost bid flow, both consumer rejections (due to timing constraints) and timeouts (bids unprocessed until round end) are legitimate business logic behaviors. Since the Promise pattern already allows callers to check and handle errors appropriately, callers can decide whether to log at higher levels based on their specific use case. This change eliminates false alarms in monitoring while maintaining error visibility through the Promise API. * lint * Add descriptions to seq-coordinator config options * Fix L2PricingState UpdatePricingModel panic if chain owner sets insane parameters * Add option to enable PreImages * Enable Golang linter misspell (for spell check) * Update to modern golangci-lint version and enable misspell * Add nil check to SendExpressLaneTransaction RPC method * Increase auctioneer gas margin, lower log level for transient falures * WasmModuleRoot only needed when validating * Remove extraneous `--node.feed.output.disable-signing` parameter Means the same thing as `--node.feed.output.signed`, so just use that. * update bold pin * Test Push0 for eth_simulateV1 * Fix eth_simulateV1 and add test for the same * update geth * Tests exact ink usage of HostIOs Separate program gas-usage tests in two sets: * One set of tests compares the HostIO ink usage with the equivalent EVM opcode gas usage. The objective of these tests is to ensure Stylus HostIOs don't diverge too much from EVM opcodes. These tests make a rough comparison between them. The expected values are not exact and, depending on the HostIO, the error margins can be huge. * The other set compares the HostIO ink usage with exact hardcoded values. The objective of these tests is to ensure there isn't a regression when changing the Stylus implementation. * Solve race in TestRequestForwardingToArchiveNodes system_test * add documentation * Fix ink usage tests When using a Rust Stylus program, updating the Rust compiler can produce a different wasm and then change the amount of ink charged for a few HostIOs. * Add more cases to storage ink-usage test * Add ink-usage test case with clear between loads * Add more cases for read-return-data ink-usage test * Use simpler wasm program in call ink test * Add more cases for account_code ink-usage test * feat: Live tracing Implement go-ethereum's live tracing feature in Nitro - Add the live tracer to go-ethereum/eth/tracers/live. Rebuild nitro - To activate a tracer pass --node.trace.tracername and optionally --node.vmtrace.jsonconfig - Support --dev and production mode, in both sequencer and non-sequencer mode - Changes made 1. Import the tracers/live directory in cmd/nitro.go to call init() functions on tracers 2. Add config params to the nitro binary to activate a tracer from bash 3. Call LiveDirectory.New() in func GetBlockchain() in gethexec/blockchain.go to activate the tracer based on config. The tracer is attached to vmconfig 4. Read the tracer in executionengine.go. Pass it to block_processor.go 5. Missing hook calls OnBlockStart and OnBlockEnd in block_processor.go 6. If tracer hooks are provided and isMsgForPrefetch is false then attach hooks to the stateDB, pass them to vm.Config to spawn an EVM instance with tracing enabled. - Credits: OffchainLabs#2934 * Fix OnBlockStart and OnBlockEnd live tracer hooks not firing * ignore cursor rules * gas dimension trace tests: proof of concept integration test live tracer * gas dimension trace tests: simple calculation passing * gas dimension trace tests: rename file * gas dimension trace tests: SLOAD cold, warm * gas dimension trace tests: BALANCE,EXTCODESIZE,EXTCODEHASH warm and cold * gas dimension trace tests: EXTCODECOPY with memory expansion * gas dimension trace tests: use core/params values and one-dimensional check * gas dimension trace tests: enumerate list of all remaining test cases * gas dimension trace tests: log test cases * gas dimension trace tests: write header comments for all the sstore tests, including doing the math for the expected values. * gas dimension trace tests: refactor reduce code duplication on deploy and call function * gas dimension trace tests: refactor get specific opcode log pattern * gas dimension trace tests: single SSTORE test cases * gas dimension trace tests: multiple SSTORE in a row test cases * gas dimension trace tests: SELFDESTRUCT tests * gas dimension trace tests: DELEGATECALL tests * gas dimension trace tests: STATICCALL tests * gas dimension trace tests: tweak comments + start on CALL * gas dimension trace tests: split into separate files it was becoming far too unweildy in one file for navigation * gas dimension trace tests: CALL tests * gas dimension trace tests: refactor CALL tests order, comment headers * gas dimension trace tests: CALLCODE tests * gas dimension trace tests: CREATE + CREATE2 opcodes * gas dimension trace tests: boilerplate for gas_dim_tx_opcode tests The test itself is failing * gas dimension trace tests: computation-only passing, boilerplate BALANCE add boilerplate for the read only opcodes like BALANCE, SLOAD, etc * gas dimension trace tests: use tracer L1/L2GasUsed + ChildExecutionGas Adapt tests to take advantage of the now exposed L1GasUsed and L2GasUsed values in the tracer output. Check against it in the gas_dim_tx_op_state tests. Change all the tests to handle child execution gas separately from one dimensional gas for CALL, CALLCODE STATICCALL, DELEGATECALL, CREATE, and CREATE2. * gas dimension trace tests: tx opcodes by dimension one sstore test is still failing, need to examine further * gas dimension trace tests: use capped adjusted gas refund * gas dimension trace tests: fix linter issues * gas dimension trace tests: Resolve conflicts from OffchainLabs#3066 and OffchainLabs#3138 * gas dimension trace tests: tests and solgen added to contracts subrepo * gas dimension trace tests: add tests for INVALID and REVERT opcodes * gas dimension trace tests: @tsahee says set MaxRevertGasReject = 0 * gas dimension trace tests: use t.Parallel() speeds up gas-dimension tests from 263.030s -> 133.336s * gas dimension trace tests: address reviewer nits * chore: use contract-local * chore: move gas-dimension-test-contracts Co-authored-by: relyt29 <[email protected]> * chore: restore nitro-contracts pin * chore: remove unused hardhat for now * format: fix * fix: include yulFilePathsGasDimensions * refactor: move mocks to contracts-local * fix: missing contract * fix: build contracts-local * fix: try mocksgen for mocks_legacy_gen * Revert "fix: try mocksgen for mocks_legacy_gen" This reverts commit 0ef85f4. * fix: mocks-contracts-legacy * fix: build Simple in paris to avoid PUSH0 * chore: remove hardcode oz lib * chore: use foundry lib for oz * refactor: mocksgen to localgen * refactor: use makefile in contract-local * fix: localgen import * chore: getting rid of boldmocksgen * refactor: use precompile interface submodule * format: fix * fix: use mocksgen in bold solgen * Refactor HostIO Pricing * Refactor the HostIO pricing to gather all base prices in the pricing.rs file. * General code improvements to maximize readability. Close NIT-3434 Co-authored-by: Raul Jordan <[email protected]> * We have been experiencing a lot of CI flakiness The working theory is that the hardware is just overwhelmed while building the tests and this is because of intensive I/O and system resources, so we're moving our three heaviest workloads to a 32 core 128GB machine. * Fix the "runs-on" string It should be one of the standard box sizes. The "label" in the UI for the runner has nothing to do with the profile of the box (even though for the other two hosted runners, we have labels set that match.) * Go back to using arbitrator-ci * pin submodules * Celestia DA - Fix anytrust fallback issues - Add `Type()` to dapWritter to differentiate which dapWritter is being used (`celestia` or `anytrust`) - pin contracts branch and nitro testnode branch Update .gitmodules Update download-machine.sh Update Dockerfile Fix interface methods Merge branch 'celestia-v3.1.2' into celestia-v3.2.1 Pin contracts and testnode * remove Celestia config * adapt celestia to daProvider client * fix Dockerfile and pin submodules * fix dasserver * Update Dockerfile * pin submodules * update geth * Update or add copyright and license headers * Add copyright and license headers * Fix Makefile and one solidity contract * Fix the path to the gas-dimensions files * Update openzeppelin-contracts pin * feat: add static foundry profile for gas sensitive build * fix: replicate old config * Revert "fix: replicate old config" This reverts commit 19893c3. * fix: use 0.8.17 for Simple.sol * fix: replicate old config * chore: use solc 0.8.30 for gasdim * fix: use paris for default * specify solidity version in naming Co-authored-by: gzeon <[email protected]> * gas dimension trace tests: precompile tests * refactor: copy Precompile.sol from contracts to contracts-local * gas dimension trace tests: precompile tests pass for SLOAD, SSTORE, CALL * gas dimension trace tests: call into stylus, but tracer has errors * gas dimension trace tests: passing stylus and precompile tests * fix: linter formatting * update go-ethereum pin * revert 2674370 - move back openzeppelin-contracts pin to v4.7.3 * revert 2674370 - move back openzeppelin-contracts pin to v4.7.3 * fix allowed wasm module roots check * Add a line for the submodule-pin-check for oppenzeplin-contracts This way, the presubmit check won't fail. * Add a submodule pin check for the openzeppelin-contracts submodule This has to be merged on the `master` branch before the check can pass in a branch. * Limit parallel jobs within make to 8 Currently, we very frequently see the build step in the CI workflow run into: `error: Resource temporarily unavailable (os error 11)` This is typically because there are too many open file descriptors, and that's possible given that `-j` doesn't limit the number of jobs that make will try to run in parallel. 8 might still be too many, but let's try this value first, and then ramp down until we find a consistently healthy CI sweet spot. * Submodule pin check only works with branch names, not tags This is something we should try to remember for the future. * Update the openzeppelin-pin to the release-v4.7 branch This is only a few commits ahead of the v4.7.3 label, so hopefully, it won't break the build, and it should allow the new submodule-pin-check.yml to work with this configuraiton after OffchainLabs#3323 * Add missing account lookups for trace Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue. Also pulls in OffchainLabs/go-ethereum#470 OffchainLabs/go-ethereum#469 OffchainLabs/go-ethereum#459 * Remove stateless_block_validator public function for reader, streamer and tracker * Fix the Docker build for contracts-local In the context of a Docker build, the contracts-local directory is not part of a git repo, which causes forge install to fail. Also, the foundryup line is needed to get the more recent versions of the solc compiler. * fix ci * The foundryup can't run in the Makefile or it will break CI The thing we really need is for folks to be using a modern enough version of foundry to have the newer solc compiler. So, that's being done in the Dockerfile itself now. * Update nitro-testnode * pin celestia contracts and testnode * pin deps * Fix make buil * Update contracts * Fix build-replay-env * Fix tests * Fix tests * Lint files * fix prepare-espresso-crypto-helper * Fix * Don't rebuild espresso-crypto * Lint fix * Remove escape hatch test * Fix tests * Skeptical fix * Self-hosted ci runner * Update bold * Revert "Update bold" This reverts commit 0ebb448. * Update bold * Update tx-streamer to enqueue batches of messages to espresso (#648) * fix enqueued * pull last confirmed pos from database * push logs * fix log * Fix CI --------- Co-authored-by: Jeremy <[email protected]> (cherry picked from commit a34cf64) * Fix * Fix linter * Update contract submodule path * Fix * Revert cmd/nitro/init.go * Revert "Fix" This reverts commit 1c06f5c. * Update bold submodule --------- Co-authored-by: Aman Sanghi <[email protected]> Co-authored-by: Aman Sanghi <[email protected]> Co-authored-by: Raul Jordan <[email protected]> Co-authored-by: Tristan Wilson <[email protected]> Co-authored-by: Joshua Colvin <[email protected]> Co-authored-by: Tristan-Wilson <[email protected]> Co-authored-by: Pepper Lebeck-Jobe <[email protected]> Co-authored-by: Gabriel de Quadros Ligneul <[email protected]> Co-authored-by: Ganesh Vanahalli <[email protected]> Co-authored-by: Tsahi Zidenberg <[email protected]> Co-authored-by: rollchad <[email protected]> Co-authored-by: relyt29 <[email protected]> Co-authored-by: gzeon <[email protected]> Co-authored-by: Gabriel de Quadros Ligneul <[email protected]> Co-authored-by: Pepper Lebeck-Jobe <[email protected]> Co-authored-by: Diego <[email protected]> Co-authored-by: ǝuᴉu ʎʇuǝʍʇ ʇʎlǝɹ <[email protected]> Co-authored-by: gzeon <[email protected]> Co-authored-by: Maciej Kulawik <[email protected]> Co-authored-by: Sneh Koul <[email protected]>
Zaptoss
pushed a commit
to distributed-lab/nitro-espresso-integration
that referenced
this pull request
Aug 11, 2025
* Add option to enable PreImages * Enable Golang linter misspell (for spell check) * Update to modern golangci-lint version and enable misspell * Add nil check to SendExpressLaneTransaction RPC method * Increase auctioneer gas margin, lower log level for transient falures * WasmModuleRoot only needed when validating * Remove extraneous `--node.feed.output.disable-signing` parameter Means the same thing as `--node.feed.output.signed`, so just use that. * update bold pin * Test Push0 for eth_simulateV1 * Fix eth_simulateV1 and add test for the same * update geth * Tests exact ink usage of HostIOs Separate program gas-usage tests in two sets: * One set of tests compares the HostIO ink usage with the equivalent EVM opcode gas usage. The objective of these tests is to ensure Stylus HostIOs don't diverge too much from EVM opcodes. These tests make a rough comparison between them. The expected values are not exact and, depending on the HostIO, the error margins can be huge. * The other set compares the HostIO ink usage with exact hardcoded values. The objective of these tests is to ensure there isn't a regression when changing the Stylus implementation. * Solve race in TestRequestForwardingToArchiveNodes system_test * add documentation * Fix ink usage tests When using a Rust Stylus program, updating the Rust compiler can produce a different wasm and then change the amount of ink charged for a few HostIOs. * Add more cases to storage ink-usage test * Add ink-usage test case with clear between loads * Add more cases for read-return-data ink-usage test * Use simpler wasm program in call ink test * Add more cases for account_code ink-usage test * feat: Live tracing Implement go-ethereum's live tracing feature in Nitro - Add the live tracer to go-ethereum/eth/tracers/live. Rebuild nitro - To activate a tracer pass --node.trace.tracername and optionally --node.vmtrace.jsonconfig - Support --dev and production mode, in both sequencer and non-sequencer mode - Changes made 1. Import the tracers/live directory in cmd/nitro.go to call init() functions on tracers 2. Add config params to the nitro binary to activate a tracer from bash 3. Call LiveDirectory.New() in func GetBlockchain() in gethexec/blockchain.go to activate the tracer based on config. The tracer is attached to vmconfig 4. Read the tracer in executionengine.go. Pass it to block_processor.go 5. Missing hook calls OnBlockStart and OnBlockEnd in block_processor.go 6. If tracer hooks are provided and isMsgForPrefetch is false then attach hooks to the stateDB, pass them to vm.Config to spawn an EVM instance with tracing enabled. - Credits: OffchainLabs#2934 * Fix OnBlockStart and OnBlockEnd live tracer hooks not firing * ignore cursor rules * gas dimension trace tests: proof of concept integration test live tracer * gas dimension trace tests: simple calculation passing * gas dimension trace tests: rename file * gas dimension trace tests: SLOAD cold, warm * gas dimension trace tests: BALANCE,EXTCODESIZE,EXTCODEHASH warm and cold * gas dimension trace tests: EXTCODECOPY with memory expansion * gas dimension trace tests: use core/params values and one-dimensional check * gas dimension trace tests: enumerate list of all remaining test cases * gas dimension trace tests: log test cases * gas dimension trace tests: write header comments for all the sstore tests, including doing the math for the expected values. * gas dimension trace tests: refactor reduce code duplication on deploy and call function * gas dimension trace tests: refactor get specific opcode log pattern * gas dimension trace tests: single SSTORE test cases * gas dimension trace tests: multiple SSTORE in a row test cases * gas dimension trace tests: SELFDESTRUCT tests * gas dimension trace tests: DELEGATECALL tests * gas dimension trace tests: STATICCALL tests * gas dimension trace tests: tweak comments + start on CALL * gas dimension trace tests: split into separate files it was becoming far too unweildy in one file for navigation * gas dimension trace tests: CALL tests * gas dimension trace tests: refactor CALL tests order, comment headers * gas dimension trace tests: CALLCODE tests * gas dimension trace tests: CREATE + CREATE2 opcodes * gas dimension trace tests: boilerplate for gas_dim_tx_opcode tests The test itself is failing * gas dimension trace tests: computation-only passing, boilerplate BALANCE add boilerplate for the read only opcodes like BALANCE, SLOAD, etc * gas dimension trace tests: use tracer L1/L2GasUsed + ChildExecutionGas Adapt tests to take advantage of the now exposed L1GasUsed and L2GasUsed values in the tracer output. Check against it in the gas_dim_tx_op_state tests. Change all the tests to handle child execution gas separately from one dimensional gas for CALL, CALLCODE STATICCALL, DELEGATECALL, CREATE, and CREATE2. * gas dimension trace tests: tx opcodes by dimension one sstore test is still failing, need to examine further * gas dimension trace tests: use capped adjusted gas refund * gas dimension trace tests: fix linter issues * gas dimension trace tests: Resolve conflicts from OffchainLabs#3066 and OffchainLabs#3138 * gas dimension trace tests: tests and solgen added to contracts subrepo * gas dimension trace tests: add tests for INVALID and REVERT opcodes * gas dimension trace tests: @tsahee says set MaxRevertGasReject = 0 * gas dimension trace tests: use t.Parallel() speeds up gas-dimension tests from 263.030s -> 133.336s * gas dimension trace tests: address reviewer nits * chore: use contract-local * chore: move gas-dimension-test-contracts Co-authored-by: relyt29 <[email protected]> * chore: restore nitro-contracts pin * chore: remove unused hardhat for now * format: fix * fix: include yulFilePathsGasDimensions * refactor: move mocks to contracts-local * fix: missing contract * fix: build contracts-local * fix: try mocksgen for mocks_legacy_gen * Revert "fix: try mocksgen for mocks_legacy_gen" This reverts commit 0ef85f4. * fix: mocks-contracts-legacy * fix: build Simple in paris to avoid PUSH0 * chore: remove hardcode oz lib * chore: use foundry lib for oz * refactor: mocksgen to localgen * refactor: use makefile in contract-local * fix: localgen import * chore: getting rid of boldmocksgen * refactor: use precompile interface submodule * format: fix * fix: use mocksgen in bold solgen * Refactor HostIO Pricing * Refactor the HostIO pricing to gather all base prices in the pricing.rs file. * General code improvements to maximize readability. Close NIT-3434 Co-authored-by: Raul Jordan <[email protected]> * We have been experiencing a lot of CI flakiness The working theory is that the hardware is just overwhelmed while building the tests and this is because of intensive I/O and system resources, so we're moving our three heaviest workloads to a 32 core 128GB machine. * Fix the "runs-on" string It should be one of the standard box sizes. The "label" in the UI for the runner has nothing to do with the profile of the box (even though for the other two hosted runners, we have labels set that match.) * Go back to using arbitrator-ci * update geth * Update or add copyright and license headers * Add copyright and license headers * Fix Makefile and one solidity contract * Fix the path to the gas-dimensions files * Update openzeppelin-contracts pin * feat: add static foundry profile for gas sensitive build * fix: replicate old config * Revert "fix: replicate old config" This reverts commit 19893c3. * fix: use 0.8.17 for Simple.sol * fix: replicate old config * chore: use solc 0.8.30 for gasdim * fix: use paris for default * specify solidity version in naming Co-authored-by: gzeon <[email protected]> * gas dimension trace tests: precompile tests * refactor: copy Precompile.sol from contracts to contracts-local * gas dimension trace tests: precompile tests pass for SLOAD, SSTORE, CALL * gas dimension trace tests: call into stylus, but tracer has errors * gas dimension trace tests: passing stylus and precompile tests * fix: linter formatting * update go-ethereum pin * revert 2674370 - move back openzeppelin-contracts pin to v4.7.3 * revert 2674370 - move back openzeppelin-contracts pin to v4.7.3 * fix allowed wasm module roots check * Add a line for the submodule-pin-check for oppenzeplin-contracts This way, the presubmit check won't fail. * Add a submodule pin check for the openzeppelin-contracts submodule This has to be merged on the `master` branch before the check can pass in a branch. * Limit parallel jobs within make to 8 Currently, we very frequently see the build step in the CI workflow run into: `error: Resource temporarily unavailable (os error 11)` This is typically because there are too many open file descriptors, and that's possible given that `-j` doesn't limit the number of jobs that make will try to run in parallel. 8 might still be too many, but let's try this value first, and then ramp down until we find a consistently healthy CI sweet spot. * Submodule pin check only works with branch names, not tags This is something we should try to remember for the future. * Update the openzeppelin-pin to the release-v4.7 branch This is only a few commits ahead of the v4.7.3 label, so hopefully, it won't break the build, and it should allow the new submodule-pin-check.yml to work with this configuraiton after OffchainLabs#3323 * Add missing account lookups for trace Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue. Also pulls in OffchainLabs/go-ethereum#470 OffchainLabs/go-ethereum#469 OffchainLabs/go-ethereum#459 * Remove stateless_block_validator public function for reader, streamer and tracker * Fix the Docker build for contracts-local In the context of a Docker build, the contracts-local directory is not part of a git repo, which causes forge install to fail. Also, the foundryup line is needed to get the more recent versions of the solc compiler. * fix ci * The foundryup can't run in the Makefile or it will break CI The thing we really need is for folks to be using a modern enough version of foundry to have the newer solc compiler. So, that's being done in the Dockerfile itself now. * Update submodule geth * Handle merge conflicts * Fix go import * go mod tidy * Fix * Fix staker * Fix caff node * Attempt to update version of bold. * Current state of merge In this commit make build is working, but make build-replay env is currently failing. I haven't been able to run the tests yet. * Update contracts submodule commit * Update submodules * Add bindings * Fix tests * Update contracts-legacy commit * Fix merge errors * Result of go mod tidy * Lint * Update CI * add logging to caff node test to determine reason for panic * Fix caff node * Fix caff node config test * Fix golangci.yml * Lint files * Lint files * Lint files * Lint files * Fix data poster.go * Lint files * Lint files * Fix * Fix * Fix TestOversizedMessages * Update CI time for Espresso E2E test * Fix tests * Add time for CI * Fix free disk space * Fix test and remove one from the skip list * Remove escape hatch test * Faster CI * buildjet * Go * WIP * Github path * CI upterm * ubuntu latest * Limit parallel * Skip one recource consuming test * Avoid writing logs to files * Skip tests, use self-hosted runner * Fix * Fix redis tests * Remove espresso-network-go * Address comments --------- Co-authored-by: Aman Sanghi <[email protected]> Co-authored-by: Aman Sanghi <[email protected]> Co-authored-by: Tristan Wilson <[email protected]> Co-authored-by: Tristan-Wilson <[email protected]> Co-authored-by: Pepper Lebeck-Jobe <[email protected]> Co-authored-by: Raul Jordan <[email protected]> Co-authored-by: Joshua Colvin <[email protected]> Co-authored-by: Gabriel de Quadros Ligneul <[email protected]> Co-authored-by: Ganesh Vanahalli <[email protected]> Co-authored-by: Tsahi Zidenberg <[email protected]> Co-authored-by: rollchad <[email protected]> Co-authored-by: relyt29 <[email protected]> Co-authored-by: gzeon <[email protected]> Co-authored-by: Gabriel de Quadros Ligneul <[email protected]> Co-authored-by: Pepper Lebeck-Jobe <[email protected]> Co-authored-by: ǝuᴉu ʎʇuǝʍʇ ʇʎlǝɹ <[email protected]> Co-authored-by: gzeon <[email protected]> Co-authored-by: Maciej Kulawik <[email protected]> Co-authored-by: Jeremy <[email protected]> Co-authored-by: Sneh Koul <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces two native and two live tracers,
txGasDimensionLogger
andtxGasDimensionByOpcode
, that on a per-opcode basis track the consumption of gas across multiple dimensions:txGasDimensionLogger
The logger is based off the default geth opcode struct logger. It will return each opcode, and its associated gas dimensional consumption, e.g.
txGasDimensionByOpcode
The transaction gas dimensions by opcode tracer returns the total sum of gas consumed by each different opcode, split out by dimension, e.g:
Tracer Invocation
The tracers need to have a parameter of a transaction hash to operate upon. If you have that and your node either is an archive node with the state or the transaction hash you're tracing is in the last 128 blocks then you can run the tracer like so:
Live Tracers
The live tracers are identical use the native tracers under the hood. The difference with the live tracers is that they dump the output of the tracer to disk in protobuf format to then in in order to save on storage space relative to json. Python or whatever can then generate serialization code from the .proto file to ingest the protobuf output.
Live Tracer Invocation
The live tracers can be run like so:
For example, this will create a directory called
gas-dimension-logs
in the root of the repo that will have each block as a subfolder, and a protobuf dumped file for the gas dimensions by transaction. The tracer expects a chain configuration because I was unable to get the chain configuration in the tracer creation context without having a bigger diff. So it expects it to be passed in by json - you can use the nitro repository's chain info as a reference for a chain configuration if you don't have one.this PR is identical to and replaces #457.