Skip to content

Conversation

relyt29
Copy link
Collaborator

@relyt29 relyt29 commented May 13, 2025

This PR introduces two native and two live tracers, txGasDimensionLogger and txGasDimensionByOpcode, that on a per-opcode basis track the consumption of gas across multiple dimensions:

  1. Computation - opcodes that operate on the stack, memory, simple math
  2. State access - reading and writing to existing state, storage slots, accounts tree etc
  3. State growth - expanding the amount of state in existence, either by setting a value in a smart contract from 0->Non-Zero or by creating a new account to have to track in the accounts tree
  4. History growth - logging log events which are stored in the history (as opposed to the state)

txGasDimensionLogger

The logger is based off the default geth opcode struct logger. It will return each opcode, and its associated gas dimensional consumption, e.g.

{                                                                                               
  "jsonrpc": "2.0",                                                                             
  "id": 1,                                                                                      
  "result": {                                                                                   
    "gasUsed": 994729,                                                                          
    "gasUsedForL1": 968000,                                                                     
    "gasUsedForL2": 26729,                                                                      
    "intrinsicGas": 21064,                                                                      
    "adjustedRefund": 0,                                                                        
    "failed": false,                                                                            
    "txHash": "0x23eb17459bc5cbdea76949afa741a4cdcec06183d26de389c6d3b168be3800be",             
    "blockTimestamp": 1746807439,                                                               
    "blockNumber": 17,                                                                          
    "dim": [                                                                                    
      {                                                                                         
        "pc": 0,                                                                                
        "op": "PUSH1",                                                                          
        "depth": 1,                                                                             
        "cost": 3,                                                                              
        "cpu": 3                                                                                
      },    
      {                                                                                         
        "pc": 903,                                                                              
        "op": "SSTORE",                                                                         
        "depth": 1,                                                                             
        "cost": 100,                                                                            
        "rw": 100,                                                                              
        "refund": 4800                                                                          
      },  

txGasDimensionByOpcode

The transaction gas dimensions by opcode tracer returns the total sum of gas consumed by each different opcode, split out by dimension, e.g:

                                                                                                                                                                                             
  "jsonrpc": "2.0",                                                                                                                                                                           
  "id": 1,                                                                                                                                                                                    
  "result": {                                                                                                                                                                                 
    "gasUsed": 1003063,                                                                                                                                                                       
    "gasUsedForL1": 968000,                                                                                                                                                                   
    "gasUsedForL2": 35063,                                                                                                                                                                    
    "intrinsicGas": 21064,                                                                                                                                                                    
    "adjustedRefund": 8765,                                                                                                                                                                   
    "failed": false,                                                                                                                                                                          
    "txHash": "0xe6d0c5c6e330b7d993855afd610d01fc1d8f8257a70b83684a2d24a9418afc5a",                                                                                                           
    "blockTimestamp": 1746646719,                                                                                                                                                             
    "blockNumber": 18,                                                                                                                                                                        
    "dimensions": {                                                                                                                                                                           
      "ADD": {                                                                                                                                                                                
        "gas1d": 6,                                                                                                                                                                              
        "cpu": 6             
      },
      "SLOAD": {                                                                                                                                                                                 
        "gas1d": 2200,                                                                                                                                                                              
        "cpu": 200,                                                                                                                                                                              
        "rw": 2000                                                                                                                                                                               
      },                                                                                                                                                                                         
      "SSTORE": {                                                                                                                                                                                
        "gas1d": 20100,                                                                                                                                                                             
        "cpu": 0,                                                                                                                                                                                
        "rw": 100,                                                                                                                                                                               
        "growth": 20000,                                                                                                                                                                             
        "refund": 19900                                                                                                                                                                              
      },   

Tracer Invocation

The tracers need to have a parameter of a transaction hash to operate upon. If you have that and your node either is an archive node with the state or the transaction hash you're tracing is in the last 128 blocks then you can run the tracer like so:

export TX_HASH_TO_LOOK_AT="0x348f7124e88620273caee233f25472009c5ba92a2d6743ae161045da8c533f6c";
 
curl "http://127.0.0.1:8547" \
-X POST \
-H "Content-Type: application/json" \
--data '{  
    "method":"debug_traceTransaction",
    "params":["'"$TX_HASH_TO_LOOK_AT"'", {"tracer": "txGasDimensionByOpcode"}],
    "id":1,
    "jsonrpc":"2.0"
}' | jq > tx.byOpcode.json;   
                                     
curl "http://127.0.0.1:8547" \
-X POST \
-H "Content-Type: application/json" \
--data '{
    "method":"debug_traceTransaction",
    "params":["'"$TX_HASH_TO_LOOK_AT"'", {"tracer":"txGasDimensionLogger"}],
    "id":1,
    "jsonrpc":"2.0"
}' | jq > tx.Logger.json;

Live Tracers

The live tracers are identical use the native tracers under the hood. The difference with the live tracers is that they dump the output of the tracer to disk in protobuf format to then in in order to save on storage space relative to json. Python or whatever can then generate serialization code from the .proto file to ingest the protobuf output.

Live Tracer Invocation

The live tracers can be run like so:

./target/bin/nitro --dev \
    --execution.vmtrace.tracer-name "txGasDimensionByOpcode" \
    --execution.vmtrace.json-config '{"path":"gas-dimension-logs","chainConfig":{"chainId":42161,"homesteadBlock":0,"daoForkBlock":null,"daoForkSupport":true,"eip150Block":0,"eip150Hash":"0x0000000000000000000000000000000000000000000000000000000000000000","eip155Block":0,"eip158Block":0,"byzantiumBlock":0,"constantinopleBlock":0,"petersburgBlock":0,"istanbulBlock":0,"muirGlacierBlock":0,"berlinBlock":0,"londonBlock":0,"clique":{"period":0,"epoch":0},"arbitrum":{"EnableArbOS":true,"AllowDebugPrecompiles":false,"DataAvailabilityCommittee":false,"InitialArbOSVersion":6,"InitialChainOwner":"0xd345e41ae2cb00311956aa7109fc801ae8c81a52","GenesisBlockNum":0}}}'

For example, this will create a directory called gas-dimension-logs in the root of the repo that will have each block as a subfolder, and a protobuf dumped file for the gas dimensions by transaction. The tracer expects a chain configuration because I was unable to get the chain configuration in the tracer creation context without having a bigger diff. So it expects it to be passed in by json - you can use the nitro repository's chain info as a reference for a chain configuration if you don't have one.

this PR is identical to and replaces #457.

@relyt29 relyt29 self-assigned this May 13, 2025
@cla-bot cla-bot bot added the s CLA signed label May 13, 2025
@relyt29 relyt29 force-pushed the gas-dimension-tracing branch from c7b583c to 122c51f Compare May 21, 2025 14:53
@relyt29 relyt29 marked this pull request as draft May 23, 2025 14:16
@relyt29 relyt29 force-pushed the gas-dimension-tracing branch 2 times, most recently from f8fb608 to c4f9da3 Compare May 23, 2025 16:34
amsanghi
amsanghi previously approved these changes May 29, 2025
Copy link
Contributor

@amsanghi amsanghi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Mostly looks, have tagged @eljobe to get his input on the protobuf parts.
Also the lint is failing due to some minor issue.

@relyt29 relyt29 marked this pull request as ready for review June 2, 2025 16:33
relyt29 added 18 commits June 5, 2025 13:27
Support live tracing and by RPC, support all pure-CPU opcodes.
Support the simple state access opcodes.

Rest still TODO
1. EXTCODECOPY with no memory expansion, and warm storage access
2. EXTCODECOPY with memory expansion and warm storage access
3. EXTCODECOPY with memory expansion and cold storage access
4. EXTCODECOPY with no memory expansion and cold storage access
Cold access list SLOAD
Warm access list SLOAD
all of the LOG opcodes with both indexed and unindexed data
Selfdestruct to:
* cold, empty account target, sender has balance
* warm, empty account target, sender has balance
* warm, code/value at target, sender has balance
* warm, code/value at target, no money to send
* cold, code/value at target, sender has balance
Tested:
* calling to a contract that is cold but has code
* calling to a contract that is cold and has no code
Tested:
* calling to a contract that is warm and has code
* calling to a contract that is cold and has code
* calling to a contract that is warm and has no code
* calling to a contract that is cold and has no code
Tested by hand:
* Delegatecall to warm address, to a contract with code
* Delegatecall to cold address, to a contract with code
* Delegatecall to cold address, to a contract with no code
* Delegatecall to warm address, to a contract with no code
Tested by Hand:
* warm access list+ target has code + zero value
* warm access list+ target has code + positive value
* warm access list+ target has no code + zero value
* warm access list+ target has no code + positive value
* cold access list+ target has code + zero value
* cold access list+ target has code + positive value
* cold access list+ target has no code + zero value
* cold access list+ target has no code + positive value
Tested By hand:
* CREATE
* CREATE2
* CREATE sending value
* CREATE2 sending value

sending value does not affect gas dimensions but I tested it anyways
Tested by hand in variations:
* writing to a new slot
* writing to the same slot twice with the same value
* writing to the same slot twice with a different value
* writing to a slot that was uninitialized and setting it back to zero
* writing to a slot that was cold but set to non-zero and setting it to
  nonzero
@relyt29 relyt29 force-pushed the gas-dimension-tracing branch from aa024a9 to f116416 Compare June 5, 2025 17:28
@eljobe eljobe self-assigned this Jun 7, 2025
eljobe
eljobe previously approved these changes Jun 7, 2025
@eljobe eljobe merged commit 08cdbb9 into master Jun 17, 2025
15 checks passed
@eljobe eljobe deleted the gas-dimension-tracing branch June 17, 2025 13:01
joshuacolvin0 added a commit to OffchainLabs/nitro that referenced this pull request Jun 18, 2025
Sneh1999 added a commit to EspressoSystems/nitro-espresso-integration that referenced this pull request Jul 10, 2025
* Add option to enable PreImages

* Enable Golang linter misspell (for spell check)

* Update to modern golangci-lint version and enable misspell

* Add nil check to SendExpressLaneTransaction RPC method

* Increase auctioneer gas margin, lower log level for transient falures

* WasmModuleRoot only needed when validating

* Remove extraneous `--node.feed.output.disable-signing` parameter

Means the same thing as `--node.feed.output.signed`, so just use that.

* update bold pin

* Test Push0 for eth_simulateV1

* Fix eth_simulateV1 and add test for the same

* update geth

* Tests exact ink usage of HostIOs

Separate program gas-usage tests in two sets:
* One set of tests compares the HostIO ink usage with the equivalent EVM
  opcode gas usage. The objective of these tests is to ensure Stylus
  HostIOs don't diverge too much from EVM opcodes. These tests make a
  rough comparison between them. The expected values are not exact and,
  depending on the HostIO, the error margins can be huge.
* The other set compares the HostIO ink usage with exact hardcoded
  values. The objective of these tests is to ensure there isn't a
  regression when changing the Stylus implementation.

* Solve race in TestRequestForwardingToArchiveNodes system_test

* add documentation

* Fix ink usage tests

When using a Rust Stylus program, updating the Rust compiler can produce
a different wasm and then change the amount of ink charged for a few
HostIOs.

* Add more cases to storage ink-usage test

* Add ink-usage test case with clear between loads

* Add more cases for read-return-data ink-usage test

* Use simpler wasm program in call ink test

* Add more cases for account_code ink-usage test

* feat: Live tracing

Implement go-ethereum's live tracing feature in Nitro

- Add the live tracer to go-ethereum/eth/tracers/live.
Rebuild nitro

- To activate a tracer pass --node.trace.tracername
and optionally --node.vmtrace.jsonconfig

- Support --dev and production mode, in both sequencer and
non-sequencer mode

- Changes made
  1. Import the tracers/live directory in cmd/nitro.go
  to call init() functions on tracers
  2. Add config params to the nitro binary to activate a tracer
  from bash
  3. Call LiveDirectory.New() in func GetBlockchain() in
  gethexec/blockchain.go to activate the tracer based on config.
  The tracer is attached to vmconfig
  4. Read the tracer in executionengine.go. Pass it to
 block_processor.go
  5. Missing hook calls OnBlockStart and OnBlockEnd in
  block_processor.go
  6. If tracer hooks are provided and isMsgForPrefetch
  is false then attach hooks to the stateDB, pass them to vm.Config
  to spawn an EVM instance with tracing enabled.

- Credits: OffchainLabs#2934

* Fix OnBlockStart and OnBlockEnd live tracer hooks not firing

* ignore cursor rules

* gas dimension trace tests: proof of concept integration test live tracer

* gas dimension trace tests: simple calculation passing

* gas dimension trace tests: rename file

* gas dimension trace tests: SLOAD cold, warm

* gas dimension trace tests: BALANCE,EXTCODESIZE,EXTCODEHASH warm and cold

* gas dimension trace tests: EXTCODECOPY with memory expansion

* gas dimension trace tests: use core/params values and one-dimensional check

* gas dimension trace tests: enumerate list of all remaining test cases

* gas dimension trace tests: log test cases

* gas dimension trace tests: write header comments for all the sstore
tests, including doing the math for the expected values.

* gas dimension trace tests: refactor reduce code duplication on deploy
and call function

* gas dimension trace tests: refactor get specific opcode log pattern

* gas dimension trace tests: single SSTORE test cases

* gas dimension trace tests: multiple SSTORE in a row test cases

* gas dimension trace tests: SELFDESTRUCT tests

* gas dimension trace tests: DELEGATECALL tests

* gas dimension trace tests: STATICCALL tests

* gas dimension trace tests: tweak comments + start on CALL

* gas dimension trace tests: split into separate files

it was becoming far too unweildy in one file for navigation

* gas dimension trace tests: CALL tests

* gas dimension trace tests: refactor CALL tests order, comment headers

* gas dimension trace tests: CALLCODE tests

* gas dimension trace tests: CREATE + CREATE2 opcodes

* gas dimension trace tests: boilerplate for gas_dim_tx_opcode tests

The test itself is failing

* gas dimension trace tests: computation-only passing, boilerplate BALANCE

add boilerplate for the read only opcodes like BALANCE, SLOAD, etc

* gas dimension trace tests: use tracer L1/L2GasUsed + ChildExecutionGas

Adapt tests to take advantage of the now exposed L1GasUsed and L2GasUsed
values in the tracer output. Check against it in the gas_dim_tx_op_state
tests. Change all the tests to handle child execution gas separately
from one dimensional gas for CALL, CALLCODE STATICCALL, DELEGATECALL,
CREATE, and CREATE2.

* gas dimension trace tests: tx opcodes by dimension

one sstore test is still failing, need to examine further

* gas dimension trace tests: use capped adjusted gas refund

* gas dimension trace tests: fix linter issues

* gas dimension trace tests: Resolve conflicts from OffchainLabs#3066 and OffchainLabs#3138

* gas dimension trace tests: tests and solgen added to contracts subrepo

* gas dimension trace tests: add tests for INVALID and REVERT opcodes

* gas dimension trace tests: @tsahee says set MaxRevertGasReject = 0

* gas dimension trace tests: use t.Parallel()

speeds up gas-dimension tests from 263.030s -> 133.336s

* gas dimension trace tests: address reviewer nits

* chore: use contract-local

* chore: move gas-dimension-test-contracts

Co-authored-by: relyt29 <[email protected]>

* chore: restore nitro-contracts pin

* chore: remove unused hardhat for now

* format: fix

* fix: include yulFilePathsGasDimensions

* refactor: move mocks to contracts-local

* fix: missing contract

* fix: build contracts-local

* fix: try mocksgen for mocks_legacy_gen

* Revert "fix: try mocksgen for mocks_legacy_gen"

This reverts commit 0ef85f4.

* fix: mocks-contracts-legacy

* fix: build Simple in paris to avoid PUSH0

* chore: remove hardcode oz lib

* chore: use foundry lib for oz

* refactor: mocksgen to localgen

* refactor: use makefile in contract-local

* fix: localgen import

* chore: getting rid of boldmocksgen

* refactor: use precompile interface submodule

* format: fix

* fix: use mocksgen in bold solgen

* Refactor HostIO Pricing

* Refactor the HostIO pricing to gather all base prices in the
  pricing.rs file.
* General code improvements to maximize readability.

Close NIT-3434

Co-authored-by: Raul Jordan <[email protected]>

* We have been experiencing a lot of CI flakiness

The working theory is that the hardware is just overwhelmed while building the
tests and this is because of intensive I/O and system resources, so we're moving
our three heaviest workloads to a 32 core 128GB machine.

* Fix the "runs-on" string

It should be one of the standard box sizes. The "label" in the UI for the
runner has nothing to do with the profile of the box (even though for the other
two hosted runners, we have labels set that match.)

* Go back to using arbitrator-ci

* update geth

* Update or add copyright and license headers

* Add copyright and license headers

* Fix Makefile and one solidity contract

* Fix the path to the gas-dimensions files

* Update openzeppelin-contracts pin

* feat: add static foundry profile for gas sensitive build

* fix: replicate old config

* Revert "fix: replicate old config"

This reverts commit 19893c3.

* fix: use 0.8.17 for Simple.sol

* fix: replicate old config

* chore: use solc 0.8.30 for gasdim

* fix: use paris for default

* specify solidity version in naming

Co-authored-by: gzeon <[email protected]>

* gas dimension trace tests: precompile tests

* refactor: copy Precompile.sol from contracts to contracts-local

* gas dimension trace tests: precompile tests pass for SLOAD, SSTORE, CALL

* gas dimension trace tests: call into stylus, but tracer has errors

* gas dimension trace tests: passing stylus and precompile tests

* fix: linter formatting

* update go-ethereum pin

* revert 2674370 - move back openzeppelin-contracts pin to v4.7.3

* revert 2674370 - move back openzeppelin-contracts pin to v4.7.3

* fix allowed wasm module roots check

* Add a line for the submodule-pin-check for oppenzeplin-contracts

This way, the presubmit check won't fail.

* Add a submodule pin check for the openzeppelin-contracts submodule

This has to be merged on the `master` branch before the check can pass in a
branch.

* Limit parallel jobs within make to 8

Currently, we very frequently see the build step in the CI workflow run into:
`error: Resource temporarily unavailable (os error 11)`

This is typically because there are too many open file descriptors, and that's
possible given that `-j` doesn't limit the number of jobs that make will try to
run in parallel.

8 might still be too many, but let's try this value first, and then ramp down
until we find a consistently healthy CI sweet spot.

* Submodule pin check only works with branch names, not tags

This is something we should try to remember for the future.

* Update the openzeppelin-pin to the release-v4.7 branch

This is only a few commits ahead of the v4.7.3 label, so hopefully, it won't
break the build, and it should allow the new submodule-pin-check.yml to work
with this configuraiton after OffchainLabs#3323

* Add missing account lookups for trace

Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue.

Also pulls in
OffchainLabs/go-ethereum#470

OffchainLabs/go-ethereum#469

OffchainLabs/go-ethereum#459

* Remove stateless_block_validator public function for reader, streamer and tracker

* Fix the Docker build for contracts-local

In the context of a Docker build, the contracts-local directory is not part of
a git repo, which causes forge install to fail.

Also, the foundryup line is needed to get the more recent versions of the solc
compiler.

* fix ci

* The foundryup can't run in the Makefile or it will break CI

The thing we really need is for folks to be using a modern enough version of
foundry to have the newer solc compiler. So, that's being done in the Dockerfile
itself now.

* Update submodule geth

* Handle merge conflicts

* Fix go import

* go mod tidy

* Fix

* Fix staker

* Fix caff node

* Attempt to update version of bold.

* Current state of merge

In this commit make build is working, but make build-replay env is
currently failing. I haven't been able to run the tests yet.

* Update contracts submodule commit

* Update submodules

* Add bindings

* Fix tests

* Update contracts-legacy commit

* Fix merge errors

* Result of go mod tidy

* Lint

* Update CI

* add logging to caff node test to determine reason for panic

* Fix caff node

* Fix caff node config test

* Fix golangci.yml

* Lint files

* Lint files

* Lint files

* Lint files

* Fix data poster.go

* Lint files

* Lint files

* Fix

* Fix

* Fix TestOversizedMessages

* Update CI time for Espresso E2E test

* Fix tests

* Add time for CI

* Fix free disk space

* Fix test and remove one from the skip list

* Remove escape hatch test

* Faster CI

* buildjet

* Go

* WIP

* Github path

* CI upterm

* ubuntu latest

* Limit parallel

* Skip one recource consuming test

* Avoid writing logs to files

* Skip tests, use self-hosted runner

* Fix

* Fix redis tests

* Remove espresso-network-go

* Address comments

---------

Co-authored-by: Aman Sanghi <[email protected]>
Co-authored-by: Aman Sanghi <[email protected]>
Co-authored-by: Tristan Wilson <[email protected]>
Co-authored-by: Tristan-Wilson <[email protected]>
Co-authored-by: Pepper Lebeck-Jobe <[email protected]>
Co-authored-by: Raul Jordan <[email protected]>
Co-authored-by: Joshua Colvin <[email protected]>
Co-authored-by: Gabriel de Quadros Ligneul <[email protected]>
Co-authored-by: Ganesh Vanahalli <[email protected]>
Co-authored-by: Tsahi Zidenberg <[email protected]>
Co-authored-by: rollchad <[email protected]>
Co-authored-by: relyt29 <[email protected]>
Co-authored-by: gzeon <[email protected]>
Co-authored-by: Gabriel de Quadros Ligneul <[email protected]>
Co-authored-by: Pepper Lebeck-Jobe <[email protected]>
Co-authored-by: ǝuᴉu ʎʇuǝʍʇ ʇʎlǝɹ <[email protected]>
Co-authored-by: gzeon <[email protected]>
Co-authored-by: Maciej Kulawik <[email protected]>
Co-authored-by: Jeremy <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Sneh1999 added a commit to EspressoSystems/nitro-espresso-integration that referenced this pull request Jul 17, 2025
* Allow validator failures to be able to retry in case of failure

* skip l3 tests until we have stronger ci machines

* include delayed message lookup

* gci

* Refactor validator wallet

* Add zero-config HA coordination for Timeboost auctioneer

Implements automatic primary/secondary failover for multiple auctioneer instances
using a single Redis coordination key. This ensures high availability without
requiring any additional configuration - just run multiple instances with the
same Redis URL.

The design generates a unique ID for each auctioneer on startup, with all instances
competing for a single "auctioneer.chosen" key in Redis. Only the primary instance
that holds this key consumes bids from the Redis stream. The lock value stores both
the instance ID and timestamp in the format "id:timestamp" to enable stale lock
detection without relying on Redis TTL mechanisms. When acquiring stale locks, the
system uses a delete + SetNX pattern to ensure atomic acquisition.

For failover protection, the system employs timestamp-based staleness detection that
doesn't depend on Redis passive expiration timing. The delete + SetNX pattern prevents
race conditions by ensuring only one instance can take over at a time. The stale lock
timeout is set to 3x the IdletimeToAutoclaim duration to ensure proper coordination.

Failover Timeline Example:
  T=0s: Primary auctioneer crashes
  T=1s: Primary's unacked messages become autoclaimable (IdletimeToAutoclaim)
  T=3s: Secondary detects stale lock and acquires it
  T=3s+: Secondary starts consuming messages

This timing ensures that all messages are autoclaimable before the secondary takes
over, preserving message ordering during failover and preventing split-brain scenarios.
The result is a clean handoff with approximately 3 second failover time.

Regarding auction resolution, it's still possible that an auctioneer could crash when
it was supposed to resolve the auction, and the new primary that comes up doesn't
resolve it because resolution happens at a fixed time. Further work is needed if we
want to handle this case - we'd need to check for resolution events and retry if no
resolution occurred within the expected window. The auction contract does prevent
double resolution, ensuring correctness even in edge cases.

For S3 storage, bids are stored locally before being uploaded to S3 on a default
15 minute interval. During failover, any bids not yet uploaded by a crashed primary
may be lost. This is acceptable as the data is primarily for research purposes.

The implementation includes comprehensive failover tests covering basic failover
scenarios, multiple instance coordination, concurrent start handling, rapid recovery
prevention, stale lock detection, invalid lock format recovery, and message
reprocessing after failover.

Usage (no config changes needed):
  # Start primary
  ./nitro --timeboost.auctioneer.redis-url="redis://localhost:6379"

  # Start secondary (identical command)
  ./nitro --timeboost.auctioneer.redis-url="redis://localhost:6379"

The first instance to start becomes primary. If it fails, the secondary
automatically takes over after detecting the stale lock.

* Lower log levels for pubsub producer errors

Changed consumer error and timeout logs from Error to Debug level in the pubsub
producer. These aren't always system errors - in the timeboost bid flow, both
consumer rejections (due to timing constraints) and timeouts (bids unprocessed
until round end) are legitimate business logic behaviors.

Since the Promise pattern already allows callers to check and handle errors
appropriately, callers can decide whether to log at higher levels based on
their specific use case. This change eliminates false alarms in monitoring
while maintaining error visibility through the Promise API.

* lint

* Add descriptions to seq-coordinator config options

* Fix L2PricingState UpdatePricingModel panic if chain owner sets insane parameters

* Add option to enable PreImages

* Enable Golang linter misspell (for spell check)

* Update to modern golangci-lint version and enable misspell

* Add nil check to SendExpressLaneTransaction RPC method

* Increase auctioneer gas margin, lower log level for transient falures

* WasmModuleRoot only needed when validating

* Remove extraneous `--node.feed.output.disable-signing` parameter

Means the same thing as `--node.feed.output.signed`, so just use that.

* update bold pin

* Test Push0 for eth_simulateV1

* Fix eth_simulateV1 and add test for the same

* update geth

* Tests exact ink usage of HostIOs

Separate program gas-usage tests in two sets:
* One set of tests compares the HostIO ink usage with the equivalent EVM
  opcode gas usage. The objective of these tests is to ensure Stylus
  HostIOs don't diverge too much from EVM opcodes. These tests make a
  rough comparison between them. The expected values are not exact and,
  depending on the HostIO, the error margins can be huge.
* The other set compares the HostIO ink usage with exact hardcoded
  values. The objective of these tests is to ensure there isn't a
  regression when changing the Stylus implementation.

* Solve race in TestRequestForwardingToArchiveNodes system_test

* add documentation

* Fix ink usage tests

When using a Rust Stylus program, updating the Rust compiler can produce
a different wasm and then change the amount of ink charged for a few
HostIOs.

* Add more cases to storage ink-usage test

* Add ink-usage test case with clear between loads

* Add more cases for read-return-data ink-usage test

* Use simpler wasm program in call ink test

* Add more cases for account_code ink-usage test

* feat: Live tracing

Implement go-ethereum's live tracing feature in Nitro

- Add the live tracer to go-ethereum/eth/tracers/live.
Rebuild nitro

- To activate a tracer pass --node.trace.tracername
and optionally --node.vmtrace.jsonconfig

- Support --dev and production mode, in both sequencer and
non-sequencer mode

- Changes made
  1. Import the tracers/live directory in cmd/nitro.go
  to call init() functions on tracers
  2. Add config params to the nitro binary to activate a tracer
  from bash
  3. Call LiveDirectory.New() in func GetBlockchain() in
  gethexec/blockchain.go to activate the tracer based on config.
  The tracer is attached to vmconfig
  4. Read the tracer in executionengine.go. Pass it to
 block_processor.go
  5. Missing hook calls OnBlockStart and OnBlockEnd in
  block_processor.go
  6. If tracer hooks are provided and isMsgForPrefetch
  is false then attach hooks to the stateDB, pass them to vm.Config
  to spawn an EVM instance with tracing enabled.

- Credits: OffchainLabs#2934

* Fix OnBlockStart and OnBlockEnd live tracer hooks not firing

* ignore cursor rules

* gas dimension trace tests: proof of concept integration test live tracer

* gas dimension trace tests: simple calculation passing

* gas dimension trace tests: rename file

* gas dimension trace tests: SLOAD cold, warm

* gas dimension trace tests: BALANCE,EXTCODESIZE,EXTCODEHASH warm and cold

* gas dimension trace tests: EXTCODECOPY with memory expansion

* gas dimension trace tests: use core/params values and one-dimensional check

* gas dimension trace tests: enumerate list of all remaining test cases

* gas dimension trace tests: log test cases

* gas dimension trace tests: write header comments for all the sstore
tests, including doing the math for the expected values.

* gas dimension trace tests: refactor reduce code duplication on deploy
and call function

* gas dimension trace tests: refactor get specific opcode log pattern

* gas dimension trace tests: single SSTORE test cases

* gas dimension trace tests: multiple SSTORE in a row test cases

* gas dimension trace tests: SELFDESTRUCT tests

* gas dimension trace tests: DELEGATECALL tests

* gas dimension trace tests: STATICCALL tests

* gas dimension trace tests: tweak comments + start on CALL

* gas dimension trace tests: split into separate files

it was becoming far too unweildy in one file for navigation

* gas dimension trace tests: CALL tests

* gas dimension trace tests: refactor CALL tests order, comment headers

* gas dimension trace tests: CALLCODE tests

* gas dimension trace tests: CREATE + CREATE2 opcodes

* gas dimension trace tests: boilerplate for gas_dim_tx_opcode tests

The test itself is failing

* gas dimension trace tests: computation-only passing, boilerplate BALANCE

add boilerplate for the read only opcodes like BALANCE, SLOAD, etc

* gas dimension trace tests: use tracer L1/L2GasUsed + ChildExecutionGas

Adapt tests to take advantage of the now exposed L1GasUsed and L2GasUsed
values in the tracer output. Check against it in the gas_dim_tx_op_state
tests. Change all the tests to handle child execution gas separately
from one dimensional gas for CALL, CALLCODE STATICCALL, DELEGATECALL,
CREATE, and CREATE2.

* gas dimension trace tests: tx opcodes by dimension

one sstore test is still failing, need to examine further

* gas dimension trace tests: use capped adjusted gas refund

* gas dimension trace tests: fix linter issues

* gas dimension trace tests: Resolve conflicts from OffchainLabs#3066 and OffchainLabs#3138

* gas dimension trace tests: tests and solgen added to contracts subrepo

* gas dimension trace tests: add tests for INVALID and REVERT opcodes

* gas dimension trace tests: @tsahee says set MaxRevertGasReject = 0

* gas dimension trace tests: use t.Parallel()

speeds up gas-dimension tests from 263.030s -> 133.336s

* gas dimension trace tests: address reviewer nits

* chore: use contract-local

* chore: move gas-dimension-test-contracts

Co-authored-by: relyt29 <[email protected]>

* chore: restore nitro-contracts pin

* chore: remove unused hardhat for now

* format: fix

* fix: include yulFilePathsGasDimensions

* refactor: move mocks to contracts-local

* fix: missing contract

* fix: build contracts-local

* fix: try mocksgen for mocks_legacy_gen

* Revert "fix: try mocksgen for mocks_legacy_gen"

This reverts commit 0ef85f4.

* fix: mocks-contracts-legacy

* fix: build Simple in paris to avoid PUSH0

* chore: remove hardcode oz lib

* chore: use foundry lib for oz

* refactor: mocksgen to localgen

* refactor: use makefile in contract-local

* fix: localgen import

* chore: getting rid of boldmocksgen

* refactor: use precompile interface submodule

* format: fix

* fix: use mocksgen in bold solgen

* Refactor HostIO Pricing

* Refactor the HostIO pricing to gather all base prices in the
  pricing.rs file.
* General code improvements to maximize readability.

Close NIT-3434

Co-authored-by: Raul Jordan <[email protected]>

* We have been experiencing a lot of CI flakiness

The working theory is that the hardware is just overwhelmed while building the
tests and this is because of intensive I/O and system resources, so we're moving
our three heaviest workloads to a 32 core 128GB machine.

* Fix the "runs-on" string

It should be one of the standard box sizes. The "label" in the UI for the
runner has nothing to do with the profile of the box (even though for the other
two hosted runners, we have labels set that match.)

* Go back to using arbitrator-ci

* pin submodules

* Celestia DA

- Fix anytrust fallback issues
- Add `Type()` to dapWritter to differentiate which dapWritter is being used (`celestia` or `anytrust`)
- pin contracts branch and nitro testnode branch

Update .gitmodules

Update download-machine.sh

Update Dockerfile

Fix interface methods

Merge branch 'celestia-v3.1.2' into celestia-v3.2.1

Pin contracts and testnode

* remove Celestia config

* adapt celestia to daProvider client

* fix Dockerfile and pin submodules

* fix dasserver

* Update Dockerfile

* pin submodules

* update geth

* Update or add copyright and license headers

* Add copyright and license headers

* Fix Makefile and one solidity contract

* Fix the path to the gas-dimensions files

* Update openzeppelin-contracts pin

* feat: add static foundry profile for gas sensitive build

* fix: replicate old config

* Revert "fix: replicate old config"

This reverts commit 19893c3.

* fix: use 0.8.17 for Simple.sol

* fix: replicate old config

* chore: use solc 0.8.30 for gasdim

* fix: use paris for default

* specify solidity version in naming

Co-authored-by: gzeon <[email protected]>

* gas dimension trace tests: precompile tests

* refactor: copy Precompile.sol from contracts to contracts-local

* gas dimension trace tests: precompile tests pass for SLOAD, SSTORE, CALL

* gas dimension trace tests: call into stylus, but tracer has errors

* gas dimension trace tests: passing stylus and precompile tests

* fix: linter formatting

* update go-ethereum pin

* revert 2674370 - move back openzeppelin-contracts pin to v4.7.3

* revert 2674370 - move back openzeppelin-contracts pin to v4.7.3

* fix allowed wasm module roots check

* Add a line for the submodule-pin-check for oppenzeplin-contracts

This way, the presubmit check won't fail.

* Add a submodule pin check for the openzeppelin-contracts submodule

This has to be merged on the `master` branch before the check can pass in a
branch.

* Limit parallel jobs within make to 8

Currently, we very frequently see the build step in the CI workflow run into:
`error: Resource temporarily unavailable (os error 11)`

This is typically because there are too many open file descriptors, and that's
possible given that `-j` doesn't limit the number of jobs that make will try to
run in parallel.

8 might still be too many, but let's try this value first, and then ramp down
until we find a consistently healthy CI sweet spot.

* Submodule pin check only works with branch names, not tags

This is something we should try to remember for the future.

* Update the openzeppelin-pin to the release-v4.7 branch

This is only a few commits ahead of the v4.7.3 label, so hopefully, it won't
break the build, and it should allow the new submodule-pin-check.yml to work
with this configuraiton after OffchainLabs#3323

* Add missing account lookups for trace

Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue.

Also pulls in
OffchainLabs/go-ethereum#470

OffchainLabs/go-ethereum#469

OffchainLabs/go-ethereum#459

* Remove stateless_block_validator public function for reader, streamer and tracker

* Fix the Docker build for contracts-local

In the context of a Docker build, the contracts-local directory is not part of
a git repo, which causes forge install to fail.

Also, the foundryup line is needed to get the more recent versions of the solc
compiler.

* fix ci

* The foundryup can't run in the Makefile or it will break CI

The thing we really need is for folks to be using a modern enough version of
foundry to have the newer solc compiler. So, that's being done in the Dockerfile
itself now.

* Update nitro-testnode

* pin celestia contracts and testnode

* pin deps

* Fix make buil

* Update contracts

* Fix build-replay-env

* Fix tests

* Fix tests

* Lint files

* fix prepare-espresso-crypto-helper

* Fix

* Don't rebuild espresso-crypto

* Lint fix

* Remove escape hatch test

* Fix tests

* Skeptical fix

* Self-hosted ci runner

* Update bold

* Revert "Update bold"

This reverts commit 0ebb448.

* Update bold

* Update tx-streamer to enqueue batches of messages to espresso (#648)

* fix enqueued

* pull last confirmed pos from database

* push logs

* fix log

* Fix CI

---------

Co-authored-by: Jeremy <[email protected]>
(cherry picked from commit a34cf64)

* Fix

* Fix linter

* Update contract submodule path

* Fix

* Revert cmd/nitro/init.go

* Revert "Fix"

This reverts commit 1c06f5c.

* Update bold submodule

---------

Co-authored-by: Aman Sanghi <[email protected]>
Co-authored-by: Aman Sanghi <[email protected]>
Co-authored-by: Raul Jordan <[email protected]>
Co-authored-by: Tristan Wilson <[email protected]>
Co-authored-by: Joshua Colvin <[email protected]>
Co-authored-by: Tristan-Wilson <[email protected]>
Co-authored-by: Pepper Lebeck-Jobe <[email protected]>
Co-authored-by: Gabriel de Quadros Ligneul <[email protected]>
Co-authored-by: Ganesh Vanahalli <[email protected]>
Co-authored-by: Tsahi Zidenberg <[email protected]>
Co-authored-by: rollchad <[email protected]>
Co-authored-by: relyt29 <[email protected]>
Co-authored-by: gzeon <[email protected]>
Co-authored-by: Gabriel de Quadros Ligneul <[email protected]>
Co-authored-by: Pepper Lebeck-Jobe <[email protected]>
Co-authored-by: Diego <[email protected]>
Co-authored-by: ǝuᴉu ʎʇuǝʍʇ ʇʎlǝɹ <[email protected]>
Co-authored-by: gzeon <[email protected]>
Co-authored-by: Maciej Kulawik <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Zaptoss pushed a commit to distributed-lab/nitro-espresso-integration that referenced this pull request Aug 11, 2025
* Add option to enable PreImages

* Enable Golang linter misspell (for spell check)

* Update to modern golangci-lint version and enable misspell

* Add nil check to SendExpressLaneTransaction RPC method

* Increase auctioneer gas margin, lower log level for transient falures

* WasmModuleRoot only needed when validating

* Remove extraneous `--node.feed.output.disable-signing` parameter

Means the same thing as `--node.feed.output.signed`, so just use that.

* update bold pin

* Test Push0 for eth_simulateV1

* Fix eth_simulateV1 and add test for the same

* update geth

* Tests exact ink usage of HostIOs

Separate program gas-usage tests in two sets:
* One set of tests compares the HostIO ink usage with the equivalent EVM
  opcode gas usage. The objective of these tests is to ensure Stylus
  HostIOs don't diverge too much from EVM opcodes. These tests make a
  rough comparison between them. The expected values are not exact and,
  depending on the HostIO, the error margins can be huge.
* The other set compares the HostIO ink usage with exact hardcoded
  values. The objective of these tests is to ensure there isn't a
  regression when changing the Stylus implementation.

* Solve race in TestRequestForwardingToArchiveNodes system_test

* add documentation

* Fix ink usage tests

When using a Rust Stylus program, updating the Rust compiler can produce
a different wasm and then change the amount of ink charged for a few
HostIOs.

* Add more cases to storage ink-usage test

* Add ink-usage test case with clear between loads

* Add more cases for read-return-data ink-usage test

* Use simpler wasm program in call ink test

* Add more cases for account_code ink-usage test

* feat: Live tracing

Implement go-ethereum's live tracing feature in Nitro

- Add the live tracer to go-ethereum/eth/tracers/live.
Rebuild nitro

- To activate a tracer pass --node.trace.tracername
and optionally --node.vmtrace.jsonconfig

- Support --dev and production mode, in both sequencer and
non-sequencer mode

- Changes made
  1. Import the tracers/live directory in cmd/nitro.go
  to call init() functions on tracers
  2. Add config params to the nitro binary to activate a tracer
  from bash
  3. Call LiveDirectory.New() in func GetBlockchain() in
  gethexec/blockchain.go to activate the tracer based on config.
  The tracer is attached to vmconfig
  4. Read the tracer in executionengine.go. Pass it to
 block_processor.go
  5. Missing hook calls OnBlockStart and OnBlockEnd in
  block_processor.go
  6. If tracer hooks are provided and isMsgForPrefetch
  is false then attach hooks to the stateDB, pass them to vm.Config
  to spawn an EVM instance with tracing enabled.

- Credits: OffchainLabs#2934

* Fix OnBlockStart and OnBlockEnd live tracer hooks not firing

* ignore cursor rules

* gas dimension trace tests: proof of concept integration test live tracer

* gas dimension trace tests: simple calculation passing

* gas dimension trace tests: rename file

* gas dimension trace tests: SLOAD cold, warm

* gas dimension trace tests: BALANCE,EXTCODESIZE,EXTCODEHASH warm and cold

* gas dimension trace tests: EXTCODECOPY with memory expansion

* gas dimension trace tests: use core/params values and one-dimensional check

* gas dimension trace tests: enumerate list of all remaining test cases

* gas dimension trace tests: log test cases

* gas dimension trace tests: write header comments for all the sstore
tests, including doing the math for the expected values.

* gas dimension trace tests: refactor reduce code duplication on deploy
and call function

* gas dimension trace tests: refactor get specific opcode log pattern

* gas dimension trace tests: single SSTORE test cases

* gas dimension trace tests: multiple SSTORE in a row test cases

* gas dimension trace tests: SELFDESTRUCT tests

* gas dimension trace tests: DELEGATECALL tests

* gas dimension trace tests: STATICCALL tests

* gas dimension trace tests: tweak comments + start on CALL

* gas dimension trace tests: split into separate files

it was becoming far too unweildy in one file for navigation

* gas dimension trace tests: CALL tests

* gas dimension trace tests: refactor CALL tests order, comment headers

* gas dimension trace tests: CALLCODE tests

* gas dimension trace tests: CREATE + CREATE2 opcodes

* gas dimension trace tests: boilerplate for gas_dim_tx_opcode tests

The test itself is failing

* gas dimension trace tests: computation-only passing, boilerplate BALANCE

add boilerplate for the read only opcodes like BALANCE, SLOAD, etc

* gas dimension trace tests: use tracer L1/L2GasUsed + ChildExecutionGas

Adapt tests to take advantage of the now exposed L1GasUsed and L2GasUsed
values in the tracer output. Check against it in the gas_dim_tx_op_state
tests. Change all the tests to handle child execution gas separately
from one dimensional gas for CALL, CALLCODE STATICCALL, DELEGATECALL,
CREATE, and CREATE2.

* gas dimension trace tests: tx opcodes by dimension

one sstore test is still failing, need to examine further

* gas dimension trace tests: use capped adjusted gas refund

* gas dimension trace tests: fix linter issues

* gas dimension trace tests: Resolve conflicts from OffchainLabs#3066 and OffchainLabs#3138

* gas dimension trace tests: tests and solgen added to contracts subrepo

* gas dimension trace tests: add tests for INVALID and REVERT opcodes

* gas dimension trace tests: @tsahee says set MaxRevertGasReject = 0

* gas dimension trace tests: use t.Parallel()

speeds up gas-dimension tests from 263.030s -> 133.336s

* gas dimension trace tests: address reviewer nits

* chore: use contract-local

* chore: move gas-dimension-test-contracts

Co-authored-by: relyt29 <[email protected]>

* chore: restore nitro-contracts pin

* chore: remove unused hardhat for now

* format: fix

* fix: include yulFilePathsGasDimensions

* refactor: move mocks to contracts-local

* fix: missing contract

* fix: build contracts-local

* fix: try mocksgen for mocks_legacy_gen

* Revert "fix: try mocksgen for mocks_legacy_gen"

This reverts commit 0ef85f4.

* fix: mocks-contracts-legacy

* fix: build Simple in paris to avoid PUSH0

* chore: remove hardcode oz lib

* chore: use foundry lib for oz

* refactor: mocksgen to localgen

* refactor: use makefile in contract-local

* fix: localgen import

* chore: getting rid of boldmocksgen

* refactor: use precompile interface submodule

* format: fix

* fix: use mocksgen in bold solgen

* Refactor HostIO Pricing

* Refactor the HostIO pricing to gather all base prices in the
  pricing.rs file.
* General code improvements to maximize readability.

Close NIT-3434

Co-authored-by: Raul Jordan <[email protected]>

* We have been experiencing a lot of CI flakiness

The working theory is that the hardware is just overwhelmed while building the
tests and this is because of intensive I/O and system resources, so we're moving
our three heaviest workloads to a 32 core 128GB machine.

* Fix the "runs-on" string

It should be one of the standard box sizes. The "label" in the UI for the
runner has nothing to do with the profile of the box (even though for the other
two hosted runners, we have labels set that match.)

* Go back to using arbitrator-ci

* update geth

* Update or add copyright and license headers

* Add copyright and license headers

* Fix Makefile and one solidity contract

* Fix the path to the gas-dimensions files

* Update openzeppelin-contracts pin

* feat: add static foundry profile for gas sensitive build

* fix: replicate old config

* Revert "fix: replicate old config"

This reverts commit 19893c3.

* fix: use 0.8.17 for Simple.sol

* fix: replicate old config

* chore: use solc 0.8.30 for gasdim

* fix: use paris for default

* specify solidity version in naming

Co-authored-by: gzeon <[email protected]>

* gas dimension trace tests: precompile tests

* refactor: copy Precompile.sol from contracts to contracts-local

* gas dimension trace tests: precompile tests pass for SLOAD, SSTORE, CALL

* gas dimension trace tests: call into stylus, but tracer has errors

* gas dimension trace tests: passing stylus and precompile tests

* fix: linter formatting

* update go-ethereum pin

* revert 2674370 - move back openzeppelin-contracts pin to v4.7.3

* revert 2674370 - move back openzeppelin-contracts pin to v4.7.3

* fix allowed wasm module roots check

* Add a line for the submodule-pin-check for oppenzeplin-contracts

This way, the presubmit check won't fail.

* Add a submodule pin check for the openzeppelin-contracts submodule

This has to be merged on the `master` branch before the check can pass in a
branch.

* Limit parallel jobs within make to 8

Currently, we very frequently see the build step in the CI workflow run into:
`error: Resource temporarily unavailable (os error 11)`

This is typically because there are too many open file descriptors, and that's
possible given that `-j` doesn't limit the number of jobs that make will try to
run in parallel.

8 might still be too many, but let's try this value first, and then ramp down
until we find a consistently healthy CI sweet spot.

* Submodule pin check only works with branch names, not tags

This is something we should try to remember for the future.

* Update the openzeppelin-pin to the release-v4.7 branch

This is only a few commits ahead of the v4.7.3 label, so hopefully, it won't
break the build, and it should allow the new submodule-pin-check.yml to work
with this configuraiton after OffchainLabs#3323

* Add missing account lookups for trace

Pulls in OffchainLabs/go-ethereum#472 to fix tracing issue.

Also pulls in
OffchainLabs/go-ethereum#470

OffchainLabs/go-ethereum#469

OffchainLabs/go-ethereum#459

* Remove stateless_block_validator public function for reader, streamer and tracker

* Fix the Docker build for contracts-local

In the context of a Docker build, the contracts-local directory is not part of
a git repo, which causes forge install to fail.

Also, the foundryup line is needed to get the more recent versions of the solc
compiler.

* fix ci

* The foundryup can't run in the Makefile or it will break CI

The thing we really need is for folks to be using a modern enough version of
foundry to have the newer solc compiler. So, that's being done in the Dockerfile
itself now.

* Update submodule geth

* Handle merge conflicts

* Fix go import

* go mod tidy

* Fix

* Fix staker

* Fix caff node

* Attempt to update version of bold.

* Current state of merge

In this commit make build is working, but make build-replay env is
currently failing. I haven't been able to run the tests yet.

* Update contracts submodule commit

* Update submodules

* Add bindings

* Fix tests

* Update contracts-legacy commit

* Fix merge errors

* Result of go mod tidy

* Lint

* Update CI

* add logging to caff node test to determine reason for panic

* Fix caff node

* Fix caff node config test

* Fix golangci.yml

* Lint files

* Lint files

* Lint files

* Lint files

* Fix data poster.go

* Lint files

* Lint files

* Fix

* Fix

* Fix TestOversizedMessages

* Update CI time for Espresso E2E test

* Fix tests

* Add time for CI

* Fix free disk space

* Fix test and remove one from the skip list

* Remove escape hatch test

* Faster CI

* buildjet

* Go

* WIP

* Github path

* CI upterm

* ubuntu latest

* Limit parallel

* Skip one recource consuming test

* Avoid writing logs to files

* Skip tests, use self-hosted runner

* Fix

* Fix redis tests

* Remove espresso-network-go

* Address comments

---------

Co-authored-by: Aman Sanghi <[email protected]>
Co-authored-by: Aman Sanghi <[email protected]>
Co-authored-by: Tristan Wilson <[email protected]>
Co-authored-by: Tristan-Wilson <[email protected]>
Co-authored-by: Pepper Lebeck-Jobe <[email protected]>
Co-authored-by: Raul Jordan <[email protected]>
Co-authored-by: Joshua Colvin <[email protected]>
Co-authored-by: Gabriel de Quadros Ligneul <[email protected]>
Co-authored-by: Ganesh Vanahalli <[email protected]>
Co-authored-by: Tsahi Zidenberg <[email protected]>
Co-authored-by: rollchad <[email protected]>
Co-authored-by: relyt29 <[email protected]>
Co-authored-by: gzeon <[email protected]>
Co-authored-by: Gabriel de Quadros Ligneul <[email protected]>
Co-authored-by: Pepper Lebeck-Jobe <[email protected]>
Co-authored-by: ǝuᴉu ʎʇuǝʍʇ ʇʎlǝɹ <[email protected]>
Co-authored-by: gzeon <[email protected]>
Co-authored-by: Maciej Kulawik <[email protected]>
Co-authored-by: Jeremy <[email protected]>
Co-authored-by: Sneh Koul <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants