Skip to content

Conversation

LouisTsai-Csie
Copy link
Collaborator

@LouisTsai-Csie LouisTsai-Csie commented Sep 16, 2025

🗒️ Description

Background for reviewers:

In some benchmark tests, there are not only benchmark transactions but also setup transactions that create the scenario for the benchmark target. For example, in the SSTORE benchmark we first need a setup transaction that deploys as many contracts as possible, and then a separate transaction to perform the storage updates. This feature helps gas-limit testing distinguish between the setup and execution phases.

This PR extends transaction metadata, which labeling phases such as setup, testing, and cleanup. Currently, operations like deploy_contract or fund_eoa are tagged as setup phase, while transactions/blocks included in state_test or blockchain_test are tagged as testing. However, the current labeling is not always precise enough.

Take test_worst_blockhash as an example. The first 256 blocks should be classified as setup, while only the final block should count as the testing phase. Without this distinction, we might mistakenly include the setup blocks in benchmark accounting.

def test_worst_blockhash(
    blockchain_test: BlockchainTestFiller,
    pre: Alloc,
    gas_benchmark_value: int,
):
    """Test running a block with as many blockhash accesses to the oldest allowed block as possible."""
    # Create 256 dummy blocks to fill the blockhash window.
    blocks = [Block()] * 256

    # Always ask for the oldest allowed BLOCKHASH block.
    execution_code = Op.PUSH1(1) + While(
        body=Op.POP(Op.BLOCKHASH(Op.DUP1)),
    )
    execution_code_address = pre.deploy_contract(code=execution_code)
    op_tx = Transaction(
        to=execution_code_address,
        gas_limit=gas_benchmark_value,
        sender=pre.fund_eoa(),
    )
    blocks.append(Block(txs=[op_tx]))

    blockchain_test(
        pre=pre,
        post={},
        blocks=blocks,
    )

The design introduces a test_phase attribute at both the block and transaction level (currently supporting execution and setup phases). This attribute is used during execution (see transaction_post.py for details), and the metadata is updated accordingly.

🔗 Related Issues or PRs

This is the follow-up PR for #1945, more description could be found in issue #2137.
Related discussion: 1, 2

✅ Checklist

  • All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:
    uvx --with=tox-uv tox -e lint,typecheck,spellcheck,markdownlint
  • All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
  • All: Considered adding an entry to CHANGELOG.md.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can apply labels).
  • Tests: Ran mkdocs serve locally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.
  • Tests: For PRs implementing a missed test case, update the post-mortem document to add an entry the list.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or tests/static have been assigned @ported_from marker.

@LouisTsai-Csie LouisTsai-Csie self-assigned this Sep 16, 2025
@LouisTsai-Csie LouisTsai-Csie added scope:fw Scope: Framework (evm|tools|forks|pytest) scope:fill Scope: fill command scope:execute Scope: Changes to the execute command labels Sep 16, 2025
@LouisTsai-Csie
Copy link
Collaborator Author

Note: I have not yet tested against execute command. I will provide some result later

@danceratopz danceratopz self-requested a review September 25, 2025 13:41
Copy link
Collaborator

@fselmo fselmo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @LouisTsai-Csie. I don't have much to add on the implementation. This looks great. I'm going to test this early next week but just had some minor observations at first pass and left as comments.

I wanted to just get a bit of clarity on the scope on how the metadata of each phase will be used. I think we're capturing the data in a good way and I will continue my review next week, but I'd like to understand all the ways we currently plan to use this kind of metadata to get a better understanding for the design.

I think we mentioned being able to call execute on a network with a setup phase first, and later call it with the execution phase. Is there any other case I'm missing so I can understand the end goal on capturing this split? Thanks!

cc: @marioevz

assert len(manager.setup_blocks) == 0
assert len(manager.execution_transactions) == 0
assert len(manager.execution_blocks) == 0
assert manager.get_current_phase() == TestPhase.EXECUTION
Copy link
Collaborator

@fselmo fselmo Oct 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should have a test with two managers which tests that changing the phase of one does not impact the other (no context muddying). This will be the case for sure because the phase is set on the instance, but just to sanity check against point 2 in this comment in the old PR in case the phase logic ever changes.

It could be simple... something like entering a setup context block with manager1, checking that its context changed to setup, and making sure manager2.get_current_phase() is still execution (default).

Thoughts?

block_access_list: Bytes | None = Field(None)
"""EIP-7928: Block-level access lists (serialized)."""
"""
EIP-7928: Block-level access lists (serialized).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Any reason this was changed? There's an empty space here but we can probably put it back to one line? 👀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
scope:execute Scope: Changes to the execute command scope:fill Scope: fill command scope:fw Scope: Framework (evm|tools|forks|pytest)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants