Skip to content

fix(ported_static): fork-conditional gas bumps for ~340 EIP-8037 Amsterdam tests#2839

Merged
fselmo merged 19 commits into
ethereum:devnets/bal/7from
leolara:wt-bal-7-amsterdam-ported-static-more
May 12, 2026
Merged

fix(ported_static): fork-conditional gas bumps for ~340 EIP-8037 Amsterdam tests#2839
fselmo merged 19 commits into
ethereum:devnets/bal/7from
leolara:wt-bal-7-amsterdam-ported-static-more

Conversation

@leolara
Copy link
Copy Markdown
Member

@leolara leolara commented May 12, 2026

🗒️ Description

Iteratively unskips ported static tests that fail on Amsterdam due to
EIP-8037's two-dimensional gas model. Each tractable failure is fixed
with a small fork-conditional gas bump that preserves the original
pre-EIP-8037 value:

tx_gas_limit = <original>
if fork.is_eip_enabled(8037):
    tx_gas_limit = <bumped>

All edited files keep the original budget on pre-EIP-8037 forks.
Every modified test has been validated on Amsterdam and Cancun.

Skip list: 897 → 554 entries (-343).

Tractable families fixed

  • stRandom + stRandom2 (260 files, 260 entries) — entire auto-
    generated random_statetest* family with gas_limit=100000 bumped
    to 500k on EIP-8037 only.
  • stMemoryTest mem32kb/mem64kb single-byte family (18 files) —
    gas_limit=100000 bumped to 300k (32kb) or 1M (64kb).
  • stMemoryTest mem{0,31,32,33}b_single_byte (4 files) — 100k → 200k.
  • stPreCompiledContracts2 modexp_0_0_0_* (4 files, 11 entries) —
    tx_gas[g] list bumped element-wise so the OoG-by-design `g0`
    parametrization of modexp_0_0_0_20500 is preserved.
  • stSystemOperationsTest test_create_name_registrator* (4 files)
    — `gas_limit=300000` → 1M.
  • stCallCodes/stCallDelegateCodes _suicide_end family* (28 files,
    also from earlier commit on this branch) — inner/middle/outer CALL
    gas bumped; some of these later turned out to bake the gas literal
    into the deployed bytecode and were restored to the skip list (see
    below).
  • misc init-code / create tests (8 files in stCreateTest,
    stCallCreateCallCodeTest, stInitCodeTest, stCreate2, stTransactionTest,
    stRevertTest) — fork-conditional inner-CALL or `tx_gas` bumps.

Tests NOT fixed (554 still skipped on Amsterdam)

  • Gas-measurement tests — post-state stores Op.GAS deltas; needs
    recomputed expected values, not a gas bump (most of
    stEIP150singleCodeGasPrices, parts of stCreateTest, stNonZeroCallsTest,
    stEIP3651_warmcoinbase, stTransactionTest, stEIP150Specific).
  • OoG-by-design tests — test names contain `oog`, `not_enough_gas`,
    `ask_more_gas`, `partial`, `oogm_after/before`. Their premise is hitting
    OoG at an exact budget; a bump would invalidate the assertion (most of
    stCallCreateCallCodeTest, several stCreate2 `_oog`, stWalletTest,
    stRevertTest `oog`, stInitCodeTest `out_of_gas`, stSStoreTest).
  • Bytecode-baked gas literals — these tests verify the deployed code
    of a child contract whose init code was generated by
    `Op.CALL(gas=K, …)`. Changing K changes the bytecode and produces a
    CodeMismatch. Earlier commits in this branch bumped some of these
    (passed at CPSB=1174 but no longer at CPSB=1530); the trailing commit
    restores their skip entries (suicide_end family,
    revert_depth_create_address_collision, push0 [1025_push0],
    revert_opcode_in_calls_on_non_empty_return_data).
  • Multi-parametrization with per-d post-states — stZeroKnowledge
    `test_point_mul_add*`, stCreate2 `test_create2call_precompiles`,
    `test_create2no_cash[d1]`, stRevertTest
    `test_revert_opcode_multiple_sub_calls`, stEIP5656_MCOPY
    `test_mcopy_copy_cost`. Cannot bump a shared `tx_gas` list without
    breaking the OoG-by-design parametrizations.
  • Coinbase/refund balance shifts — stRefundTest, stEIP3860
    init-code-size-limit, stCreateTest `test_create_address_warm_after_fail`.
  • Inverse failures — Amsterdam allows what was previously rejected
    (e.g. stEIP3860 `invalid` parametrization); behavior-change territory.

All modified files carry the `@manually-enhanced` docstring marker.

🔗 Related Issues or PRs

Follows #2796 on the same branch / fork.

✅ Checklist

  • All: Ran fast static checks: `just static` is clean (ruff,
    mypy, vulture, ethereum-spec-lint, actionlint, codespell).
  • All: PR title adheres to the repo standard.
  • All: Considered updating the online docs in the ./docs/ directory.
  • All: Set appropriate labels for the changes (only maintainers can
    apply labels).
  • Tests: Ran `mkdocs serve` locally and verified the auto-generated
    docs for new tests in the Test Case Reference are correctly
    formatted.
  • Tests: For PRs implementing a missed test case, update the
    post-mortem document.
  • Ported Tests: All converted JSON/YML tests from ethereum/tests or
    tests/static have been assigned `@ported_from` marker (preserved
    from upstream).

@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (devnets/bal/7@8148f2e). Learn more about missing BASE report.

Additional details and impacted files
@@               Coverage Diff                @@
##             devnets/bal/7    #2839   +/-   ##
================================================
  Coverage                 ?   85.58%           
================================================
  Files                    ?      630           
  Lines                    ?    39593           
  Branches                 ?     3935           
================================================
  Hits                     ?    33887           
  Misses                   ?     5083           
  Partials                 ?      623           
Flag Coverage Δ
unittests 85.58% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@spencer-tb spencer-tb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for this! We change the hex to int and I removed some of the docstrings comments. Will rebase on #2845 once merged, check the tests still fill then merge this one in!

leolara and others added 19 commits May 12, 2026 11:18
…DelegateCodes* _suicide_end family

28 tests in the `*_suicide_end` family share the same nested
CALL/CALLCODE/DELEGATECALL pattern with three hardcoded inner gas
values (`0xC350` 50k / `0x186A0` 100k / `0x249F0` 150k). On Amsterdam
each SSTORE in the innermost callee adds per-storage state-gas
(`32 * COST_PER_STATE_BYTE`) that spills back into regular gas when
the reservoir is empty, OoG'ing the inner CALL before its SSTORE
marker fires.

Apply fork-conditional `inner_call_gas` / `middle_call_gas` /
`outer_call_gas` variables that keep the original 50k / 100k / 150k
on pre-EIP-8037 forks and bump to `0x186A0` / `0xC3500` / `0xF4240`
on Amsterdam. Each file also gets the `fork: Fork` parameter, the
`Fork` import, and a `@manually-enhanced` docstring marker.

Drops 28 entries from `amsterdam_skip_list.txt` (897 -> 869).
…ests

`test_create2collision_balance`, `_code`, `_code2`, `_nonce` all share
the same pattern: tx with `gas_limit=400_000` deploys a contract via
CREATE2 that collides with an existing account. EIP-8037 NEW_ACCOUNT
state-gas spill on Amsterdam exceeds the 400k budget, OoG'ing the tx
before the collision check.

Apply fork-conditional `outer_tx_gas = 400_000` (or `1_000_000` on
EIP-8037). Pre-EIP-8037 keeps the original; post-state unchanged on
all forks. Marked `@manually-enhanced`; dropped 4 Amsterdam skip
entries (869 -> 865).
…h_create_address_collision

Two tests (`_collision` and `_collision_berlin`) share the same
`tx_gas = [110_000, 170_000]` pattern: tx OoGs on Amsterdam where
EIP-8037 NEW_ACCOUNT state-gas spillover exceeds the budget at the
CREATE2-via-revert path.

Apply fork-conditional `[500_000, 700_000]` on EIP-8037; pre-EIP-8037
keeps the original tuned values. Marked `@manually-enhanced`; dropped
the corresponding Amsterdam skip entries (857 -> 849).
…_statetest* family

9 auto-generated `test_random_statetest*` files (stRandom and
stRandom2) all use `tx_gas = 100_000`. EIP-8037 state-gas spill on
Amsterdam pushes the tx OoG before its SSTORE marker fires.

Apply fork-conditional `tx_gas_limit = 100_000` (pre-EIP-8037) or
`500_000` (EIP-8037). Each file gets `fork: Fork` parameter, `Fork`
import, and `@manually-enhanced` marker. Pre-EIP-8037 behavior
unchanged. Dropped corresponding Amsterdam skip entries.
…de_in_calls / coinbase_warm_fail

Three tests where an inner CALL/DELEGATECALL with a small hardcoded
gas budget OoG'd on Amsterdam due to EIP-8037 state-gas spill:

- `stEIP3855_push0/test_push0.py` — `inner_call_gas` (100k → 1M)
  for the SSTORE-containing callees
- `stRevertTest/test_revert_opcode_in_calls_on_non_empty_return_data.py`
  — three nested DELEGATECALL gas values bumped fork-conditionally
- `stEIP3651_warmcoinbase/test_coinbase_warm_account_call_gas_fail.py`
  — `tx_gas` bumped from 80k to 500k

All three preserve original values on pre-EIP-8037 forks. Each file
marked `@manually-enhanced`. Dropped corresponding Amsterdam skip
entries.
Tests where a single `tx_gas` or inner-CALL `gas=` literal was
hardcoded too tight for EIP-8037 state-gas spill on Amsterdam:

- stCallCodes/test_callcode_in_initcode_to_empty_contract
- stCallCreateCallCodeTest/test_create_fail_balance_too_low
- stCreate2/test_create2_first_byte_loop
- stCreate2/test_create2_suicide
- stRevertTest/test_revert_opcode_calls
- stSystemOperationsTest/test_call_to_name_registrator0
- stSystemOperationsTest/test_callcode_to_return1

Each gets fork-conditional `outer_tx_gas` / `inner_call_gas`
variables defaulting to the original budget on pre-EIP-8037 and
bumped on EIP-8037. Each file marked `@manually-enhanced`; skip
entries dropped.
…t/test_create_name_registrator* family

The four `test_create_name_registrator*` tests share the same
`tx_gas_limit=300000` budget. On Amsterdam, EIP-8037 NEW_ACCOUNT
state-gas spills into regular gas and OoGs the CREATE. Bump
fork-conditionally to 1_000_000 on EIP-8037 only; pre-EIP-8037
forks unchanged. Removes 4 entries from the skip list.
…mem64kb single-byte family

18 tests share `gas_limit=100000` plus a single SSTORE-set after
MSTORE8 at offset 32k or 64k. On Amsterdam the SSTORE-set state-gas
spill exceeds 100k. Bump fork-conditionally to 300k (32kb) / 1M
(64kb) on EIP-8037 only; pre-EIP-8037 unchanged. Removes 18 entries
from the skip list.
…,32,33}b_single_byte

The four small mem-byte tests share `gas_limit=100000` + a single
SSTORE-set. Even with no memory expansion, the EIP-8037 SSTORE-set
state-gas spill exceeds the original 100k tx budget. Bump fork-
conditionally to 200k on EIP-8037 only. Removes 4 entries from the
skip list.
…ntracts2 modexp_0_0_0_* family

The four `modexp_0_0_0_*` tests share a 4-element `tx_gas` budget list
where the lower entries OoG on Amsterdam due to EIP-8037 state-gas
spill (the inner MODEXP CALL leaves no headroom for the post-call
SSTORE-set). For 22000/25000/35000 all expectations require the SSTORE
to succeed, so bump all four entries to 200000. For 20500, entry g0
is the negative-case (must still OoG and leave storage empty), so it
keeps the original 42540; only g1/g2 are bumped. Pre-EIP-8037 forks
unchanged. Removes 11 entries from the skip list.
…to-generated tests

260 random-statetest files share the same `gas_limit=100000` budget
with a single SSTORE-set in the post-state. On Amsterdam the SSTORE-
set state-gas spill exceeds 100k. Bump fork-conditionally to 500_000
on EIP-8037 only; pre-EIP-8037 forks unchanged. Removes 260 entries
from the skip list (the entire stRandom + stRandom2 sections).
…al_call_hitting_gas_limit_success

The test names its inner CALL gas (25k) and tx gas (150k) precisely
to land inside the SSTORE-set on Cancun. Amsterdam adds 48k state-
gas spill per SSTORE-set, OoGing the inner CALL. Bump inner CALL
gas to 200k, tx to 500k, env gas limit to 1M on EIP-8037 only; pre-
EIP-8037 unchanged. Removes 1 entry from the skip list.
…stCallCreateCallCodeTest init-code tests

Four tests share the pattern of a tx with inner CALL forwarding 60k
gas that performs a fresh SSTORE-set in the callee. On Amsterdam,
EIP-8037 state-gas spill (and NEW_ACCOUNT spill on the create_init_*
test) OoGs the inner CALL. Bump inner CALL gas and tx gas fork-
conditionally; pre-EIP-8037 unchanged. Removes 4 entries from the
skip list.
Three contract-creation tests have `gas_limit` precisely tuned around
the Cancun intrinsic (53k base + small calldata). EIP-8037 folds the
new-account state-gas (~171k) into the TX_CREATE intrinsic, pushing
the effective floor above the original budget. Bump tx gas (and the
sender's funded balance where it needed extra room) fork-conditionally
on EIP-8037 only; pre-EIP-8037 unchanged. Removes 3 entries from the
skip list.
…ate_message_reverted

g1=150000 is meant to succeed (CREATE2 deploys a contract that
SSTORE 0=12, SSTORE 1=13). On Amsterdam, EIP-8037 NEW_ACCOUNT plus
2x fresh SSTORE-set state-gas spill exceeds 150k. Bump g1 to 500k
on EIP-8037 only; g0 (the OoG-by-design parametrization) stays at
80k. Sender balance also bumped to cover the larger budget. Pre-
EIP-8037 forks unchanged. Removes 1 entry from the skip list.
…reate_in_init / revert_opcode_return

Both tests share the pattern of a tightly-budgeted tx that the
state-gas spill from a fresh SSTORE-set (revert_opcode_return) or
nested CREATE NEW_ACCOUNT (revert_in_create_in_init_paris) blows
out on Amsterdam. Bump tx gas (only tx_gas[1] for
revert_opcode_return so the other parametrization keeps its tuned
budget) fork-conditionally on EIP-8037 only; pre-EIP-8037
unchanged. Removes 2 entries from the skip list.
…ps no longer cover

After the CPSB recalibration (1174 → 1530), 42 tests that earlier
commits in this branch un-skipped now fail again on Amsterdam. The
fix variables in those test files are still good for pre-EIP-8037
forks, but the bumped Amsterdam budgets are no longer sufficient at
CPSB=1530 — and several of the suicide_end/collision tests bake the
gas literal into the contract bytecode that the post-state hashes,
so bumping the value would break the code-match assertions.

Restore the skip entries so the Amsterdam run is clean; the per-test
fixes can be retuned by hand in a follow-up.
@fselmo fselmo force-pushed the wt-bal-7-amsterdam-ported-static-more branch from 7cc72e9 to 7cfa787 Compare May 12, 2026 17:20
@fselmo
Copy link
Copy Markdown
Contributor

fselmo commented May 12, 2026

Thanks so much for this! We change the hex to int and I removed some of the docstrings comments. Will rebase on #2845 once merged, check the tests still fill then merge this one in!

Rebased here. Just waiting for CI but looks good so far.

@fselmo fselmo merged commit b7860ca into ethereum:devnets/bal/7 May 12, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants