You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
-1Lines changed: 0 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -109,7 +109,6 @@ cd execution-spec-tests
109
109
uv python install 3.11
110
110
uv python pin 3.11
111
111
uv sync --all-extras
112
-
uv run solc-select use 0.8.24 --always-install
113
112
```
114
113
115
114
See [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) in the online docs if you encounter issues.
Copy file name to clipboardExpand all lines: docs/CHANGELOG.md
+11Lines changed: 11 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,6 +10,11 @@ Test fixtures for use by clients are available for each release on the [Github r
10
10
11
11
- Python 3.10 support was removed in this release ([#1808](https://github.com/ethereum/execution-spec-tests/pull/1808)).
12
12
13
+
#### 💥 Important Change for test contributors
14
+
15
+
- EEST no longer allows usage of Yul code in Python tests. From now on, please make use of our opcode wrapper. Yul code is now only allowed in the "static tests" located in `./tests/static/` (these are test cases defined by JSON and YAML files instead of Python test functions that were originally maintained in [ethereum/tests](https://github.com/ethereum/tests)).
16
+
- In order to fill the static tests (which is not the case by default), please ensure that `solc` is located in your `PATH`.
17
+
13
18
#### 💥 Important Change for `fill` Users
14
19
15
20
The output behavior of `fill` has changed ([#1608](https://github.com/ethereum/execution-spec-tests/pull/1608)):
@@ -27,6 +32,12 @@ Users can select any of the artifacts depending on their testing needs for their
27
32
28
33
### 🛠️ Framework
29
34
35
+
#### 🔀 Refactoring
36
+
37
+
- 🔀 Move `TransactionType` enum from test file to proper module location in `ethereum_test_types.transaction_types` for better code organization and reusability.
38
+
- ✨ Opcode classes now validate keyword arguments and raise `ValueError` with clear error messages.
39
+
- 🔀 This PR removes the `solc` requirement to fill Python test cases. Regular test contributors no longer need to concern themselves with `solc` and, as such, the `solc-select` dependency has been removed. The remaining tests that used Yul have been ported to the EEST opcode wrapper mini-lang and the use of Yul in Python tests is no longer supported. Maintainers only: To fill the "static" JSON and YAML tests (`./tests/static/`) locally, `solc` (ideally v0.8.24) must be available in your PATH.
40
+
30
41
#### `fill`
31
42
32
43
- ✨ Add the `ported_from` test marker to track Python test cases that were converted from static fillers in [ethereum/tests](https://github.com/ethereum/tests) repository ([#1590](https://github.com/ethereum/execution-spec-tests/pull/1590)).
Copy file name to clipboardExpand all lines: docs/dev/porting_legacy_tests.md
+2-4Lines changed: 2 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -85,16 +85,14 @@ Follow the hyperlinks for lost base coverage (`LBC`) to address coverage gaps. H
85
85
86
86
It's important to note that coverage helps identify missing test paths. If you believe the coverage loss is due to differences in "setup" code between frameworks and doesn't impact the feature you're testing, explain this in your PR. A team member can help with the review.
87
87
88
-
Also note that yul tests and possibly other tests used `CALLDATALOAD` that might no longer needed when designing a test with python. But we must always investigate if an opcode is not covered anymore to see if its okay.
89
-
90
-
For example, review the [discussion in this PR.](https://github.com/ethereum/execution-spec-tests/pull/975#issuecomment-2528792289)
88
+
For example, review the [discussion in this PR] to see an example of why and how coverage loss can occur.(https://github.com/ethereum/execution-spec-tests/pull/975#issuecomment-2528792289)
91
89
92
90
## Resolving Coverage Gaps from Yul Compilation
93
91
94
92
When porting tests from ethereum/tests, you may encounter coverage gaps that are false positives. This commonly occurs because:
95
93
96
94
-**Original tests** often used Yul to define smart contracts, and solc compilation introduces additional opcodes that aren't specifically under test
97
-
-**EEST ports**typically use the explicit EEST Opcode mini-language, which is more precise in opcode definition
95
+
-**EEST ports** use the explicit EEST Opcode mini-language, which is more precise in opcode definition
98
96
99
97
If coverage analysis shows missing opcodes that were only present due to Yul compilation artifacts (not the actual feature being tested), this can be resolved during PR review by adding the `coverage_missed_reason` parameter:
Copy file name to clipboardExpand all lines: docs/getting_started/installation.md
+2-12Lines changed: 2 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -26,27 +26,17 @@ If installed via `curl`, `uv` will download Python for your target platform if o
26
26
27
27
Clone [execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and install its dependencies. We recommend using Python 3.12, the following uses `uv` to download and configures 3.12 to be the Python version used in execution-spec-tests:
Then follow [this guide](./installation_troubleshooting.md#problem-exception-failed-to-compile-yul-source) to build the `solc` binary from source and copy it to the expected location.
39
+
Static tests/maintainers only: To learn how to build the `solc` binary from source (optional) follow [this guide](./installation_troubleshooting.md#problem-exception-failed-to-compile-yul-source).
Copy file name to clipboardExpand all lines: docs/getting_started/installation_troubleshooting.md
+6-13Lines changed: 6 additions & 13 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,11 +38,11 @@ This page provides guidance on how to troubleshoot common issues that may arise
38
38
39
39
### Problem: `CERTIFICATE_VERIFY_FAILED`
40
40
41
-
!!! danger "Problem: `Failed to install solc ... CERTIFICATE_VERIFY_FAILED`"
42
-
When running either `uv run solc-select use 0.8.24 --always-install` or `fill` you encounter the following error:
41
+
!!! danger "Problem: `Failed to ... CERTIFICATE_VERIFY_FAILED`"
42
+
When running `fill` you might encounter the following error:
43
43
44
44
```bash
45
-
Exit: Failed to install solc version 0.8.24: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)>
45
+
Exit: Failed to ...: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)>
46
46
```
47
47
48
48
=== "Ubuntu"
@@ -66,15 +66,8 @@ This page provides guidance on how to troubleshoot common issues that may arise
66
66
67
67
### Problem: `Exception: failed to compile yul source`
68
68
69
-
!!! danger "Problem: `Running fill fails with tests that contain yul source code` on ARM platforms"
70
-
If you're using `x86_64`, try to manually run `solc-select` to install the required version of the `solc` binary:
71
-
72
-
```shell
73
-
uv run solc-select use 0.8.24 --always-install
74
-
```
75
-
76
-
Otherwise, this can happen when you're using an ARM64 OS but followed the x86-64 installation guide.
77
-
To resolve the issue you must build solidity from source (avoid 0.8.24 as it might require building z3 from source as well).
69
+
!!! danger "Problem: `Running fill on static_tests fails with tests that contain yul source code` on ARM platforms"
70
+
To resolve the issue you must acquire the `solc` binary e.g. by building solidity from source. The guide below installs v0.8.28 but you might prefer to choose a different version.
78
71
79
72
!!! success "Solution: Build solc from source"
80
73
The following steps have been tested on Ubuntu 24.04.2 LTS ARM64:
@@ -88,7 +81,7 @@ This page provides guidance on how to troubleshoot common issues that may arise
Running `uv run solc --version` should now return the expected version. Verify that `fill` works by running `uv run fill -m yul_test` in the execution-spec-tests root folder.
84
+
Running `uv run solc --version` should now return the expected version.
92
85
93
86
## Problem: `ValueError: unsupported hash type ripemd160`
Copy file name to clipboardExpand all lines: docs/running_tests/test_formats/blockchain_test_engine_x.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,7 +14,7 @@ The key optimization is that **clients need only be started once per group** ins
14
14
15
15
Instead of including large pre-allocation state in each test fixture, this format references a pre-allocation groups folder (`pre_alloc`) which contains all different pre-allocation combinations organized by group.
16
16
17
-
A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`BlockchainTestEngineXFixture`](#BlockchainTestEngineXFixture) test object, with the key string representing the test name.
17
+
A single JSON fixture file is composed of a JSON object where each key-value pair is a different [`BlockchainTestEngineXFixture`](#blockchaintestenginexfixture) test object, with the key string representing the test name.
18
18
19
19
The JSON file path plus the test name are used as the unique test identifier.
20
20
@@ -48,7 +48,7 @@ Each file in the `pre_alloc` folder corresponds to a pre-allocation group identi
48
48
49
49
## Consumption
50
50
51
-
For each [`BlockchainTestEngineXFixture`](#BlockchainTestEngineXFixture) test object in the JSON fixture file, perform the following steps:
51
+
For each [`BlockchainTestEngineXFixture`](#blockchaintestenginexfixture) test object in the JSON fixture file, perform the following steps:
52
52
53
53
1.**Load Pre-Allocation Group**:
54
54
- Read the appropriate file from the `pre_alloc` folder in the same directory
0 commit comments