8000 Fix non-deterministic test generation by anumukul · Pull Request #4641 · ethereum/consensus-specs · GitHub
[go: up one dir, main page]
Skip to content

Fix non-deterministic test generation #4641

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix non-deterministic test generation #4641

wants to merge 2 commits into from

Conversation

anumukul
Copy link
@anumukul anumukul commented Oct 7, 2025

This PR fixes non-deterministic test generation that caused different outputs when running make reftests on different systems.

Changes

  1. Proposer slashing tests: Added random.seed() calls at the start of three test functions to ensure deterministic slot selection
  2. SSZ progressive containers: Sorted the iteration over MODIFIED_PROGRESSIVE_CONTIANERS by class name to ensure consistent ordering

Fixes #4639

@anumukul
Fix non-deterministic test generation
03b51c4 
Add deterministic random seeding to proposer slashing tests and sort
SSZ progressive container iteration to ensure consistent test output
across different systems.

- Add random.seed() to three proposer slashing tests in gloas
- Sort MODIFIED_PROGRESSIVE_CONTIANERS iteration by class name

Fixes ethereum#4639
@jtraglia
Copy link
Member
jtraglia commented Oct 7, 2025 •
edited
Loading

Hey @anumukul thanks for the PR!

(0) Could you run make lint to fix the linter?

(1) Could you include a fix for this typo I noticed? Unrelated to your fix.

consensus-specs/tests/generators/runners/ssz_generic_cases/ssz_progressive_container.py

Line 135 in bfd90bd

MODIFIED_PROGRESSIVE_CONTIANERS: Sequence[type[View]] = {

(2) What about sanity/blocks/pyspec_tests/mix_blob_tx_and_non_blob_tx tests?

(3) Did you test these yourself? For example, from the consensus-specs directory, you can do something like:

  • make clean
  • rm -rf ../consensus-spec-tests # or move it elsewhere
  • make reftests runner=operations

And then compare the consensus-spec-tests between multiple runs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make reftest generation reproducible
2 participants
@anumukul @jtraglia
0