ci: remove commit count limit from `test-each-commit` #34383

l0rinc · 2026-01-22T21:38:21Z

Problem

test-each-commit currently tests only a limited number of ancestor commits in a PR, so failures introduced deeper in the commit stack might be missed.

Fix

Remove the max-count limit so test-each-commit runs the full build + unit + functional test flow on every non-head PR commit, while keeping the PR tip excluded because it is already covered by the normal CI jobs.

Examples

Example failure 10 commits deep: https://github.com/l0rinc/bitcoin/actions/runs/21390976651/job/61577575033?pr=105 in ci: compile untested ancestor commits as minimal sanity-check - failing l0rinc/bitcoin#105
Example pass with >7 dummy commits: https://github.com/l0rinc/bitcoin/actions/runs/21392557521/job/61595159841?pr=106 in ci: compile untested ancestor commits as minimal sanity-check - passing l0rinc/bitcoin#106

Note: this PR has gone through a few iterations, the latest one just extends the existing job.

DrahtBot · 2026-01-22T21:38:33Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.

Type	Reviewers
ACK	maflcko

If your review is incorrectly listed, please copy-paste  into the comment that the bot should ignore.

DrahtBot · 2026-01-22T22:14:12Z

🚧 At least one of the CI tasks failed.
_{Task lint: https://github.com/bitcoin/bitcoin/actions/runs/21265870531/job/61204792016}
_{LLM reason (✨ experimental): Lint failure: a file is not executable (ci-compile-each-commit-exec.py has 644 permission).}

Hints

Try to run the tests locally, according to the documentation. However, a CI failure may still
happen due to a number of reasons, for example:

Possibly due to a silent merge conflict (the changes in this pull request being
incompatible with the current code in the target branch). If so, make sure to rebase on the latest
commit of the target branch.
A sanitizer issue, which can only be found by compiling with the sanitizer and running the
affected test.
An intermittent issue.

Leave a comment here, if you need help tracking down a confusing failure.

maflcko · 2026-01-23T08:48:58Z

ci-test-each-commit-exec.py

unrelated note: I think this script is missing the functional test runner --failfast option. A failure will lead to a massive log output with all failures collected, which is hard to read and debug. It would be better to just print the first failure.

More generally, I wonder if it wouldn't be easier to just run it on all commits. I guess the timeout could be raised to the max (6 hours), possibly using a cirrus runner.

If a pull request doesn't finish after 6 hours on a 16 core machine, then it is probably too large and should be split up?

If it isn't possible to split up, one or two reviewers can run the script locally?

l0rinc · 2026-01-23T17:13:00Z

Thanks @maflcko, split the previous test-each-commit into two, updated PR title&description (kept old branch name though).

maflcko · 

Ah, with "it" I meant "pull request", not "the ci task here".

If a pull request doesn't finish after 6 hours on a 16 core machine, then it is probably too large and should be split up?

Was there ever a pull request in history that wouldn't compile and test all commits in 6 hours on a modern 16 core machine?

l0rinc · 2026-01-26T08:20:14Z

Was there ever a pull request in history that wouldn't compile and test all commits in 6 hours on a modern 16 core machine?

let's find out

l0rinc · 2026-01-26T08:21:16Z

the leveldb and crc32 updates don't always compile, but I couldn't find any other

edit: e.g. first commit of #33641 fails with util/env_posix.cc:857:24: error: no template named 'is_standard_layout_v' in namespace 'std'; did you mean 'is_standard_layout'? And 9a5d297 also fails with old cmake requirement.

maflcko · 2026-01-26T08:32:33Z

the leveldb and crc32 updates don't always compile, but I couldn't find any other

edit: e.g. first commit of #33641 fails with util/env_posix.cc:857:24: error: no template named 'is_standard_layout_v' in namespace 'std'; did you mean 'is_standard_layout'? And 9a5d297 also fails with old cmake requirement.

That pull request has a single commit, so it seems impossible for it to time out. Also, if it timed-out the current CI task would time out as well.

If you want to actually test this, my recommendation would be to only look at pulls with more than 6 commits.

l0rinc · 2026-01-26T08:54:32Z

you mean that f21162d was already skipped by CI?

maflcko · 2026-01-26T09:08:19Z

you mean that f21162d was already skipped by CI?

The commit is both a child of a merge commit and also a subtree commit of a different repo. You can also check the behavior here https://github.com/bitcoin/bitcoin/actions/runs/21197620806/job/60976836032?pr=34363, which is another subtree bump.

l0rinc · 2026-01-26T16:05:24Z

Thanks for the feedback, the latest version of the PR simply removes the 6 commit limit from test-each-commit job.

maflcko · 2026-01-27T08:44:53Z

This is missing a test (sample CI output).

Also, when increasing the commit count to check, this should run on a larger runner, I'd say. Maybe ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md with double the CPU threads?

l0rinc · 2026-01-27T09:11:27Z

This is missing a test (sample CI output).

Added back the reproducer examples to the PR description.

Maybe ghcr.io/cirruslabs/ubuntu-runner-amd64:24.04-md

Thanks, added this one with standard ubuntu fallback, similarly to the lint job.

double the CPU threads

It seems to me we're already doing that:

bitcoin/.github/ci-test-each-commit-exec.py

Line 59 in fa6db67

str(num_procs * 2),

Also applied a DrahtBot LLM suggestion.

Extend `test-each-commit` to run on every non-head pull request commit. The PR tip is excluded because it is already covered by other CI jobs. Runner was changed to a performant cirrus runner. Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>

maflcko · 2026-01-28T16:44:46Z

lgtm ACK 22a6d89

DrahtBot added the Tests label Jan 22, 2026

l0rinc force-pushed the l0rinc/ci-compile-untested-commits branch from 1a6a888 to 4640108 Compare January 22, 2026 22:13

DrahtBot added the CI failed label Jan 22, 2026

l0rinc marked this pull request as ready for review January 22, 2026 23:18

DrahtBot removed the CI failed label Jan 23, 2026

l0rinc marked this pull request as draft January 23, 2026 16:15

l0rinc force-pushed the l0rinc/ci-compile-untested-commits branch 2 times, most recently from 292a097 to 052728f Compare January 23, 2026 17:06

DrahtBot added the CI failed label Jan 23, 2026

l0rinc changed the title ~~ci: compile remaining untested ancestor commits~~ ci: run unit + functional tests on all ancestor commits Jan 23, 2026

l0rinc marked this pull request as ready for review January 23, 2026 17:13

DrahtBot removed the CI failed label Jan 23, 2026

This was referenced Jan 25, 2026

kernel, logging: Pass Logger instances to kernel objects #30342

Draft

kernel, refactor: return error status on all fatal errors #29700

Draft

Add util::ResultPtr class #26022

Draft

refactor: Use util::Result class for wallet loading #25722

Draft

l0rinc force-pushed the l0rinc/ci-compile-untested-commits branch from 052728f to d5964ce Compare January 26, 2026 16:03

l0rinc changed the title ~~ci: run unit + functional tests on all ancestor commits~~ ci: remove commit count limit from test-each-commit Jan 26, 2026

l0rinc force-pushed the l0rinc/ci-compile-untested-commits branch from d5964ce to 512b3a3 Compare January 27, 2026 09:09

l0rinc force-pushed the l0rinc/ci-compile-untested-commits branch from 512b3a3 to 22a6d89 Compare January 27, 2026 09:25

DrahtBot added CI failed and removed CI failed labels Jan 27, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ci: remove commit count limit from `test-each-commit` #34383

ci: remove commit count limit from `test-each-commit` #34383

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ci: remove commit count limit from test-each-commit #34383

Are you sure you want to change the base?

ci: remove commit count limit from test-each-commit #34383

Conversation

Uh oh!

Problem

Fix

Examples

Uh oh!

Uh oh!

Reviews

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ci: remove commit count limit from `test-each-commit` #34383

ci: remove commit count limit from `test-each-commit` #34383