assert: check test failure message for Empty and NotEmpty #1745
+198
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ccoVeille
commented
May 17, 2025
dolmen
requested changes
May 22, 2025
9117894 to
9580a2c
Compare
Please review |
9580a2c to
0011d03
Compare
Only the tests are updated, code is unchanged. Previously, the tests were checking only the result of the asserter. Using captureTestingT helper allows to check the error message
0011d03 to
559d23a
Compare
dolmen
approved these changes
May 23, 2025
dolmen
requested changes
May 23, 2025
I'm unsure what you meant, could you explain a bit more ? Do you want me to check the error messages returned by require.Empty and require.NotEmpty ? require.Empty and require.NotEmpty are simple wrappers around assert.Empty and assert.NotEmpty, the code is very limited. I feel like testing the returned values of assert.Empty and assert.NotEmpty is enough. I don't plan to maintain two sets of tests, one in assert package and one in require one. Unfortunately, I checked code and for now, captureTestingT is not available in require package I could start a refactoring. captureTestingT could be moved in an internal package to be available for both. TL;DR; the PR seems good enough for me Of course, if I misunderstood what you meant, feel free to come back to me. |
|
I think that given require.Empty and require.NotEmpty are generated wrappers for their assertion counterparts, that testing them as well might not be cost effective. I've not read the code enough to leave a definitive review yet. It's on my list. 👀 |
|
@dolmen could you give me some feedbacks about my last post, and also consider braken one. |
dolmen
approved these changes
May 28, 2025
apricote
pushed a commit
to hetznercloud/fleeting-plugin-hetzner
that referenced
this pull request
Aug 27, 2025
…nercloud/fleeting-plugin-hetzner!273) This MR contains the following updates: | Package | Change | Age | Confidence | |---|---|---|---| | [github.com/stretchr/testify](https://github.com/stretchr/testify) | `v1.10.0` -> `v1.11.0` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>stretchr/testify (github.com/stretchr/testify)</summary> ### [`v1.11.0`](https://github.com/stretchr/testify/releases/tag/v1.11.0) [Compare Source](stretchr/testify@v1.10.0...v1.11.0) #### What's Changed ##### Functional Changes v1.11.0 Includes a number of performance improvements. - Call stack perf change for CallerInfo by [@​mikeauclair](https://github.com/mikeauclair) in [#​1614](stretchr/testify#1614) - Lazily render mock diff output on successful match by [@​mikeauclair](https://github.com/mikeauclair) in [#​1615](stretchr/testify#1615) - assert: check early in Eventually, EventuallyWithT, and Never by [@​cszczepaniak](https://github.com/cszczepaniak) in [#​1427](stretchr/testify#1427) - assert: add IsNotType by [@​bartventer](https://github.com/bartventer) in [#​1730](stretchr/testify#1730) - assert.JSONEq: shortcut if same strings by [@​dolmen](https://github.com/dolmen) in [#​1754](stretchr/testify#1754) - assert.YAMLEq: shortcut if same strings by [@​dolmen](https://github.com/dolmen) in [#​1755](stretchr/testify#1755) - assert: faster and simpler isEmpty using reflect.Value.IsZero by [@​dolmen](https://github.com/dolmen) in [#​1761](stretchr/testify#1761) - suite: faster methods filtering (internal refactor) by [@​dolmen](https://github.com/dolmen) in [#​1758](stretchr/testify#1758) ##### Fixes - assert.ErrorAs: log target type by [@​craig65535](https://github.com/craig65535) in [#​1345](stretchr/testify#1345) - Fix failure message formatting for Positive and Negative asserts in [#​1062](stretchr/testify#1062) - Improve ErrorIs message when error is nil but an error was expected by [@​tsioftas](https://github.com/tsioftas) in [#​1681](stretchr/testify#1681) - fix Subset/NotSubset when calling with mixed input types by [@​siliconbrain](https://github.com/siliconbrain) in [#​1729](stretchr/testify#1729) - Improve ErrorAs failure message when error is nil by [@​ccoVeille](https://github.com/ccoVeille) in [#​1734](stretchr/testify#1734) - mock.AssertNumberOfCalls: improve error msg by [@​3scalation](https://github.com/3scalation) in [#​1743](stretchr/testify#1743) ##### Documentation, Build & CI - docs: Fix typo in README by [@​alexandear](https://github.com/alexandear) in [#​1688](stretchr/testify#1688) - Replace deprecated io/ioutil with io and os by [@​alexandear](https://github.com/alexandear) in [#​1684](stretchr/testify#1684) - Document consequences of calling t.FailNow() by [@​greg0ire](https://github.com/greg0ire) in [#​1710](stretchr/testify#1710) - chore: update docs for Unset [#​1621](stretchr/testify#1621) by [@​techfg](https://github.com/techfg) in [#​1709](stretchr/testify#1709) - README: apply gofmt to examples by [@​alexandear](https://github.com/alexandear) in [#​1687](stretchr/testify#1687) - refactor: use %q and %T to simplify fmt.Sprintf by [@​alexandear](https://github.com/alexandear) in [#​1674](stretchr/testify#1674) - Propose Christophe Colombier (ccoVeille) as approver by [@​brackendawson](https://github.com/brackendawson) in [#​1716](stretchr/testify#1716) - Update documentation for the Error function in assert or require package by [@​architagr](https://github.com/architagr) in [#​1675](stretchr/testify#1675) - assert: remove deprecated build constraints by [@​alexandear](https://github.com/alexandear) in [#​1671](stretchr/testify#1671) - assert: apply gofumpt to internal test suite by [@​ccoVeille](https://github.com/ccoVeille) in [#​1739](stretchr/testify#1739) - CI: fix shebang in .ci.\*.sh scripts by [@​dolmen](https://github.com/dolmen) in [#​1746](stretchr/testify#1746) - assert,require: enable parallel testing on (almost) all top tests by [@​dolmen](https://github.com/dolmen) in [#​1747](stretchr/testify#1747) - suite.Passed: add one more status test report by [@​Ararsa-Derese](https://github.com/Ararsa-Derese) in [#​1706](stretchr/testify#1706) - Add Helper() method in internal mocks and assert.CollectT by [@​dolmen](https://github.com/dolmen) in [#​1423](stretchr/testify#1423) - assert.Same/NotSame: improve usage of Sprintf by [@​ccoVeille](https://github.com/ccoVeille) in [#​1742](stretchr/testify#1742) - mock: enable parallel testing on internal testsuite by [@​dolmen](https://github.com/dolmen) in [#​1756](stretchr/testify#1756) - suite: cleanup use of 'testing' internals at runtime by [@​dolmen](https://github.com/dolmen) in [#​1751](stretchr/testify#1751) - assert: check test failure message for Empty and NotEmpty by [@​ccoVeille](https://github.com/ccoVeille) in [#​1745](stretchr/testify#1745) - deps: fix dependency cycle with objx (again) by [@​dolmen](https://github.com/dolmen) in [#​1567](stretchr/testify#1567) - assert.Empty: comprehensive doc of "Empty"-ness rules by [@​dolmen](https://github.com/dolmen) in [#​1753](stretchr/testify#1753) - doc: improve godoc of top level 'testify' package by [@​dolmen](https://github.com/dolmen) in [#​1760](stretchr/testify#1760) - assert.ErrorAs: simplify retrieving the type name by [@​ccoVeille](https://github.com/ccoVeille) in [#​1740](stretchr/testify#1740) - assert.EqualValues: improve test coverage to 100% by [@​dolmen](https://github.com/dolmen) in [#​1763](stretchr/testify#1763) - suite.Run: simplify running of Setup/TeardownSuite by [@​renzoarreaza](https://github.com/renzoarreaza) in [#​1769](stretchr/testify#1769) - assert.CallerInfo: micro optimization by using LastIndexByte by [@​dolmen](https://github.com/dolmen) in [#​1767](stretchr/testify#1767) - assert.CallerInfo: micro cleanup by [@​dolmen](https://github.com/dolmen) in [#​1768](stretchr/testify#1768) - assert: refactor Test*FileExists and Test*DirExists tests to enable parallel testing by [@​dolmen](https://github.com/dolmen) in [#​1766](stretchr/testify#1766) - suite.Run: refactor handling of stats for improved readability by [@​dolmen](https://github.com/dolmen) in [#​1764](stretchr/testify#1764) - tests: improve captureTestingT helper by [@​ccoVeille](https://github.com/ccoVeille) in [#​1741](stretchr/testify#1741) - build(deps): bump actions/che 7440 ckout from 4 to 5 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​1778](stretchr/testify#1778) #### New Contributors - [@​greg0ire](https://github.com/greg0ire) made their first contribution in [#​1710](stretchr/testify#1710) - [@​techfg](https://github.com/techfg) made their first contribution in [#​1709](stretchr/testify#1709) - [@​mikeauclair](https://github.com/mikeauclair) made their first contribution in [#​1614](stretchr/testify#1614) - [@​cszczepaniak](https://github.com/cszczepaniak) made their first contribution in [#​1427](stretchr/testify#1427) - [@​architagr](https://github.com/architagr) made their first contribution in [#​1675](stretchr/testify#1675) - [@​tsioftas](https://github.com/tsioftas) made their first contribution in [#​1681](stretchr/testify#1681) - [@​siliconbrain](https://github.com/siliconbrain) made their first contribution in [#​1729](stretchr/testify#1729) - [@​bartventer](https://github.com/bartventer) made their first contribution in [#​1730](stretchr/testify#1730) - [@​Ararsa-Derese](https://github.com/Ararsa-Derese) made their first contribution in [#​1706](stretchr/testify#1706) - [@​renzoarreaza](https://github.com/renzoarreaza) made their first contribution in [#​1769](stretchr/testify#1769) - [@​3scalation](https://github.com/3scalation) made their first contribution in [#​1743](stretchr/testify#1743) **Full Changelog**: <stretchr/testify@v1.10.0...v1.11.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi4xMCIsInVwZGF0ZWRJblZlciI6IjQxLjgyLjEwIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
project-mirrors-bot-tu bot
pushed a commit
to project-mirrors/forgejo-runner
that referenced
this pull request
Aug 28, 2025
This PR contains the following updates: | Package | Change | Age | Confidence | |---|---|---|---| | [github.com/stretchr/testify](https://github.com/stretchr/testify) | `v1.10.0` -> `v1.11.1` | [](https://docs.renovatebot.com/merge-confidence/) | [](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>stretchr/testify (github.com/stretchr/testify)</summary> ### [`v1.11.1`](https://github.com/stretchr/testify/releases/tag/v1.11.1) [Compare Source](stretchr/testify@v1.11.0...v1.11.1) This release fixes [#​1785](stretchr/testify#1785) introduced in v1.11.0 where expected argument values implementing the stringer interface (`String() string`) with a method which mutates their value, when passed to mock.Mock.On (`m.On("Method", <expected>).Return()`) or actual argument values passed to mock.Mock.Called may no longer match one another where they previously did match. The behaviour prior to v1.11.0 where the stringer is always called is restored. Future testify releases may not call the stringer method at all in this case. #### What's Changed - Backport [#​1786](stretchr/testify#1786) to release/1.11: mock: revert to pre-v1.11.0 argument matching behavior for mutating stringers by [@​brackendawson](https://github.com/brackendawson) in [#​1788](stretchr/testify#1788) **Full Changelog**: <stretchr/testify@v1.11.0...v1.11.1> ### [`v1.11.0`](https://github.com/stretchr/testify/releases/tag/v1.11.0) [Compare Source](stretchr/testify@v1.10.0...v1.11.0) #### What's Changed ##### Functional Changes v1.11.0 Includes a number of performance improvements. - Call stack perf change for CallerInfo by [@​mikeauclair](https://github.com/mikeauclair) in [#​1614](stretchr/testify#1614) - Lazily render mock diff output on successful match by [@​mikeauclair](https://github.com/mikeauclair) in [#​1615](stretchr/testify#1615) - assert: check early in Eventually, E 8000 ventuallyWithT, and Never by [@​cszczepaniak](https://github.com/cszczepaniak) in [#​1427](stretchr/testify#1427) - assert: add IsNotType by [@​bartventer](https://github.com/bartventer) in [#​1730](stretchr/testify#1730) - assert.JSONEq: shortcut if same strings by [@​dolmen](https://github.com/dolmen) in [#​1754](stretchr/testify#1754) - assert.YAMLEq: shortcut if same strings by [@​dolmen](https://github.com/dolmen) in [#​1755](stretchr/testify#1755) - assert: faster and simpler isEmpty using reflect.Value.IsZero by [@​dolmen](https://github.com/dolmen) in [#​1761](stretchr/testify#1761) - suite: faster methods filtering (internal refactor) by [@​dolmen](https://github.com/dolmen) in [#​1758](stretchr/testify#1758) ##### Fixes - assert.ErrorAs: log target type by [@​craig65535](https://github.com/craig65535) in [#​1345](stretchr/testify#1345) - Fix failure message formatting for Positive and Negative asserts in [#​1062](stretchr/testify#1062) - Improve ErrorIs message when error is nil but an error was expected by [@​tsioftas](https://github.com/tsioftas) in [#​1681](stretchr/testify#1681) - fix Subset/NotSubset when calling with mixed input types by [@​siliconbrain](https://github.com/siliconbrain) in [#​1729](stretchr/testify#1729) - Improve ErrorAs failure message when error is nil by [@​ccoVeille](https://github.com/ccoVeille) in [#​1734](stretchr/testify#1734) - mock.AssertNumberOfCalls: improve error msg by [@​3scalation](https://github.com/3scalation) in [#​1743](stretchr/testify#1743) ##### Documentation, Build & CI - docs: Fix typo in README by [@​alexandear](https://github.com/alexandear) in [#​1688](stretchr/testify#1688) - Replace deprecated io/ioutil with io and os by [@​alexandear](https://github.com/alexandear) in [#​1684](stretchr/testify#1684) - Document consequences of calling t.FailNow() by [@​greg0ire](https://github.com/greg0ire) in [#​1710](stretchr/testify#1710) - chore: update docs for Unset [#​1621](stretchr/testify#1621) by [@​techfg](https://github.com/techfg) in [#​1709](stretchr/testify#1709) - README: apply gofmt to examples by [@​alexandear](https://github.com/alexandear) in [#​1687](stretchr/testify#1687) - refactor: use %q and %T to simplify fmt.Sprintf by [@​alexandear](https://github.com/alexandear) in [#​1674](stretchr/testify#1674) - Propose Christophe Colombier (ccoVeille) as approver by [@​brackendawson](https://github.com/brackendawson) in [#​1716](stretchr/testify#1716) - Update documentation for the Error function in assert or require package by [@​architagr](https://github.com/architagr) in [#​1675](stretchr/testify#1675) - assert: remove deprecated build constraints by [@​alexandear](https://github.com/alexandear) in [#​1671](stretchr/testify#1671) - assert: apply gofumpt to internal test suite by [@​ccoVeille](https://github.com/ccoVeille) in [#​1739](stretchr/testify#1739) - CI: fix shebang in .ci.\*.sh scripts by [@​dolmen](https://github.com/dolmen) in [#​1746](stretchr/testify#1746) - assert,require: enable parallel testing on (almost) all top tests by [@​dolmen](https://github.com/dolmen) in [#​1747](stretchr/testify#1747) - suite.Passed: add one more status test report by [@​Ararsa-Derese](https://github.com/Ararsa-Derese) in [#​1706](stretchr/testify#1706) - Add Helper() method in internal mocks and assert.CollectT by [@​dolmen](https://github.com/dolmen) in [#​1423](stretchr/testify#1423) - assert.Same/NotSame: improve usage of Sprintf by [@​ccoVeille](https://github.com/ccoVeille) in [#​1742](stretchr/testify#1742) - mock: enable parallel testing on internal testsuite by [@​dolmen](https://github.com/dolmen) in [#​1756](stretchr/testify#1756) - suite: cleanup use of 'testing' internals at runtime by [@​dolmen](https://github.com/dolmen) in [#​1751](stretchr/testify#1751) - assert: check test failure message for Empty and NotEmpty by [@​ccoVeille](https://github.com/ccoVeille) in [#​1745](stretchr/testify#1745) - deps: fix dependency cycle with objx (again) by [@​dolmen](https://github.com/dolmen) in [#​1567](stretchr/testify#1567) - assert.Empty: comprehensive doc of "Empty"-ness rules by [@​dolmen](https://github.com/dolmen) in [#​1753](stretchr/testify#1753) - doc: improve godoc of top level 'testify' package by [@​dolmen](https://github.com/dolmen) in [#​1760](stretchr/testify#1760) - assert.ErrorAs: simplify retrieving the type name by [@​ccoVeille](https://github.com/ccoVeille) in [#​1740](stretchr/testify#1740) - assert.EqualValues: improve test coverage to 100% by [@​dolmen](https://github.com/dolmen) in [#​1763](stretchr/testify#1763) - suite.Run: simplify running of Setup/TeardownSuite by [@​renzoarreaza](https://github.com/renzoarreaza) in [#​1769](stretchr/testify#1769) - assert.CallerInfo: micro optimization by using LastIndexByte by [@​dolmen](https://github.com/dolmen) in [#​1767](stretchr/testify#1767) - assert.CallerInfo: micro cleanup by [@​dolmen](https://github.com/dolmen) in [#​1768](stretchr/testify#1768) - assert: refactor Test*FileExists and Test*DirExists tests to enable parallel testing by [@​dolmen](https://github.com/dolmen) in [#​1766](stretchr/testify#1766) - suite.Run: refactor handling of stats for improved readability by [@​dolmen](https://github.com/dolmen) in [#​1764](stretchr/testify#1764) - tests: improve captureTestingT helper by [@​ccoVeille](https://github.com/ccoVeille) in [#​1741](stretchr/testify#1741) - build(deps): bump actions/checkout from 4 to 5 by [@​dependabot](https://github.com/dependabot)\[bot] in [#​1778](stretchr/testify#1778) #### New Contributors - [@​greg0ire](https://github.com/greg0ire) made their first contribution in [#​1710](stretchr/testify#1710) - [@​techfg](https://github.com/techfg) made their first contribution in [#​1709](stretchr/testify#1709) - [@​mikeauclair](https://github.com/mikeauclair) made their first contribution in [#​1614](stretchr/testify#1614) - [@​cszczepaniak](https://github.com/cszczepaniak) made their first contribution in [#​1427](stretchr/testify#1427) - [@​architagr](https://github.com/architagr) made their first contribution in [#​1675](stretchr/testify#1675) - [@​tsioftas](https://github.com/tsioftas) made their first contribution in [#​1681](stretchr/testify#1681) - [@​siliconbrain](https://github.com/siliconbrain) made their first contribution in [#​1729](stretchr/testify#1729) - [@​bartventer](https://github.com/bartventer) made their first contribution in [#​1730](stretchr/testify#1730) - [@​Ararsa-Derese](https://github.com/Ararsa-Derese) made their first contribution in [#​1706](stretchr/testify#1706) - [@​renzoarreaza](https://github.com/renzoarreaza) made their first contribution in [#​1769](stretchr/testify#1769) - [@​3scalation](https://github.com/3scalation) made their first contribution in [#​1743](stretchr/testify#1743) **Full Changelog**: <stretchr/testify@v1.10.0...v1.11.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC), Automerge - Between 12:00 AM and 03:59 AM ( * 0-3 * * * ) (UTC). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS44Mi4xMCIsInVwZGF0ZWRJblZlciI6IjQxLjgyLjEwIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJLaW5kL0Nob3JlIl19--> <!--start release-notes-assistant--> <!--URL:https://code.forgejo.org/forgejo/runner--> - other - [PR](https://code.forgejo.org/forgejo/runner/pulls/915): <!--number 915 --><!--line 0 --><!--description VXBkYXRlIG1vZHVsZSBnaXRodWIuY29tL3N0cmV0Y2hyL3Rlc3RpZnkgdG8gdjEuMTEuMQ==-->Update module github.com/stretchr/testify to v1.11.1<!--description--> <!--end release-notes-assistant--> Reviewed-on: https://code.forgejo.org/forgejo/runner/pulls/915 Reviewed-by: earl-warren <earl-warren@noreply.code.forgejo.org> Co-authored-by: Renovate Bot <bot@kriese.eu> Co-committed-by: Renovate Bot <bot@kriese.eu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Repost of #1737.
check test failure message for Empty and NotEmpty
Changes
Only the tests are updated, code is unchanged.
Previously, the tests were checking only the result of the asserter.
Using captureTestingT helper allows to check the error message
Motivation
It allows us to make sure the error message reported are OK.
Related issues
This is a proof of concept PR that follows up #1737 that was closed because:
it was not using a fork
there were a lot of changes to review.
chore: improve tests #1737