8000 Drop pytest warning config in nightly tests by QuLogic · Pull Request #23130 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]
Skip to content

Drop pytest warning config in nightly tests #23130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 25, 2022
Merged

Drop pytest warning config in nightly tests #23130

merged 2 commits into from
May 25, 2022

Conversation

QuLogic
Copy link
Member
@QuLogic QuLogic commented May 25, 2022 •
edited
Loading

PR Summary

Apparently, NumPy or whatever dependency has fixed the warnings that we needed to ignore. Tests with nightlies enabled seem to be passing just fine.

And having this file is causing pytest to drop some config we used to have, making it load files we don't want.

Fixes #23084.

PR Checklist

Tests and Styling

  • Has pytest style unit tests (and pytest passes).
  • Is Flake 8 compliant (install flake8-docstrings and run flake8 --docstring-convention=all).

Documentation

  • [n/a] New features are documented, with examples if plot related.
  • [n/a] New features have an entry in doc/users/next_whats_new/ (follow instructions in README.rst there).
  • [n/a] API changes documented in doc/api/next_api_changes/ (follow instructions in README.rst there).
  • [n/a] Documentation is sphinx and numpydoc compliant (the docs should build without error).

@QuLogic
Drop pytest warning config in nightly tests
fb84a0f 
Apparently, NumPy or whatever dependency has fixed the warnings that we
needed to ignore. And having this file is causing pytest to drop some
config we used to have, making it load files we don't want.
@QuLogic QuLogic added this to the v3.6.0 milestone May 25, 2022
@QuLogic
Copy link
Member Author
QuLogic commented May 25, 2022

Actually, looking a bit closer, I see warnings from examples/scales/log_test.py, which we definitely don't want to run.

Locally, I see pytest --collect-only | rg examples shows warnings from the examples, but they aren't collected. That's because they have top-level code, but no test functions/classes. So I wonder if we should add a pytest.ini by default that tells pytest to ignore those directories.

And then also turn warnings into errors in the nightly build (by mostly reverting this PR), so we catch things from upstream?

@QuLogic
Copy link
Member Author
QuLogic commented May 25, 2022

After some further investigation, I think I see the reason for the inconsistency. For stuff in lib, we have a conftest.py that configures warnings into errors and ignores a few:

matplotlib/lib/matplotlib/testing/conftest.py

Lines 20 to 25 in fb84a0f

("filterwarnings", "error"),
("filterwarnings",
"ignore:.*The py23 module has been deprecated:DeprecationWarning"),
("filterwarnings",
r"ignore:DynamicImporter.find_spec\(\) not found; "
r"falling back to find_module\(\):ImportWarning"),

We don't have that for examples, so that's why the warnings depend on the config in pytest.ini. The distutils warning is not ignored above though, and I assume from that that the upstream code has been fixed to not trigger those.

What that means is that I'm going to keep the commit above, dropping the pytest.ini in the nightly test. But I am also going to add a pytest.ini that tells pytest to only look in lib, as we don't want it trying to import examples, tutorials, etc.

@QuLogic
Tell Pytest to only look in lib by default
d9fbc7d 
This prevents it from trying to look at examples or other documentation,
some of which may match the test file glob, but won't contain tests.
greglucas
greglucas reviewed May 25, 2022
View reviewed changes
pytest.ini
@@ -0,0 +1,7 @@
[pytest]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong preference here, but would it make sense to add this to the setup.cfg since we already have that? It seems like a lot of configuration stuff is getting put into that and pyproject.tomls now.

Copy link
Member
@timhoffm timhoffm May 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let’s for now stick with pytest.ini. Long term this should go into pyproject.toml. However creating it only for pytest may be a bit confusing. Setup.cfg is not really recommended for pytest https://docs.pytest.org/en/6.2.x/customize.html#setup-cfg.

@timhoffm timhoffm merged commit 3048d7b into matplotlib:main May 25, 2022
@QuLogic QuLogic deleted the fix-nightly-again branch May 25, 2022 19:26
@greglucas greglucas mentioned this pull request Jun 7, 2022
Closed
@matthewfeickert matthewfeickert mentioned this pull request Jun 8, 2022
Merged
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@greglucas greglucas greglucas approved these changes

@timhoffm timhoffm timhoffm approved these changes

Assignees
No one assigned
Labels
topic: testing
Projects
None yet
Milestone
v3.6.0
Development

Successfully merging this pull request may close these issues.

[TST] Upcoming dependency test failures
3 participants
@QuLogic @timhoffm @greglucas
0