10000 CI: Add job that runs tests with pre-release dependencies by EwoutH · Pull Request #217 · quaquel/EMAworkbench · GitHub
[go: up one dir, main page]
Skip to content

CI: Add job that runs tests with pre-release dependencies #217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Mar 8, 2023
Merged

CI: Add job that runs tests with pre-release dependencies #217

merged 3 commits into from
Mar 8, 2023

Conversation

EwoutH
Copy link
Collaborator
@EwoutH EwoutH commented Dec 5, 2022

Adds a job which installs and runs the tests with pre-release pip dependencies (using pip install --pre). This allows to catch deprecation warnings and failures in new versions to be detected earlier.

It also updates the default job to use Python 3.11, effectively migrating the Windows and macOS jobs from Python 3.10 to 3.11.

@coveralls
Copy link
coveralls commented Dec 5, 2022 •
edited
Loading

Coverage Status

Coverage remained the same at 81.139% when pulling 9f76759 on ci-pip-pre-and-py311 into 4636fdb on master.

@EwoutH
Copy link
Collaborator Author
EwoutH commented Dec 6, 2022

Something really weird is happening where the pytest gets stuck on 13%. The last test passed is

test/test_analysis/test_feature_scoring.py::FeatureScoringTestCase::test_prepare_outcomes PASSED [ 13%]

and after that it keeps running indefinitely without finishing (failing or otherwise).

In earlier testing the test directly after that

test/test_analysis/test_logistic_regression.py::LogitTestCase::test_logit FAILED [ 14%]

failed, as can be seen here in the log. The test fails because of an NumPy error as can be seen here:

>       return masked_where(~(np.isfinite(getdata(a))), a, copy=copy)
E       TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

This might be either a regression in NumPy, or expected behavirour for NumPy 1.24 (currently tested is 1.24.0rc2).

@quaquel Fixing this error might help the pytest to also continue running. Do you have any idea why this is failing and if it's expected?

@quaquel
Copy link
Owner
quaquel commented Dec 6, 2022

without checking the test code because I don't have time for that right now, my hunch is that this is a datatype issue due to ongoing changes in the datatype system in numpy. I'll try to look at this later this week (most likely Thursday) if still necessary.

@EwoutH EwoutH added the ci label Dec 8, 2022
@quaquel
Copy link
Owner
quaquel commented Dec 9, 2022

looks good and can be merged

@EwoutH
Copy link
Collaborator Author
EwoutH commented Dec 18, 2022

NumPy 1.24.0 was just released, so I triggered a CI run, and now all jobs fail on the failed test case above: https://github.com/quaquel/EMAworkbench/actions/runs/3726755907

How important is this failing test? Do we need to issue a bug fix release after fixing this bug?

It at least proves the usefulness of having an early warning CI job.

@quaquel
Copy link
Owner
quaquel commented Dec 19, 2022

It would be worth issuing a bug fix release. I'll try to look at the issue today.

@EwoutH
Copy link
Collaborator Author
EwoutH commented Dec 28, 2022

Looks like it's an interaction bug between NumPy and Pandas. See issues numpy/numpy#22826 and numpy/numpy#22720, and the bugfix PR numpy/numpy#22838.

So NumPy 1.24.1 was released with this fix, and now the CI is all over the place, with builds seemingly randomly passing or getting stuck on that test. Very strange.

@EwoutH EwoutH mentioned this pull request Feb 28, 2023
Closed
@EwoutH EwoutH force-pushed the ci-pip-pre-and-py311 branch from 9f76759 to 3b15b5d Compare February 28, 2023 08:41
EwoutH added 3 commits March 8, 2023 14:40
@EwoutH
CI: Add job that runs tests with pre-release dependencies
df6d7cb 
Adds a job which installs and runs the tests with pre-release pip dependencies. This allows to catch deprecation warnings and failures in new versions to be detected earlier.

It also updates the default job to use Python 3.11, effectively migrating the Windows and macOS jobs from Python 3.10 to 3.11.
@EwoutH
CI: Add name variable to job matrix to separate jobs
548c315
@EwoutH
CI: Fix name syntax
960cb17
@EwoutH EwoutH force-pushed the ci-pip-pre-and-py311 branch from 3b15b5d to 960cb17 Compare March 8, 2023 13:40
@EwoutH
Copy link
Collaborator Author
EwoutH commented Mar 8, 2023

@quaquel Let's merge this. Even if the Ubuntu jobs only sometimes pass, it's useful to have this as an early warning.

Also, the required tests need to be updated (or removed all together for my part, we're perfectly capable of interpreting the test results ourselves).

@quaquel quaquel merged commit c46821d into master Mar 8, 2023
@quaquel quaquel deleted the ci-pip-pre-and-py311 branch November 18, 2023 10:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@quaquel quaquel quaquel approved these changes

Assignees
No one assigned
Labels
ci
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@EwoutH @coveralls @quaquel
0