8000 ⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev ⚠️ · Issue #26154 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev ⚠️ #26154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
scikit-learn-bot opened this issue Apr 12, 2023 · 9 comments
Closed

⚠️ CI failed on Linux_Nightly.pylatest_pip_scipy_dev ⚠️ #26154

scikit-learn-bot opened this issue Apr 12, 2023 · 9 comments
Assignees
@glemaitre
Labels
Bug Build / CI

Comments

@scikit-learn-bot
Copy link
Contributor
scikit-learn-bot commented Apr 12, 2023 •
edited
Loading

CI is still failing on Linux_Nightly.pylatest_pip_scipy_dev (Jul 06, 2023)
Unable to find junit file. Please see link for details.
@github-actions github-actions bot added the Needs Triage Issue requires triage label Apr 12, 2023
@glemaitre
Copy link
Member

/take

@glemaitre
Copy 8000 link
Member

The culprit is pandas dev. I will bisect to know which commit changed the behaviour.

@glemaitre
Copy link
Member
glemaitre commented Apr 12, 2023 •
edited
Loading

So it comes from this commit: pandas-dev/pandas#52542

It comes from calling pd.concat(..., ignore_index=True) with a first dataset containing None (thus an object dtype) with a second dataset containing np.nan and float (thus a float64 dtype).

The previous behaviour cast the column as object dtype while the new behaviour is casting into float64.

I am trying to craft a minimal reproducer.

@lesteve
Copy link
Member
lesteve commented Apr 26, 2023 •
edited
Loading

There are plenty of (~280) recent errors probably due to a pandas change (and maybe numpy too?), symptoms look like this:

  • FutureWarning: is_sparse is deprecated and will be removed in a future version. Check isinstance(dtype, pd.SparseDtype) instead.
  • FutureWarning: The behavior of DataFrame concatenation with all-NA entries is deprecated. In a future version, this will no longer exclude all-NA columns when determining the result dtypes. To retain the old behavior, cast the all-NA columns to the desired dtype before the concat operation.
  • ValueError: setting an array element with a sequence.
  • DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)

Number of failures look like this (quick and dirty analysis may miss a few kind of errors):

    245 FutureWarning: is_sparse is deprecated and will be removed in a future version. Check `isinstance(dtype, pd.SparseDtype)` instead.
     19 DeprecationWarning: Conversion of an array with ndim > 0 to a scalar is deprecated, and will error in future. Ensure you extract a single element from your array before performing this operation. (Deprecated NumPy 1.25.)
     14 ValueError: setting an array element with a sequence.
      5 FutureWarning: The behavior of DataFrame concatenation with all-NA entries is deprecated. In a future version, this will no longer exclude all-NA columns when determining the result dtypes. To retain the old behavior, cast the all-NA columns to the desired dtype before the concat operation.

@lesteve lesteve mentioned this issue Apr 27, 2023
Merged
@lesteve
Copy link
Member
lesteve commented Apr 27, 2023

Opened #26287 about pandas is_sparse

@jjerphan jjerphan mentioned this issue May 1, 2023
Merged
@thomasjpfan thomasjpfan added Bug Build / CI and removed Needs Triage Issue requires triage labels May 4, 2023
This was referenced May 7, 2023
Closed
Merged
@thomasjpfan thomasjpfan mentioned this issue May 16, 2023
Merged
@lesteve
Copy link
Member
lesteve commented May 23, 2023

Seems like some np.find_common_type DeprecationWarning are coming from pandas pandas-dev/pandas#53236 and should hopefully be fixed soon.

@lesteve lesteve mentioned this issue Jun 7, 2023
Closed
@ogrisel ogrisel mentioned this issue Jun 12, 2023
Merged
@scikit-learn-bot
Copy link
Contributor Author
scikit-learn-bot commented Jun 16, 2023 •
edited
Loading

CI is no longer failing! ✅

Successful run on Jun 19, 2023

@jjerphan
Copy link
Member

For you information, SciPy is currently transitioning from the sparse matrix semantic to the sparse array semantic (see #26418 for discussing what it means for scikit-learn).

If tests using sparse data fail on pylatest_pip_scipy_dev, feel free to ping me.

@jeremiedbb
Copy link
Member

These issues have all been fixed. Let's close

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@glemaitre glemaitre

Labels
Bug Build / CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@lesteve @thomasjpfan @glemaitre @scikit-learn-bot @jjerphan @jeremiedbb
0