8000 BUG: Regression in 1.25.0 for boolean operation on pd.NA · Issue #23978 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BUG: Regression in 1.25.0 for boolean operation on pd.NA #23978

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mwaskom opened this issue Jun 18, 2023 · 3 comments
Closed

BUG: Regression in 1.25.0 for boolean operation on pd.NA #23978

mwaskom opened this issue Jun 18, 2023 · 3 comments
Labels
00 - Bug

Comments

@mwaskom
Copy link
mwaskom commented Jun 18, 2023

Describe the issue:

Not sure if this is fundamentally a pandas or numpy issue (or if I'm doing the wrong thing and missed a deprecation warning) but code that ran as expected with numpy 1.24.3 is now raising an error on 1.25.0

Reproduce the code example:

import numpy as np
import pandas as pd

s = pd.Series([1, 2, pd.NA])
np.in1d(s, [0, 1]).all()

Error message:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In [1], line 5
      2 import pandas as pd
      4 s = pd.Series([1, 2, pd.NA])
----> 5 np.in1d(s, [0, 1]).all()

File ~/miniconda/envs/py310/lib/python3.10/site-packages/numpy/lib/arraysetops.py:733, in in1d(ar1, ar2, assume_unique, invert, kind)
    731         mask = np.zeros(len(ar1), dtype=bool)
    732         for a in ar2:
--> 733             mask |= (ar1 == a)
    734     return mask
    736 # Otherwise use sorting

File ~/miniconda/envs/py310/lib/python3.10/site-packages/pandas/_libs/missing.pyx:388, in pandas._libs.missing.NAType.__bool__()

TypeError: boolean value of NA is ambiguous

Runtime information:

1.25.0
3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:38:29) [Clang 13.0.1 ]

[{'numpy_version': '1.25.0',
  'python': '3.10.6 | packaged by conda-forge | (main, Aug 22 2022, 20:38:29) '
            '[Clang 13.0.1 ]',
  'uname': uname_result(system='Darwin', node='arcturus.local', release='21.6.0', version='Darwin Kernel Version 21.6.0: Mon Aug 22 20:20:07 PDT 2022; root:xnu-8020.140.49~2/RELEASE_ARM64_T8110', machine='arm64')},
 {'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'],
                      'found': ['ASIMDHP', 'ASIMDDP'],
                      'not_found': ['ASIMDFHM']}},
 {'architecture': 'armv8',
  'filepath': '/Users/mwaskom/miniconda/envs/py310/lib/python3.10/site-packages/numpy/.dylibs/libopenblas64_.0.dylib',
  'internal_api': 'openblas',
  'num_threads': 8,
  'prefix': 'libopenblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.23'}]

Context for the issue:

Failing seaborn tests: https://github.com/mwaskom/seaborn/actions/runs/5275363773
@mwaskom
Copy link
Author
mwaskom commented Jun 18, 2023

Quite possible that I did miss a warning here as apparently this warnings from this operation are being silenced 🙈

Still not seeing anything obviously relevant in the release notes and not immediately certain what the right workaround would be.

@mwaskom
Copy link
Author
mwaskom commented Jun 18, 2023

Seems to be some relevant context here: pandas-dev/pandas#50124

@mwaskom
Copy link
Author
mwaskom commented Jun 18, 2023

Based on the linked issue this is expected / intended behavior for numpy and while pandas may improve the behavior of numpy export to avoid including pd.NA values, I gather this is something that needs to be worked around for now.

@mwaskom mwaskom closed this as not planned Won't fix, can't repro, duplicate, stale Jun 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@mwaskom
0