10000 BUG: MaskedArray.astype('uint8') with certain fill_value raises warning on ARM (Mac M3) inside Docker (Ubuntu 24.04/25.04) and leads inconsistent output · Issue #28403 · numpy/numpy · GitHub
[go: up one dir, main page]
Skip to content

BUG: MaskedArray.astype('uint8') with certain fill_value raises warning on ARM (Mac M3) inside Docker (Ubuntu 24.04/25.04) and leads inconsistent output #28403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
AlessioChecchin opened this issue Feb 28, 2025 · 1 comment
Open

BUG: MaskedArray.astype('uint8') with certain fill_value raises warning on ARM (Mac M3) inside Docker (Ubuntu 24.04/25.04) and leads inconsistent output #28403

AlessioChecchin opened this issue Feb 28, 2025 · 1 comment
Labels
00 - Bug

Comments

@AlessioChecchin
Copy link
AlessioChecchin commented Feb 28, 2025 •
edited
Loading

Describe the issue:

I encountered a warning when converting a MaskedArray to uint8 inside a Docker container running Ubuntu 24.04 or 25.04 on an Apple M3 Mac. The same code runs fine on:

  • macOS (outside Docker, on M3)
  • Windows WSL2 with Ubuntu 22.04 (both inside and outside Docker)

However, inside Docker (Ubuntu 24.04 or 25.04 on ARM), the following script produces a warning:

import numpy as np

foo = np.ma.masked_equal(np.arange(10, dtype="float32"), 5)
foo.fill_value = -1.0
bar = foo.astype("uint8")  # Generates a warning
print(bar.fill_value)
bar.fill_value = 255
print(bar.fill_value)
print(bar.filled().dtype)

Note that i found this script on another issue that was caused by another bug and seems solved.

Expected Behavior:
The script should execute without warnings, as it does on other platforms and the output shoud be consistent.
Here is the output that i got from all other platforms.

255
255
uint8

Actual Behavior:
A warning is raised when calling astype("uint8") when running in Docker on ARM.
I also noted that print(bar.fill_value) prints 0 only in Docker ARM

/opt/venv/lib/python3.13/site-packages/numpy/ma/core.py:502: RuntimeWarning: invalid value encountered in cast
  fill_value = np.asarray(fill_value, dtype=ndtype)
0
255
uint8
2.2.3

System information

  • Numpy version: 2.2.3
  • Docker: Ubuntu 24.04 or 25.04
  • Host system: macOS (Apple M3, ARM64)
  • np.show_config()
{
  "Compilers": {
    "c": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.2.1",
      "commands": "cc"
    },
    "cython": {
      "name": "cython",
      "linker": "cython",
      "version": "3.0.12",
      "commands": "cython"
    },
    "c++": {
      "name": "gcc",
      "linker": "ld.bfd",
      "version": "10.2.1",
      "commands": "c++"
    }
  },
  "Machine Information": {
    "host": {
      "cpu": "aarch64",
      "family": "aarch64",
      "endian": "little",
      "system": "linux"
    },
    "build": {
      "cpu": "aarch64",
      "family": "aarch64",
      "endian": "little",
      "system": "linux"
    }
  },
  "Build Dependencies": {
    "blas": {
      "name": "scipy-openblas",
      "found": true,
      "version": "0.3.28",
      "detection method": "pkgconfig",
      "include directory": "/opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas64/include",
      "lib directory": "/opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas64/lib",
      "openblas configuration": "OpenBLAS 0.3.28  USE64BITINT DYNAMIC_ARCH NO_AFFINITY neoversen1 MAX_THREADS=64",
      "pc file directory": "/project/.openblas"
    },
    "lapack": {
      "name": "scipy-openblas",
      "found": true,
      "version": "0.3.28",
      "detection method": "pkgconfig",
      "include directory": "/opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas64/include",
      "lib directory": "/opt/_internal/cpython-3.13.0/lib/python3.13/site-packages/scipy_openblas64/lib",
      "openblas configuration": "OpenBLAS 0.3.28  USE64BITINT DYNAMIC_ARCH NO_AFFINITY neoversen1 MAX_THREADS=64",
      "pc file directory": "/project/.openblas"
    }
  },
  "Python Information": {
    "path": "/tmp/build-env-tttxdu3h/bin/python",
    "version": "3.13"
  },
  "SIMD Extensions": {
    "baseline": [
      "NEON",
      "NEON_FP16",
      "NEON_VFPV4",
      "ASIMD"
    ],
    "found": [
      "ASIMDHP",
      "ASIMDFHM"
    ],
    "not found": [
      "SVE"
    ]
  }
}

Additional Notes:

  • The issue does not occur on Windows WSL2 (Intel i9 13900) or macOS native (M3).

I also took note of the .whl files:

  • WSL2 running Ubuntu 22.04: numpy-2.2.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • WSL2 running Ubuntu 22.04 + Docker running Ubuntu 25.05: numpy-2.2.3-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
  • macOS native (M3): numpy-2.2.3-cp312-cp312-macosx_14_0_arm64.whl
  • macOS + Docker running Ubuntu 25.05: numpy-2.2.3-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Reproduce the code example:

import numpy as np

foo = np.ma.masked_equal(np.arange(10, dtype="float32"), 5)
foo.fill_value = -1.0
bar = foo.astype("uint8")  # Generates a warning
print(bar.fill_value)
bar.fill_value = 255
print(bar.fill_value)
print(bar.filled().dtype)

Python and NumPy Versions:

3.13.2 (main, Feb 5 2025, 01:23:35) [GCC 14.2.0]

Runtime Environment:

[{
  'numpy_version': '2.2.3',
  'python': '3.13.2 (main, Feb  5 2025, 01:23:35) [GCC 14.2.0]',
  'uname': uname_result(system='Linux', node='4bcdb1f2c3ec', release='6.10.14-linuxkit', version='#1 SMP Fri Nov 29 17:22:03 UTC 2024', machine='aarch64')
},
{'simd_extensions': {'baseline': ['NEON', 'NEON_FP16', 'NEON_VFPV4', 'ASIMD'], 'found': ['ASIMDHP', 'ASIMDFHM'], 'not_found': ['SVE']}},
{
  'architecture': 'neoversen1',
  'filepath': '/opt/venv/lib/python3.13/site-packages/numpy.libs/libscipy_openblas64_-48fd33d4.so',
  'internal_api': 'openblas',
  'num_threads': 11,
  'prefix': 'libscipy_openblas',
  'threading_layer': 'pthreads',
  'user_api': 'blas',
  'version': '0.3.28'
}]
@TTsangSC
Copy link

Just wanna give this issue a boost as a fellow M3 user on Python 3.13, who has submitted a MaskedArray.fill_value issue in the last month, which has received zero attention so far. Maybe the lack of interest reflects a small actual user-base for numpy.ma, which explains why e.g. numpy.ma.core.mvoid hasn't even received the courtesy of an updated repr()1 in numpy 2.x, unlike other scalar types...

Back to the issue, I looked at line 502 and tried to experiment with np.asarray without using numpy.ma:

>>> with warnings.catch_warnings():                                                                                                                                               
...     warnings.simplefilter('error', RuntimeWarning)                                                                                                                            
...     for f in [2, 4, 8]:  # There's no float8                                                                                                                                  
...         for u in [1, 2, 4, 8]:                                                                                                                                                
...             try:                                                                                                                                                              
...                 np.asarray(np.dtype(f'f{f}').type(-1), dtype=f'u{u}')                                                                                                         
...             except RuntimeWarning:                                                                                                                                            
...                 print(f'Warned: f{f} -> u{u}')                                                                                                                                
...                                                                                                                                                                               
array(255, dtype=uint8)                                                                                                                                                           
array(65535, dtype=uint16)                                                                                                                                                        
Warned: f2 -> u4                                                                                                                                                                  
Warned: f2 -> u8                                                                                                                                                                  
array(255, dtype=uint8)                                                                                                                                                           
array(65535, dtype=uint16)                                                                                                                                                        
Warned: f4 -> u4                                                                                                                                                                  
Warned: f4 -> u8                                                                                                                                                                  
array(255, dtype=uint8)                                                                                                                                                           
array(65535, dtype=uint16)                                                                                                                                                        
Warned: f8 -> u4                                                                                                                                                                  
Warned: f8 -> u8

Which however seems to be a separate issue to yours, given that yours happened during downcasting to a smaller uint, and mine happened consistently when casting from arbitrary negative float to uint32 or 64.

Unfortunately spelunking though both the Python and the C codebase returned no result on "invalid value encountered in cast". However, in the 1.24.0 release notes they did mention that:

Note that for float to int casts, the exact warnings that are given may be platform dependent. For example:

arr = np.full(100, fill_value=1000, dtype=np.float64)
arr.astype(np.int8)

May give a result equivalent to (the intermediate cast means no warning is given):

arr.astype(np.int64).astype(np.int8)

May return an undefined result, with a warning set:

RuntimeWarning: invalid value encountered in cast

The precise behavior is subject to the C99 standard and its implementation in both software and hardware.

So I guess (1) we just ran into some platform-dependent UB here, (2) the warning wasn't even issued from numpy itself, and (3) given the release notes, the inconsistencies are expected and not intended to be fixed.

Footnotes

  1. Most scalars (incl. numpy.void) used to have identical repr()s to their corresponding builtin objects up to numpy 1.x. ↩

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
00 - Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@AlessioChecchin @TTsangSC
0