multiple inheritance of static C api types is not supported by CPython, yet we do it anyway #11998
Comments
|
CPython becomes aware of our hack if I try to add a metaclass to 'np.generic', and complains that 'np.generic' and 'float' have mismatching layouts |
|
I would be in favor of removing all inheritance from Python types as part of a dtype refactor/clean-up. Is there any way we could do a deprecation cycle for this? |
At the risk of straying a little off-topic, it would be really useful to define discrete engineering tasks related to the proposed dtype overhaul that the core devs think would be useful (even as experiments likely to fail). Perhaps this is one such engineering task. I know there are some broader documents & discussions floating around. |
|
@eric-wieser Could you provide example code that detects the issue? |
|
I have a patch somewhere that brings it up - will make a PR out of it for the sake of producing CI logs |
|
The Python C-API docs were changed. Multiple inheritance was allowed 12 years ago and was not discouraged like it is now. The only requirement was that the binary ABIs were compatible --- which they did. This is re-writing history to claim otherwise. We did not "cross-our fingers" . This was supported behavior that was changed in Python and NumPy did not update because I would be the only one who knew how and why this was implemented and have not been involved due to lack of time and funding. It's a maintenance issue, and a movement of Python issue. This should now be cleaned up, yes, as Python has changed its stance in Python 3. This was carefully considered and implemented to ensure NumPy floats behaved like Python floats as much as possible. |
This is an API change that should only really happen in a major release. It is a real semantic change that should be signaled by a major version bump. |
|
@teoliphant We are unlikely to have a major release anytime soon, maybe never. Apart from whether or not it was good python practice in the past, the double inheritance makes NumPy float64s oddballs in the NumPy float universe. It was a wonderful thing when NumPy ints no longer inherited from python ints when Python 8000 3 came along and I think it would be nice to extend that to float64. |
Which is exactly the problem and why it would be good to change it. |
|
Obviously, I think we should not fear the major version number bump. Even if the major version is to publish a "minimum API for NumPy" and not do major code re-writes. In other words, there is a minimum concept of a NumPy 2.0 that does not take unbounded effort. That should be the start. There are enough of these little problems to fix that are not hard to get a large majority of supporters for. |
|
As it stands, removing double inheritance is relatively minor compared to other changes that have happened in .x release (i.e. NumPy doesn't really adhere to semantic versioning). I cannot immediately think of problems it would cause. And if it did, those can be easily fixed. |
|
Sure, but why make the change if it means little? Linux hasn't had a "real" new version release since 2.6.0, and the only reason the version/major numbers change were that Linus thought the release numbers were getting too big. We aren't there yet... I suppose we could pick an arbitrary number, say 25, and release NumPy 2.0.0 instead of 1.26.0, just because, but the current thinking is to just evolve, which means steady change rather than jumps. I think that the Python 3 experience, as well as the several failed NumPy 2 attempts, have led to that approach. |
|
That said, I can see jumping the major release number for marketing reasons, so to speak. NumPy has come a long way since 1.0, new people are doing the development, the governance and organization are changing, and it might be nice to recognize all that by going to 2.0. |
That is possibly a good idea. We should then not tie it to some backwards-incompatible change though, we're pretty much all agreed indeed that gradual evolution and basing deprecations based on "pain for users" is the way to go. 2.0 could mark some random point in time (for marketing reasons), or for example the completion of a major feature like |
That was not my intent - thanks for delivering the correct history! |
Uh oh!
There was an error while loading. Please reload this page.
Our scalar type classes are such that:
issubclass(np.int_, int) and issubclass(np.int_, np.integer)(py2 only)issubclass(np.float_, float) and issubclass(np.float_, np.floating)issubclass(np.complex_, float) and issubclass(np.complex_, np.complexfloating)We do this in the C api by manually copying across some
tp_*slots, settingtp_basesto a tuple, and crossing our fingers. However, the CPython API docs say (emphasis mine):We do not do this, so it seems we're relying on internal behavior we're not supposed to.
The text was updated successfully, but these errors were encountered: