API: Bump NPY_MAXDIMS to 64 and NPY_MAXARGS to 128 #24072
Conversation
|
Ah, I forgot the worst case initially. Added to the release notes also, but if a downstream library uses However, I could only see a few usages of |
|
If we are already touching this, I would be in favor of removing the use of a fixed size altogether. The only places we really need this constant are when allocating a temporary buffer for some calculations on the strides. Currently this is a static allocation on the stack, it could be done dynamically instead. In |
|
Maybe there should be some artificial upper limit so people do not do crazy things, say 1024. |
|
I don't disagree, but don't really want to dive into it right now. I would just remove the Something the old PR fails to mention, though: This breaks iteration ABI, the legacy iterate has to stay limited to 32 dims (which will affect some functions). |
|
Going to just close this for now. Maybe the solution is to just remove both |
|
Cross-linking to #5744; this topic was moved forward for NumPy 2.0. |
Both come up often enough, there was once a PR to bump
NPY_MAXARGSto 256, but right now I thought, maybe try this and someone else can bump it more ;). (of course bumping requires a major release, though).This increases stack usage and the size of some objects (mainly iterators) of course, otherwise I don't really expect much of an impact, there are a couple of places where this type of arrays are fully zeroed out, but even the worst case should be a very small (few percent) slowdown.
Effect:
NPY_MAXDIMSin a struct in their headers. This would be problematic if their downstream compiles with NumPy 1.x still.I think it is realistic that this does nothing except bumping the stack use a bit, although I guess it is not unlikely that some downstream libs may not support 64 dims right away.
(I wouldn't mind adjusting some of the places where these are used to use
alloca, e.g. in ufuncs we almost always have 2, but always have huge working stack area).The discussion then suggested 256, and this was the old PR: #4840
I am not sure, for MAXDIMs, removing the limit entirely is hard, for MAXARGS, it seems like much more to do than back in the day, but at least for the iterator probably not so bad. I.e. it would be nice to not have MAXARGS at all (or maybe have it, but only to switch between stack allocation and malloc.
Until then, does just bumping seem good, should we go with 256. I honestly have no opinion, I did this first remembering it was higher, then saw they suggested 256...