8000 MAINT,ENH: Reorganize buffered iteration setup by seberg · Pull Request #27883 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

MAINT,ENH: Reorganize buffered iteration setup #27883

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Dec 17, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8d69f05
MAINT: Refactor buffered loop to do an initial setup
seberg Nov 28, 2024
7d19f2d
TST: Fixup tests to pass (for now not correct)
seberg Nov 30, 2024
809cf0c
BUG: Avoid a coresize of 0
seberg Dec 2, 2024
9302b30
TST: The nullpointer doesn't print uniformly accross platforms
seberg Dec 2, 2024
54d967f
BUG: Add missing pointer dereference
seberg Dec 2, 2024
87ad5a2
BUG: Fix coreoffset mechanism and unset buf-reuse on different (parti…
seberg Dec 2, 2024
67cddf7
TST: Forgot one more nditer debug_print() fixup
seberg Dec 2, 2024
facd237
Apply suggestions from code review
seberg Dec 2, 2024
22f335c
MAINT: Remove bufnever logic from allocate-arrays (we do it later any…
seberg Dec 2, 2024
ac282fc
MAINT: Rename to BUF_SINGLESTRIDE and use the new gap in the flags
seberg D 8000 ec 2, 2024
6ed3b99
MAINT: Remove unnecessary fixed-stride logic
seberg Dec 3, 2024
162a951
MAINT: Fill in correct buffer transferstride always
seberg Dec 3, 2024
0cc37b9
MAINT: Don't use static zero (initialize by caller instead)
seberg Dec 3, 2024
786e546
DOC: Improve the function documentation somewhat.
seberg Dec 3, 2024
d09be24
BUG: Fix typo in new assert
seberg Dec 3, 2024
92ecbaf
MAINT: Reinstante the CONTIG flag (as buggy as it was)
seberg Dec 5, 2024
5bad357
remove debug print (was this part of the sanitizer problem?!)
seberg Dec 5, 2024
67cb5c5
TST: See if deleting a, b makes santizer tests pass
seberg Dec 6, 2024
5508cc5
MAINT: Move buffersize init (new code always limits it to itersize)
seberg Dec 10, 2024
be6749d
DOC: Explain buffer-setup cost more (maybe should rethink it...)
seberg Dec 12, 2024
426934d
DOC: Expand/change iteration explanation based on Marten's review
seberg Dec 15, 2024
1836fd1
MAINT: More review comments (mostly code clarifications/simplifications)
seberg Dec 15, 2024
e758983
MAINT: Smaller changes based on review
seberg Dec 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
MAINT: Fill in correct buffer transferstride always
Also clarifies the comment, since it may be a bit confusing:
The outer stride being zero means that there must be nonzero
inner strides (otherwise they are all zero and we don't use reduce
logic).
But the inner stride being zero is not meaningful.
  • Loading branch information
seberg committed Dec 3, 2024
commit 162a951e84fb150a13b561c49d25f136e7bdba03
5 changes: 3 additions & 2 deletions numpy/_core/src/multiarray/nditer_api.c
Original file line number Diff line number Diff line change
Expand Up @@ -1863,8 +1863,9 @@ npyiter_fill_buffercopy_params(

if ((opitflags & NPY_OP_ITFLAG_REDUCE) && (NAD_STRIDES(outer_axisdata)[iop] != 0)) {
/*
* Reduce with inner stride ==0 (outer !=0), we buffer the outer stride
* which also means buffering only outersize items.
* Reduce with all inner strides ==0 (outer !=0). We buffer the outer
* stride which also means buffering only outersize items.
* (If the outer stride is 0, some inner ones are guaranteed nonzero.)
*/
assert(NAD_STRIDES(axisdata)[iop] == 0);
*ndim_transfer = 1;
Expand Down
19 changes: 14 additions & 5 deletions numpy/_core/src/multiarray/nditer_constr.c
Original file line number Diff line number Diff line change
Expand Up @@ -3381,19 +3381,28 @@ npyiter_allocate_transfer_functions(NpyIter *iter)
npy_intp *strides = NAD_STRIDES(axisdata), op_stride;
NpyIter_TransferInfo *transferinfo = NBF_TRANSFERINFO(bufferdata);

npy_intp sizeof_axisdata = NIT_AXISDATA_SIZEOF(itflags, ndim, nop);
NpyIter_AxisData *reduce_axisdata = NIT_INDEX_AXISDATA(axisdata, bufferdata->outerdim);
npy_intp *reduce_strides = NAD_STRIDES(reduce_axisdata);

/* combined cast flags, the new cast flags for each cast: */
NPY_ARRAYMETHOD_FLAGS cflags = PyArrayMethod_MINIMAL_FLAGS;
NPY_ARRAYMETHOD_FLAGS nc_flags;

for (iop = 0; iop < nop; ++iop) {
npyiter_opitflags flags = op_itflags[iop];

/*
* Reduction operands may be buffered with a different stride,
* so we must pass NPY_MAX_INTP to the transfer function factory.
* Reduce operands buffer the outer stride if it is nonzero; compare
* `npyiter_fill_buffercopy_params`.
* (Inner strides cannot _all_ be zero if the outer is, but some can be.)
*/
// TODO: This should not be the case anymore!? (was it ever?!?)
op_stride = (flags & NPY_OP_ITFLAG_REDUCE) ? NPY_MAX_INTP :
strides[iop];
if ((op_itflags[iop] & NPY_OP_ITFLAG_REDUCE) && reduce_strides[iop] != 0) {
op_stride = reduce_strides[iop];
}
else {
op_stride = strides[iop];
}

/*
* If we have determined that a buffer may be needed,
Expand Down
0