10000 BUG: longdouble with elsize 12 is never uint alignable by mattip · Pull Request #12611 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

BUG: longdouble with elsize 12 is never uint alignable #12611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Dec 26, 2018

Conversation

mattip
Copy link
Member
@mattip mattip commented Dec 25, 2018

Fix #12607

It seems we hit an alignment assert on 32bit linux with longdouble dtype (which has elsize 12). Fix it in two stages (the first commit only extends CI to test asserts):

  1. Add -UNDEBUG to the CFLAGS so that the assert statements are checked (debug mode)
  2. Find and fix the underlying alignment issue

@mattip mattip requested a review from ahaldane December 25, 2018 18:57
@mattip
Copy link
Member Author
mattip commented Dec 25, 2018

Commit 1 succeeded - the 32 bit test run crashed.

Fix added for longdouble in the next commit and tests are passing, but I am not sure it belongs there since longdouble is not a uint.

@mattip
Copy link
Member Author
mattip commented Dec 26, 2018

@ahaldane The end result is that any dtype with a non-power-of-two elsize (like longdouble on platforms where it is 12) will always be non-aligned? Is that the desired outcome?

return 1;
}
else {
/* always return false for alignment == 0, which means unaligned */
return 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is alignment == 0 even a legal argument to this function?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe there is a better way to organize it... but the idea is that npy_uint_alignment, which normally returns the uint alignment, returns 0 if the type is not uint-alignable.

Then, it turned out to be convenient to allow the result of npy_uint_alignment to be passed directly to raw_array_is_aligned.

An alternative would be to check the return value of npy_uint_alignment explicitly everywhere it is used. But since it is pretty much always used as an arg to raw_array_is_aligned it seemed less complicated this way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changed the comment

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@@ -88,7 +88,8 @@ broadcast_strides(int ndim, npy_intp *shape,
/*
* Checks whether a data pointer + set of strides refers to a raw
* array whose elements are all aligned to a given alignment.
* alignment should be a power of two.
* alignment should be a power of two, or may be the sentinel value 0 to mean
* unaligned, in which case false is always returned.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment confuses me - I read this as saying that is_aligned(..., alignment = 0) means is_unaligned(...)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not misreading you, that is what the comment is meant to say. If you pass an alignment of 0, the function always returns "not aligned". Do you mean I should say "not aligned" instead of "unaligned"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. the return value means "not aligned", but the meaning of the alignment argument is closer to "unalignable"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixing

@ahaldane
Copy link
Member

The end result is that any dtype with a non-power-of-two elsize (like longdouble on platforms where it is 12) will always be non-aligned? Is that the desired outcome?

That's the idea, except to be precise it will be non-uint aligned.

The logic is that the aligned copy code in lowelevel_strided_loops.c.src can only handle datatypes whose elsize is a power of two, as it does something like (*((uint64 *)dst)) = *((uint64 *)src);. See https://github.com/numpy/numpy/blob/master/numpy/core/src/multiarray/lowlevel_strided_loops.c.src#L133

Actually, it now occurs to me this might be relaxed a bit: An elsize of 12 could be uint-aligned to 4 bytes, so we could use a uint32 to copy such a type, maybe.

@mattip
Copy link
Member Author
mattip commented Dec 26, 2018

this might be relaxed a bit: An elsize of 12 could be uint-aligned to 4 bytes, so we could use a uint32 to copy such a type

Let's leave this as a bugfix, and think about enhancements for another PR.

@mattip mattip changed the title BUG: fix alignment assert on 32 bit for longdouble BUG: longdouble with elsize 12 is never uint alignable Dec 26, 2018
@eric-wieser
Copy link
Member
8000

An elsize of 12 could be uint-aligned to 4 bytes, so we could use a uint32 to copy such a type, maybe.

That sounds like a good idea to me.

Copy link
Member
@eric-wieser eric-wieser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me now

@eric-wieser
Copy link
Member

Can we add a test case for this in any easy way? Does the V12 dtype show this problem too?

@mattip mattip added this to the 1.16.1 release milestone Dec 26, 2018
@mattip
Copy link
Member Author
mattip commented Dec 26, 2018

I don't see where we expose the result of raw_array_is_aligned. There are IsAligned and IsUIntAligned, but those also are not exposed to python. We could expose them to python via _mutliarray_tests. Is that required for this bugfix, where we know that without the fix the 32 bit chroot crashes, and with it it passes?

@eric-wieser
Copy link
Member

Nope, what you have is fine - I missed that the 32-bit build now catches the problem. Was just wondering if there's a platform-agnostic way this bug manifested itself.

@charris charris added the 09 - Backport-Candidate PRs tagged should be backported label Dec 26, 2018
@charris charris merged commit 65709eb into numpy:master Dec 26, 2018
@charris
Copy link
Member
charris commented Dec 26, 2018

Thanks Matti, Allan.

@charris
Copy link
Member
charris commented Dec 27, 2018

It is easier to backport these things if (internal) merges are done fast forward. Putting the test last, or at least squashed with the fix, is also helpful if a bisect needs to be done in the future.

@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Dec 27, 2018
@charris charris removed this from the 1.16.0 release milestone Dec 27, 2018
@ahaldane
Copy link
Member

I think the merge was done automatically by github on Matti's github repo which I made a PR to. I'm not sure what the best way is for us to collaborate on PRs.

@charris
Copy link
Member
charris commented Dec 27, 2018

During the work on __array_ufunc__ I would periodically rebase so that the merges were fast forward. In this case, I ended up checking out the pr and a simple rebase, took care of the merge. Not sure why it works, but I'm happy.

@mattip mattip deleted the align-longdouble branch May 20, 2019 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1.6.0rc1 3.7 debug: test failure: test_multiarray.py::TestAlignment::test_various_alignments Aborted
4 participants
0