-
-
Notifications
You must be signed in to change notification settings - Fork 10.9k
Fix #5339: cache dtype.__hash__. #5343
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this unfortunately breaks the ABI and causes forward compatibility issues with cython.
though I am playing with breaking ABI by hiding the ufunc structure (at least partially) maybe we could break this too.
But if we do we should do it properly and add a pointer to a private object or something similar.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't adding a field at the end be fine, ABI-wise? We don't allow
subclassing. What cython issue are you thinking of?
I doubt there's any legitimate reason for anyone outside numpy to access
anything besides a few fields at the beginning anyway though (in
particularly itemsize), and I've also been pondering plans to rewrite the
dtype struct. So I wouldn't rule that out either. Rather than using a
pimpl, though, it might be sufficient to just hide the private fields
behind a private struct definition that only we can cast to.
On 4 Dec 2014 06:47, "Julian Taylor" notifications@github.com wrote:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That sounds reasonable. Do you have a patch for that somewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
extending structs breaks programs embedding this struct into other structures (unlikely) and cython checks the size of these types and complains if they are different than what the program was built with, thats just an annoyance but one that comes up very often.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't finished my ufunc patch yet, though it goes along the lines that njsmith suggested, keep the current structure and internally use a larger one. Additionally it might be interesting to mangle the public part in the development version to trigger failures for people that use the internals. For release the mangling can be reverted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm less knowledgeable than you guys :) What should be the way forward?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
one way would be to implement the hiding of the object internals, another wait until someone else does it, but that might take a while.
Unfortunately the dtype internals are probably far more commonly used than the ufunc internals I want to hide. This probably gives us much less wiggle room for breaking the ABI.
Unless someone finds a way to do so without breaking it, maybe some kind of magic number in one of the existing fields could work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe I am also just overly cautious, in light of no better idea if you want to work on it I think the approach of adding a second private struct that embeds the public struct is worthwhile. Its only problem is that we technically can't trust the hash to be correct if others do unusual stuff with the public part.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would the general scheme work like PyArray_Fields currently does? Which dtype fields should be private vs. public exactly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ping on my questions above :)