WIP: handling of nan in unique, fixes #2111 #5487
Conversation
Works for floats, but not for complex or structured dtypes
|
☔ The latest upstream changes (presumably #7742) made this pull request unmergeable. Please resolve the merge conflicts. |
| # gh-2111: NaNs never compare equal, so they need special handling, | ||
| # but they always sort to the end | ||
| if issubclass(ar.dtype.type, np.inexact) and np.isnan(aux[-1]): | ||
| nanidx = np.searchsorted(aux, np.nan) |
There was a problem hiding this comment.
what if you replace this with nanidx = np.nonzero(np.isnan(aux))[0] ?
Edit: ... and the next line with flag[nanindex[:-1] + 1] = False
There was a problem hiding this comment.
Well, that creates an intermediate array of the same size as aux, so it's O(n) in both space and time. The solution above is O(log n) in time and O(1) in space, so it will almost certainly have better performance. It could be argued that your proposed code may be more understandable to a beginner, but I don't think np.searchsorted should be considered advanced or obscure, and should be perfectly fine to use, especially in numpy's source code.
So I guess I like my code better... ;-)
There was a problem hiding this comment.
@mattip's solution has the benefit of working for all types. So maybe we want both?
|
@jaimefrio this is marked WIP and has been open for a while. Do you plan on returning to this ? |
|
I'm going to close this, it is stalled and needs rebase. @jaimefrio Please make a new PR if you want to pursue this. |
Works for floats, but not for complex or structured dtypes