8000 Fix percentile and quantile by mhvk · Pull Request #46 · seberg/numpy · GitHub
[go: up one dir, main page]

Skip to content

Fix percentile and quantile #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 28, 2023
Merged

Fix percentile and quantile #46

merged 1 commit into from
Sep 28, 2023

Conversation

mhvk
Copy link
@mhvk mhvk commented Jul 12, 2023

Discussion in main thread

q = np.true_divide(q, 100)
# Use dtype of array if possible (e.g., if q is a python int or float)
# by making the divisor have the dtype of the data array.
q = np.true_divide(q, a.dtype.type(100) if a.dtype.kind == "f" else 100)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, feels a bit brittle, but I have to sleep on it probably (also doesn't generalize easily to user DTypes, but OK).

Maybe in this case it is actually easier to just track was_pyscalar = type(q) in (int, float, complex) and then apply if was_pyscalar: pos = float(pos) at the end of the calculation.

Beyond figuring this out, one poitn I am annoying about it is, that I think this is an important problem to get eyes on, because percentile can't be the only function in NumPy (and even more so downstream!) to run into this.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem with was_pyscalar is that I think the expectation should be that the result keeps the array input dtype, but only if it is float.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, by pos there, I don't mean the end-result, but the interpolation point value that is used for the last bit of the calculation. I.e. the intermediate result just before mixing it with the values from a.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I see. Indeed, that could work too. I think I slightly prefer to trying to deal with things up front - I'm trying to think of these functions as gufuncs and following the same logic...

q = np.asanyarray(q)
# Use dtype of array if possible (e.g., if q is a python int or float).
if isinstance(q, (int, float)) and a.dtype.kind == "f":
q = np.asanyarray(q, dtype=np.result_type(a, q))
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also just use this stanza for percentile to keep things more uniform.

Note that this would not work for user types either given the check for a.dtype.kind == "f", but I did have to include that since otherwise an example with a datetime broke.

@seberg
Copy link
Owner
seberg commented Sep 28, 2023

Going to merge and roll with this for now. I am not sure I like it, but I suspect it is good enough either way.
There is probably a point that q should be just be ignored completely for the purpose of the result type, in which case we don't have to cast it up-front, because can just force-cast it later on (which might lose special casting behaviors, but I think it is fine here because those are only really relevant for integers).

@seberg seberg merged commit da8f360 into seberg:nep50-finalize Sep 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0