8000 ENH/BUG: Allow multinomial to check pvals with other float types by bashtage · Pull Request #16732 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

ENH/BUG: Allow multinomial to check pvals with other float types #16732

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

bashtage
Copy link
Contributor
@bashtage bashtage commented Jul 2, 2020

Add additional check when original input is an array that does not have dtype double

closes #8317

@bashtage bashtage force-pushed the multinomial-pvals-other-dtype branch from b6243eb to 66ac658 Compare July 2, 2020 15:35
@bashtage
Copy link
Contributor Author
bashtage commented Jul 2, 2020

The condition is kind of a mouthful, so may not be the best approach. The alternative would be to use a fused type for kahan_sum which would then be checked also using a fused type for the comparison value.

@bashtage bashtage force-pushed the multinomial-pvals-other-dtype branch from 66ac658 to a8f6a0a Compare July 2, 2020 16:12
@bashtage
Copy link
Contributor Author
bashtage commented Jul 2, 2020

The two commits use different approaches.

  1. The first uses a complex test that is only run on np.floating that aren't doubles.
  2. The second adds a new kahan_check function that takes a cython.floating fused type to handle the float separate from the double.

Are either of these reasonable ways to handle it? The third option, which I haven't implemented would be

  1. Leave the test alone, but check if pvals is a np.floating array, and if so, add an additional message to the error stating that np.sum(pvals.astype(double)[:-1]) must be <= 1. This doesn't fix it but would prevent anyone from running into it again.

@seberg
Copy link
Member
seberg commented Jul 2, 2020

My first impression is that the last version looks slightly nicer than the previous. Does the 1+1e-12 threshold really make sense for float32 though?
Maybe your last version is actually the most practical one, although it is somewhat annoying that there is probably no easy recipe of how to fix a float32 or similar input which almost works...

@bashtage
Copy link
Contributor Author
bashtage commented Jul 2, 2020

Does the 1+1e-12 threshold really make sense for float32 though?

No. It is just 1.0. Could use the value of np.nextafter(np.float32(1.0), np.float32(2.0)).

Maybe your last version is actually the most practical one, although it is somewhat annoying that there is probably no easy recipe of how to fix a float32 or similar input which almost works...

Is last here 3 or 2?

@seberg
Copy link
Member
seberg commented Jul 2, 2020

Oh, yeah, I was thinking about option 3, which I wouldn't mind if there was some simple solution to give to the user running into the problem...

@bashtage bashtage force-pushed the multinomial-pvals-other-dtype branch from 2244453 to bce9a67 Compare July 2, 2020 17:43
@bashtage
Copy link
Contributor Author
bashtage commented Jul 2, 2020

I've not included the 3rd version in the 3rd commit. It is easily the simplest.

Add additional check when original input is an array that does not have dtype double

closes numpy#8317
Use a fused type and a kahan sum to check condition
Improve error message when the sum of pvals is larger than 1
when the input data is an ndarray
@bashtage bashtage force-pushed the multinomial-pvals-other-dtype branch from 95b0a34 to 359d04f Compare February 18, 2021 16:39
@bashtage bashtage closed this Feb 24, 2021
bashtage added a commit to bashtage/numpy that referenced this pull request Feb 26, 2021
Improve error message when the sum of pvals is larger than 1
when the input data is an ndarray

closes numpy#8317
xref numpy#16732
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

multinomial casts input to np.float64
3 participants
0