8000 BUG: Fixed 'midpoint' interpolation of np.percentile in odd cases. by madphysicist · Pull Request #7129 · numpy/numpy · GitHub
[go: up one dir, main page]

Skip to content

BUG: Fixed 'midpoint' interpolation of np.percentile in odd cases. #7129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

madphysicist
Copy link
Contributor

interpolation='midpoint' must return the same as interpolation='higher' and interpolation='lower' when the two are the same, not 'lower' + 0.5 as it was doing.

Another way of putting it: the 50th percentile of an array with an odd number of elements is the middle of the (sorted) array, no matter what the interpolation argument is.

I updated the test to deal with odd instead of even number of elements in the input array.

If this fix gets backported, it should go back as far as 1.0.9, when the interpolation keyword was introduced.

@@ -2055,7 +2055,7 @@ def compare_results(res, desired):
assert_array_equal(res[i], desired[i])


class TestScoreatpercentile(TestCase):
class TestPercentile(TestCase):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There does not appear to be any good reason to name the test otherwise than with the name of the function being tested in this case. Perhaps the function name changed at some point and the test was updated?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes,I think this was ported from a scipy function with that name.

@charris
Copy link
Member
charris commented Jan 29, 2016

@seberg I think it can go in, but perhaps note in Compatibility just so folks will notice the change.

@seberg
Copy link
Member
seberg commented Jan 29, 2016

Agreed, @madphysicist can you add a short note in doc/release/1.12.0-notes.rst? Then we can put this in.

@njsmith
Copy link
Member
njsmith commented Jan 29, 2016

Given that this is an incorrect-results bug, I'm tempted to relax my usual
hard-line stance and vote for backporting to 1.11... though given it's
existed forever, I doubt an extra few months will make much difference one
way or the other.
On Jan 29, 2016 11:48 AM, "seberg" notifications@github.com wrote:

Agreed, @madphysicist https://github.com/madphysicist can you add a
short note in doc/release/1.12.0-notes.rst? Then we can put this in.


Reply to this email directly or view it on GitHub
#7129 (comment).

@charris
Copy link
Member
charris commented Jan 31, 2016

@madphysicist Could you squash the commits into one?

EDIT: into two, leave the notes bit separate.

@charris charris added this to the 1.11.0 release milestone Jan 31, 2016
'midpoint' must return the same as 'higher' and 'lower' when the two
are the same, not 'lower' + 0.5 as it was doing.
@madphysicist madphysicist force-pushed the percentile-midpoint-interpolation branch from db0e816 to 9ec694b Compare February 1, 2016 01:04
@madphysicist
Copy link
Contributor Author

I really don't think many people use other interpolators than the default one of 'linear', and if they do, it is most likely to do 'higher' or 'lower', not 'midpoint'. Otherwise I am sure someone would have noticed the incorrect behavior by now. I only noticed it because I was trying to figure out what exactly a percentile is based on some simple cases :)

@madphysicist
Copy link
Contributor Author

Also, I just checked out nanpercentile in numpy/lib/nan/nanfunctions.py. It will not be needing an update since it uses regular percentile to do the actual computation.

@charris
Copy link
Member
charris commented Feb 1, 2016

Ah, so you don't think this needs a backport?

@madphysicist
Copy link
Contributor Author

I don't think a backport would hurt, but I think the odds of it helping anything much are extremely slim. It's really not my decision.

@charris
Copy link
Member
charris commented Feb 1, 2016

Well, lets leave it for 1.12 then. It is easy to get carried away with backports.

@charris charris removed this from the 1.11.0 release milestone Feb 1, 2016
charris added a commit that referenced this pull request Feb 1, 2016
…olation

BUG: Fixed 'midpoint' interpolation of np.percentile in odd cases.
@charris charris merged commit ea97756 into numpy:master Feb 1, 2016
@madphysicist madphysicist deleted the percentile-midpoint-interpolation branch February 1, 2016 13:06
@juliantaylor
Copy link
Contributor

@charris this is a bugfix and 0.11 is not released yet, I don't see a reason to delay this, there is no evidence yet software is relying on the incorrect result. The pandas issue was just adapting the testsuite to the correct result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants
0