8000 Add new example for plotting a confidence_ellipse by CarstenSchelp · Pull Request #13570 · matplotlib/matplotlib · GitHub
[go: up one dir, main page]

Skip to content

Add new example for plotting a confidence_ellipse #13570

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Apr 2, 2019
Merged

Add new example for plotting a confidence_ellipse #13570

merged 21 commits into from
Apr 2, 2019

Conversation

CarstenSchelp
Copy link
Contributor
@CarstenSchelp CarstenSchelp commented Mar 2, 2019

PR Summary

New statistics-example with a robust and exact way to plot a confidence ellipse of a two-dimensional dataset.

PR Checklist

  • Has Pytest style unit tests
  • Code is Flake 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

anntzer
anntzer previously requested changes Mar 2, 2019
kwargs : `~matplotlib.patches.Patch` properties

author : Carsten Schelp
license: GNU General Public License v3.0 (https://github.com/CarstenSchelp/CarstenSchelp.github.io/blob/master/LICENSE)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a lawyer so pinging @tacaswell on this, but I think GPL won't be acceptable. Anyone can dismiss once resolved.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And most other examples don’t have an author block....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a lawyer so pinging @tacaswell on this, but I think GPL won't be acceptable. Anyone can dismiss once resolved.

When no other example bothers with licenses I won't, either. I just thought that the python license was GPLv3.0 compatible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh - matplotlib has a license of its own. Either way - I removed the license block. I hope that it is ok now if I resolve this conversation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anntzer: Hi, thanks to the community this is pull request is approaching a state that might be called 'mature'. Do you agree that your change request has been addressed appropriately? It was about that probably problematic license-tag in the docstring.

@anntzer
Copy link
Contributor
anntzer commented Mar 2, 2019

Thanks for the PR!

There are a few styling issues; we don't apply PEP8 extremely strictly on the examples but I'd suggest you run flake8 on the file and fix as many of the style points it raises.

@CarstenSchelp
Copy link
Contributor Author

Thank you for the reviews! I have applied the suggested changes.

@timhoffm
Copy link
Member
timhoffm commented Mar 3, 2019

This is technically ok, but it's a lot of code (see current doc build for the rendered result).

Putting all the code into a single code block might be a bit scary to the reader. Have you considered splitting the example into multiple sections? See e.g. https://matplotlib.org/devdocs/gallery/lines_bars_and_markers/joinstyle.html (source is in examples/lines_bars_and_markers/joinstyle.py).

@CarstenSchelp
Copy link
Contributor Author

@timhoffm Hi Tim, you are right - it even scared me! Thanks for pointing me to that example. The one I picked to get started did not have to use more than one code block. I will be back with a more friendly layout shortly.

@CarstenSchelp
Copy link
Contributor Author

I hope that I managed to make the example more user-friendly, now.

@jklymak
Copy link
Member
jklymak commented Mar 3, 2019

Ellipse takes x,y width height and angle, so why do you use transforms to supply those parameters?

@CarstenSchelp
Copy link
Contributor Author

Ellipse takes x,y width height and angle, so why do you use transforms to supply those parameters?

Hi @jklymak,
I appreciate your question, thank you!
I could move the rotation out of sight by specifying the angle when creating the ellipse like such after line 65:

    ellipse = Ellipse((0, 0),
        width=ell_radius_x * 2,
        height=ell_radius_y * 2,
        angle=45,
        **kwargs)

The effect is that the same transform would be applied a code-layer lower.

As for the translation transform - it cannot be worked away by specifying (mean_x, mean_y) instead of the origin (0, 0) as the center of the ellipse. The scale transform is relative to the origin and the ellipse shifts to the wrong place when it is scaled while it is already in its final place. Some implementations of scale() allow the user to specify a center of the scaling transform other than the origin. But this one does not. (Also this would mean nothing else than shifting the ellipse back and forth once more.)

Executing the transforms explicitly does not degrade efficiency as far as I can tell. Also I like to see all transforms in the same code-layer rather than having some executed here and some there.
The way I coded this also corresponds nicely with the "recipe" that I give in the article that explains this whole approach.
I hope this explanation is helpful. Do let me know when I am not entirely clear!

@CarstenSchelp
Copy link
Contributor Author

Aah! One wrong click! Sorry for this closing-reopening hubbub. Went by accident.

Copy link
Member
@timhoffm timhoffm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much nicer now. I've proposed some additional improvements 😄

@jklymak
Copy link
Member
jklymak commented Mar 3, 2019

The effect is that the same transform would be applied a code-layer lower.

Yes, but the naive user's tendency would be to not apply a transform. If you need one then great, but explain why. I'm not convinced you do, but I could certainly be wrong...

As for the translation transform - it cannot be worked away by specifying (mean_x, mean_y) instead of the origin (0, 0) as the center of the ellipse.

I don't understand why you need a scale transform at all.

The way I coded this also corresponds nicely with the "recipe" that I give in the article that explains this whole approach.

Do you link the article? Because I'm not following why you don't know the width and height a-priori. If you can't do this any other way than by specifying transforms it would be good to explain it clearly to the user, because otherwise they will wonder why you aren't just rotating at the start.

@CarstenSchelp
Copy link
Contributor Author

Do you link the article?

The link to the article is in the docstring of the method. But in fact, the approach does need some explanation and maybe the link should be in a more prominent place. Like the first heading that goes "The plotting function itself ...".
If that is not enough then I could place some comment in the code like "This is a normalized covariance, the slope of the ascending axis of its ellipse is always 1 (=45deg).", "Now scale the normalized ellipse back to size - that way, the angle takes care of itself." … and so forth.
I assume that users who wants to understand what's going on will read the article if the link is somehow findable. I'll let this rest for the night and come up with something some time tomorrow (guess what timezome I'm in :-)

Copy link
Member
@timhoffm timhoffm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already quite good. Some additional recommendations to make the code even more compact and clear.

< 67E6 /form>
@jklymak
Copy link
Member
jklymak commented Mar 4, 2019

First, I agree that an example like this is useful.

Second, I understand what you are doing, and I understand why you want to do it this way, but I feel it is a misuse of the transform stack. Instead I think you should consider getting the eigenvectors and eigenvalues and using those to calculate the angle, width, and height as outlined in the first answer to this stack overflow question:

https://stackoverflow.com/questions/12301071/multidimensional-confidence-intervals/12321306#12321306

@CarstenSchelp
Copy link
Contributor Author

@jklymak: Thank you for really diving into this, Jody.
To me transforms are a computationally efficient and intuitive way to get things done. They are not particularly obscure, low level or heavy. The transforms API is great, too - resulting in very readable code.
If I typed this out in numpy it would actually look cluttered and scary.
So applying transforms here does not look like misuse to me.
Also, half of the beauty of this approach to get the ellipse right is that by simply scaling the normalized ellipse, the angle takes care of itself. There is really no need to do "atan2(...)".
Using Ellipse(angle=...) alone requires me to provide an angle that will implicitly emerge from other operations. So, within this approach, Ellipse(angle=...) is not the right tool for the job.
I hope that I am making enough sense , here. Can you find rhyme and reason in my point of view?

timhoffm and others added 4 commits March 9, 2019 15:06
TODO: pass facecolor parameter to Ellipse() explicitly.

Co-Authored-By: CarstenSchelp <carstenschelp@mp.nl>
…s '.T'

Co-Authored-By: CarstenSchelp <carstenschelp@mp.nl>
Co-Authored-By: CarstenSchelp <carstenschelp@mp.nl>
@anntzer anntzer dismissed their stale review March 10, 2019 19:27

comments handled

@anntzer
Copy link
Contributor
anntzer commented Mar 10, 2019

I haven't looked too much into it, but after a cursory look I think I do like the approach of using transforms to achieve the desired result.

@ImportanceOfBeingErnest
Copy link
Member

If there is a controversity about the method being used, maybe one can add another function showing the eigenvector decomposition at the bottom of the example and show how both methods would yield the same result?

Copy link
Member
@timhoffm timhoffm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only minor style issues left.

I don't engage in the discussion on eigenvector vs. transform. Both have their pros and cons. Either is fine by me.

# Different number of standard deviations
# """""""""""""""""""""""""""""""""""""""
#
# A plot with n_std = 3 (gray), 2 (blue) and 1 (red)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could be some thing more descriptive like "A plot with multiple ellipses describing different standard deviations". The exact numbers and colors are explained in the legend and don't need to be described here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I didn't see this one. I will take a look shortly.

[-0.2, 0.35]
])
mu = np.array([0, 0]).T
scale = np.array([8, 5]).T
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Be consistent in the way you define mu and scale (see above).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is a controversity about the method being used, maybe one can add another function showing the eigenvector decomposition at the bottom of the example and show how both methods would yield the same result?

So far all comments and suggestions were aiming for an example as concise and readable as possible. This is a good thing and I would not add anything extra.
The core of this example is to show a particular way to get that ellipse and to explain why this works (via a link, that is). There are other ways to get that ellipse and the linked article links to one, in its turn, too. If somebody wants to show another way that should happen in another example, I think. Should be a quick job with some - perfectly legitimate - copy and paste.

@timhoffm
Copy link
Member
timhoffm commented Mar 12, 2019

For easier review: Current doc build

timhoffm and others added 5 commits March 14, 2019 09:09
Remove another facecolot='none' that apparently slipped through.

Co-Authored-By: CarstenSchelp <carstenschelp@mp.nl>
Remove superfluous string interpolation-'f' that slipped through.

Co-Authored-By: CarstenSchelp <carstenschelp@mp.nl>
Copy link
Member
@timhoffm timhoffm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO this is good to go. We can still bring it in 3.1.

Latest doc build

@CarstenSchelp thanks a lot!

@CarstenSchelp
Copy link
Contributor Author

Likewise, Tim, and everyone else! Nice too see that people find it worthwhile!

@anntzer
Copy link
Contributor
anntzer commented Apr 2, 2019

Small markup improvements remain possible, but this is already better than 99% of the examples, so I'm not going to nitpick :) merging.
Thanks for the great PR!

@anntzer anntzer added this to the v3.1.0 milestone Apr 2, 2019
@anntzer anntzer merged commit 4eb452a into matplotlib:master Apr 2, 2019
meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request Apr 2, 2019
tacaswell added a commit that referenced this pull request Apr 2, 2019
…570-on-v3.1.x

Backport PR #13570 on branch v3.1.x (Add new example for plotting a confidence_ellipse)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants
0