8000 Examples shouldn't use weighted macro-averaged P/R/F · Issue #6847 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Examples shouldn't use weighted macro-averaged P/R/F #6847

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
jnothman opened this issue Jun 1, 2016 · 6 comments
Closed

Examples shouldn't use weighted macro-averaged P/R/F #6847

jnothman opened this issue Jun 1, 2016 · 6 comments
Labels
Easy Well-defined and straightforward way to resolve

Comments

@jnothman
Copy link
Member
jnothman commented Jun 1, 2016

We've moved away from the default precision/recall/f-score being a prevalence-weighted macro-average (average="weighted") to requiring the user to specify an option, on the basis that the weighted average is somewhat non-standard / not in textbooks. The example at examples/model_selection/grid_search_digits.py still uses the weighted average, as do some docstrings. We should identify if a more standard averaging option is appropriate to illustrate the point in each case.
@jnothman jnothman added Easy Well-defined and straightforward way to resolve Need Contributor labels Jun 1, 2016
@aakashpatel25
Copy link

I can take it if it is not already taken. Thanks.

@jnothman
Copy link
Member Author
jnothman commented Jun 4, 2016

You're welcome

On 4 June 2016 at 03:30, aakashpatel25 notifications@github.com wrote:

I can take it if it is not already taken. Thanks.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#6847 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEz64PXnGmwrZLt49vNIEi6R4OIMZUpks5qIGSZgaJpZM4IrFYu
.

@HashCode55
Copy link
Contributor
HashCode55 commented Jun 13, 2016 •
edited
Loading

@jnothman I'd like to take this issue if its already not taken... but just want a little bit of explanation what needs to be done.

@jnothman
Copy link
Member Author

@aakashpatel25 will have to speak for whether it's taken. Find places where weighted average is used, and choose a more appropriate metric variant. Update the test output to match.

@HashCode55
Copy link
Contributor
HashCode55 commented Jun 14, 2016 •
edited
Loading

@jnothman So, if I understand correctly after reading through the codebase, grid_search_digits.py is using prediction_weighted for example where it could have used precision_macro (as I guess the data is balanced in that case) and what needs to be done is to alter all the places like this?

@jnothman
Copy link
Member Author

That sort of thing.

On 15 June 2016 at 04:06, Mehul Ahuja notifications@github.com wrote:

@jnothman https://github.com/jnothman So, if I understand correctly
after reading through the codebase, grid_search_digits.py is using
prediction_weighted for example where it could have used
'precision_macro' (as I guess the data is balanced in that case) and what
needs to be done is to alter all the places like this?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#6847 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAEz60-P7GVXT0Hfv-Bd6IYZR2CTPD47ks5qLu21gaJpZM4IrFYu
.

@HashCode55 HashCode55 mentioned this issue Jun 18, 2016
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Easy Well-defined and straightforward way to resolve
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jnothman @aakashpatel25 @HashCode55
0