8000 brier_score_loss error · Issue #11245 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

brier_score_loss error #11245

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
pukhlyakova opened this issue Jun 12, 2018 · 3 comments · Fixed by #13628
Closed

brier_score_loss error #11245

pukhlyakova opened this issue Jun 12, 2018 · 3 comments · Fixed by #13628

Comments

@pukhlyakova
Copy link

Hello. I think I found a mistake in brier_score_loss.
When you have a target = 1 and a prediction = 1 too, brier_score_loss should be 0 (best result), but it gives 1. Why is it happening? Because _check_binary_probabilistic_predictions gets target with only one class and convert it to 0. And metric calculates for target = 0 and prediction = 1. The same problem for target = 1 and prediction = 0. brier_score_loss is 0 (the best result), but it should be 1.

Examples:
Approx = [0, 0, 0, 0]
Target = [1, 1, 1, 1]
Weight = [1, 1, 1, 1]
brier_score_loss(Target, Approx, sample_weight=Weight)
result is 0

Approx = [1, 1, 1, 1]
Target = [1, 1, 1, 1]
Weight = [1, 1, 1, 1]
brier_score_loss(Target, Approx, sample_weight=Weight)
result is 1

Maybe we should fix it? Thank you.
@jnothman
Copy link
Member
jnothman commented Jun 12, 2018 •
edited
Loading

Thanks. Duplicate issue of #9300, #8459

@jnothman
Copy link
Member

Sorry, just amended those links. #9301 is likely to be merged for next release

@pukhlyakova
Copy link
Author

Thank you

@qinhanmin2014 qinhanmin2014 mentioned this issue Apr 12, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[MRG] FIX Correct brier_score_loss when there's only one class in y_true
2 participants
@jnothman @pukhlyakova
0