FIX f1_score with zero_division=1 on binary classes #27165
Conversation
sklearn/metrics/_classification.py
Outdated
| pred_sum + true_sum, 0 | ||
| ) | ||
| denom[denom_mask] = 1 # avoid division by 0 | ||
| f_score = numer / denom |
There was a problem hiding this comment.
Considering the issue reported in #27189, I think that we can express it without relying on the precision and recall:
# The score is defined as:
# score = (1 + beta**2) * precision * recall / (beta**2 * precision + recall)
# Therefore, you can express the score in terms of confusion matrix entries as:
# score = (1 + beta**2) * tp / ((1 + beta**2) * tp + beta**2 * fn + fp)
denom = beta2 * true_sum + pred_sum
f_score = _prf_divide(
(1 + beta2) * tp_sum,
denom,
"fscore",
"true nor predicted",
average,
warn_for,
zero_division,
)In this case, we handle it as any other by zero division.
|
The warning raise in |
|
By using my proposed formulation, we have two tests that start to fail because instead of having the value provided by |
|
@glemaitre Should we remove the warning that we are specifically raising for f1 # warn for f-score only if zero_division is warn, it is in warn_for
# and BOTH prec and rec are ill-defined
if zero_division == "warn" and ("f-score",) == warn_for:
if (pred_sum[true_sum == 0] == 0).any():
_warn_prf(average, "true nor predicted", "F-score is", len(true_sum)) |
|
Indeed since now the warning will be handled in the new |
Reference Issues/PRs
Fixes #26965
What does this implement/fix? Explain your changes.
Any other comments?
CC: @glemaitre Could you kindly have a look to see if this makes sense? I am not totally sure this is the correct fix so marking the PR as draft.