8000 DOC move d2_log_loss_score in the classification metrics section (#28… · jeromedockes/scikit-learn@ff3ad59 · GitHub
[go: up one dir, main page]
Skip to content

Commit ff3ad59

Browse files
OmarManzoorOmar Salman
OmarManzoor
and
Omar Salman
authored
DOC move d2_log_loss_score in the classification metrics section (scikit-learn#28938)
Co-authored-by: Omar Salman <omar.salman@arbisoft>
1 parent 3915d1d commit ff3ad59

File tree

3 files changed

+69
-45
lines changed

3 files changed

+69
-45
lines changed

‎doc/modules/classes.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -982,6 +982,7 @@ details.
982982
metrics.classification_report
983983
metrics.cohen_kappa_score
984984
metrics.confusion_matrix
985+
metrics.d2_log_loss_score
985986
metrics.dcg_score
986987
metrics.det_curve
987988
metrics.f1_score

‎doc/modules/model_evaluation.rst

Lines changed: 67 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,7 @@ Scoring Function
7777
'roc_auc_ovo' :func:`metrics.roc_auc_score`
7878
'roc_auc_ovr_weighted' :func:`metrics.roc_auc_score`
7979
'roc_auc_ovo_weighted' :func:`metrics.roc_auc_score`
80+
'd2_log_loss_score' :func:`metrics.d2_log_loss_score`
8081

8182
**Clustering**
8283
'adjusted_mutual_info_score' :func:`metrics.adjusted_mutual_info_score`
@@ -377,6 +378,7 @@ Some also work in the multilabel case:
377378
recall_score
378379
roc_auc_score
379380
zero_one_loss
381+
d2_log_loss_score
380382

381383
And some work with binary and multilabel (but not multiclass) problems:
382384

@@ -1986,6 +1988,71 @@ see the example below.
19861988

19871989
|details-end|
19881990

1991+
.. _d2_score_classification:
1992+
1993+
D² score for classification
1994+
---------------------------
1995+
1996+
The D² score computes the fraction of deviance explained.
1997+
It is a generalization of R², where the squared error is generalized and replaced
1998+
by a classification deviance of choice :math:`\text{dev}(y, \hat{y})`
1999+
(e.g., Log loss). D² is a form of a *skill score*.
2000+
It is calculated as
2001+
2002+
.. math::
2003+
2004+
D^2(y, \hat{y}) = 1 - \frac{\text{dev}(y, \hat{y})}{\text{dev}(y, y_{\text{null}})} \,.
2005+
2006+
Where :math:`y_{\text{null}}` is the optimal prediction of an intercept-only model
2007+
(e.g., the per-class proportion of `y_true` in the case of the Log loss).
2008+
2009+
Like R², the best possible score is 1.0 and it can be negative (because the
2010+
model can be arbitrarily worse). A constant model that always predicts
2011+
:math:`y_{\text{null}}`, disregarding the input features, would get a D² score
2012+
of 0.0.
2013+
2014+
|details-start|
2015+
**D2 log loss score**
2016+
|details-split|
2017+
2018+
The :func:`d2_log_loss_score` function implements the special case
2019+
of D² with the log loss, see :ref:`log_loss`, i.e.:
2020+
2021+
.. math::
2022+
2023+
\text{dev}(y, \hat{y}) = \text{log_loss}(y, \hat{y}).
2024+
2025+
Here are some usage examples of the :func:`d2_log_loss_score` function::
2026+
2027+
>>> from sklearn.metrics import d2_log_loss_score
2028+
>>> y_true = [1, 1, 2, 3]
2029+
>>> y_pred = [
2030+
... [0.5, 0.25, 0.25],
2031+
... [0.5, 0.25, 0.25],
2032+
... [0.5, 0.25, 0.25],
2033+
... [0.5, 0.25, 0.25],
2034+
... ]
2035+
>>> d2_log_loss_score(y_true, y_pred)
2036+
0.0
2037+
>>> y_true = [1, 2, 3]
2038+
>>> y_pred = [
2039+
... [0.98, 0.01, 0.01],
2040+
... [0.01, 0.98, 0.01],
2041+
... [0.01, 0.01, 0.98],
2042+
... ]
2043+
>>> d2_log_loss_score(y_true, y_pred)
2044+
0.981...
2045+
>>> y_true = [1, 2, 3]
2046+
>>> y_pred = [
2047+
... [0.1, 0.6, 0.3],
2048+
... [0.1, 0.6, 0.3],
2049+
... [0.4, 0.5, 0.1],
2050+
... ]
2051+
>>> d2_log_loss_score(y_true, y_pred)
2052+
-0.552...
2053+
2054+
|details-end|
2055+
19892056
.. _multilabel_ranking_metrics:
19902057

19912058
Multilabel ranking metrics
@@ -2826,51 +2893,6 @@ Here are some usage examples of the :func:`d2_absolute_error_score` function::
28262893

28272894
|details-end|
28282895

2829-
|details-start|
2830-
**D² log loss score**
2831-
|details-split|
2832-
2833-
The :func:`d2_log_loss_score` function implements the special case
2834-
of D² with the log loss, see :ref:`log_loss`, i.e.:
2835-
2836-
.. math::
2837-
2838-
\text{dev}(y, \hat{y}) = \text{log_loss}(y, \hat{y}).
2839-
2840-
The :math:`y_{\text{null}}` for the :func:`log_loss` is the per-class
2841-
proportion.
2842-
2843-
Here are some usage examples of the :func:`d2_log_loss_score` function::
2844-
2845-
>>> from sklearn.metrics import d2_log_loss_score
2846-
>>> y_true = [1, 1, 2, 3]
2847-
>>> y_pred = [
2848-
... [0.5, 0.25, 0.25],
2849-
... [0.5, 0.25, 0.25],
2850-
... [0.5, 0.25, 0.25],
2851-
... [0.5, 0.25, 0.25],
2852-
... ]
2853-
>>> d2_log_loss_score(y_true, y_pred)
2854-
0.0
2855-
>>> y_true = [1, 2, 3]
2856-
>>> y_pred = [
2857-
... [0.98, 0.01, 0.01],
2858-
... [0.01, 0.98, 0.01],
2859-
... [0.01, 0.01, 0.98],
2860-
... ]
2861-
>>> d2_log_loss_score(y_true, y_pred)
2862-
0.981...
2863-
>>> y_true = [1, 2, 3]
2864-
>>> y_pred = [
2865-
... [0.1, 0.6, 0.3],
2866-
... [0.1, 0.6, 0.3],
2867-
... [0.4, 0.5, 0.1],
2868-
... ]
2869-
>>> d2_log_loss_score(y_true, y_pred)
2870-
-0.552...
2871-
2872-
|details-end|
2873-
28742896
.. _visualization_regression_evaluation:
28752897

28762898
Visual evaluation of regression models

‎sklearn/tests/test_public_functions.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,7 @@ def _check_function_param_validation(
234234
"sklearn.metrics.consensus_score",
235235
"sklearn.metrics.coverage_error",
236236
"sklearn.metrics.d2_absolute_error_score",
237+
"sklearn.metrics.d2_log_loss_score",
237238
"sklearn.metrics.d2_pinball_score",
238239
"sklearn.metrics.d2_tweedie_score",
239240
"sklearn.metrics.davies_bouldin_score",

0 commit comments

Comments
 (0)
0