@@ -20,8 +20,10 @@ model:
20
20
This is discussed on section :ref: `scoring_parameter `.
21
21
22
22
* **Metric functions **: The :mod: `metrics ` module implements functions
23
- assessing prediction errors for specific purposes. This is discussed in
24
- the section :ref: `prediction_error_metrics `.
23
+ assessing prediction error for specific purposes. These metrics are detailed
24
+ in sections on :ref: `classification_metrics `,
25
+ :ref: `multilabel_ranking_metrics `, :ref: `regression_metrics ` and
26
+ :ref: `clustering_metrics `.
25
27
26
28
Finally, :ref: `dummy_estimators ` are useful to get a baseline
27
29
value of those metrics for random predictions.
@@ -42,7 +44,7 @@ Model selection and evaluation using tools, such as
42
44
controls what metric they apply to estimators evaluated.
43
45
44
46
Common cases: predefined values
45
- --------------------------------
47
+ -------------------------------
46
48
47
49
For the most common usecases, you can simply provide a string as the
48
50
``scoring `` parameter. Possible values are:
@@ -91,22 +93,31 @@ predicted values. These are detailed below, in the next sections.
91
93
92
94
.. _scoring :
93
95
94
- Defining your scoring strategy from score functions
96
+ Defining your scoring strategy from metric functions
95
97
-----------------------------------------------------
96
98
97
- The scoring parameter can be a callable that takes model predictions and
98
- ground truth.
99
+ The module :mod: ` sklearn.metric ` also exposes a set of simple functions
100
+ measuring a prediction error given ground truth and prediction:
99
101
100
- However, if you want to use a scoring function that takes additional parameters, such as
101
- :func: `fbeta_sco
8000
re `, you need to generate an appropriate scoring object. The
102
- simplest way to generate a callable object for scoring is by using
103
- :func: `make_scorer `.
104
- That function converts score functions (discussed below in :ref: `prediction_error_metrics `) into callables that can be
105
- used for model evaluation.
102
+ - functions ending with ``_score `` return a value to
103
+ maximize (the higher the better).
106
104
107
- One typical use case is to wrap an existing scoring function from the library
108
- with non default value for its parameters such as the ``beta `` parameter for the
109
- :func: `fbeta_score ` function::
105
+ - functions ending with ``_error `` or ``_loss `` return a
106
+ value to minimize (the lower the better).
107
+
108
+ Metrics available for various machine learning tasks are detailed in sections
109
+ below.
110
+
111
+ Many metrics are not given names to be used as ``scoring `` values,
112
+ sometimes because they require additional parameters, such as
113
+ :func: `fbeta_score `. In such cases, you need to generate an appropriate
114
+ scoring object. The simplest way to generate a callable object for scoring
115
+ is by using :func: `make_scorer `. That function converts metrics
116
+ into callables that can be used for model evaluation.
117
+
118
+ One typical use case is to wrap an existing metric function from the library
119
+ with non default value for its parameters, such as the ``beta `` parameter for
120
+ the :func: `fbeta_score ` function::
110
121
111
122
>>> from sklearn.metrics import fbeta_score, make_scorer
112
123
>>> ftwo_scorer = make_scorer(fbeta_score, beta=2)
@@ -138,6 +149,8 @@ from a simple python function::
138
149
* any additional parameters, such as ``beta `` in an :func: `f1_score `.
139
150
140
151
152
+ .. _diy_scoring :
153
+
141
154
Implementing your own scoring object
142
155
------------------------------------
143
156
You can generate even more flexible model scores by constructing your own
@@ -154,24 +167,10 @@ the following two rules:
154
167
``estimator ``'s predictions on ``X `` which reference to ``y ``.
155
168
Again, higher numbers are better.
156
169
157
- .. _prediction_error_metrics :
158
-
159
- Function for prediction-error metrics
160
- ======================================
161
-
162
- The module :mod: `sklearn.metric ` also exposes a set of simple functions
163
- measuring a prediction error given ground truth and prediction:
164
-
165
- - functions ending with ``_score `` return a value to
166
- maximize (the higher the better).
167
-
168
- - functions ending with ``_error `` or ``_loss `` return a
169
- value to minimize (the lower the better).
170
-
171
170
.. _classification_metrics :
172
171
173
172
Classification metrics
174
- -----------------------
173
+ =======================
175
174
176
175
.. currentmodule :: sklearn.metrics
177
176
@@ -228,7 +227,7 @@ And some work with binary and multilabel indicator format:
228
227
In the following sub-sections, we will describe each of those functions.
229
228
230
229
Accuracy score
231
- ..............
230
+ --------------
232
231
233
232
The :func: `accuracy_score ` function computes the
234
233
`accuracy <http://en.wikipedia.org/wiki/Accuracy_and_precision >`_, the fraction
@@ -271,7 +270,7 @@ In the multilabel case with binary label indicators: ::
271
270
the dataset.
272
271
273
272
Confusion matrix
274
- ................
273
+ ----------------
275
274
276
275
The :func: `confusion_matrix ` function computes the `confusion matrix
277
276
<http://en.wikipedia.org/wiki/Confusion_matrix> `_ to evaluate
@@ -313,7 +312,7 @@ from the :ref:`example_model_selection_plot_confusion_matrix.py` example):
313
312
314
313
315
314
Classification report
316
- ......................
315
+ ----------------------
317
316
318
317
The :func: `classification_report ` function builds a text report showing the
319
318
main classification metrics. Here a small example with custom ``target_names ``
@@ -348,7 +347,7 @@ and inferred labels::
348
347
grid search with a nested cross-validation.
349
348
350
349
Hamming loss
351
- .............
350
+ -------------
352
351
353
352
The :func: `hamming_loss ` computes the average Hamming loss or `Hamming
354
353
distance <http://en.wikipedia.org/wiki/Hamming_distance> `_ between two sets
@@ -395,7 +394,7 @@ In the multilabel case with binary label indicators: ::
395
394
396
395
397
396
Jaccard similarity coefficient score
398
- .....................................
397
+ -------------------------------------
399
398
400
399
The :func: `jaccard_similarity_score ` function computes the average (default)
401
400
or sum of `Jaccard similarity coefficients
@@ -432,7 +431,7 @@ In the multilabel case with binary label indicators: ::
432
431
.. _precision_recall_f_measure_metrics :
433
432
434
433
Precision, recall and F-measures
435
- .................................
434
+ ---------------------------------
436
435
437
436
The `precision <http://en.wikipedia.org/wiki/Precision_and_recall#Precision >`_
438
437
is intuitively the ability of the classifier not to label as
@@ -639,7 +638,7 @@ Then the metrics are defined as:
639
638
640
639
641
640
Hinge loss
642
- ...........
641
+ -----------
643
642
644
643
The :func: `hinge_loss ` function computes the average
645
644
`hinge loss function <http://en.wikipedia.org/wiki/Hinge_loss >`_. The hinge
@@ -673,7 +672,8 @@ with a svm classifier::
673
672
674
673
675
674
Log loss
676
- ........
675
+ --------
676
+
677
677
The log loss, also called logistic regression loss or cross-entropy loss,
678
678
is a loss function defined on probability estimates.
679
679
It is commonly used in (multinomial) logistic regression and neural networks,
@@ -725,7 +725,7 @@ The log loss is non-negative.
725
725
726
726
727
727
Matthews correlation coefficient
728
- .................................
728
+ ---------------------------------
729
729
730
730
The :func: `matthews_corrcoef ` function computes the Matthew's correlation
731
731
coefficient (MCC) for binary classes (quoting the `Wikipedia article on the
@@ -761,7 +761,7 @@ function:
761
761
.. _roc_metrics :
762
762
763
763
Receiver operating characteristic (ROC)
764
- .......................................
764
+ ---------------------------------------
765
765
766
766
The function :func: `roc_curve ` computes the `receiver operating characteristic
767
767
curve, or ROC curve (quoting
@@ -857,7 +857,7 @@ if predicted outputs have been binarized.
857
857
.. _zero_one_loss :
858
858
859
859
Zero one loss
860
- ..............
860
+ --------------
861
861
862
862
The :func: `zero_one_loss ` function computes the sum or the average of the 0-1
863
863
classification loss (:math: `L_{0 -1 }`) over :math: `n_{\text {samples}}`. By
@@ -903,7 +903,7 @@ In the multilabel case with binary label indicators: ::
903
903
.. _multilabel_ranking_metrics :
904
904
905
905
Multilabel ranking metrics
906
- --------------------------
906
+ ==========================
907
907
908
908
.. currentmodule :: sklearn.metrics
909
909
@@ -912,7 +912,8 @@ associated with it. The goal is to give high scores and better rank to
912
912
the ground truth labels.
913
913
914
914
Label ranking average precision
915
- ...............................
915
+ -------------------------------
916
<
EED3
code class="diff-text syntax-highlighted-line addition">+
916
917
The :func: `label_ranking_average_precision_score ` function
917
918
implements the label ranking average precision (LRAP). This metric is linked to
918
919
the :func: `average_precision_score ` function, but is based on the notion of
@@ -955,7 +956,7 @@ Here a small example of usage of this function::
955
956
.. _regression_metrics :
956
957
957
958
Regression metrics
958
- -------------------
959
+ ===================
959
960
960
961
.. currentmodule :: sklearn.metrics
961
962
@@ -966,7 +967,7 @@ to handle the multioutput case: :func:`mean_absolute_error`,
966
967
967
968
968
969
Explained variance score
969
- .........................
970
+ -------------------------
970
971
971
972
The :func: `explained_variance_score ` computes the `explained variance
972
973
regression score <http://en.wikipedia.org/wiki/Explained_variation> `_.
@@ -991,7 +992,7 @@ function::
991
992
0.957...
992
993
993
994
Mean absolute error
994
- ...................
995
+ -------------------
995
996
996
997
The :func: `mean_absolute_error ` function computes the `mean absolute
997
998
error <http://en.wikipedia.org/wiki/Mean_absolute_error> `_, which is a risk
@@ -1021,7 +1022,7 @@ Here a small example of usage of the :func:`mean_absolute_error` function::
1021
1022
1022
1023
1023
1024
Mean squared error
1024
- ...................
1025
+ -------------------
1025
1026
1026
1027
The :func: `mean_squared_error ` function computes the `mean square
1027
1028
error <http://en.wikipedia.org/wiki/Mean_squared_error> `_, which is a risk
@@ -1056,7 +1057,7 @@ function::
1056
1057
evaluate gradient boosting regression.
1057
1058
1058
1059
R² score, the coefficient of determination
1059
- ...........................................
1060
+ -------------------------------------------
1060
1061
1061
1062
The :func: `r2_score ` function computes R², the `coefficient of
1062
1063
determination <http://en.wikipedia.org/wiki/Coefficient_of_determination> `_.
@@ -1092,31 +1093,17 @@ Here a small example of usage of the :func:`r2_score` function::
1092
1093
for an example of R² score usage to
1093
1094
evaluate Lasso and Elastic Net on sparse signals.
1094
1095
1096
+ .. _clustering_metrics :
1097
+
1095
1098
Clustering metrics
1096
1099
======================
1097
1100
1098
- The :mod: `sklearn.metrics ` implements several losses, scores and utility
1099
- function for more information see the :ref: `clustering_evaluation `
1100
- section.
1101
-
1102
-
1103
- Biclustering metrics
1104
- ====================
1105
-
1106
- The :mod: `sklearn.metrics ` module implements bicluster scoring
1107
- metrics. For more information see the :ref: `biclustering_evaluation `
1108
- section.
1109
-
1110
-
1111
1101
.. currentmodule :: sklearn.metrics
1112
1102
1113
- .. _clustering_metrics :
1114
-
1115
- Clustering metrics
1116
- -------------------
1117
-
1118
1103
The :mod: `sklearn.metrics ` implements several losses, scores and utility
1119
- functions. For more information see the :ref: `clustering_evaluation ` section.
1104
+ functions. For more information see the :ref: `clustering_evaluation `
1105
+ section for instance clustering, and :ref: `biclustering_evaluation ` for
1106
+ biclustering.
1120
1107
1121
1108
1122
1109
.. _dummy_estimators :
0 commit comments