8000 ENH Add CalibrationDisplay plotting class (#17443) · scikit-learn/scikit-learn@da36f72 · GitHub
[go: up one dir, main page]

Skip to content

Commit da36f72

Browse files
Lucy Liuogrisel
andauthored
ENH Add CalibrationDisplay plotting class (#17443)
Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>
1 parent c3db2cd commit da36f72

File tree

9 files changed

+990
-200
lines changed

9 files changed

+990
-200
lines changed

doc/modules/calibration.rst

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,11 +30,25 @@ approximately 80% actually belong to the positive class.
3030
Calibration curves
3131
------------------
3232

33-
The following plot compares how well the probabilistic predictions of
34-
different classifiers are calibrated, using :func:`calibration_curve`.
33+
Calibration curves (also known as reliability diagrams) compare how well the
34+
probabilistic predictions of a binary classifier are calibrated. It plots
35+
the true frequency of the positive label against its predicted probability,
36+
for binned predictions.
3537
The x axis represents the average predicted probability in each bin. The
3638
y axis is the *fraction of positives*, i.e. the proportion of samples whose
37-
class is the positive class (in each bin).
39+
class is the positive class (in each bin). The top calibration curve plot
40+
is created with :func:`CalibrationDisplay.from_estimators`, which uses
41+
:func:`calibration_curve` to calculate the per bin average predicted
42+
probabilities and fraction of positives.
43+
:func:`CalibrationDisplay.from_estimator`
44+
takes as input a fitted classifier, which is used to calculate the predicted
45+
probabilities. The classifier thus must have :term:`predict_proba` method. For
46+
the few classifiers that do not have a :term:`predict_proba` method, it is
47+
possible to use :class:`CalibratedClassifierCV` to calibrate the classifier
48+
outputs to probabilities.
49+
50+
The bottom histogram gives some insight into the behavior of each classifier
51+
by showing the number of samples in each predicted probability bin.
3852

3953
.. figure:: ../auto_examples/calibration/images/sphx_glr_plot_compare_calibration_001.png
4054
:target: ../auto_examples/calibration/plot_compare_calibration.html
@@ -161,6 +175,8 @@ mean a better calibrated model.
161175
:class:`CalibratedClassifierCV` supports the use of two 'calibration'
162176
regressors: 'sigmoid' and 'isotonic'.
163177

178+
.. _sigmoid_regressor:
179+
164180
Sigmoid
165181
^^^^^^^
166182

doc/modules/classes.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1123,7 +1123,7 @@ See the :ref:`visualizations` section of the user guide for further details.
11231123
metrics.DetCurveDisplay
11241124
metrics.PrecisionRecallDisplay
11251125
metrics.RocCurveDisplay
1126-
1126+
calibration.CalibrationDisplay
11271127 8000

11281128
.. _mixture_ref:
11291129

doc/visualizations.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ values of the curves.
6565
* :ref:`sphx_glr_auto_examples_miscellaneous_plot_roc_curve_visualization_api.py`
6666
* :ref:`sphx_glr_auto_examples_miscellaneous_plot_partial_dependence_visualization_api.py`
6767
* :ref:`sphx_glr_auto_examples_miscellaneous_plot_display_object_visualization.py`
68+
* :ref:`sphx_glr_auto_examples_calibration_plot_compare_calibration.py`
6869

6970
Available Plotting Utilities
7071
============================
@@ -90,6 +91,7 @@ Display Objects
9091

9192
.. autosummary::
9293

94+
calibration.CalibrationDisplay
9395
inspection.PartialDependenceDisplay
9496
metrics.ConfusionMatrixDisplay
9597
metrics.DetCurveDisplay

doc/whats_new/v1.0.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,9 @@ Changelog
152152
:class:`calibration.CalibratedClassifierCV` can now properly be used on
153153
prefitted pipelines. :pr:`19641` by :user:`Alek Lefebvre <AlekLefebvre>`.
154154

155+
- |Feature| :func:`calibration.CalibrationDisplay` added to plot
156+
calibration curves. :pr:`17443` by :user:`Lucy Liu <lucyleeow>`.
157+
155158
- |Fix| Fixed an error when using a ::class:`ensemble.VotingClassifier`
156159
as `base_estimator` in ::class:`calibration.CalibratedClassifierCV`.
157160
:pr:`20087` by :user:`Clément Fauchereau <clement-f>`.

0 commit comments

Comments
 (0)
0