Additional metrics in sklearn.metrics.classification_report
#21000
Comments
sklearn.metrics.classification_reportsklearn.metrics.classification_report
|
With #19556 on the go, I think that we will be OK regarding the metrics. However, I wanted to have this issue opened to discuss the API of |
|
That will be awesome. They are many ways to filter out unwanted metrics, depending on the output format of the If users have an option to retrieve the output as a dictionary, then they have more control over the output, like
|
|
This is another feature and I think it is discuss there: #19012 |
|
That's great! I can see that it has been a while since you first bought up this idea. |
Basically, we lack a bit of reviewers' time. |
|
@glemaitre i would like to work on this feature if i get the chance to do the same as classification report is most used way to get details about the test set. |
|
I would advise not to start working on this issue since that there are no feedbacks from @scikit-learn/core-devs regarding the API |
|
@glemaitre would you mind creating an issue which would cover all-ish the open issues we have and have your proposed APIs there, so that the rest of us can have a look and take the API discussion from there? |
|
Oh, there was quite some panache when this issue appeared 2 years ago, any developments since then? It strikes me as an inconsistency with the general API design that the I could imagine the reasons to include the fact that the bottom three rows of the default report are, to some extent, "custom" (hard-coded): so maybe a bit of additional logic will be needed to decide which scorer affords which summary statistics. When I try to interpret the current behavior in terms of the strings returned by So one extension of the API that would feel quite natural would be to add two new parameters where the current default would correspond to classification_report(y_test, y_pred, scoring=["precision", "recall", "f1"], average=[None, "macro", "weighted"])The Needless to say that all of this would most conveniently be handled by a |
Describe the workflow you want to enable
Metrics are Extremely Important for benchmarking models' performance. But in scikit-learn, it is not easy to extract these metrics from a multiclass classification model. In the past, I had to combine different pieces of functions within the
sklearn.metricsmodule in order to derive my own. But it is counterproductive, so I end up staying away from scikit-learn, and using other libraries such as pycm, mlxtend, yellowbrick, ... etc to get the job done. It is not perfect and still requires some customization, but they are more complete, and much easier to use than scikit-learn. So it would be great if scikit-learn can improve a little more in this area, so that we can focus more on modeling than customizing the code.Describe your proposed solution
metrics.classification_reportis a good start. The currentmetrics.classification_reportreturnIt is great, but far from complete. The following 2 are very important as well
Once we can cover these 4 metrics for the multiclass classification
we can pretty much derive the rest of the metrics, such as
These 2 are quite important as well, but it can be tricky to get for multiclass classification
The pycm library is quite comprehensive. Perhaps you may consider integrating some of their goodies into scikit-learn.
Describe alternatives you've considered, if relevant
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: