Closed
Description
Describe the bug
When running check_estimators on an estimator with poor_score tag, enforcing that all provided classes are observed in the test output of check_classifiers_predictions is too stringent.
Steps/Code to Reproduce
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.utils.estimator_checks import check_estimator
class TestPoorScore(LogisticRegression):
def decision_function(self, X):
scores = super().decision_function(X=X)
shape = scores.shape
new_scores = np.ones(shape[0])
if scores.ndim > 1:
new_scores = np.column_stack((new_scores, np.zeros((shape[0], shape[1] - 1))))
return new_scores
def _more_tags(self):
return dict(poor_score=True)
check_estimator(TestPoorScore)Expected Results
all checks pass (if poor_score is set the same evaluation as ComplementNB is used)
Actual Results
Traceback (most recent call last):
File "test.py", line 34, in <module>
check_estimator(TestPoorScore)
File "/home/scgraham/repos/scikit-learn/sklearn/utils/estimator_checks.py", line 466, in check_estimator
check(estimator)
File "/home/scgraham/repos/scikit-learn/sklearn/utils/estimator_checks.py", line 2188, in check_classifiers_classes
check_classifiers_predictions(X, y_, name, classifier_orig)
File "/home/scgraham/repos/scikit-learn/sklearn/utils/_testing.py", line 317, in wrapper
return fn(*args, **kwargs)
File "/home/scgraham/repos/scikit-learn/sklearn/utils/estimator_checks.py", line 2140, in check_classifiers_predictions
assert_array_equal(np.unique(y), np.unique(y_pred))
File "/home/scgraham/.local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 935, in assert_array_equal
assert_array_compare(operator.__eq__, x, y, err_msg=err_msg,
File "/home/scgraham/.local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 765, in assert_array_compare
raise AssertionError(msg)
AssertionError:
Arrays are not equal
(shapes (2,), (1,) mismatch)
x: array(['one', 'two'], dtype='<U3')
y: array(['two'], dtype='<U3')Versions
System:
python: 3.8.2 (default, Mar 26 2020, 15:53:00) [GCC 7.3.0]
executable: /home/scgraham/miniconda3/envs/sklearn/bin/python
machine: Linux-4.4.0-18362-Microsoft-x86_64-with-glibc2.10
Python dependencies:
pip: 20.0.2
setuptools: 46.1.1.post20200323
sklearn: 0.23.dev0
numpy: 1.18.1
scipy: 1.4.1
Cython: 0.29.15
pandas: 1.0.3
matplotlib: None
joblib: 0.14.1
Built with OpenMP: True