8000 Ensure estimators pass check_estimator with default parameters? · Issue #9050 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]
Skip to content

Ensure estimators pass check_estimator with default parameters? #9050

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
amueller opened this issue Jun 8, 2017 · 4 comments
Closed

Ensure estimators pass check_estimator with default parameters? #9050

amueller opened this issue Jun 8, 2017 · 4 comments
Labels
Needs Decision - Close Requires decision for closing

Comments

@amueller
Copy link
Member
amueller commented Jun 8, 2017

Apart from the hard-coded estimators in the common tests, we also set some parameters, ostensibly for faster tests.
But some estimators don't pass the tests with default parameters. I just ran into BaggingClassifier that has inconsistent predict_log_proba (which is mostly inf).
We should either make the tests more robust or the estimators.
@amueller
Copy link
Member Author
amueller commented Jun 8, 2017

on the other hand, not all tests pass when set_checking_parameters (or set_testing_parameters or whatever it is called now) is run... not sure I understand how currently the tests pass

@rth
Copy link
Member
rth commented Jun 8, 2017

Linking other related issues: #8683 , #6981 , #6715 ..

@amueller
Copy link
Member Author

This is not currently possible since the default parameters make assumptions on the number of features, in particular in SelectKBest and GaussianRandomProjection. I'm not entirely sure if this is something we should fix or not.

@cmarmo cmarmo added the Needs Decision - Close Requires decision for closing label Dec 17, 2021
@thomasjpfan
Copy link
Member

At this point, _set_checking_parameters is used to:

  1. Run tests faster.
  2. Adopt the parameters to fit the data that is generated for check_estimator. When we change the default, the _set_checking_parameters has comments describing why.

The estimator tags greatly expanded the test coverage in the common tests. With that in mind, I am closing this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Needs Decision - Close Requires decision for closing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@amueller @rth @cmarmo @thomasjpfan
0