8000 MAINT run common tests on IterativeImputer by glemaitre · Pull Request #24930 · scikit-learn/scikit-learn · GitHub
[go: up one dir, main page]

Skip to content

MAINT run common tests on IterativeImputer #24930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 16, 2022

Conversation

glemaitre
Copy link
Member

Related to #24923

When debugging #24923, I found out that we don't run the common tests on IterativeImputer because we don't explicitly enable it.

Copy link
Member
@lesteve lesteve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With this change, calling sklearn.utils.discovery.all_estimators has the side effect of enabling IterativeImputer.

Maybe this is not something we really want, and trying to do something similar in a test file would be more appropriate (even if there is still some side-effect)?

@glemaitre
Copy link
Member Author

Maybe this is not something we really want, and trying to do something similar in a test file would be more appropriate (even if there is still some side-effect)?

We internally use all_estimators only for testing purposes (docstring, common tests, etc.). If someone else is using this function, I don't see in which way it would be an issue to get an experimental feature since even an experimental feature should be compatible with check_estimator.

The contract of all_estimators state that is all estimators that inherited from BaseEstimator and are not declared privately or in the test modules.

@lesteve
Copy link
Member
lesteve commented Nov 15, 2022

A bit convoluted and maybe not important enough to care, but this is what I meant. With the current changes in this PR, the following snippet runs without error:

from sklearn.utils.discovery import all_estimators

estimators = all_estimators()

# now I can import IterativeImputer without doing "from sklearn.experimental import enable_iterative_imputer
from sklearn.impute import IterativeImputer

@betatim
Copy link
Member
betatim commented Nov 15, 2022

Can we make a context manager that enables a particular/all experimental estimators? It is a bit weird to have this side effect, and I think it makes it impossible to not have the experimental estimator enabled when calling all_estimators()?

I had a quick look in the pytest documentation for a way to import something only within a particular test but couldn't find anything. If there was something like this it would be another nice solution.

@lesteve
Copy link
Member
lesteve commented Nov 15, 2022

A (maybe?) simpler option would be to add from sklearn.experimental import enable_iterative_imputer in sklearn/tests/test_common.py, which would still run tests for the IterativeImputer while restricting the the side-effect to our tests (full disclosure: we are already importing enable_halving_search_cv in sklearn/tests/test_common.py)

@betatim
Copy link
Member
betatim commented Nov 15, 2022

That does sound like a simpler way, especially if there is precedent for doing this.

For completeness and so I don't lose it. This is the kinda context manager I was thinking of:

@contextmanager
def experimental():
    from sklearn.impute._iterative import IterativeImputer
    from sklearn import impute
    setattr(impute, "IterativeImputer", IterativeImputer)
    impute.__all__ += ["IterativeImputer"]
    yield
    delattr(impute, "IterativeImputer")
    impute.__all__.remove("IterativeImputer")


with experimental():
    from sklearn.imputer import IterativeImputer
    print(IterativeImputer)

from sklearn.impute import IterativeImputer # raises ImportError

@glemaitre
Copy link
Member Author

here is precedent for doing this.

I might be the precedent :)

Fine with the solution of @lesteve

@@ -45,14 +45,17 @@

import sklearn

# make it possible to discover experimental estimators when calling `all_estimators`
from sklearn.experimental import enable_iterative_imputer # noqa
from sklearn.experimental import enable_halving_search_cv # noqa
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
from sklearn.experimental import enable_halving_search_cv # noqa
from sklearn.experimental import enable_halving_search_cv # noqa
from sklearn.experimental import enable_hist_gradient_boosting # noqa

While we are at it, also add hist gradient boosting?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HistGradientBoosting* does not need the experimental import anymore it is there just for backward-compatibility (and warns that it is not needed anymore) ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok. I should have run the code :D

Copy link
Member
@jeremiedbb jeremiedbb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I checked and confirm that the tests now run with the experimental estimators

@jeremiedbb jeremiedbb merged commit 4795098 into scikit-learn:main Nov 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0