[MRG + 1] Add refit_time_ attribute to model selection #11310
Conversation
In particular to GridSearchCV and RandomizedSearchCV. Implements scikit-learn#8833.
sklearn/model_selection/_search.py
Outdated
| refit_time_ : float | ||
| Seconds used for refitting the best model on the whole dataset. | ||
|
|
||
| This is present only if ``refit`` is set to ``True``. |
There was a problem hiding this comment.
I think you mean "if ``refit`` is not False"
There was a problem hiding this comment.
Thanks a lot, I just fixed the wording.
|
I wasn't sure if the tests/CI will pass, therefore I decided that this PR isn't ready yet. If that's wrong I'm happy to change the name prior to the tests passing. |
|
There appears to be an issue with time measurement and Windows. Can I disable the check that the refit time is larger than zero for Windows? |
doc/whats_new/v0.20.rst
Outdated
| :class:`model_selection.RandomizedSearchCV` if ``refit`` is set to ``True``. | ||
| This will allow measuring the complete time it takes to perform | ||
| hyperparameter optimization and refitting the best model on the whole | ||
| dataset. |
There was a problem hiding this comment.
add :user: thing and PR # (since there's no issue).
| assert_true(hasattr(search, "refit_time_")) | ||
| assert_true(isinstance(search.refit_time_, float)) | ||
| if sys.version_info[0] >= 3: | ||
| assert_greater(search.refit_time_, 0) |
There was a problem hiding this comment.
why not >= as for the other parameters?
There was a problem hiding this comment.
Just did this - why didn't I realize this before... Please excuse the extra commits.
|
Unit tests are passing except for one test on travis-ci which fails because of a server error on mldata.org. |
|
ugh that's annoying. I restarted that one. |
Reference Issues/PRs
Resolves #8833.
What does this implement/fix? Explain your changes.
This PR implements a new
refit_time_attribute which will be stored in GridSearchCV and RandomizedSearchCV ifrefitis set toTrue. This will allow measuring the complete time it takes to perform + hyperparameter optimization and refitting the best model on the whole dataset and will be particularly helpful if the hyperparameter optimization is carried out in parallel.