[MRG] Set noise tunable by default #209
Conversation
| nu=2.5)), | ||
| normalize_y=True, alpha=alpha, random_state=random_state) | ||
| kernel=cov_amplitude * matern + noise, | ||
| normalize_y=True, random_state=random_state) |
There was a problem hiding this comment.
By default alpha in GaussianProcessRegressor is positive, which means noise can be controlled in two ways. Shall we fix alpha=0.0 instead here, since we have the noise component?
There was a problem hiding this comment.
Yes. But 10**-10 is still a small value no?. It is added to the covariance here (https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/gaussian_process/gpr.py#L382)
|
I am yet to see the implications of this on any example but I'm unlikely to get there before tomorrow. |
|
For the first time, I'm happy to see a build fail... |
|
I think I broke something on sklearn master with this scikit-learn/scikit-learn@c31ad7a :/ Could you check if you are still able to run the bayesian optimisation notebook? The repr of "res.models" breaks. |
|
Meh... :/ I'll check later... |
|
Couldn't help it.
|
|
Minimal code to reproduce: matern = Matern(length_scale=np.ones(1))
ConstantKernel() * matern |
|
See: scikit-learn/scikit-learn#7259 for a fix |
|
@glouppe I've pushed a temporary workaround and I've modified the example, which makes me think. should we keep alpha (maybe renamed to noise) with an option="auto"? It would be easier for people to play with their custom kernels without having to add a |
| @@ -26,6 +26,6 @@ def test_gp_minimize(): | |||
| yield (check_minimize, bench4, 0.0, | |||
| [("-2", "-1", "0", "1", "2")], search, acq, 0.05, 10) | |||
| yield (check_minimize, branin, 0.39, [(-5.0, 10), (0.0, 15.)], | |||
| search, acq, 0.1, 50) | |||
There was a problem hiding this comment.
I changed this because tests were failing.
|
Looks ok. Feel free to merge if green. |
https://papers.nips.cc/paper/4522-practical-bayesian-optimization-of-machine-learning-algorithms.pdf advocates the use of setting noise as a tunable hyperparameter on the GP kernel.