Multiple hyperparameters for Gaussian Process #2520
Labels
Comments
|
Sorry for the super slow response cycle. We definitely have to look into this. I'm not sure there is currently great support for multiple hyperparameters and this needs a definite refactoring. |
|
Closing. A new implementation of gaussian process has just been merged (see #4270) and offers this feature. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
The gaussian process module recently merged some code that allowed multifeature things. That is a great step, but there is an almost equally important thing, and it is having multiple hyperparams.
All kernels in GPs have a set of hyperparams theta. In scikits, it seems like this set is restricted to cardinality of one--that is, you can only have one hyperparameter. Note that this is not the same as having an element in theta that is multidimensional. Currently in the scikits code this is allowed. For example, in the squared exponential kernel I can have:
k(x,x') = exp( -theta * (x-x')^2 )
and theta can be a vector/array whose elements correspond to different dimensions/features. However, I cannot have this:
k(x,x') = theta_0 * exp( -theta_1 * (x-x')^2 )
because in order for this to make sense. I would need theta to actually be a matrix, where a row corresponds to one hyperparam and the columns on that row are the features. Then, for that second kernel above, I would have a theta matrix that had 2 rows.
This is what makes sense to me, but it does not work. Sometimes I have run into problems with the optimizer because I have written bad kernels, but I know theoretically this is not the case. The kernel I made up above isn't necessarily valid, I don't know. The one I actually tried was a periodic kernel, where instead of (x-x')^2 I used sin(x-x')^2. That works. But theoretically you can add any two valid kernels together and get a new valid kernel. This works too, but then I'm stuck without a hyperparam for the sin part.
So if I'm doing this multiple hyperparms thing wrong, I'd like to know. If this is not actually a feature, I'd like to make it one and I need some advice on how to extend the code for this. I'm a bit of a noob with this code so any help or tips you guys have would be helpful.
The text was updated successfully, but these errors were encountered: