API for non continuous inputs #59
Comments
|
These are two existing ways of specifying such things, space = hp.choice('a',
[
('case 1', 1 + hp.lognormal('c1', 0, 1)),
('case 2', hp.uniform('c2', -10, 10))
])param_grid = [
{'C': [1, 10, 100, 1000], 'kernel': ['linear']},
{'C': [1, 10, 100, 1000], 'gamma': [0.001, 0.0001], 'kernel': ['rbf']},
]It would be good to adopt an existing one if it isn't too awkward for us. We could reuse some of the code for the sampling etc and as a user I don't have to get used to yet another slightly different way of specifying choices. |
|
Good idea indeed. I would be in favour of following as closely as possible elements from the scikit-learn API. I am not sure if it really fits though, as we would lack a way to specify the scale. |
|
An idea: param_grid = [
{'C': [1, 10, 100, 1000], # categorical C
'kernel': ['linear']},
{'C': skopt.range(1,1000), # continuos C
'gamma': [0.001, 0.0001], 'kernel': ['rbf']},
]If you pass a list/tuple we assume it is categorical, to specify a range you have to use param_grid = [
{'C': [1, 10, 100, 1000], # categorical C because len()>2
'kernel': ['linear']},
{'C': [1,1000], # continuos C because len()==2
# categorical gamma, special notation to distinguish 2 element categorical
'gamma': skopt.categorical(0.001, 0.0001),
# single element or multi element list of non numerical -> categorical
'kernel': ['rbf'],
'something_categorical': ['hello', 'world']},
] |
|
Hmmm, why not. Maybe take the same as in https://github.com/hyperopt/hyperopt/wiki/FMin#21-parameter-expressions but with some syntax shortcuts? (a list == |
|
We are not limiting ourselves to ML hyperparamters right? Why not something simplistic like where |
|
We will not able to support true categorical parameters like a SVM kernel for instance but we should advise the users to manually provide and get the lower and upper bounds rather than complicate the API for this specific use case. |
|
While working on #62 I realize that categorical is not the same as discrete. For instance, knowing that a DecisionTree with |
|
I'd like to be able to support something like the SVM RBF use case. This means supporting the use case where the value of a variable I like having one "object" that describes the whole search space instead of splitting it up/having to specify masks. At least I'd hope we can find a way to write down the whole search space that is easy for humans to write down without making a mistake. So for me finding a good syntax is the top priority. |
|
I was just afraid that it was becoming a bit too verbose, but I don't find a better option. How do you propose to handle the distinction between categorical and discrete? |
|
Could we use: skopt.categorical(0.001, 0.0001) # two choices
(10, 100) # integer values in the range 10 - 100
(10., 100.) # continuos values in the range 10 - 100
(1, 2, 4, 8) # categorical
("hello", "world") # categoryWould that work? |
At the moment, input values are assumed to live within a bounded continuous range. We should think about an API on how to specify integer and symbolic values as well, and what would be the consequences for the algorithms we implemented so far.
The text was updated successfully, but these errors were encountered: