10000 Surprising values returned by gp_minimize from categories · Issue #821 · scikit-optimize/scikit-optimize · GitHub
[go: up one dir, main page]
Skip to content
This repository was archived by the owner on Feb 28, 2024. It is now read-only.

Surprising values returned by gp_minimize from categories #821

Closed
gar1t opened this issue Jan 31, 2020 · 4 comments
Closed

Surprising values returned by gp_minimize from categories #821

gar1t opened this issue Jan 31, 2020 · 4 comments
Labels
Bug
Milestone
0.7.1

Comments

@gar1t
Copy link
gar1t commented Jan 31, 2020

7720fda is an ostensible fix for #752 but the results from gp_minimize now return invalid category values.

Sample to recreate:

import skopt

def f(params):
    return 0

dims = [[1, 2, 3]]

res = skopt.gp_minimize(
    f,
    dims,
    n_calls=1,
    n_random_starts=1,
)

print(res.x_iters)

Output:

[['1']]

Note that the suggested value is a string value, which is not one of the category values.

The transform to strings is predicated on the ability to reverse transform the values, but this isn't happening here.
@gar1t gar1t changed the title Invalid values returned by gp_minimize from categories Surprising values returned by gp_minimize from categories Jan 31, 2020
@gar1t
Copy link
Author
gar1t commented Jan 31, 2020

Corrected the title as I might be breaking the abstraction here. If x_iters is an internal structure (from the docs, it's the location of function evaluation for each iteration) then it would be helpful to have the type information for use in a call to space.inverse_transform() to get the suggested values. As it is there's no way to know from the return value what values were used in calls to func.

@gar1t
Copy link
Author
gar1t commented Jan 31, 2020 •
edited
Loading

Hehe, I take it back, from what I'm seeing in the 0.7 release, this is extremely invalid :)

Here's a clearer example:

import skopt

def f(params):
    print(params)
    return 0

dims = [[1]]

res = skopt.gp_minimize(
    f,
    dims,
    n_calls=1,
    n_random_starts=1,
)

Output is always ['1'].

gar1t pushed a commit to guildai/guildai that referenced this issue Jan 31, 2020
Peg scikit-optimize to 0.5.2 for now
4880198 
See scikit-optimize/scikit-optimize#821 -
this breaks categories as all values are converted by skopt to
strings.
@holgern holgern added the Bug label Jan 31, 2020
@holgern holgern added this to the 0.7.1 milestone Jan 31, 2020
@holgern
Copy link
Contributor
holgern commented Jan 31, 2020

The problem is that there is not an inverse_transform performed when the function f is called.

from skopt.space import Space, Categorical, Integer, Real, Dimension
space= Space([[1,2,3]])
dimension = space.dimensions[0]
dims = Categorical(dimension.categories,
                                dimension.prior,
                                name=dimension.name,
                                transform="identity")
print(dims)
# Categorical(categories=('1', '2', '3'), prior=None)
print(dims._rvs.rvs(size=1))
# 1
print(dims.rvs(1))
# ['2']

There seems to me a problem with a missing inverse_transform

@holgern holgern mentioned this issue Jan 31, 2020
Merged
holgern added a commit that referenced this issue Jan 31, 2020
@holgern
Merge pull request #823 from holgern/fix_categorical_space
f4f2ffe 
[MRG] Fix categorical space (issue #821)
@holgern
Copy link
Contributor
holgern commented Jan 31, 2020

Will be fixed in 0.7.1
6F8A

@holgern holgern closed this as completed Jan 31, 2020
@xmatthias xmatthias mentioned this issue Nov 4, 2020
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Bug
Projects
None yet
Milestone
0.7.1
Development

No branches or pull requests
2 participants
@gar1t @holgern
0