Scikit optimize with several parameters #1058

IgnacioEG · 2021-09-17T09:22:43Z

IgnacioEG
Sep 17, 2021

I have to optimize a black-box problem that depends on external software (no function definition neither derivatives) that is quite expensive to evaluate. It depends on several variables, some of them are real and some other are integers.

I think Scikit Optimize may be a good choice. I am newbie with Scikit Optimize.

I was wondering if the following example (from the Scikit Optimize documentation) may be adapted to my actual problem. Being "f" an external function that provides the cost of a given set of parameters. Here it is a dummy function just to be reproducible. But, instead of depending just on "x", make it dependable on "y" and "z" being one of them restricted to integer values.

I have seen some other examples of Scikit Optimize oriented to hyperparameter optimization (based on Scikit Learn), but they seem less clear for me.

Here is the minimum reproducible example (that crash):

import numpy as np
from skopt import gp_minimize
from skopt.space import Integer
from skopt.space import Real


np.random.seed(123)
def f(x,y,z):
    return (np.sin(5 * x[0]) * (1 - np.tanh(x[0] ** 2)) *np.random.randn() * 0.1-y[0]**2+z[0]**2)

search_space = list()
search_space.append(Real(-2, 2, name='x'))
search_space.append(Integer(-2, 2, name='y'))
search_space.append(Real(0, 2, name='z'))

res = gp_minimize(f, search_space, n_calls=20)
print("x*=%.2f, y*=%.2f, f(x*,y*)=%.2f" % (res.x[0],res.y[0],res.z[0], res.fun))

Best regards and thank you

QuentinSoubeyran · 2021-09-29T16:50:52Z

QuentinSoubeyran
Sep 29, 2021

As per gp_minimize documentation, the function f() to optimize should take a single argument, which is a list of argument.

In your code, you are mixing two usages: f() takes normal argument, but x is subscripted (x[0]) as if it was a list of arguments, instead of one specific argument.

Here is a script where f() has been fixed:

import numpy as np
import skopt

np.random.seed(123)

dimensions = [
    (-2.0, 2.0),
    (-2, 2),
    (0.0, 2.0),
]


def f(args):
    x, y, z = args
    return (
        np.sin(5 * x) * (1 - np.tanh(x ** 2)) * np.random.randn() * 0.1
        - y ** 2
        + z ** 2
    )

result = skopt.gp_minimize(f, dimensions, n_calls=20)
print(f"Lowest value is {result.fun=}, at {result.x=}")

If you want to use a standard function, you can use the skopt.utils.use_named_args decorator:

import numpy as np
import skopt
from skopt import space, utils

np.random.seed(123)

dimensions = [
    space.Real(-2.0, 2.0, name="x"),
    space.Integer(-2, 2, name="y"),
    space.Real(0.0, 2.0, name="z"),
]


@utils.use_named_args(dimensions)
def f(x, y, z):
    return (
        np.sin(5 * x) * (1 - np.tanh(x ** 2)) * np.random.randn() * 0.1
        - y ** 2
        + z ** 2
    )


result = skopt.gp_minimize(f, dimensions, n_calls=20)
print(f"Lowest value is {result.fun=}, at {result.x=}")

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Scikit optimize with several parameters #1058

Uh oh!

{{title}}

Uh oh!

Replies: 1 comment

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Scikit optimize with several parameters #1058

Uh oh!

IgnacioEG Sep 17, 2021

Replies: 1 comment

Uh oh!

QuentinSoubeyran Sep 29, 2021

IgnacioEG
Sep 17, 2021

QuentinSoubeyran
Sep 29, 2021