Is there a way to restrict the parameter space differently at each iteration? #730

michaelyli · 2018-11-07T23:20:47Z

I'd like to only compute acquisition function values for a subset of the parameters within the total parameter space. Is there an easy way for me to do this? In other words, could I restrict the parameter space considered to some input set of parameters?

iaroslav-ai · 2019-04-21T18:58:48Z

You can always create a new instance of Optimizer / or surrogate directly from sratch with your observations, where you drop or add any of the dimensions that are necessary for your cause. This will restrict your estimator / acquisition function to work only with the parameters that you require. Feel free to reopen if this does not address your issue.

sqbl · 2019-04-21T19:30:04Z

First of: @iaroslav-ai, you really rock tonight! I've never seen as many updates on one night.

But, with this question: Setting a new instance of the Optimizer with one of the previous dimensions left out (to mimic that a parameter in that dimension is set to a set-point) - would this not remodel everything - and "forget" about potential cross-interactions between any of the other remaining dimensions.
I do not have the coding skills to do this, but my intuition of the problem would be to go along the lines of making a model like "expected_minimum" with full flexibility in all but one dimension and then calculate an aquisition function. Ofcourse for this to work, we need to make the "expected_minimum-look-alike" through a bayesian process, that returns a std.
Could this work?

iaroslav-ai · 2019-04-21T19:34:32Z

Ah I see. I understood that some dimensions are to be dropped; But I guess that you just want to fix some, and calculate acquisition function over the rest of parameters? I think I had a code snippet that does someting similar, I take a look. But I remember that it was not straightforward at least

iaroslav-ai · 2019-04-21T19:34:54Z

Thanks for a positive comment btw :)

iaroslav-ai · 2019-04-21T19:44:15Z

Hm the best I can come up with is #325 and this corresponding snippet. What this does though is allow a fixed "context" vector, which is fixed for optimization of acquisition function. This is done in context of Transfer Learning. Maybe something for you to look at, but not exactly fixing arbitrary dimensions of the acquisition function / surrogate, but only values in the context vector.

I guess if I was in your position I would look at the latest surrogate that Optimizer produces, and use that one directly; You could define a function of your own, that takes a subset of dimensions, and for the rest appends fixed values. You could use minimize from scipy in gradient free mode, to avoid calculating the gradients.

Beyond that, I do not have anything better at the moment :/

nxorable · 2020-09-08T12:34:01Z

Would a simple API update be to allow the caller to constrain the space of expected_minimum_random_sampling to a subspace on a subsequent call?

iaroslav-ai closed this as completed Apr 21, 2019

iaroslav-ai reopened this Apr 21, 2019

nxorable mentioned this issue Sep 7, 2020

Expose additional variables to the surrogate model #944

Open

kernc linked a pull request Nov 17, 2020 that will close this issue

ENH: Add constrained optimization #971

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Is there a way to restrict the parameter space differently at each iteration? #730

Is there a way to restrict the parameter space differently at each iteration? #730

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Is there a way to restrict the parameter space differently at each iteration? #730

Is there a way to restrict the parameter space differently at each iteration? #730

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!