Step 1:

Add essential libraries which are mainly used to manipulate the data inside the objective function and to set up the path such that the classes we wrote are reachable.

# Import essential libraries
import sys
sys.path.append("..")
import numpy as np

Step 2

Import the main class which will take care of running the entire routine Gaussian Process + Bayesian Optimization.

# Import our library
import core.BO as BO

Step 3

Define an objective function and the variables

# Define objective function
# If you want to maximize remember to negate the output
def f(X, noise=0.0):
    fval = -np.sin(3*X) - X**2 + 0.7*X
    fval_noise = noise * np.random.randn(*X.shape)
    return fval + fval_noise

# Define variables
fvars = [{'name':'var1', 'type':'continuous', 'domain': (-1.0, 2.0)}]

Step 4

Define the Kernel you are going to use (all the kernel supported by GPy are avaiable here).
Define initial samples which means how many points are chosen randomly to start the Gaussian Process.
Create the Bayesian Optimization object (GPyOpt like).

# Define kernel
kernel = 'Matern32'

# Define initial samples
init_samples=5

# Create the BayesianOptimization object
my_bo=BO.BayesianOptimization(fvars, f, normalize_X=True, init_n_samples=init_samples, samples_generator='random', ARD=True, kernel=kernel)

Step 5

Run the optimization specifying the max number of steps and if you want step-by-step plot (just for debug). In the end print the max value found (since we are trying to maximize the function in this problem).

%matplotlib inline

# Run the optimization
my_bo.run_optimization(max_iter=20, plot=True)

# Print the min
print(np.max(my_bo.Y))

from IPython.display import Image
Image(filename='imgs/plots_init.png')

After 20 iterations...

Image(filename='imgs/plots_end.png')

Comparison with GPyOpt

%matplotlib inline

import numpy as np
import GPy
import GPyOpt
from GPyOpt.methods import BayesianOptimization

bounds = np.array([[-1.0, 2.0]])
noise = 0.0

def f(X, noise=0.0):
    return -np.sin(3*X) - X**2 + 0.7*X + noise * np.random.randn(*X.shape)

X_init = np.array([[-0.9], [1.1]])
Y_init = f(X_init)

kernel = GPy.kern.Matern52(input_dim=1, variance=1.0, lengthscale=1.0)
bds = [{'name': 'X', 'type': 'continuous', 'domain': bounds.ravel()}]

optimizer = BayesianOptimization(f=f, 
                                 domain=bds,
                                 model_type='GP',
                                 kernel=kernel,
                                 acquisition_type ='EI',
                                 acquisition_jitter = 0.01,
                                 X=X_init,
                                 Y=-Y_init,
                                 noise_var = noise**2,
                                 exact_feval=False,
                                 normalize_Y=False,
                                 maximize=True)

optimizer.run_optimization(max_iter=10)
optimizer.plot_acquisition()
print(min(optimizer.Y_best))

Image(filename='imgs/gpyopt.png')