ABSTRACT:  In this talk, we consider the problem of optimizing an expensive black-box objective function when a finite budget of total evaluations is prescribed. We use a Bayesian optimization approach. At each iteration, the optimal solution strategy is to evaluate the design leading to the maximum expected improvement accumulated over the remaining steps.This lookahead strategy is the solution of an intractable dynamic programming problem. We demonstrate how to approximate the solution of this dynamic programming problem using rollout, and propose rollout heuristics specifically designed for the Bayesian optimization setting. We also show how to extend this lookahead strategy to account for black-box inequality constraints. We present numerical experiments showing that the resulting lookahead algorithms for optimization with a finite budget perform favorably compared to several popular greedy Bayesian optimization algorithms, both in the unconstrained and constrained case.