The Expected Improvement (EI) method is a widely-used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model, which results in suboptimal solutions. We propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration. Under certain prior specifications, we prove the global convergence of HEI over a broad function space, and derive global convergence rates under smoothness assumptions on the objective function. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian procedure while avoiding expensive Markov-chain Monte Carlo sampling. Numerical experiments and a toy semiconductor optimization application show the improvement of HEI over existing black-box optimization methods.