Least Squares High-Dimensional Model Averaging

Abstract

This paper aims to propose an efficient method to implement the model averaging estimation in the high-dimensional framework. To this end, we first consider the orthogonal greedy algorithm proposed by Ing and Lai (2011) to construct a set of nested models. We then determine the optimal weights by incorporating the risk inflation factor into the Mallow Model Averaging Criterion (Hansen, 2007) to address the uncertainty arising from the variable selection. The resultant estimator is called HDMMA estimator. We show in this paper that HDMMA estimator asymptotically attains the minimax rate. Hence, it is asymptotically optimal in the sense that it achieves the same order of the lowest risk over a rich class of model average estimators whose weights reside in the continuous extension of a discrete index set. Numerical studies in this paper suggest the effectiveness of HDMMA in terms of attaining the minimax rates. Keywords: High-dimension, Model averaging, Greedy algorithm