Robust stepwise regression methods for high-dimensional variable selection

Abstract

Stepwise regression is a classical and very popular variable screening method which has been widely accepted by practical analysts. Wang (2009) showed that forward regression with an extended Bayesian information criterion can identify theoretically all relevant predictors consistently under an ultrahigh-dimensional setup with regular assumptions. Ing and Lai (2011) further introduced a fast stepwise regression method which has oracle property under a strong sparsity assumption. Both methods showed very impressive performances in each own simulation scenarios respectively. However, each method performed unsatisfactorily under some of the other’s simulation schemes. It indicates that these screening methods of elegant theoretical properties may be sensitive to their own assumptions. This study was motivated to develop a more robust stepwise method for screening high-dimensional data in practice. The idea is to establish a new stopping rule other than the conventional information criteria for lessening model assumptions. Numerical simulations studies have shown encouraging performance of the proposed methods in comparison with several popular penalized likelihood techniques for various models in the literature.