A variable screening approach for linear regression based on a randomized algorithm

Abstract

When the data size is large, even performing a standard statistical analysis can become challenging. For instance, in linear regression, if the number of predictors is very large and exceeds the sample size, the usual least squares estimation will fail. A common practice for solving the above problem is to perform variable screening and variable cleaning. In this talk, we focus on the screening step and propose an algorithm involves randomization. The proposed method incorporates a random partitioning step, which improves the variable screening accuracy. Our simulation results show that the proposed approach works well.