Bayesian Adaptive LASSO for Linear Models

Abstract

The focus of this talk is the selection of informative covariates in linear regression model. We introduce a novel method to select variables, which is referred to as the Bayesian adaptive LASSO. This proposal is motivated by a particular hierarchical Bayesian model that is able to provide adaptive information to identify important covariates to be included in the final model. Unlike others?€? approaches, we do not directly use posterior model probability for variable selection. Instead, we adopt the posterior information to construct an estimation criterion that shares the key feature of the adaptive LASSO, which advocates that the penalties of unimportant covariates should be larger than those of important covariates. In particular, we design an efficient MCMC algorithm to handle data sets involving a large number of covariates. Alternatively, the Bayesian adaptive LASSO can be solved by an efficient algorithm for LASSO. We compare the proposed method with its several Bayesian competitors through simulation experiments. The results show that the proposed method performs better in terms of the prediction accuracy and the relative model error. We also discuss the extension of the Bayesian adaptive LASSO in the generalized linear model. Finally, the proposed method is illustrated with three real-world data sets.