Bayesian factor-adjusted sparse regression

Abstract

Many sparse regression methods rely on an assumption that the covariates are weakly correlated, which hardly holds in many economic and financial datasets. To relax this assumption, we model the strongly correlated covariates by a factor structure: strong correlations among covariates are modeled by common factors, while the remaining variations of covariates are modeled as idiosyncratic components. We then propose a factor-adjusted sparse regression model and develop a semi-Bayesian estimation method for it. Posterior contraction rate and model selection consistency are established by a non-asymptotic analysis. Experimental studies show that the proposed method outperforms its Lasso analogue, manifests insensitivity to overestimates of the number of common factors, pays a negligible price when covariates are uncorrelated, scales up well with increasing sample size, dimensionality and sparsity, and converges fast to the posterior distribution. An application to the U.S. bond risk premia lends further support to the proposed model and method.