High-Dimensional Multivariate Linear Regression with Weighted Nuclear Norm Regularization

Abstract

A low-rank matrix estimation problem when the data is assumed to be generated from the multivariate linear regression model is considered. To induce the low-rank coefficient matrix, we employ the weighted nuclear norm (WNN) penalty defined as the weighted sum of the singular values of the matrix. The weights are set in a non-decreasing order, which yields the non-convexity of the WNN objective function in the parameter space. Such objective function has been applied in many applications, but studies on the estimation properties of the estimator from the objective function are limited. We propose an efficient algorithm under the framework of alternative directional method of multipliers (ADMM) to estimate the coefficient matrix. The estimator from the suggested algorithm converges to a stationary point of an augmented Lagrangian function. Under the orthogonal design setting, effects of the weights for estimating the singular values of ground-truth coefficient matrix are derived. Under the Gaussian design setting, a minimax convergence rate on the estimation error is derived. We also propose a generalized cross-validation (GCV) criterion for selecting the tuning parameter and an iterative algorithm for updating the weights. Simulations and a real data analysis demonstrate the competitive performance of our new method. Several extensions of the proposed method with applications will also be discussed.

Please click here for participating the talk online

Download