Adaptive Variable Selection on High-Dimensional Non-Parametric Regression Models

Abstract

Sparse linear models that incorporate non-parametric elements have gained significant prominence because of their flexibility in high-dimensional applications.
While standard estimation frameworks often rely on  basis expansion and assume an underlying sparse structure to achieve parsimony, the practical implementation of these models is hindered by the fact that both the degree of sparsity and the functional smoothness are generally unknown.
Conventional approaches often circumvent this by pre-specifying the number of basis functions, focusing solely on variable selection.This paper addresses this gap by investigating the convergence properties of the Orthogonal Greedy Algorithm (OGA) across diverse regimes of sparsity and smoothness. We propose a unified framework that utilizes a high-dimensional Akaike's information criterion (HDAIC) to jointly determine the optimal number of OGA iterations and the  basis dimension. We show that the resulting procedure, OGA combined with HDAIC (OGA+HDAIC), is adaptive in the sense that it automatically achieves the optimal trade-off between variance and squared bias without prior knowledge of the sparsity or the smoothness levels.

線上視訊請點選連結