Prediction and feature selection are essential topics for statistical and machine learning (ML) methods when dealing with high-dimensional data. However, they come with limitations: statistical methods may exhibit lower predictive ability than ML methods, while ML methods are often criticized as black boxes. In this paper, we introduce a tree-based algorithm, the Multipath Chebyshev Greedy Algorithm (MPCGA), which enhances the predictive performance of statistical methods and feature selection capabilities under model misspecification. This algorithm extends the Chebychev’s Greedy Algorithm (CGA) and High Dimensional Information Criterion (HDIC) into a tree-expanded structure, allowing for the simultaneous consideration of multiple models. MPCGA outperforms traditional statistical methods when models are misspecified, while maintaining high feature selection precision. Furthermore, we propose accelerated algorithms to boost the computational speed of MPCGA handling indicator features in binary outcome cases. The paper includes a case study on a lung cancer dataset, demonstrating that utilizing ML methods with the feature set selected by MPCGA leads to suitable results.