Boosting: A Stochastic Recursive Optimization Algorithm

Abstract

Boosting is one of the most successful recent ensemble classifiers. It attracts considerable attentions because its impressive empirical performance and easy implementation. However, many of its theoretical properties await further investigation. Friedman, Hastie and Tibshirani (2000) views boosting as a procedure builds additive logistic regression model via Newton-like updates for optimizing the exponential criteria. Inspired by this interpretation, we consider boosting as a stochastic recursive optimization algorithm. The new interpretation provides good theoretical framework for further investigation. Implications on weak base hypothesis assumption and consistency will be addressed.