A Bridge between Boosting Method and Kernel Machine

Abstract

In this talk, boosting methods are studied from a viewpoint of kernel machines. This natural connection has already been revealed by defining a kernel function associated with the set of weak learners, which we call the WL kernel (Weak Learner kernel). We review this connection with respect to a kernel exponential family, and propose two important extensions of boosting methods for classification problems. First proposal is a new simple regularized boosting, which is confirmed to be valid through some experiments on real data. The other is a new simple kernel function from the investigation of the RKHS of decision stumps, which is one of the most widely-used weak learners. Several experiments confirm the efficiency and the validity of the proposed algorithm with the new kernel function.