Decision Trees - Maximum Likelihood Approach

Abstract

Decision trees have been proved to be valuable tools for description and classification of data. Many decision trees based on least squares optimization criterion (least squares regression tree) have been implemented in the past thirty years. On the contrary, decision trees based on the maximum likelihood principle have only been used to study censored survival data. One limitation of using the least squares optimization criterion to construct trees, however, is the data type for the response variable must be continuous. For discrete response, other optimization criteria such as Entropy, Gini index, and classification accuracy are used. This limitation can be overcome with the proposed maximum likelihood criterion since the maximum likelihood criterion can deal with both types of response variables. In addition, many standard likelihood related methods such as the AIC and BIC model selection criteria can be naturally incorporated into the tree method development. Simulation study showed the proposed maximum likelihood approach can build better tree models than the traditional least squares approach. This presentation will include four parts. The first part is an introduction section on existing methods for decision trees. The second part is a description on the maximum likelihood regression tree. The third part is a simulation study to compare the proposed approach with the existing least squares approach on four different types of model settings. The last part includes several real world examples.