Inference for ROC Curves Based on Estimated Predictive Indices

Abstract

The receiver operating characteristic (ROC) curve is used in many disciplines to gauge the effectiveness of binary classifiers. In this paper we propose a method for conducting inference about various features of ROC curves. Examples include testing ROC curve dominance between two classifiers or testing hypotheses about the area under the curve (AUC). To do this, we propose an estimator for the ROC curve and show that it converges weakly to a zero mean Gaussian process. A simulation method is proposed to approximate the limiting process. An important feature of our method is that it leads to asymptotically valid inference even if the ROC curve is constructed based on an estimated predictive index. This is not always the case with “off-the-shelf” methods that are commonly used by practitioners.