Backtesting, Prequential Analysis and Prediction Process

Abstract

In the prequential framework, data are observed sequentially in time, and at each time it is required to make a prediction about the distribution of the next observation conditional on the past, based on the data up to that point.? Some time later, we need to check whether the sequence of our forecasts are consistent with that of observations.? This ex-post examination of predictive performance is called a backtesting in finance.? Through the recent controversy on backtestability issue, we now know that, depending on which aspect of the conditional distribution we try to predict, we are faced with varying difficulty in devising backtesting procedures.? In this talk, some attempts are made to extend Davis' calibration concept to larger classes of statistical functions (with values in an space).? Comparison of two probability forecasting systems under absolute continuity condition may be interpreted in terms of the corresponding prediction processes which always posse! ? ?ss Markov property, and we explore its implications.? Computation for a few simple examples from time series analysis will be shown to exemplify the theory.? Finally, the possibility of extensions to the case with auxiliary random variables (covariates) and to the continuous-time case will be discussed.