The information criteria such as AIC and BIC are not always applicable. For example, when the maximum likelihood estimate of parameters cannot be obtained, their values cannot be computed. A statistical model is said to be regular when the parameters that minimize the KL divergence are unique and the Fisher information matrix is non-degenerate. Dr. Sumio Watanabe devised WAIC as an information criterion that can be applied even when the model is not regular. WAIC is justified in the same way as AIC. It calculates a value called the empirical loss, defined using the posterior average instead of the maximum likelihood estimate, and hence derives the general asymptotic posterior distribution (which converges to the normal distribution in the regular case). Then, using the method of resolution of singularities from algebraic geometry, a learning coefficient value, denoted as $\lambda$ which is $\d/2$ in the regular case, is defined by expanding the number of parameters $d$. The first half of the lecture will explain the complex Watanabe theory in an accessible manner. In the second half, we will introduce LS (Learning under Singularity, Liu and Suzuki, 2024), which improves existing methods like WBIC and SBIC that have been extended to irregular cases. Time permitting, further results from our group will be discussed.

Please click here for participating the talk online.