Last-Layer Adaptive Scoring-rule Tuning for Pre-Trained Regression Networks

Abstract

We study post-hoc predictive intervals for pre-trained regression networks under three operational constraints: no retraining, fixed architecture, no ensembling. We propose LAST (Last-layer Adaptive Scoring-rule Tuning): a Bayesian linear regression on the trained features with a single predictive-variance scale τ selected by leave-one-out CRPS, paired with optional split-conformal calibration. Our technical contributions are (i) a closed-form Sherman–Morrison evaluation of the leave-one-out CRPS objective that extends the classical ridge-LOO identity from MSE cross-validation to scoring-rule cross-validation, reducing the τ-selection cost to O(nd_L^2  +Kn) for a K-point grid (no retraining, no nested CV); and (ii) uniform consistency of τ ̂_n at the parametric rate. Empirically, LAST is competitive with last-layer Laplace with conformal calibration on standard tabular regression but decisively wins on density-quality deliverables under heavy-tailed noise; LAST also delivers better far-OOD conditional coverage than constant width MAP with conformal calibration on gap data and heteroscedastic data.

Keywords: post-hoc uncertainty quantification; last-layer Bayesian inference; leave-one-out cross-validation; strictly proper scoring rules; continuous ranked probability score (CRPS).

Please click here for participating the talk online.