Model Selection for High-dimensional Regressions

Abstract

We propose two methods for the classification of binary observations which are applicable to information retrieval. First, we discuss the discovery of minimal sets of features which are necessary for explaining all observations, and the detection of hidden logical patterns in the data which are capable of distinguishing between two groups. Combinations of such patterns are used for developing a classification procedure. Second, we present a classification model which uses probabilistic logical patterns and maximum entropy distribution. Classification experiments by simulation and by using the TREC collection are discussed.