Conditions for Existence of Uniformly Consistent Classifiers

Abstract

We consider the statistical problem of binary classification, which means attaching a random observation X from a separable metric space E to one of the two classes, 0 or 1. We prove that the consistent estimation of conditional probability p(X)= P(Y=1 X) , where Y is the true class of X, is equivalent to the consistency of a class of empirical classifiers. We then investigate for what classes P there exist an estimate p that is consistent uniformly in p P. We show that this holds if and only if P is a totally bounded subset of L1(Eμ), where μ is the distribution of X. In the case, where E is countable, we give a complete characterization of classes π, allowing consistent estimation of p, uniform in (μ,p)ϵπ.