Rank-Based Multiple Classifier Decision Combination: A Theoretical Study

Abstract

This study presents a theoretical investigation of the rank-based multiple classifier decision problem for closed-set pattern identification. The problem of combining the decisions of more than one classifiers with raw outputs in the form of candidate class rankings is considered and formulated as a general discrete optimization problem with an objective function based on the total probability of correct decision. This formulation uses certain performance statistics about the joint behavior of the ensemble of classifiers, which need to be estimated from cross-validation data. An initial approach leads to an integer (binary) programming problem with a simple and global optimum solution but of prohibitive dimensionality. Therefore, we present a partitioning formalism under which this dimensionality can be reduced by incorporating our prior knowledge about the problem domain and the structure of the training data. It is also shown that formalism can effectively explain a number of successfully used combination approaches in the literature.