Evaluation of content-based image descriptors by statistical methods

Evaluation of visual information retrieval systems is usually performed byexecuting test queries and computing recall- and precision-like measures based onpredefined media collections and ground truth information. This process is complex andtime consuming. For the evaluation of feature transformations (transformation of visualmedia objects to feature vectors) it would be desirable to have simpler methods available aswell. In this paper we introduce a supplementary evaluation procedure for features that isfounded on statistical data analysis. A second novelty is that we make use of the existingvisual MPEG-7 descriptors to judge the characteristics of feature transformations. Theproposed procedure is divided into four steps: (1) feature extraction, (2) merging withMPEG-7 data and normalisation, (3) statistical data analysis and (4) visualisation andinterpretation. Three types of statistical methods are used for evaluation: (1) univariatedescription (moments, etc.), (2) identification of similarities between feature elements (e.g.cluster analysis) and (3) identification of dependencies between variables (e.g. factoranalysis). Statistical analysis provides beneficial insights into the structure of features thatcan be exploited for feature redesign. Application and advantages of the proposed approachare shown in a number of toy examples.