Open set classification in the domain of license plate type images using deep learning

License plate type recognition is a classification problem on an open set of classes. Its purpose is to distinguish between different backgrounds of license plates mounted on vehicles. Knowing the license plate type allows to identify the country in which the respective license plate is registered. Modern approaches which solve open set classification problems make use of deep learning to train an intermediate embedding space in which a subsequent classification is performed. This embedding space essentially represents the result of a non-linear feature reduction. Within it, each class is represented by a compact cluster which is separable from the other classes. The embedding space is only trained with a subset of the known classes. The remaining known classes are incorporated without retraining the network. Instances of classes which are unknown to the system are identified using thresholding. The main contribution of this work is the analysis of the class distributions within the embedding space, which has been largely neglected by previous work. For this purpose, new benchmarks which allow a qualitative comparison between different learned embedding spaces are introduced. The identifying characters of a license plate are challenging for license plate type recognition because the model tends towards learning specific characters as features. To overcome this problem, randomized masking of the characters during training is proposed. Furthermore, investigations about which features the trained network is sensitive to are carried out using gradient-based saliency map techniques. A commonly used embedding space classifier assumes class distributions of homogeneous variance as a prior under the open set restriction. The analysis of the class distributions in the embedding space shows that this prior is not met for the used license plate type dataset. The experiments reveal that the class distributions even show a significant difference from multivariate Gaussians, which are capable of modeling more complex distributions in the shape of hyper-ellipsoids. The computed saliency maps visualize that the model learns reasonable features for most of the license plate types. However, for a few of them, primitive features which a human would not consider for classification are exploited.