Interactive tracking of markers for facial palsy analysis

The human face provides a rich source of information from muscular movement and nerval actuation to properties of skin and facial characteristics. This information can be exploited to diagnose and quantify facial impairments. Facial palsy is one of these impairments, and is caused by restrictions of the nerval actuation of muscles responsible for facial expressions. The main symptoms of this condition are asymmetrical facial movement and partial facial paralysis. To measure its progress and to compare pre-surgical with post-surgical conditions, medical physicians require different clinical measures extracted from those locations of the face which provide most information about the facial expression. These locations are indicated by small artificial markers which are placed on the patient's face before an evaluation session. A video of the patient is then recorded which is used to localize these markers in every frame. This task is currently performed manually by an operator and can take up to five hours for a single video. Object tracking refers to a research field which deals with the estimation of the position of one or many objects from an image sequence. Its methods have been applied successfully to different applications, ranging from video surveillance to robotics. Traditionally, illumination, changes in pose and occlusion are considered as the main problems when tracking artificial objects of interest. While the associated tracking methods proved themselves able to deal with these problems in recent years, tracking objects from the medical perspective are still partly unexplored. Just like all natural objects, the human face has a high potential for deformation and is characterized by an irregular texture. Additionally, not only one, but multiple objects/markers have to be tracked simultaneously, which imposes additional difficulty by ensuring that markers can be uniquely identified in every frame. The thesis explores the possibility of tracking the artificial facial markers semi-automatically by applying different, state-of-the-art tracking schemes to the presented problem. The tracking schemes are based on a sequential Bayes estimation technique, the so called particle filter, which assesses a set of hypothesis using their congruence with the target model. Hence, the location of each marker can be accurately estimated and occlusions handled efficiently. To improve the accuracy and to reset lost markers, the clinical operator can interact with the tracking system. The results showed that our chosen methods are superior in both the number of interactions and accuracy when compared with traditional trackers which use only a single hypothesis concerning the marker locations.
Additionally, it is shown that the evaluated schemes are able to replace the task of manual tracking while preserving a high accuracy. As a result, the time to locate the markers is decreased by around 2/3 with an accuracy of around 3-4 pixels towards the available ground truth.
Additionally, only around 2 % of the evaluated frames required operator intervention.