Augmented Reality video : situated video compositions in panorama-based Augmented Reality applications

The rapid development of mobile devices such as smart phones has led to new possibilities in the context of Mobile Augmented Reality (AR). While there exists a broad range of AR applications providing static content, such as textual annotations, there is still a lack of supporting dynamic content, such as video, in the field of Mobile AR. In this work a novel approach to record and replay video content composited in-situ with a live view of the real environment, with respect to the user's view onto the scene, is presented. The proposed technique works in real-time on currently available mobile phones, and uses a panorama-based tracker to create visually seamless and spatially registered overlays of video content, hence giving end users the chance to re-experience past events at a different point of time. To achieve this, a temporal foreground-background segmentation of video footage is applied and it is shown how the segmented information can be precisely registered in real-time in the camera view of a mobile phone.
Furthermore, the user interface and video post effects implemented in a first prototype within a skateboard training application are presented.
To evaluate the proposed system, a user study was conducted. The results are given at the end of this work along with an outlook on possible future work.