AR Record & Replay: Situated Compositing of Video Content in Mobile Augmented Reality

In this paper we present a novel approach to record and replay video content composited in-situ with a live view of the real environment. Our real-time technique works on mobile phones, and uses an panorama-based tracker to create visually seamless and spatially registered overlay of video content. We apply a temporal foreground-background segmentation of video footage and show how the segmented information can be precisely registered in real-time in the camera view of a mobile phone. We describe the user interface and the video post effects implemented in our prototype as well as our approach with a skateboard training application. Our technique can also be used with online video material and supports the creation of augmented situated documentaries.