A vision-based system for fingertip detection on tracked interactive surfaces

Multi-touch sensing on interactive tabletops and other flat surfaces has become a major trend in the field of human-computer interaction over the past years. The main objective is to provide a touch interface for the direct manipulation of digital content by multiple users at the same time. Within these terms the appropriate design of the interactive surface as well as the automatic detection and tracking of fingertips are crucial.
Popular techniques for fingertip and touch detection use specific contact-sensitive computer hardware that is either relying on optical sensing in a controlled environment or capacitive surface technology.
Since such hardware is usually custom-made, those interaction systems are mostly expensive, inconvenient to move, install and operate and not scalable. To overcome these drawbacks, a number of multi-touch researchers strive for alternative techniques to provide more adjustable interfaces. Here, everyday surfaces shall be augmented with the functionality of touch-based user interfaces, while using none but off-the-shelf and affordable vision hardware and relying on state-of-the-art computer vision methods and algorithms.
This work starts off with the description, discussion and evaluation of common surface hardware technologies as well as existing techniques based on simple video hardware. After that, a set of straightforward computer vision algorithms is selected in order to develop a stand-alone software application. The application is capable of continuously tracking a rectangular surface as well as detecting multiple fingertips that hover above its top. This work is concluded by providing relevant empirical results on computer vision-based rectangle and fingertip detection in natural indoor environments.