Multi-User Mixed Reality System for flexible First Responder Training

Effective action and the correct assessment of dangerous situations – especially in large-scale disaster operations – are essential for first-aiders from the fire brigade, rescue service and army. In order to acquire these skills, large-scale training with different first aiders in an environment that is as realistic as possible is necessary. However, such scenarios are often too dangerous, expensive or complex to simulate realistically enough – which reduces the training benefit. Mixed reality (MR) technologies can realistically depict virtual hazards within a physical training environment. There are currently isolated solutions with MR concepts, but these offer insufficient freedom of movement. Interaction between first responders and real/virtual objects is neglected.

Adaptation options for the exercise leader – to practice different variants of a scenario – are limited. The aim of MRespond is to research technologies that bring about a significant improvement in MR training in the areas of freedom of movement, interaction and dynamic adaptability. Their suitability for use is checked with the support of the users. To this end, two large-scale deployment scenarios – a building fire and a scenario involving chemical hazards – are being developed and implemented. Management and operational forces have to deal with dangerous situations in an interdisciplinary manner. The main focus here is on the free mobility of all trainees in outdoor and multi-storey indoor areas, the interaction between real and virtual objects (e.g for the use of usual equipment) as well as the adaptability of the scenarios. This requires the conception of a robust localization and tracking of the trainees indoors and outdoors as well as suitable object recognition algorithms. Both visible (fire, smoke, …) and invisible hazards (e.g. chemical substances detectable with measuring devices) can be placed virtually on the site.

Emergency puppets represent casualties and are overlaid with virtual injuries to show realistic injury patterns.

The actions of the trainees have an influence on virtual elements (e.g. smoke extraction when windows are opened, triage decisions). A trainer interface makes it possible to adapt the training process, place hazards and evaluate actions. The acceptance of training with virtual elements, training effectiveness and legal aspects of training standards are examined. Furthermore, research is carried out into the influence of professional experience and the gender of the trainees on success and whether diversity-specific differences (gender, age, ethnicity) occur in the treatment of virtual patients.

Results