Semi-Autonomous, Virtual Reality Based Robotic Telemanipulation for the Execution of Peg-In-Hole Assembly Tasks

Robotic arms demonstrate superior speed and precision when performing tasks in challenging environments. However, creating fully autonomous robots remains challenging, particularly for tasks that demand the kind of subtle perception and complex decision-making that humans use naturally. This paper investigates the integration of Virtual Reality (VR) with robotic telemanipulation, aiming to boost cooperation between humans and robots in the execution of peg-in-hole assembly tasks. The introduced semi-autonomous framework utilizes visual object detection technology to identify the types and 6D poses of objects, offering affordances to the user. The robot autonomously plans its movement toward the designated goal location before switching to the real-time telemanipulation scheme. This allows the user to either approve the pre-determined task execution plan or take over the robot control and make necessary adjustments. Following a comparative user study between the proposed framework and a pure telemanipulation system, the effectiveness of this approach is evaluated and demonstrated.