Accurate Registration of 3D Models with Organs in Augmented Reality Through Inertial Sensors

Augmented Reality (AR) can improve the accuracy of surgical procedures by displaying 3D reconstructions of organs directly on the patient’s body. Moreover, it can improve the surgeon’s level of attention by freeing him from the task of mentally associating information from various sources. A crucial aspect for the reliability of an AR system is the registration process, which aims to guarantee the correspondence between the 3D model and the real organ during the whole operation. The research work introduced in this paper aims to improve the accuracy of the registration process by updating the alignment between real and virtual organs even in presence of deformations induced by the insertion of surgical instruments or small displacements that organs might undergo during surgery. By detecting even imperceptible movements, the inertial sensors allow dynamic updating of the alignment between real and virtual organs to ensure an error of a few millimetres. The solution introduced in this study could therefore constitute a first step towards improving precision in AR-guided surgery, although more in-depth studies are needed to assess the optimal positioning of the sensors, which in the future could be done automatically by an application that suggests it in the preparatory phase of surgery.