Research Interests:
Research Interests: Psychology and Gesture
Built application for the paper "A Unity3D-based Framework for Interaction-Free Calibration of Optical See-Through Head-Mounted Displays". The application provides a Unity3D-based framework for the implementation of two proposed... more
Built application for the paper "A Unity3D-based Framework for Interaction-Free Calibration of Optical See-Through Head-Mounted Displays". The application provides a Unity3D-based framework for the implementation of two proposed fully interaction-free calibration methods for optical see-through head-mounted displays. Please refer to the README file for deployment and test instructions.
Built application for the paper "A Unity3D-based Framework for Interaction-Free Calibration of Optical See-Through Head-Mounted Displays". The application provides a Unity3D-based framework for the implementation of two proposed... more
Built application for the paper "A Unity3D-based Framework for Interaction-Free Calibration of Optical See-Through Head-Mounted Displays". The application provides a Unity3D-based framework for the implementation of two proposed fully interaction-free calibration methods for optical see-through head-mounted displays. Please refer to the README file for deployment and test instructions.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Aortic valve replacement is the only definitive treatment for aortic stenosis, a highly prevalent condition in elderly population. Minimally invasive surgery brought numerous benefits to this intervention, and robotics recently provided... more
Aortic valve replacement is the only definitive treatment for aortic stenosis, a highly prevalent condition in elderly population. Minimally invasive surgery brought numerous benefits to this intervention, and robotics recently provided additional improvements in terms of telemanipulation, motion scaling, and smaller incisions. Difficulties in obtaining a clear and wide field of vision is a major challenge in minimally invasive aortic valve surgery: surgeon orientates with difficulty because of lack of direct view and limited spaces. This work focuses on the development of a computer vision methodology, for a three-eyed endoscopic vision system, to ease minimally invasive instrument guidance during aortic valve surgery. Specifically, it presents an efficient image stitching method to improve spatial awareness and overcome the orientation problems which arise when cameras are decentralized with respect to the main axis of the aorta and are nonparallel oriented. The proposed approach ...
Research Interests:
Research Interests:
Research Interests:
Research Interests:
The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the... more
The adoption of extended reality solutions is growing rapidly in the healthcare world. Augmented reality (AR) and virtual reality (VR) interfaces can bring advantages in various medical-health sectors; it is thus not surprising that the medical MR market is among the fastest-growing ones. The present study reports on a comparison between two of the most popular MR head-mounted displays, Magic Leap 1 and Microsoft HoloLens 2, for the visualization of 3D medical imaging data. We evaluate the functionalities and performance of both devices through a user-study in which surgeons and residents assessed the visualization of 3D computer-generated anatomical models. The digital content is obtained through a dedicated medical imaging suite (Verima imaging suite) developed by the Italian start-up company (Witapp s.r.l.). According to our performance analysis in terms of frame rate, there are no significant differences between the two devices. The surgical staff expressed a clear preference fo...
Research Interests:
Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and... more
Visual augmented reality (AR) has the potential to improve the accuracy, efficiency and reproducibility of computer-assisted orthopaedic surgery (CAOS). AR Head-mounted displays (HMDs) further allow non-eye-shift target observation and egocentric view. Recently, a markerless tracking and registration (MTR) algorithm was proposed to avoid the artificial markers that are conventionally pinned into the target anatomy for tracking, as their use prolongs surgical workflow, introduces human-induced errors, and necessitates additional surgical invasion in patients. However, such an MTR-based method has neither been explored for surgical applications nor integrated into current AR HMDs, making the ergonomic HMD-based markerless AR CAOS navigation hard to achieve. To these aims, we present a versatile, device-agnostic and accurate HMD-based AR platform. Our software platform, supporting both video see-through (VST) and optical see-through (OST) modes, integrates two proposed fast calibration procedures using a specially designed calibration tool. According to the camera-based evaluation, our AR platform achieves a display error of 6.31 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 2.55 arcmin for VST and 7.72 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 3.73 arcmin for OST. A proof-of-concept markerless surgical navigation system to assist in femoral bone drilling was then developed based on the platform and Microsoft HoloLens 1. According to the user study, both VST and OST markerless navigation systems are reliable, with the OST system providing the best usability. The measured navigation error is 4.90 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.04 mm, 5.96 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 2.22<inline-formula><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> for the VST system, and 4.36 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 0.80 mm, 5.65 <inline-formula><tex-math notation="LaTeX">$\pm$</tex-math></inline-formula> 1.42<inline-formula><tex-math notation="LaTeX">$^\circ$</tex-math></inline-formula> for the OST system.
Research Interests:
Research Interests:
Research Interests:
The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are... more
The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance analysis. This is the case of projected augmented reality, which, compared to head-worn AR displays, offers the advantages of being simultaneously accessible by multiple users whilst preserving user awareness of the environment and feeling of immersion. This work provides a general evaluation of a head-mounted projector for precision manual tasks and a detailed description of the experimental protocol applied for investigating spatial and temporal registration and their combination in a user test. The results show that the accuracy and latency of the device make it suitable for guiding precision tasks.
Research Interests:
Research Interests:
Research Interests:
In recent years, huge progress has been made in the management of brain tumors, due to the availability of imaging devices, which provide fundamental anatomical and pathological information not only for diagnostic purposes [...]
Research Interests:
Research Interests:
In the context of image-guided surgery, augmented reality (AR) represents a ground-breaking enticing improvement, mostly when paired with wearability in the case of open surgery. Commercially available AR head-mounted displays (HMDs),... more
In the context of image-guided surgery, augmented reality (AR) represents a ground-breaking enticing improvement, mostly when paired with wearability in the case of open surgery. Commercially available AR head-mounted displays (HMDs), designed for general purposes, are increasingly used outside their indications to develop surgical guidance applications with the ambition to demonstrate the potential of AR in surgery. The applications proposed in the literature underline the hunger for AR-guidance in the surgical room together with the limitations that hinder commercial HMDs from being the answer to such a need. The medical domain demands specifically developed devices that address, together with ergonomics, the achievement of surgical accuracy objectives and compliance with medical device regulations. In the framework of an EU Horizon2020 project, a hybrid video and optical see-through augmented reality headset paired with a software architecture, both specifically designed to be se...
Research Interests:
Research Interests:
Research Interests:
In the context of a European project, we identified over 150 requirements for the development of an augmented reality (AR) head-mounted display (HMD) specifically tailored to support highly challenging manual surgical procedures. The... more
In the context of a European project, we identified over 150 requirements for the development of an augmented reality (AR) head-mounted display (HMD) specifically tailored to support highly challenging manual surgical procedures. The requirements were established by surgeons from different specialties and by industrial players working in the surgical field who had strong commitments to the exploitation of this technology. Some of these requirements were specific to the project, while others can be seen as key requirements for the implementation of an efficient and reliable AR headset to be used to support manual activities in the peripersonal space. The aim of this work is to describe these ergonomic requirements that impact the mechanical design of the HMDs, the possible innovative solutions to these requirements, and how these solutions have been used to implement the AR headset in surgical navigation. We also report the results of a preliminary qualitative evaluation of the AR he...
Research Interests:
In the context of Augmented Reality (AR), industrial pioneers and early adopters have considered the wearable optical see-through (OST) displays as proper and effective tools in applications spanning from manufacturing and maintenance up... more
In the context of Augmented Reality (AR), industrial pioneers and early adopters have considered the wearable optical see-through (OST) displays as proper and effective tools in applications spanning from manufacturing and maintenance up to the entertainment field and the medical area, because they provide the user with an egocentric viewpoint maintaining the quality of the visual perception of the real world.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
Augmented reality (AR) Head-Mounted Displays (HMDs) are emerging as the most efficient output medium to support manual tasks performed under direct vision. Despite that, technological and human-factor limitations still hinder their... more
Augmented reality (AR) Head-Mounted Displays (HMDs) are emerging as the most efficient output medium to support manual tasks performed under direct vision. Despite that, technological and human-factor limitations still hinder their routine use for aiding high-precision manual tasks in the peripersonal space. To overcome such limitations, in this work, we show the results of a user study aimed to validate qualitatively and quantitatively a recently developed AR platform specifically conceived for guiding complex 3D trajectory tracing tasks. The AR platform comprises a new-concept AR video see-through (VST) HMD and a dedicated software framework for the effective deployment of the AR application. In the experiments, the subjects were asked to perform 3D trajectory tracing tasks on 3D-printed replica of planar structures or more elaborated bony anatomies. The accuracy of the trajectories traced by the subjects was evaluated by using templates designed ad hoc to match the surface of the...
Research Interests:
Research Interests:
Research Interests:
Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual... more
Augmented reality (AR) has been successfully providing surgeons an extensive visual information of surgical anatomy to assist them throughout the procedure. AR allows surgeons to view surgical field through the superimposed 3D virtual model of anatomical details. However, open surgery presents new challenges. This study provides a comprehensive overview of the available literature regarding the use of AR in open surgery, both in clinical and simulated settings. In this way, we aim to analyze the current trends and solutions to help developers and end/users discuss and understand benefits and shortcomings of these systems in open surgery. We performed a PubMed search of the available literature updated to January 2018 using the terms (1) "augmented reality" AND "open surgery", (2) "augmented reality" AND "surgery" NOT "laparoscopic" NOT "laparoscope" NOT "robotic", (3) "mixed reality" AND "open surge...
Research Interests:
Research Interests:
Augmented Reality (AR) has already proven its worth in various applications in the medical domain. However, most of the solutions proposed were bound to specific hardware or software configurations, and/or their application was limited to... more
Augmented Reality (AR) has already proven its worth in various applications in the medical domain. However, most of the solutions proposed were bound to specific hardware or software configurations, and/or their application was limited to specific cases, thus lacking in flexibility. In this paper, we present a software framework suitable for AR video see-through systems conceived for medical applications: our solution allows merging of real world images grabbed by one or more external cameras with computer-generated sceneries coregistered to the acquired images. The software framework is highly configurable and extensible thanks to the employment of two text configuration files that make it suitable for many typologies of potential applications. The proposed solution can be easily adapted to functioning with different tracking and AR visualization modalities. The versatility of the software for video see-through AR applications was already tested on various medical applications, in conjunction with head-mounted displays or with external spatial displays.
Research Interests:
Research Interests:
Research Interests:
Research Interests:
The human perception of the three-dimensional world is influenced by the mutual integration of physiological and psychological depth cues, whose complexity is still an unresolved issue per se. Even more so if we wish to mimic the... more
The human perception of the three-dimensional world is influenced by the mutual integration of physiological and psychological depth cues, whose complexity is still an unresolved issue per se. Even more so if we wish to mimic the perceptive efficiency of the human visual system within augmented reality (AR) based surgical navigation systems. In this work we present a novel and ergonomic AR interaction paradigm that aids the manual placement of a non-tracked rigid body in space by manually minimizing the reprojection residuals between a set of corresponding virtual and real feature points. Our paradigm draws its inspiration from the general problem of estimating camera pose from a set of n-correspondences, i.e. perspective-n-point problem. In a recent work, positive results were achieved in terms of geometric error by applying the proposed strategy on the validation of a wearable AR system to aid manual maxillary repositioning.
Research Interests:
ABSTRACT
Research Interests:
Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following... more
Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following deformations impressed on actual anatomy. A proof-of-concept solution is described providing AR visualization of hidden deformable tubular structures using nitinol tubes sensorized with electromagnetic sensors. This system was tested in vitro on a setup comprised of sensorized cystic, left and right hepatic, and proper hepatic arteries. In the trial session, the surgeon deformed the tubular structures with surgical forceps in 10 positions. The mean, standard deviation, and maximum misalignment between virtual and real arteries were 0.35, 0.22, and 0.99 mm, respectively. The alignment accuracy obtained demonstrates the feasibility of the approach, which can be adopted in advanced AR simulations, in particular as an aid to the identification and isolatio...
Research Interests: Computer Science, Visualization, Augmented Reality, Medicine, General Surgery, and 13 moreVirtual Reality, Software, Computer assisted orthopaedic surgery, Humans, Electromagnetic Radiation, Calibration, Surgical Simulation, Clinical Sciences, Computer User Interface Design, Reproducibility of Results, Simulation Training, Cholecystectomy, and Laparoscopy
Magnetic guide of endovascular devices or magnetized therapeutic microparticles to the specific target in the arterial tree is increasingly studied, since it could improve treatment efficacy and reduce side effects. Most proposed systems... more
Magnetic guide of endovascular devices or magnetized therapeutic microparticles to the specific target in the arterial tree is increasingly studied, since it could improve treatment efficacy and reduce side effects. Most proposed systems use external permanent magnets attached to robotic manipulators or magnetic resonance imaging (MRI) systems to guide internal carriers to the region of treatment. We aim to simplify this type of procedures, avoiding or reducing the need of robotic arms and MRI systems in the surgical scenario. On account of this we investigated the use of a wearable stereoscopic video see-through augmented reality system to show the hidden vessel to the surgeon; in this way, the surgeon is able to freely move the external magnet, following the showed path, to lead the endovascular magnetic device towards the desired position. In this preliminary study, we investigated the feasibility of such an approach trying to guide a magnetic capsule inside a vascular mannequin....
Research Interests:
Research Interests:
Stereoscopic video see though AR systems permit accurate marker video based registration. To guarantee accurate registration, cameras are normally rigidly blocked while the user could require changing their vergence. We propose a solution... more
Stereoscopic video see though AR systems permit accurate marker video based registration. To guarantee accurate registration, cameras are normally rigidly blocked while the user could require changing their vergence. We propose a solution working with lightweight hardware that, without the need for a new calibration of the cameras relative pose after each vergence adjustment, guarantees registration accuracy using pre-determined calibration data.
Research Interests:
Research Interests:
Wearable Video See-Through (VST) devices for Augmented Reality (AR) and for obtaining a Magnified View are taking hold in the medical and surgical fields. However, these devices are not yet usable in daily clinical practice, due to... more
Wearable Video See-Through (VST) devices for Augmented Reality (AR) and for obtaining a Magnified View are taking hold in the medical and surgical fields. However, these devices are not yet usable in daily clinical practice, due to focusing problems and a limited depth of field. This study investigates the use of liquid-lens optics to create an autofocus system for wearable VST visors. The autofocus system is based on a Time of Flight (TOF) distance sensor and an active autofocus control system. The integrated autofocus system in the wearable VST viewers showed good potential in terms of providing rapid focus at various distances and a magnified view.
Research Interests:
Research Interests:
Cryosurgery is a technique of growing popularity involving tissue ablation under controlled freezing. Technological advancement of devices along with surgical technique improvements have turned cryosurgery from an experimental to an... more
Cryosurgery is a technique of growing popularity involving tissue ablation under controlled freezing. Technological advancement of devices along with surgical technique improvements have turned cryosurgery from an experimental to an established option for treating several diseases. However, cryosurgery is still limited by inaccurate planning based primarily on 2D visualization of the patient’s preoperative images. Several works have been aimed at modelling cryoablation through heat transfer simulations; however, most software applications do not meet some key requirements for clinical routine use, such as high computational speed and user-friendliness. This work aims to develop an intuitive platform for anatomical understanding and pre-operative planning by integrating the information content of radiological images and cryoprobe specifications either in a 3D virtual environment (desktop application) or in a hybrid simulator, which exploits the potential of the 3D printing and augmen...
Research Interests:
Augmented Reality (AR) based on head-mounted displays (HMDs) represent the most ergonomic and efficient solution for aiding complex manual tasks performed under direct vision and it is considered an enabling technology of the fourth... more
Augmented Reality (AR) based on head-mounted displays (HMDs) represent the most ergonomic and efficient solution for aiding complex manual tasks performed under direct vision and it is considered an enabling technology of the fourth industrial revolution (Industry 4.0). This work intends to give a harmonized and consistent overview of the workflow carried out so far in the framework of the European Project VOSTARS towards the development of a novel AR HMD specifically designed to aiding complex manual tasks within arm’s reach. First results and future developments are being presented.
Research Interests:
Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a... more
Neuronavigation has become an essential neurosurgical tool in pursuing minimal invasiveness and maximal safety, even though it has several technical limitations. Augmented reality (AR) neuronavigation is a significant advance, providing a real-time updated 3D virtual model of anatomical details, overlaid on the real surgical field. Currently, only a few AR systems have been tested in a clinical setting. The aim is to review such devices. We performed a PubMed search of reports restricted to human studies of in vivo applications of AR in any neurosurgical procedure using the search terms ``Augmented reality'' and ``Neurosurgery.'' Eligibility assessment was performed independently by two reviewers in an unblinded standardized manner. The systems were qualitatively evaluated on the basis of the following: neurosurgical subspecialty of application, pathology of treated lesions and lesion locations, real data source, virtual data source, tracking modality, registration technique, visualization processing, display type, and perception location. Eighteen studies were included during the period 1996 to September 30, 2015. The AR systems were grouped by the real data source: microscope (8), hand- or head-held cameras (4), direct patient view (2), endoscope (1), and X-ray fluoroscopy (1) head-mounted display (1). A total of 195 lesions were treated: 75 (38.46 {\%}) were neoplastic, 77 (39.48 {\%}) neurovascular, and 1 (0.51 {\%}) hydrocephalus, and 42 (21.53 {\%}) were undetermined. Current literature confirms that AR is a reliable and versatile tool when performing minimally invasive approaches in a wide range of neurosurgical diseases, although prospective randomized studies are not yet available and technical improvements are needed.