Search Results (355)

Some of the barriers preventing virtual reality (VR) from being widely adopted are the cost and unfamiliarity of VR systems. Here, we propose that in many cases, the specialized controllers shipped with most VR head-mounted displays can be replaced by a regular smartphone, cutting the cost of the system, and allowing users to interact in VR using a device they are already familiar with. To achieve this, we developed SmartVR Pointer, an approach that uses smartphones to replace the specialized controllers for two essential operations in VR: selection and navigation by teleporting. In SmartVR Pointer, a camera mounted on the head-mounted display (HMD) is tilted downwards so that it points to where the user will naturally be holding their phone in front of them. SmartVR Pointer supports three selection modalities: tracker based, gaze based, and combined/hybrid. In the tracker-based SmartVR Pointer selection, we use image-based tracking to track a QR code displayed on the phone screen and then map the phone’s position to a pointer shown within the field of view of the camera in the virtual environment. In the gaze-based selection modality, the user controls the pointer using their gaze and taps on the phone for selection. The combined technique is a hybrid between gaze-based interaction in VR and tracker-based Augmented Reality. It allows the user to control a VR pointer that looks and behaves like a mouse pointer by moving their smartphone to select objects within the virtual environment, and to interact with the selected objects using the smartphone’s touch screen. The touchscreen is used for selection and dragging. The SmartVR Pointer is simple and requires no calibration and no complex hardware assembly or disassembly. We demonstrate successful interactive applications of SmartVR Pointer in a VR environment with a demo where the user navigates in the virtual environment using teleportation points on the floor and then solves a Tetris-style key-and-lock challenge. Full article

To fully harness the potential of wind turbine systems and meet high power demands while maintaining top-notch power quality, wind farm managers run their systems 24 h a day/7 days a week. However, due to the system’s large size and the complex interactions of its many components operating at high power, frequent critical failures occur. As a result, it has become increasingly important to implement predictive maintenance to ensure the continued performance of these systems. This paper introduces an innovative approach to developing a head-mounted fault display system that integrates predictive capabilities, including deep learning long short-term memory neural networks model integration, with anomaly explanations for efficient predictive maintenance tasks. Then, a 3D virtual model, created from sampled and recorded data coupled with the deep neural diagnoser model, is designed. To generate a transparent and understandable explanation of the anomaly, we propose a novel methodology to identify a possible subset of characteristic variables for accurately describing the behavior of a group of components. Depending on the presence and risk level of an anomaly, the parameter concerned is displayed in a piece of specific information. The system then provides human operators with quick, accurate insights into anomalies and their potential causes, enabling them to take appropriate action. By applying this methodology to a wind farm dataset provided by Energias De Portugal, we aim to support maintenance managers in making informed decisions about inspection, replacement, and repair tasks. Full article

Many situations can induce dizziness in healthy participants, be it when riding a carrousel or when making head movements while wearing a head-mounted display. Everybody—maybe with the exception of vestibular loss patients—is prone to dizziness, albeit to widely varying degrees. Some people get dizzy after a single rotation around the body axis, while others can perform multiple pirouettes without the slightest symptoms. We have developed a form of vestibular habituation training with the purpose of reducing proneness to dizziness. The training consists of a short (8 min) exercise routine which is moderate enough that it can easily be integrated into a daily routine. Twenty volunteers performed the training over the course of two weeks. We measured subjective dizziness before and after each daily session. We also performed several vestibular tests before (pre-test) and after (post-test) the two-week training period. They included exposure to a rotating and pitching visual environment while standing upright, as well as a physical rotation that was abruptly stopped. The results show that the dizziness induced during a given daily session decreased over the course of the two weeks. The dizziness induced by the rotating visual stimulus was significantly less after completion of the training period compared with the initial pre-test. Also, postural stability and post-rotatory spinning sensations had improved when comparing the post-test with the pre-test. We conclude that a short regular vestibular training can significantly improve proneness to dizziness. Full article

First-person VR- and MR-based Action Observation research has thus far yielded both positive and negative findings in studies observing such tools’ potential to teach motor skills. Teaching drumming, particularly polyrhythms, is a challenging motor skill to learn and has remained largely unexplored in the field of Action Observation. In this contribution, a multimodal tool designed to teach rudimental and polyrhythmic drumming was developed and tested in a 20-subject study. The tool presented subjects with a first-person MR perspective via a head-mounted display to provide users with visual exposure to both virtual content and their physical surroundings simultaneously. When compared against a control group practicing via video demonstrations, results showed increased rhythmic accuracy across four exercises. Specifically, a difference of 239 ms (z-ratio = 3.520, p < 0.001) was found between the timing errors of subjects who practiced with our multimodal mixed reality development compared to subjects who practiced with video, demonstrating the potential of such affordances. This research contributes to ongoing work in the fields of Action Observation and Mixed Reality, providing evidence that Action Observation techniques can be an effective practice method for drumming. Full article

Teaching root canal anatomy has traditionally been reliant on static methods, but recent studies have explored the potential of advanced technologies like augmented reality (AR) to enhance learning and address the limitations of traditional training methods, such as the requirement for spatial imagination and the inability to simulate clinical scenarios fully. This study evaluated the potential of AR as a tool for teaching root canal anatomy in preclinical training in endodontics for predoctoral dental students. Six cone beam computed tomography (CBCT) images of teeth were selected. Board-certified endodontist and radiologist recorded the tooth type and classification of root canals. Then, STereoLithography (STL) files of the same images were imported into a virtual reality (VR) application and viewed through a VR head-mounted display. Forty-three third-year dental students were asked questions about root canal anatomy based on the CBCT images, and then, after the AR model. The time to respond to each question and feedback was recorded. Student responses were paired, and the difference between CBCT and AR scores was examined using a paired-sample t-test and set to p = 0.05. Students demonstrated a significant improvement in their ability to answer questions about root canal anatomy after utilizing the AR model (p < 0.05). Female participants demonstrated significantly higher AR scores compared to male participants. However, gender did not significantly influence overall test scores. Furthermore, students required significantly less time to answer questions after using the AR model (M = 4.09, SD = 3.55) compared to the CBCT method (M = 15.21, SD = 8.01) (p < 0.05). This indicates that AR may improve learning efficiency alongside comprehension. In a positive feedback survey, 93% of students reported that the AR simulation led to a better understanding of root canal anatomy than traditional CBCT interpretation. While this study highlights the potential of AR in learning root canal anatomy, further research is needed to explore its long-term impact and efficacy in clinical settings. Full article

Cloud virtual reality (VR) is attracting attention in terms of its lightweight head-mounted display (HMD), providing telepresence and mobility. However, it is still in the research stages due to motion-to-photon (MTP) latency, the need for high-speed network infrastructure, and large-scale traffic processing problems. These problems are expected to be partially solved through edge computing, but the limited computing resource capacity of the infrastructure presents new challenges. In particular, in order to efficiently provide multi-user content such as remote meetings on edge devices, resource provisioning is needed that considers the application’s traffic patterns and computing resource requirements at the same time. In this study, we present a microservice architecture (MSA)-based application to provide multi-user cloud VR in edge computing and propose a scheme for planning an efficient service deployment considering the characteristics of each service. The proposed scheme not only guarantees the MTP latency threshold for all users but also aims to reduce networking and computing resource waste. The proposed scheme was evaluated by simulating various scenarios, and the results were compared to several studies. It was confirmed that the proposed scheme represents better performance metrics than the comparison schemes in most cases from the perspectives of networking, computing, and MTP latency. Full article

Aim of the Study: To assess how virtual reality (VR) patient-specific simulations can support decision-making processes and improve care in pediatric urology, ultimately improving patient outcomes. Patients and Methods: Children diagnosed with urological conditions necessitating complex procedures were retrospectively reviewed and enrolled in the study. Patient-specific VR simulations were developed with medical imaging specialists and VR technology experts. Routine CT images were utilized to create a VR environment using advanced software platforms. The accuracy and fidelity of the VR simulations was validated through a multi-step process. This involved comparing the virtual anatomical models to the original medical imaging data and conducting feedback sessions with pediatric urology experts to assess VR simulations’ realism and clinical relevance. Results: A total of six pediatric patients were reviewed. The median age of the participants was 5.5 years (IQR: 3.5–8.5 years), with an equal distribution of males and females across both groups. A minimally invasive laparoscopic approach was performed for adrenal lesions (n = 3), Wilms’ tumor (n = 1), bilateral nephroblastomatosis (n = 1), and abdominal trauma in complex vascular and renal malformation (ptotic and hypoplastic kidney) (n = 1). Key benefits included enhanced visualization of the segmental arteries and the deep vascularization of the kidney and adrenal glands in all cases. The high depth perception and precision in the orientation of the arteries and veins to the parenchyma changed the intraoperative decision-making process in five patients. Preoperative VR patient-specific simulation did not offer accuracy in studying the pelvic and calyceal anatomy. Conclusions: VR patient-specific simulations represent an empowering tool in pediatric urology. By leveraging the immersive capabilities of VR technology, preoperative planning and intraoperative navigation can greatly impact surgical decision-making. As we continue to advance in medical simulation, VR holds promise in educational programs to include even surgical treatment of more complex urogenital malformations. Full article

The aim of this study is to understand the experiences of university students who took part in a pilot program for an experiential learning opportunity in immersive virtual reality (iVR). Experiential learning opportunities are essential for students who will be expected to apply their knowledge in a professional setting. Head-mounted display devices were distributed to university students and individuals with developmental disabilities at a partnering community organization. The university students met community partners in a virtual world and interacted with them to learn about their partners’ self-selected goals related to communication and job skills. A mixed methods analysis of survey responses and journal entries was conducted. Students reported an overall positive experience with iVR and indicated an interest in pursuing future opportunities to include iVR in their learning. Full article

In the global context, advancements in technology and science have rendered virtual, augmented, and mixed-reality technologies capable of transforming clinical care and medical environments by offering enhanced features and improved healthcare services. This paper aims to present a mixed reality-based system to control a robotic wheelchair for people with limited mobility. The test group comprised 11 healthy subjects (six male, five female, mean age 35.2 ± 11.7 years). A novel platform that integrates a smart wheelchair and an eye-tracking-enabled head-mounted display was proposed to reduce the cognitive requirements needed for wheelchair movement and control. The approach’s effectiveness was demonstrated by evaluating our system in realistic scenarios. The demonstration of the proposed AR head-mounted display user interface for controlling a smart wheelchair and the results provided in this paper could highlight the potential of the HoloLens 2-based innovative solutions and bring focus to emerging research topics, such as remote control, cognitive rehabilitation, the implementation of patient autonomy with severe disabilities, and telemedicine. Full article

Surgical education demands extensive knowledge and skill acquisition within limited time frames, often limited by reduced training opportunities and high-pressure environments. This review evaluates the effectiveness of extended reality-based head-mounted display (ExR-HMD) technology in surgical education, examining its impact on educational outcomes and exploring its strengths and limitations. Data from PubMed, Cochrane Library, Web of Science, ScienceDirect, Scopus, ACM Digital Library, IEEE Xplore, WorldCat, and Google Scholar (Year: 2014–2024) were synthesized. After screening, 32 studies comparing ExR-HMD and traditional surgical training methods for medical students or residents were identified. Quality and bias were assessed using the Medical Education Research Study Quality Instrument, Newcastle–Ottawa Scale-Education, and Cochrane Risk of Bias Tools. Results indicate that ExR-HMD offers benefits such as increased immersion, spatial awareness, and interaction and supports motor skill acquisition theory and constructivist educational theories. However, challenges such as system fidelity, operational inconvenience, and physical discomfort were noted. Nearly half the studies reported outcomes comparable or superior to traditional methods, emphasizing the importance of social interaction. Limitations include study heterogeneity and English-only publications. ExR-HMD shows promise but needs educational theory integration and social interaction. Future research should address technical and economic barriers to global accessibility. Full article

Periacetabular osteotomy (PAO) is an effective approach for the surgical treatment of developmental dysplasia of the hip (DDH). However, due to the complex anatomical structure around the hip joint and the limited field of view (FoV) during the surgery, it is challenging for surgeons to perform a PAO surgery. To solve this challenge, we propose a robot-assisted, augmented reality (AR)-guided surgical navigation system for PAO. The system mainly consists of a robot arm, an optical tracker, and a Microsoft HoloLens 2 headset, which is a state-of-the-art (SOTA) optical see-through (OST) head-mounted display (HMD). For AR guidance, we propose an optical marker-based AR registration method to estimate a transformation from the optical tracker coordinate system (COS) to the virtual space COS such that the virtual models can be superimposed on the corresponding physical counterparts. Furthermore, to guide the osteotomy, the developed system automatically aligns a bone saw with osteotomy planes planned in preoperative images. Then, it provides surgeons with not only virtual constraints to restrict movement of the bone saw but also AR guidance for visual feedback without sight diversion, leading to higher surgical accuracy and improved surgical safety. Comprehensive experiments were conducted to evaluate both the AR registration accuracy and osteotomy accuracy of the developed navigation system. The proposed AR registration method achieved an average mean absolute distance error (mADE) of 1.96 ± 0.43 mm. The robotic system achieved an average center translation error of 0.96 ± 0.23 mm, an average maximum distance of 1.31 ± 0.20 mm, and an average angular deviation of 3.77 ± 0.85°. Experimental results demonstrated both the AR registration accuracy and the osteotomy accuracy of the developed system. Full article

The cognitive state of a person can be categorized using the circumplex model of emotional states, a continuous model of two dimensions: arousal and valence. The purpose of this research is to select a machine learning model(s) to be integrated into a virtual reality (VR) system that runs cognitive remediation exercises for people with mental health disorders. As such, the prediction of emotional states is essential to customize treatments for those individuals. We exploit the Remote Collaborative and Affective Interactions (RECOLA) database to predict arousal and valence values using machine learning techniques. RECOLA includes audio, video, and physiological recordings of interactions between human participants. To allow learners to focus on the most relevant data, features are extracted from raw data. Such features can be predesigned, learned, or extracted implicitly using deep learners. Our previous work on video recordings focused on predesigned and learned visual features. In this paper, we extend our work onto deep visual features. Our deep visual features are extracted using the MobileNet-v2 convolutional neural network (CNN) that we previously trained on RECOLA’s video frames of full/half faces. As the final purpose of our work is to integrate our solution into a practical VR application using head-mounted displays, we experimented with half faces as a proof of concept. The extracted deep features were then used to predict arousal and valence values via optimizable ensemble regression. We also fused the extracted visual features with the predesigned visual features and predicted arousal and valence values using the combined feature set. In an attempt to enhance our prediction performance, we further fused the predictions of the optimizable ensemble model with the predictions of the MobileNet-v2 model. After decision fusion, we achieved a root mean squared error (RMSE) of 0.1140, a Pearson’s correlation coefficient (PCC) of 0.8000, and a concordance correlation coefficient (CCC) of 0.7868 on arousal predictions. We achieved an RMSE of 0.0790, a PCC of 0.7904, and a CCC of 0.7645 on valence predictions. Full article

Neural radiance fields (NeRFs) and 3D Gaussian splatting have emerged as promising 3D reconstruction techniques recently. However, their application in virtual reality (VR), particularly in firefighting training, remains underexplored. We present an innovative VR firefighting simulation system based on 3D Gaussian Splatting technology. Leveraging these techniques, we successfully reconstruct realistic physical environments. By integrating the Unity3D game engine with head-mounted displays (HMDs), we created and presented immersive virtual fire scenes. Our system incorporates NeRF technology to generate highly realistic models of firefighting equipment. Users can freely navigate and interact with fire within the virtual fire scenarios, enhancing immersion and engagement. Moreover, by utilizing the Photon PUN2 networking framework, our system enables multi-user collaboration on firefighting tasks, improving training effectiveness and fostering teamwork and communication skills. Through experiments and surveys, it is demonstrated that the proposed VR framework enhances user experience and holds promises for improving the effectiveness of firefighting training. Full article

The impact of viewing VR videos using a head-mounted display (HMD) on tear film dynamics is examined by comparing the viewing experience of individuals using their naked eyes with that of viewers wearing contact lenses. While the impact of VR on eye dryness has been studied, there is limited research on the risks for contact lens wearers. This study aims to investigate eye dryness associated with VR use in individuals wearing soft contact lenses. Seventeen adults in their 20s (7 male, 10 female) with uncorrected visual acuity of 0.8+ participated. The non-invasive tear film break-up time (NIBUT) was assessed before and after a 20 min VR video session under two conditions: with and without soft contact lenses. The results indicated a decrease in the initial tear film break-up time and an increase in the average tear film break-up time when viewing with naked eyes, whereas viewing with contact lenses led to decreases in both parameters, with statistically significant changes observed. Although the alteration in the tear film break-up time was insignificant during VR video viewing with naked eyes, the tear film stability of individuals wearing soft contact lenses tended to decrease. Caution is advised when using soft contact lenses during VR video sessions to mitigate potential eye dryness. Full article

The goal of vehicular automation is to enhance driver comfort by reducing the necessity for active engagement in driving. This allows for the performance of non-driving-related tasks (NDRTs), with attention shifted away from the driving process. Despite this, there exists a discrepancy between current in-vehicle display configurations and the escalating demands of NDRTs. This study investigates drivers’ preferences for in-vehicle display configurations within highly automated driving contexts. Utilizing virtual reality head-mounted displays (VR-HMDs) to simulate autonomous driving scenarios, this research employs Unity 3D Shape for developing sophisticated head movement tracking software. This setup facilitates the creation of virtual driving environments and the gathering of data on visual attention distribution. Employing an orthogonal experiment, this experiment methodically analyses and categorizes the primary components of in-vehicle display configurations to determine their correlation with visual immersion metrics. Additionally, this study incorporates subjective questionnaires to ascertain the most immersive display configurations and to identify key factors impacting user experience. Statistical analysis reveals that a combination of Portrait displays with Windshield Head-Up Displays (W-HUDs) is favored under highly automated driving conditions, providing increased immersion during NDRTs. This finding underscores the importance of tailoring in-vehicle display configurations to individual needs to avoid distractions and enhance user engagement. Full article