CN119105644A - A gaze interaction method to accelerate VR target selection - Google Patents
A gaze interaction method to accelerate VR target selection Download PDFInfo
- Publication number
- CN119105644A CN119105644A CN202410879025.7A CN202410879025A CN119105644A CN 119105644 A CN119105644 A CN 119105644A CN 202410879025 A CN202410879025 A CN 202410879025A CN 119105644 A CN119105644 A CN 119105644A
- Authority
- CN
- China
- Prior art keywords
- target
- trigger
- triggering
- gaze
- auxiliary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04842—Selection of displayed objects or displayed text elements
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a staring interaction method for accelerating VR target selection, which relates to the technical field of man-machine interaction interface design and comprises the steps of S1, designing a ring menu layout triggered by target assistance and a staring method assisted by the target assistance, S2, comparing target selection task verification based on target assistance staring with technology, and S3, an encoding optimization strategy of an auxiliary trigger graph in a high-density space. According to the gaze interaction method for accelerating VR target selection, a target auxiliary fixation (TA-fixation) method is introduced in VR eye control interaction to relieve the problem of Midas touch, specifically, a ring menu known as high-efficiency is adopted, and an auxiliary trigger object is designed to provide feedback and confirmation for IO. After a user selects one IO from the pie menu, the user can watch the auxiliary trigger object to confirm the selection of the auxiliary trigger object, the user can position the auxiliary trigger object in a unit environment through the coordinate point of the central position, the adjustable accuracy threshold of continuous position detection is set to be 0.9, and the interference of blink and eye jump visual behaviors on the trigger accuracy is avoided.
Description
Technical Field
The invention relates to the technical field of man-machine interaction interface design, in particular to a staring interaction method for accelerating VR target selection.
Background
Virtual reality head mounted displays (VRHWDs) are evolving into a basic platform for general consumers to access and participate in digital content by virtue of advanced sensor technology and price increases. The immersive nature of virtual reality puts higher demands on the naturalness and cognitive efficiency of human-computer interaction behavior. However, traditional interaction methods, such as external controls, buttons, or gestures, may no longer be suitable for user experience or efficiency. Furthermore, with the continued advancement of eye tracking technology, eye-controlled interactions have become a significant hands-free solution in virtual reality/augmented reality. Compared to hand or button based methods, eye control interactions have two key advantages, speed and intuitiveness, rotation speed of the eyeball in a particular motion exceeding 300 °/s, and consistent with natural and relaxed human behavior. The importance extends to individuals with specific diseases (such as freezing syndrome or hand dysfunction) or with hands occupied in everyday situations. Recently, appleVisionPro has led to interest and potential progress in gaze-based interactive technology that enables users to interact with virtual objects through eye-hand coordination.
The interaction target triggering task is typically used in eye-controlled interactions, requiring the user to search for and locate a particular interaction object on the interface and trigger this IO using a particular gaze input technique. Common gaze input techniques include gaze, tracking, gaze gestures, and blinking, but current gaze input techniques face several challenges including the "Mi Dasi touch" problem, accuracy problems in high density space, and eye fatigue that may result from prolonged use of the eye.
We have therefore proposed a gaze interaction method that speeds VR target selection in order to solve the problems set forth above.
Disclosure of Invention
The invention aims to provide a staring interaction method for accelerating VR target selection, which aims to solve the problems of 'Mi Dasi touch' in the current market, the precision problem in a high-density space and the eye fatigue possibly caused by long-time use of eyes, which are proposed by the background technology.
In order to achieve the aim, the invention provides the following technical scheme that the staring interaction method for accelerating VR target selection comprises the steps of S1, designing a ring menu layout triggered by target assistance and a staring method assisted by the target assistance, S2, comparing target selection task verification based on target assistance staring with technology, S3, an encoding optimization strategy of an auxiliary trigger graph in a high-density space;
s1, designing a target-assisted triggered annular menu layout and a target-assisted staring method
Completing layout setting of an eye control interaction target and an auxiliary trigger graph, wherein the auxiliary trigger graph is positioned outside an extension line of a menu center point and an interaction target center point, the width position of the interaction target is kept to be 0.5 times, the shapes of the auxiliary trigger graph and the interaction target are annular, and the width of the auxiliary trigger graph is 0.5 times that of the interaction target;
Firstly, monitoring the coincidence degree of the position of the gaze point of a tested person and the position area of the interaction target within 400ms to trigger the color-changing feedback of an auxiliary trigger graph, secondly, detecting that the direction of the gaze track of the tested person is transferred to the area of the auxiliary trigger graph, and detecting that the position of the gaze point of the tested person coincides with the position of the auxiliary trigger graph within 300ms to trigger the triggering behavior of the interaction target;
s2, target selection task verification and technology comparison based on target auxiliary staring trigger
Designing annular menu layout under different space densities, carrying out target trigger task verification based on target auxiliary staring trigger, and comparing with other eye control trigger technologies to determine the dominant space trigger state and the inferior trigger space state of the target auxiliary staring trigger;
Setting an interaction target triggering task aiming at an eye control interaction interface, designing icons of an annular menu as interaction targets, designing different space layout forms including interaction target density and space distribution areas of the menu, determining eye control selection steps and platform acquisition data under the setting tasks, such as triggering time, triggering accuracy and pupil diameter data, combining the experimental platforms in the step S1 to develop the interaction target triggering task, comparing the proposed target auxiliary staring eye control triggering method with the traditional staring triggering method and eye potential triggering method, definitely including recruiting eye control interaction experimental flows of a tested experiment, a training experiment and a formal experiment, acquiring experimental platform data, and comparing adaptation scenes of different methods;
s3, coding optimization strategy of auxiliary trigger graph under high-density space
Optimizing the inferior trigger space state obtained in the step S2, enhancing the comprehensive trigger performance of the target auxiliary staring method, specifically simulating the inferior trigger space state, setting the codes (space position and size) of the multi-level auxiliary trigger graph, exploring the dominant coding mode for enhancing the trigger efficiency, and providing a multi-dimensional target auxiliary staring trigger coding strategy.
Preferably, the design of the menu interface with the annular layout captures the position, angle and movement speed data of the head of the user in real time through a plurality of high-sensitivity Inertial Measurement Units (IMUs) and depth cameras integrated in a Virtual Reality (VR) helmet, so that the position and direction of the menu interface are dynamically adjusted, synchronous binding of the menu interface and the movement of the head of the user is realized, and the menu interface can respond in time and move correspondingly in the field of view of the user when the head of the user moves.
Preferably, the annular menu is composed of an interaction target and an auxiliary trigger pattern, wherein the interaction target is positioned in the annular menu, the auxiliary trigger pattern is positioned at the outer side of the annular menu, the center of the auxiliary trigger pattern is positioned at the connecting line between the center of the menu and the center of the interaction target, and the interval between the two interaction targets and the auxiliary trigger pattern is kept 0.5 times of the width of the interaction target;
the interactive object and the auxiliary trigger graphic are set to gray, wherein one randomly occurring blue icon is set as the interactive object to be triggered.
Preferably, the target assists in triggering a trigger procedure design of the gaze trigger target;
The visual areas of the interaction target and the auxiliary triggering object are visual line detection areas, wherein the auxiliary triggering image is used as a visual prompt and a 'confirmation button' of a triggering process, the specific triggering process is that a user gazes at a blue triggering target firstly, the coincidence degree of the gaze point position of a tested person and the position area of the interaction target is monitored within 400ms to trigger the color change feedback of the auxiliary triggering image, if the detected gaze track direction is transferred to the auxiliary triggering image area, and the position of the tested gaze point coincides with the position of the auxiliary triggering image within 300ms to trigger the triggering action of the interaction target, and the successful triggering interaction target is changed into green feedback.
Preferably, the menu space layout design is realized by adjusting the interaction target density and the space distribution area of the menu;
different numbers of interactive targets (3, 4, 5, 6, 7 and 8) are arranged in the same area, and the spatial distribution area (defined as 60-degree field angle and 45-degree field angle area) of the menu is set.
Preferably, the experiment adopts a project label to trigger a task, and a participant is required to quickly and accurately find a blue interaction target in the annular menu;
The participant completes the target triggering after finding the blue target button, if the correct button is triggered, the button will turn green, otherwise, it will turn red, and the experimental platform records the triggering time, the triggering accuracy and the pupil diameter data.
Preferably, the experimental task execution stage;
The tested person executes eye control target triggering experiments, the experiments are divided into three groups, the three groups adopt a target auxiliary staring triggering method and a traditional staring triggering (only through a method of detecting the region overlapping ratio) and an eye potential triggering mode (a method of detecting the sight moving track), in the triggering process, firstly, a black fixed cross ("+") is displayed for 500 milliseconds, then after a blank screen of 1000 milliseconds, a triggering interface with a ring menu appears, at the moment, a timer is started, the task of a participant is to find a blue target button and trigger the blue target button, if a successful target icon is triggered, the icon becomes blue, if a non-target icon is triggered, the icon becomes red, once the target is triggered, the timer is stopped, the experiment program records the triggering time, and then the next experiment is started;
ending the prompting stage of the experimental task;
after the user finishes the experiment, the screen center prompts the experiment to be finished, and a main trial staff helps to take down the VR equipment and fill in the NASA-TLX questionnaire and the ASQ questionnaire;
Experimental data processing and analysis;
And collecting trigger time, trigger accuracy and pupil diameter data through experiments, wherein the trigger time in the experiments is acquired by a program timer, and is the interval between the occurrence of the moment of triggering an interactive target by a search interface and a participant, and 5 virtual interface designers preliminarily screen out the dominant space trigger state and the inferior trigger space state of the auxiliary staring trigger of the target according to the trigger time, the trigger accuracy and the pupil diameter.
Preferably, the obtained inferior trigger space result of the target auxiliary staring method sets typical optimized space parameters (namely high density and small space layout), sets the codes (space position and size) of the multi-level auxiliary trigger graphics, and develops target trigger experiments under the target auxiliary staring method based on different auxiliary graphics coding states.
Preferably, the target triggers experimental data processing;
The significance, interaction and the like of the size and the relative position of the auxiliary trigger graph on the trigger efficiency are analyzed, the trigger efficiency and the accuracy condition under each coding condition are further analyzed, visualization is carried out, the visualization result can divide the coding strategy with the efficiency priority or the accuracy priority, and the coding optimization strategy for improving the target auxiliary staring method under the high-density space is provided for the user.
Compared with the prior art, the invention has the beneficial effects that:
(1) Designing a target auxiliary staring method, constructing a virtual reality eye control interaction experimental platform, realizing the layout of an eye control interaction target and an auxiliary trigger graph and eye control interaction operation, and adjusting platform parameters to realize efficient interaction target 'selection' efficiency;
(2) The experimental platform was developed using c# and Unity engines and run at HTCVIVEProEye (a virtual reality helmet equipped with a Tobii eye tracking module, sampling rate 90HZ, tracking field angle 110 °). The display card Injeida GeForce1060 above, the processor Inteli above, and the memory above 4GB connected with the computer;
(3) Adopting a ring-shaped display fixed menu form (namely, the menu is always in the visual field of a user), wherein an interactive target is a ring-shaped icon, positioning the interactive target in a unit environment through a coordinate point of a central position, setting an adjustable accuracy threshold value of continuous position detection to be 0.9, and avoiding interference of blink and eye jump visual behaviors to trigger accuracy;
(4) Each test was line-of-sight calibrated by a HTCVIVEProEye nine-point eye-control calibration procedure. After calibration, entering an experiment prompt interface for prompting the start of a tested experiment and focusing attention, calculating the average trigger time of each participant by using statistical analysis software SPSS23.0 (IBM company, new York, U.S.), removing extreme values in an experiment result, namely values deviating from an average value by +/-0.5 standard deviations, and then carrying out mixed method analysis on three parts of data to carry out main effect analysis and interaction analysis under each index;
(5) Based on the Fitts' law, we translate the trigger time and coding parameters into throughput as an indicator of the evaluation efficiency. The evaluation index of the experiment comprises accuracy and throughput.
Drawings
FIG. 1 is a schematic diagram of the target-assisted gaze triggering method construction and optimization of the present invention;
FIG. 2 is a schematic diagram of a target-assisted gaze triggering procedure of the present invention;
FIG. 3 is a schematic diagram of three eye-controlled triggering techniques of the present invention;
FIG. 4 is a schematic view of the annular menu layout view angle and target number of the present invention;
FIG. 5 is a flow chart of a target trigger experiment of the present invention;
FIG. 6 is a graph of objective data results of a target trigger experiment of the present invention;
FIG. 7 is a graph of subjective data results of a target trigger experiment of the present invention;
FIG. 8 is a schematic diagram of an auxiliary trigger graphic encoding material modification of the present invention;
FIG. 9 is a schematic diagram of the results of the code optimization experiment of the present invention;
FIG. 10 is a schematic diagram of accuracy versus efficiency for different encodings of the present invention;
FIG. 11 is a schematic diagram of task demand code recommendations of the present invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The first embodiment of the invention provides the following technical scheme, namely a staring interaction method for accelerating VR target selection, as shown in figures 1-11, which discloses:
The experimental program was created using Unity and c# and run on HTCVIVEProEye, HTCVIVEProEye being a VR head set equipped with a Tobii eye tracking module. The head display records eye tracking information at a sampling rate of 90Hz, providing a 110 ° field of view. It is connected to a desktop computer driven by Intel Kaui 7-10800F and Inlet Kara GeForceRTX2060 graphics cards.
In order to reduce the gaze deviation during gaze triggering, we use the gaze detection tolerance method, i.e. to set an adjustable accurate threshold for continuous position detection, to ensure that the target is successfully triggered when the threshold is reached. In a formal experiment, the threshold is adjusted to 90%, and at most 10% of data is allowed to be located outside the trigger area within a specified stay time, and a ring-shaped display staring trigger menu form is adopted (namely, the menu is always in the field of view of a user), so that an interaction target is a ring-shaped icon, and the interaction target is positioned in a unite environment through a coordinate point of a central position. In an AR/VR environment, menus can be classified into three types, a surrounding gaze trigger menu (i.e., a menu that gaze triggers in the virtual world), a world gaze trigger menu (i.e., a menu that gaze triggers on a retrievable object), and a display gaze trigger menu (i.e., a menu that follows the movements of the user's head). In the display of the gaze-triggered menu, the menu gaze is triggered on the screen, always within the user's field of view, providing an intuitive and quick method of operation. However, in the "world-gaze trigger menu", the menu is attached to the interactive object, which may change the distance from the menu due to the movement of the user, resulting in inconsistency of experimental control. Furthermore, previous studies have shown that head movements have a close coupling relationship with gaze. In this study, we wanted to mitigate the effects of unintentional head movements. Therefore, we validated our experiment in the "display-gaze trigger menu" setting;
The method comprises the steps of designing a ring-shaped menu form, wherein the menu consists of an interaction target and an auxiliary trigger graph, specifically, the auxiliary trigger graph adopts a round block similar to an interaction object, is positioned on one side of the interaction target away from the circle center, and keeps a distance which is 0.3 times the width of the interaction object from the interaction target. The auxiliary trigger object is placed outside the annular menu instead of inside for two reasons, namely, to prevent visual interference with the interaction target, and to maintain the consistency and intuitiveness of the gaze movement path. The method uses the auxiliary trigger graph as a visual prompt and a 'confirmation button' of a trigger process, simplifies the selection process and solves the problem of unintentional selection frequently encountered in traditional staring type interaction, the trigger process of the target auxiliary staring trigger target is divided into four steps, namely, a first step, target interaction object searching, requiring a user to find a target interaction object in a ring menu, testing the visual searching capability, designing a prominent blue (RGB: 87,182,247) in experiments for reducing the influence of the difference of individual visual searching capability on the result, and the prominent blue is considered to attract attention from bottom to top.
Second, the target "select" phase requires the user to look at for a period of time to lock the target interactive object. In this process we set the interaction target 400ms dwell time based on a single trigger event (150-600 ms) and test experience. The third and fourth steps, the "validation" phase is accomplished by looking at the auxiliary trigger object. The user decides whether to trigger the interactive object according to the feedback of the auxiliary triggering object in the step 2. If so, the user shifts his gaze from the interactive object to the auxiliary triggering object for a dwell time of 300ms, and triggering of the interactive object is completed. Successful triggering will turn the object green, while triggering a non-target object will turn it red.
In a second embodiment, as shown in fig. 1-11, the present invention provides a gaze interaction method for accelerating VR target selection, which discloses:
The design of the menu space layout is realized by adjusting the number of menu icons and the angle of view. Setting the number of the occurrence of the interaction targets, and determining 3, 4, 5, 6, 7 and 8 icon positions, wherein the range of the occurrence positions of the interaction targets is defined as a 60-degree field angle area and a 45-degree field angle area;
The experiment adopts a project label to trigger a task, and a participant is required to quickly and accurately search for a blue target button in the annular menu. 30 participants were recruited to perform the participation experiment. The participant completes the target activation after finding the blue target button, which will turn green if the correct button is activated. Otherwise, it will turn red. The experiment platform records the triggering time, the triggering accuracy and pupil diameter data;
and (3) experimental calibration and prompting. Each test was line-of-sight calibrated by a HTCVIVEProEye nine-point eye-control calibration procedure. Entering an experiment prompt interface after calibration, and prompting the tested experiment to start and concentrate attention;
And an experimental task execution stage. The tested person executes the eye control target triggering experiment, and compares the designed target auxiliary staring triggering method with the traditional staring triggering and eye potential triggering modes. In the trigger experiment, a black gaze trigger cross ("+") was first displayed for 500 ms, followed by a 1000 ms blank screen, and a trigger interface with a ring menu was presented, at which time a timer was started. The participant's task is to find the blue target button and trigger it, if a successful target icon is triggered, the icon will turn blue, and if a non-target icon is triggered, the icon will turn red. Once the target is triggered, the timer stops. The experimental program recorded the trigger time and then started the next experiment, which was designed with a hybrid design experiment design, using menu buttons to select tasks. Data for 9000 tests were recorded in total (30 participants x3 gaze input techniques x2 field angles x5 targets x10 replicates). The experimental platform records various indicators for each trial, including trigger accuracy, trigger time, and average pupil diameter during selection. Trigger time refers to the total time from the menu interface appearance to trigger completion (including 700 ms gaze trigger time);
And ending the prompting stage of the experimental task. After the user completes the experiment, the screen center prompts the experiment to be completed, and a main trial staff helps to take down the VR equipment and fill in a 'scene back questionnaire (ASQ)' and a 'NASATLX questionnaire', and a seven-point Likester scale is adopted to collect data on the aspects of user experience and cognitive load;
Experimental data processing and analysis. The experiment collects trigger time, trigger accuracy and pupil diameter data. The experimental procedure and questionnaire design was intended to collect objective and subjective data under various conditions. Analysis of these data mainly involves two steps. First, outliers are found and then excluded from analysis. The experimenter found and removed participants (2 persons) who had significant deviations in accuracy. Furthermore, data points that exceeded twice the standard deviation of the mean (approximately 8% of the total data) were also excluded in each set of indicators. Second, the accuracy was analyzed by repeated measures of variance of the hybrid design, which included two intra-group variables (i.e., FOV and number of objects) and one inter-group variable (i.e., gaze input technique). Lattice Lin Haosi-gate correction is used when the sphericity assumption is violated. These data were analyzed using SPSS statistics 24. The results of the analysis of variance test (F represents the level of significant difference between groups, p represents the level of significance of the test) were used to elucidate the significance of each variable and whether the two independent variables affected each other. In this study, P values less than 0.05 represent statistically significant.
Experimental results and discussion
Trigger accuracy results there is a significant main effect of gaze input technique (F (1,560) =512.01, p=0.000, η 2 =0.434). The trigger accuracy of the eye gesture input is highest, followed by gaze-assisted trigger, while the trigger accuracy of gaze trigger is lowest. The target number (F (4,2240) =164.87, p=0.000, η 2 =0.227) also has a significant impact on the trigger accuracy, which gradually decreases as the target number increases. Analysis of variance also shows that there is significant interaction with the target number x eye-controlled input technique (F (8,2240) =25.36, p=0.000, η 2 =0.083). This interaction indicates that as the number of targets increases, the trigger accuracy of all three eye-controlled input techniques decreases. Especially when the number of targets exceeds 6, the accuracy of gaze triggering and target assisted gaze triggering may be significantly reduced. It follows that, with the same number of targets, gaze triggering is the lowest in accuracy and gaze gestures are the highest in accuracy. In the case where the target number is the same, the accuracy decreases when the angle of view changes from 60 ° to 45 °. No interaction between FOV and gaze input technique was observed (F (2, 560) =0.86, p=0.425, η 2 =0.003), nor was interaction between FOV and number of objects F (4,2240) =1.485, p=0.204, η 2 =0.003). This shows that the impact of the field angle on trigger accuracy is not significantly different between gaze input technology and target number.
As a result of the trigger time, the eye-controlled input technique (F (1,536) =619.86, p=0.000, η 2 =0.698) has a significant main effect on the trigger time. The average result of each gaze input technique shows that both gaze triggers and target assisted gaze triggers can be kept for a relatively short trigger time (gaze triggers faster than target assisted gaze triggers), whereas the trigger time of gaze gestures is significantly longer than both. In addition, the target numbers (F (4,2144) =8.49, p=0.000, η2=0.016) and FOV (F (1, 536) =10.26, p=0.001, η2=0.019) also significantly affect the trigger time. Post hoc analysis showed that target numbers 7 and 8 were significantly different (p < 0.05) from target numbers 4,5 and 6, while there was no significant effect between target numbers 4,5 and 6 and between target numbers 7 and 8. When the number of objects exceeds 6, the response time tends to decrease, which indicates that the user would trigger the operation faster in this case. There is also a clear interaction between the angle of view and gaze input techniques (F (2,536) =3.11, p=0.045, η 2 =0.011). The eye-gesture triggers the longest trigger time and the stationary gesture triggers the shortest time. Furthermore, the triggering time of a 60 FOV is slightly advantageous compared to a 45 FOV. However, no interaction was observed in terms of the target number x field angle (F (4,2144) =0.58, p=0.697, η 2 =0.001), the target number x gaze input technique (F (8,2144) =1.485, p=0.120, η 2 =0.006), and the target number x gaze input technique x field angle (F (8,2144) =0.97, p=0.936, η 2 =0.001). This shows that when these factors interact, their combined impact on trigger time is not significant.
Pupil diameter data results analysis of variance results indicate that the eye-controlled input technique has significant main effects (F (2,539) =12.16, p=0.000, η 2 =0.043). Post hoc comparisons show that there is a significant difference between gaze trigger and target assisted gaze trigger (p < 0.05), as well as a significant difference between gaze trigger and gaze gesture (p < 0.05). However, no significant difference (p > 0.05) was observed between the target-assisted pointing and gaze gestures. FOV (F (1, 539) =8.49, p=0.004, η 2 =0.015) and number of targets (F (4,2156) =5.89, p=0.000, η 2 =0.011). Although the main effect of these two independent variables is significant, the pupil diameter difference under different descriptive conditions is very small, with the difference between the highest and lowest points being only between 0.5. Furthermore, there is a clear interaction between the object number x gaze input technique (F (8,5126) =2.27, p=0.020, η 2 =0.008) and the FOV x object number (F (4,2156) =3.06, p=0.016, η 2 =0.006) in terms of pupil diameter. Under the same number of conditions, the pupil diameter of the gaze-triggered target slightly exceeds that of the other two techniques. However, there is no significant interaction between the field angle and gaze input method (F (2,539) =0.49, p=0.611, η 2 =0.006) and between the field angle, target number and gaze input method (F (8,2156) =1.51, p=0.149, η 2 =0.006).
Subjective questionnaire results overall average score of ASQ scale indicated that eye potential (m=10.74, sd=4.52) scored highest, while gaze trigger (m=8.80, sd=3.83) scored lowest, indicating that the user thought the user experience of eye potential was best. Two indicators of ASQ-satisfaction and task difficulty also draw similar conclusions. In analyzing ASQ scores, repeated measured one-way anova and LSD corrected post hoc multiple comparisons were employed. Through multiple comparisons after the fact, there was a significant difference in task difficulty between gaze triggering and eye potential triggering (p < 0.05), indicating that most subjects considered the latter more challenging than the former. However, the remaining indicators did not show significant differences. NASATLX total scores indicate that the cognitive load is highest with eye potential (m=24.3, sd=2.55), followed by target-assisted gaze triggering (m=22.43, sd=3.68) and gaze triggering (m=17.83, sd=3.55). Compared to the other two techniques, eye potential is significantly improved in terms of body demand (p < 0.01), whereas gaze triggering scores in terms of body demand (p < 0.05), time demand (p < 0.001) and effort (p < 0.01) are significantly lower than gaze triggering techniques and eye potential. This highlights the significant advantage of gaze triggering in terms of cognitive load. However, gaze input methods do not show significant differences in their own performance (p > 0.05) and frustration (p > 0.05). In terms of subjective ranking, the subject believes that there is no significant difference in accuracy (p > 0.05) between the three input methods. But the participant is significantly more inclined to the gaze trigger method (preference: p > 0.05) than the gaze gesture.
The target triggering efficiency under the staring triggering is highest, but the accuracy is lowest, and the accuracy is obviously reduced along with the increase of the number of targets. In contrast, the eye potential is triggered with the highest accuracy, but the trigger time is slower, and the user preference is lower. At the same time, the overall trigger performance of target-assisted gaze triggering is more balanced in terms of accuracy, trigger time and cognitive load, and better in terms of subjective accuracy and preference. At the same time, the target-assisted gaze triggering also greatly alleviates the problem of Midams contact observed in gaze triggering, especially in scenes involving a large number of targets. Furthermore, challenges presented by high-density VR space (i.e., small field angles and large number of targets) highlight the necessity of optimizing design solutions in eye-controlled interactions.
In a third embodiment, as shown in fig. 1-11, the present invention provides a gaze interaction method for accelerating VR target selection, which discloses:
Typical optimized spatial parameters (i.e., number of menu buttons and angle of view) are set, and auxiliary trigger graphic coding parameters (size and relative position of the graphic) are set. To simulate a high density space scene we have used a ring menu with eight buttons with a 45 angle of view. In terms of menu selection tasks, the experimental process remains consistent with the target trigger experiment. However, the coding method of the auxiliary trigger pattern is changed, including adjusting the size and distance of the trigger target;
And carrying out target triggering experiments under target auxiliary staring methods based on different auxiliary graph coding states. We designed a 3 (auxiliary trigger pattern sizes: S, 0.8S, and 0.6S) x 3 (distance of interaction target from auxiliary trigger pattern: 0.3D, 0.9D, and 1.5D) intra-group experiment. Specifically, the distance of the shorter distance is 0.3 times (0.3D) the width of the interactive object, and the other two distances are 0.9D and 1.5D, respectively. In the menu selection task, the participants will randomly see various combinations of sizes and distances. The test platform collects the correct rate and the data during reaction;
The target triggers the experimental data processing. The significance, interaction and the like of the size and the relative position of the auxiliary trigger pattern on the trigger efficiency are analyzed, and the trigger efficiency and accuracy condition under each coding condition are further analyzed. Based on the Fitts' law, we translate the trigger time and coding parameters into throughput as an indicator of the evaluation efficiency. The throughput is calculated as follows:
first, a difficulty Index (ID) is calculated. The fez difficulty index for each state was calculated using shannon's formula, which is commonly used for tasks involving object localization and selection.
ID=log2(D/W+1)
In experiments we used a world-locked target, providing a number of possible ways to calculate the target distance (D) and target size (W). D represents the distance from the center coordinates of the target object to the center coordinates of the auxiliary trigger object, measured in three values. With respect to W, since an irregularly shaped auxiliary trigger object is used in the experiment, we choose to use the smallest rectangular width surrounding the target for unified representation. This representation also contains three values.
Throughput calculation the throughput per trial is calculated in bits per second by calculating the ID and the Movement Time (MT). We only focus on throughput, considering MT only when "triggering". MT measures the time required for the line of sight to transfer from the target to the auxiliary trigger object after the subject has gazed at the target. Notably, the MT does not include 300 milliseconds of stay on the auxiliary trigger object.
Throughput = ID/MT
In addition, the data filtering and analysis methods employed were consistent with experiment 1. About 6% of the outlier data points were excluded from the collected dataset and the analysis of variance was repeated using the same hybrid design. The evaluation index of the experiment comprises accuracy and throughput.
Trigger test results under different coding states
And (3) triggering accuracy results, namely performing one-way analysis of variance (ANOVA) on the triggering accuracy under all conditions. Lattice Lin Haosi-gate correction is used when the sphericity assumption is violated. The results show that the distance has a significant main effect (F (2,1654) =8.04, p=0.000, η 2 =0.010). As the distance between the auxiliary trigger graphic and the interactive object increases, the trigger accuracy increases. In addition, post hoc comparisons showed significant differences between 1.5D and 0.3D (p < 0.05) and also between 1.5D and 0.9D (p < 0.05). This result shows that 0.3D and 0.5D have no significant difference in the effect on accuracy. However, no significant difference (p > 0.05) was observed between 0.3D and 0.9D. The size of the auxiliary trigger pattern (F (2,1654) =38.28, p=0.000, η 2 =0.044) also has a significant impact on the trigger accuracy. Furthermore, the smaller the size, the higher the accuracy. The interaction between the size and distance of the auxiliary trigger objects (F (2,1654) =0.55, p=0.699, η 2 =0.001) is not statistically significant.
Throughput results analysis of variance results highlight the importance of the main effect of two factors. First, the distance has a significant impact on throughput (F (2,1654) =345.44, p=0.000, η 2 =0.295), indicating that the further the distance is, the lower the throughput. Furthermore, a consistent descent pattern is exhibited under all three dimensional conditions. Second, the size of AO (F (2,1654) =382.69, p=0.000, η 2 =0.316) also has a significant impact on throughput. The average result shows that under the same distance conditions, the larger size product throughput is higher, while the smaller size product throughput is lowest. The interaction between distance and size was also significant (F (4,1654) =5.68, p=0.000, η 2 =0.014).
The visualized result can divide the coding strategy with priority of efficiency or priority of accuracy, and a personalized coding recommendation scheme is provided for the user. The experiment aims at exploring the coding optimization strategy of the auxiliary trigger object in the auxiliary staring trigger of the target and aims at improving the overall trigger performance of the menu target. However, the results of the study indicate that there is a conflict between accuracy-oriented coding strategies and efficiency-oriented coding strategies. For example, the greater the distance between the auxiliary gaze pattern and the interactive object, the more advantageous it is to increase the triggering efficiency, but the accuracy of the triggering is reduced. Also, the smaller the size of the auxiliary trigger object, the higher the trigger accuracy but the lower the trigger efficiency.
To intuitively describe the accuracy and efficiency of each distance and size under different coding strategies, we draw an accuracy-throughput scatter plot illustrating the trigger performance profile for each participant. Furthermore, we further visualize the results. First, the coordinate system is divided into uniform block areas, probability density of each area is calculated using a nuclear density analysis method, and high density areas are mapped onto the block areas. The coordinate system is divided into uniform block areas, and the probability density of each area is calculated by adopting a nuclear density analysis method. The high density areas are then mapped onto these tile areas and represented by color patches. Thus, each color block represents a particular distance-size encoding condition. Secondly, all color blocks are drawn on a chart, so that the trigger performance under all coding conditions can be intuitively seen. Then we can pinpoint the area on the chart where gaze trigger prioritizes accuracy or efficiency. Thus, we can derive the corresponding distance size encoding form. For example, FIG. 11 shows the encoded version of the efficiency priority region correspondence, including 0.3D-S, 0.3D-0.8S, and 0.9D-S. In practical design work, determining specific selection preference according to task requirements is important to balance the accuracy and efficiency of triggering tasks. The corresponding coding form can be determined by this procedure.
Although the present invention has been described with reference to the foregoing embodiments, it will be apparent to those skilled in the art that modifications may be made to the embodiments described, or equivalents may be substituted for elements thereof, and any modifications, equivalents, improvements and changes may be made without departing from the spirit and principles of the present invention.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410879025.7A CN119105644A (en) | 2024-07-02 | 2024-07-02 | A gaze interaction method to accelerate VR target selection |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410879025.7A CN119105644A (en) | 2024-07-02 | 2024-07-02 | A gaze interaction method to accelerate VR target selection |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119105644A true CN119105644A (en) | 2024-12-10 |
Family
ID=93710694
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410879025.7A Pending CN119105644A (en) | 2024-07-02 | 2024-07-02 | A gaze interaction method to accelerate VR target selection |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119105644A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180130223A1 (en) * | 2016-07-01 | 2018-05-10 | Hashplay Inc. | Method and system for determining a region of interest of a user in a virtual environment |
| CN109683705A (en) * | 2018-11-30 | 2019-04-26 | 北京七鑫易维信息技术有限公司 | The methods, devices and systems of eyeball fixes control interactive controls |
| JP2021057869A (en) * | 2019-09-30 | 2021-04-08 | 株式会社ソニー・インタラクティブエンタテインメント | Image processing device, image display system, image data transfer device, and image compression method |
| CN113534950A (en) * | 2021-03-30 | 2021-10-22 | 北京航空航天大学 | A virtual object interaction method based on mixed reality technology |
| CN114911341A (en) * | 2022-04-21 | 2022-08-16 | 中国人民解放军国防科技大学 | Target selection method and system based on secondary triggering of eye gestures |
| CN115598831A (en) * | 2021-06-28 | 2023-01-13 | 见臻科技股份有限公司（Tw） | Optical systems and related methods for providing accurate eye tracking |
| CN117632330A (en) * | 2023-10-12 | 2024-03-01 | 浙江大学 | An interactive target layout method and system for eye control interface in virtual environment |
-
2024
- 2024-07-02 CN CN202410879025.7A patent/CN119105644A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180130223A1 (en) * | 2016-07-01 | 2018-05-10 | Hashplay Inc. | Method and system for determining a region of interest of a user in a virtual environment |
| CN109683705A (en) * | 2018-11-30 | 2019-04-26 | 北京七鑫易维信息技术有限公司 | The methods, devices and systems of eyeball fixes control interactive controls |
| JP2021057869A (en) * | 2019-09-30 | 2021-04-08 | 株式会社ソニー・インタラクティブエンタテインメント | Image processing device, image display system, image data transfer device, and image compression method |
| CN113534950A (en) * | 2021-03-30 | 2021-10-22 | 北京航空航天大学 | A virtual object interaction method based on mixed reality technology |
| CN115598831A (en) * | 2021-06-28 | 2023-01-13 | 见臻科技股份有限公司（Tw） | Optical systems and related methods for providing accurate eye tracking |
| CN114911341A (en) * | 2022-04-21 | 2022-08-16 | 中国人民解放军国防科技大学 | Target selection method and system based on secondary triggering of eye gestures |
| CN117632330A (en) * | 2023-10-12 | 2024-03-01 | 浙江大学 | An interactive target layout method and system for eye control interface in virtual environment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ashbrook et al. | MAGIC: a motion gesture design tool | |
| US20170049319A1 (en) | Vision-Based Diagnosis and Treatment | |
| US10456072B2 (en) | Image interpretation support apparatus and method | |
| Vogel et al. | Hand occlusion on a multi-touch tabletop | |
| TWI802456B (en) | Optical system with improved user experiemce and accurate eye-tracking and related method | |
| Shi et al. | Exploring head-based mode-switching in virtual reality | |
| CN117632330B (en) | A method and system for interactive target layout of eye-controlled interface in virtual environment | |
| Kumar et al. | Eye-controlled interfaces for multimedia interaction | |
| CN109700472A (en) | A kind of fatigue detection method, device, equipment and storage medium | |
| Wolf et al. | Gaze comes in handy: Predicting and preventing erroneous hand actions in ar-supported manual tasks | |
| Kosovicheva et al. | Looking ahead: When do you find the next item in foraging visual search? | |
| CN114911341B (en) | Target selection method and system based on secondary triggering of eye gestures | |
| Yi et al. | DEEP: 3D gaze pointing in virtual reality leveraging eyelid movement | |
| Lim et al. | Development of gaze tracking interface for controlling 3D contents | |
| CN119105644A (en) | A gaze interaction method to accelerate VR target selection | |
| CN117742490A (en) | An eye-movement selection interaction intention recognition method | |
| Lin et al. | A novel device for head gesture measurement system in combination with eye-controlled human–machine interface | |
| EP3922166B1 (en) | Display device, display method and display program | |
| Kubota et al. | Incorporating visual field characteristics into a saliency map | |
| Niu et al. | Research on click enhancement strategy of hand-eye dual-channel human-computer interaction system: Trade-off between sensing area and area cursor | |
| Frata Furlan Peres et al. | Methods for Evaluating Immersive 3D Virtual Environments: a Systematic Literature Review | |
| Skovsgaard | Noise challenges in monomodal gaze interaction | |
| Severitt et al. | The interplay of user preference and precision in different gaze-based interaction methods | |
| Sidenmark | Coordinated eye and head movements for gaze interaction in 3D environments | |
| US20240252082A1 (en) | Method and system for providing a plurality of consecutive tests for a test person |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |