CN114035318A

CN114035318A - Compact eyeball tracking system and method

Info

Publication number: CN114035318A
Application number: CN202111360252.1A
Authority: CN
Inventors: 赵照; 闻棕择
Original assignee: Hefei Xinfoo Sensor Technology Co ltd
Current assignee: Hefei Xinfoo Sensor Technology Co ltd
Priority date: 2021-11-17
Filing date: 2021-11-17
Publication date: 2022-02-11

Abstract

The application discloses a compact eyeball tracking system and a method, the system comprises a polarization module, a Bragg grating is arranged to guide the IR light in a specific polarization state to diffract to the eyeball of a user, and an IR sensor module is used for receiving the IR light reflected by the mirror surface and the scattering of the eye, generating an image of the eye and detecting the position and the movement of the eye of the user. By utilizing the characteristic that the angle of the IR diffracted light is greatly changed compared with the incident angle of the IR light before diffraction, the installation position or angle of the Bragg grating can be set to occupy the space of the HMD as small as possible.

Description

Compact eyeball tracking system and method

Technical Field

The application relates to the technical field of head-mounted display, in particular to a compact eyeball tracking system and method.

Background

In VR, AR, or MR Head Mounted Display (HMD) devices or other wearable devices, eye tracking technology may enable the device to obtain a user eye gaze direction to enable an accurate, convenient, and intelligent way of human-computer interaction, e.g., display of images on the HMD may be controlled by the gaze direction. In addition, in the laser projection imaging process, because the design and process limitations of the MEMS scanning mirror in the prior art are very difficult to ensure higher scanning frequency to achieve a satisfactory imaging effect, the problem can be solved by means of an eyeball tracking technology, the eye gazing direction of a user is acquired by utilizing an eyeball tracking system in equipment, and an image with local high resolution only in the eye gazing direction of the glasses can be generated by adjusting the phase difference between continuous frames of projection scanning or adjusting the vertical scanning frequency in a projection visual field, so that the difficulty encountered by the MEMS scanning mirror in the prior art is overcome.

However, the addition of an eye tracking system during the development of an HMD, particularly in the optical waveguide portion, results in a corresponding increase in the volume and weight of the device, and the inclusion of optical components in the eye tracking system may also hinder the user's normal viewing path, negatively impacting the user's viewing experience and comfort during head-on.

Disclosure of Invention

In view of the above, the present application provides a compact eye tracking system and method, which is small and has no obstruction to the normal viewing path of the user.

In order to solve the technical problem, the following technical scheme is adopted in the application:

in a first aspect, the present application provides a compact eye tracking system, the system comprising:

a light source module configured to emit Infrared (IR) light to detect a user's eyeball, and emit visible light to form a visible image;

a polarization module configured to at least convert the IR light into IR light having a particular polarization state;

a light scanning module configured to direct at least IR light to propagate along an IR optical path to a diffraction grating;

an optical waveguide module configured to include at least two optical waveguides for displaying a visible image formed by visible light;

a diffraction grating configured to diffract the IR light having a specific polarization state guided by the light scanning module to a user's eyeball;

an IR light sensor module configured to receive IR light reflected by a user's eye.

Optionally, the polarization module comprises:

a fast polarization modulator configured to switch the IR light back and forth between two orthogonal linear polarization states at an appropriate switching frequency;

a quarter wave plate configured to convert the linearly polarized light from the fast polarization modulator into circularly polarized light of its corresponding handedness.

Optionally, the optical scanning module comprises at least one MEMS scanning mirror.

Alternatively, the diffraction grating may be a bragg grating configured to be sensitive to left-handed circularly polarized light or right-handed circularly polarized light, such that left-handed circularly polarized-1 order or right-handed circularly polarized +1 order diffracted light is diffracted when passing through the bragg grating.

Optionally, the diffraction grating may be configured to abut the optical waveguide.

Alternatively, the diffraction grating may be configured not to be in contact with the optical waveguide.

Optionally, the IR light path does not pass through the optical waveguide.

Optionally, the system further comprises a mirror that reflects only IR light to the diffraction grating and allows visible light to pass through.

Optionally, one face of the diffraction grating has a light absorbing material.

In a second aspect, the present application provides a compact eye tracking method, the method comprising:

emitting IR light from a light source module;

the polarization module converts the IR light into IR light with a specific polarization state;

the light scanning module guides the IR light to propagate to the diffraction grating along the IR light path;

the diffraction grating diffracts the IR light with a specific polarization state guided by the light scanning module to the eyeball of the user;

the IR sensor module receives IR light reflected by the user's eye.

Compared with the prior art, the method has the following beneficial effects:

based on the above technical solutions, the present application provides a compact eye tracking system and method, the system includes a polarization module for converting IR light into IR light with a specific polarization state, guiding the IR light with the specific polarization state to diffract to the user's eyes by arranging a bragg grating, and an IR sensor module for receiving the IR light reflected by the eyes in a specular reflection manner and a scattered reflection manner, generating an image of the eyes, and detecting the position and the movement of the eyes of the user. By utilizing the characteristic that the diffraction angle of the IR diffracted light is greatly changed from the incident angle of the IR light before diffraction, the mounting position or angle of the bragg grating can be set to occupy the space of the HMD as small as possible.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the present application, and other drawings can be obtained by those skilled in the art without creative efforts.

Fig. 1 is a schematic view of a compact eye tracking system according to an embodiment of the present application.

Fig. 2 is a schematic view of another compact eye tracking system provided by the embodiment of the present application.

FIG. 3 is a schematic diagram of IR diffraction provided by an embodiment of the present application.

Fig. 4 is a schematic view of another compact eye tracking system provided by the embodiment of the present application.

Fig. 5 is a schematic view of another compact eye tracking system provided by the embodiment of the present application.

Fig. 6 is a schematic view of a compact eyeball tracking method provided by an embodiment of the application.

Detailed Description

In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.

Fig. 1 is a schematic view of a compact eye tracking system according to an embodiment of the present application. In the example of fig. 1, the compact eye tracking system comprises a control module 10, a light source module 11, a polarization module 12, a light scanning module 13, an optical waveguide module 14, an Infrared (IR) sensor module, and an eye 16.

The light source module 11 is used to generate visible light projected in a two-dimensional field of view and forming a visible image and IR light for eye tracking. The light source module 11 may be a laser module comprising a plurality of individual lasers, which may be a red laser 112, a blue laser 113, a green laser 114 and an IR laser 111. Among them, the red laser 112, the blue laser 113, and the green laser 114 are used for visible light projection display, and the IR laser 111 is used for emitting IR light for tracking the position and direction of the eyeball.

It should be noted that the present embodiment is capable of tracking the user's eye using light waves having any wavelength and should not be limited to use with IR light. As an example, the red laser 112 may emit red laser wavelengths in the range of 600nm to 650nm, the blue laser 113 may emit blue laser wavelengths in the range of 430nm to 480nm, the green laser 114 may emit green laser wavelengths in the range of 500nm to 550nm, and the IR laser may emit IR wavelengths in the range of 780nm to 1000 nm.

The lasers of the same wavelength range may include a plurality of sub-lasers, for example, the red laser 112, the blue laser 113, and the green laser 114 may each include two sub-lasers. Multiple sub-lasers may be arranged on the same laser die via a suitable lithographic process.

The polarization module 12 is used to convert the light emitted by the laser into polarized light of a specific direction, the polarization module 12 may include a fast polarization modulator 121 and a quarter-wave plate 122, the fast polarization modulator 121 may be an LC-based polarization modulator/rotator, the light polarization is controlled by an externally applied driving voltage, and the light may be switched back and forth between two orthogonal linear polarization states at a suitable switching frequency.

The quarter-wave plate 122 converts the linear polarization from the fast polarization modulator 121 into its corresponding handedness circular polarization, i.e., left-hand circular polarization or right-hand circular polarization. The quarter-wave plate cycles between generating left-handed circularly polarized light and right-handed circularly polarized light based on the linear polarization received from the fast polarization modulator 121. It should be understood that the polarization module 12 may include additional or alternative modules and components for generating time-multiplexed switching between left-handed and right-handed circularly polarized light.

The optical scanning module 13 projects light received from the laser out of the optical scanning module, so that visible light enters the waveguide module 14 to form a visible image, and IR light is used for eye tracking. The optical scanning module 13 may comprise a single MEMS scanning mirror that is vibratable in both a horizontal axis and a vertical axis, may comprise dual MEMS scanning mirrors that are each vibratable in both a horizontal axis and a vertical axis, respectively, or may be a multi-MEMS scanning mirror module formed by a combination thereof.

This embodiment exemplifies a single axis deflection dual MEMS scanning mirror, the first scanning mirror 131 reflecting light received from the laser to the second scanning mirror 132. The first scanning mirror 131 can be configured to scan in a horizontal or x-axis direction and the second mirror scan 106 can be configured to scan in a vertical or y-axis direction such that visible light is ultimately projected throughout the two-dimensional field of view and forms a visible image and projects IR light out of the optical scanning module for eye tracking. It should be noted that the scanning mode of the first scanning mirror 131 may be vertical scanning, and the scanning mode of the second scanning mirror 132 may be horizontal scanning.

The optical waveguide module 14 is used for the transmission of red, green and blue light corresponding to an image from the input pupil to the output pupil. The optical waveguide module 14 may comprise two waveguides, in the case of two waveguides, one for guiding red and green light, and another for guiding green and blue light, or three waveguides, in the case of three waveguides, for guiding red, blue and green light, respectively, to pass from the input pupil to the output pupil.

The IR sensor module 15 is used to receive IR light from specular and diffuse reflections of the eye, generate electrical signals, convert the electrical signals to a digital signal format, and perform additional processing on the digital signals to generate an image of the user's eye, the image including position information of the eye. Therefore, when a plurality of eye images are generated, by measuring the triangular displacement of the eyes in the plurality of images, the eye position and movement of the user can be detected. The IR sensor module 15 may include one or more silicon photomultiplier tube (SiPM) sensors, may also include one or more PIN junction sensors, and may also be a hybrid of the two types of sensors. SiPM sensors employ avalanche photodiodes, operating in avalanche mode upon receiving photons, have a very fast response and high gain signal.

The control module 10 is used for controlling the working states of the scanning mirror in the light scanning module 13, the laser in the light source module 11, and the polarization component in the polarization module 12.

Fig. 2 is a schematic view of another compact eye tracking system provided by the embodiment of the present application. In the example of fig. 2, the optical waveguide module 14 is exemplified as including three waveguides, wherein the optical waveguide module 14 includes a first waveguide 143, a second waveguide 144, a third waveguide 145 and a first bragg grating 146.

The present embodiment exemplifies the case where the optical waveguide module 14 is constituted by three waveguide assemblies, and the

optical waveguides

143, 144, and 145 may be used to transmit red light, green light, and blue light corresponding to an image from the input pupil to the output pupil, respectively.

In particular, the input coupler 1431 of the waveguide 143 may be used to couple light in the red wavelength range in the corresponding image into the waveguide 143, and the output coupler 1432 of the waveguide 143 may be used to couple light in the wavelength range corresponding to red in the image out of the waveguide 143 by total internal reflection from the input coupler 1431 to the output coupler 1432. Similarly, the input coupler 1441 of the waveguide 144 may be used to couple light in the green wavelength range in the corresponding image into the waveguide 144, and the output coupler 1442 of the waveguide 144 may be used to couple light in the green wavelength range corresponding to the image out of the waveguide 144 by total internal reflection from the input coupler 1441 to the output coupler 1442. The input coupler 1451 of the waveguide 145 may be used to couple light in the blue wavelength range in the corresponding image into the waveguide 145, and the output coupler 1452 of the waveguide 145 may be used to couple light in the blue wavelength range in the corresponding image out of the waveguide 145 by total internal reflection from the input coupler 1451 to the output coupler 1452. Visible light 141 corresponding to the display image is finally projected into the human eye.

The first bragg grating 146 is used to diffract IR light of a particular polarization state. The first bragg grating 146 may be disposed on a surface of one side of the waveguide 145 and may be formed of a liquid crystal material having a grating period, a thickness, and/or an average refractive index such that the IR light wavelength satisfies a bragg condition. Which is formed by patterning a film having optical anisotropy, so that the first bragg grating 146 may be sensitive to left-handed circularly polarized light or right-handed circularly polarized light, so that left-handed circularly polarized-1 order or right-handed circularly polarized +1 order diffracted light is diffracted when passing through the first bragg grating 146.

Depending on the configuration of the polarization module 12, the first bragg grating 146 may be configured to diffract the incident light allowing left circularly polarized light to maximize-1 order or to diffract right circularly polarized light to maximize +1 order in order to match the IR light having a specific polarization state exiting the polarization module 12.

FIG. 3 is a schematic diagram of IR diffraction provided by an embodiment of the present application, and referring to FIG. 3, after the IR light passes through the MEMS scanning mirror of the optical scanning system, it is incident on the surface of the first Bragg grating 146 with a range of incident angles, such as +/-15 degrees with respect to the normal, in which φ 1 and φ 3 represent boundary incident angles of the set of IR light paths with different incident angles, and is diffracted through the first Bragg grating 146 to angles φ 2 and φ 4 with respect to the normal vector, and the diffraction angles φ 2 and φ 4 of the IR diffracted light 142 can be implemented by selecting the grating thickness, period and refractive index of the first Bragg grating 146 appropriately to ensure that the light paths of the IR diffracted light 142 with the diffraction angles φ 2 and φ 4 sufficiently cover the user's eyeball.

The compact eye tracking system design provided by the embodiment has the advantages that the thickness, the period and the refractive index of the first bragg grating 146 are reasonably regulated, so that the IR diffracted light 142 with the diffraction angles phi 2 and phi 4 can cover the user's eyes, and the requirements of the eye tracking technology are met. The first bragg grating 146 is not in the user's viewing path and does not disrupt the user viewing experience.

Fig. 4 is a schematic view of another compact eye tracking system provided by the embodiment of the present application. In the example of FIG. 4, the optical waveguide module 14 includes a second Bragg grating 147.

The second bragg grating 147 may be disposed in the optical waveguide module 14, and in particular, may be disposed closer to the user's eyeball than the third waveguide 145 that is closest to the user's eyeball side. The IR light may pass through the first waveguide 143, the second waveguide 144, and the third waveguide 145 to the second bragg grating 147 surface.

The second bragg grating 147 may be sensitive to left-handed circularly polarized light or right-handed circularly polarized light such that left-handed circularly polarized-1 order or right-handed circularly polarized +1 order diffracted light is diffracted when passing through the second bragg grating 147.

The second bragg grating 147 may be arranged to diffract incident light allowing left circularly polarized light to maximize-1 order or diffract right circularly polarized light to maximize +1 order, in order to match IR light having a specific polarization state exiting from the polarization module 12, according to the arrangement requirements of the polarization module 12.

The back surface of the second bragg grating 147, i.e., the surface opposite to the interface receiving the incident light, may be coated with a light absorbing material to perform a beam collection function.

After the IR light passes through the MEMS scanning mirror of the optical scanning system, it is incident on the surface of the second bragg grating 147 with a range of incident angles, for example +/-15 degrees from the normal, in the figure, Φ 2 and Φ 4 represent boundary incident angles of the IR light path set with different incident angles, and diffracted to angles Φ 2 to Φ 4 relative to the normal vector when passing through the second bragg grating 147, the diffraction angles Φ 2 to Φ 4 of the IR diffracted light 142 can be selected by reasonably adjusting the grating thickness, period and refractive index of the second bragg grating 147, so as to ensure that the light paths of the IR diffracted light 142 with the diffraction angles Φ 2 to Φ 4 sufficiently cover the user's eyeball.

The compact eye tracking system provided by this embodiment has the advantages that by reasonably adjusting the grating thickness, period and refractive index of the second bragg grating 147, the IR diffracted light 142 having the diffraction angles Φ 2 to Φ 4 can cover the user's eye as much as possible on the premise that the HMD space occupied by the second bragg grating 147 is small, that is, by utilizing the characteristic that the diffraction angle of the IR diffracted light 142 is greatly changed from the incident angle of the IR light before diffraction, the mounting position or angle of the second bragg grating 147 is set to occupy the HMD space as little as possible. As shown in fig. 4, the front surface of the second bragg grating 147 and the side surface of the optical waveguide module 14 may be disposed at a smaller angle as possible, so as to reduce the volume of the protrusion of the optical waveguide module 14. The second bragg grating 147 is not in the user's viewing path and does not disrupt the user viewing experience.

Fig. 5 is a schematic view of another compact eye tracking system provided by the embodiment of the present application. In the example of FIG. 5, the optical waveguide module 14 includes a third Bragg grating 148 and a mirror 149.

The mirror 149 is specially configured to reflect only IR light to the bottom of the waveguide module 14 and allow visible light to enter the input couplers in the waveguide module 14 through the mirror 149. The IR light is directed by a mirror 149 to the front of the third bragg grating 148 at the bottom of the waveguide module 14.

The third bragg grating 148 may be sensitive to left-handed circularly polarized light or right-handed circularly polarized light such that left-handed circularly polarized-1 order or right-handed circularly polarized +1 order diffracted light is diffracted as it passes through the third bragg grating 148.

Depending on the configuration of the polarization module 12, the third bragg grating 148 may be configured to diffract the incident light allowing left circularly polarized light to maximize-1 order, or to diffract right circularly polarized light to maximize +1 order, in order to match the IR light having a specific polarization state exiting the polarization module 12.

The back surface of the third bragg grating 148, i.e., the surface opposite the interface receiving the incident light, may be coated with a light absorbing material to provide beam collection.

The compact eye tracking system provided by this embodiment has the advantages that by reasonably adjusting the grating thickness, period, and refractive index of the third bragg grating 148, the IR diffracted light 142 having the diffraction angle of Φ 2 to Φ 4 can cover the user's eye as far as possible on the premise that the third bragg grating 148 occupies a small HMD space, that is, by utilizing the characteristic that the diffraction angle of the IR diffracted light 142 is greatly changed from the incident angle of the IR light before diffraction, the mounting position or angle of the third bragg grating 148 is set to occupy the HMD space as little as possible, as shown in fig. 5, the upper surface of the third bragg grating 148 can reach or approach to be parallel to the bottom surface of the optical waveguide module 14, and the requirement that the IR light covers the eye without being inclined to a certain angle is met. In addition, this design may also reduce potential IR light loss that occurs when IR light traverses the visible light guide 14. The third bragg grating 148 is not in the user's viewing path and does not disrupt the user viewing experience.

Fig. 6 is a schematic view of a compact eyeball tracking method provided by an embodiment of the application. The method includes S601 emitting IR light from a light source module; s602, the polarization module converts the IR light into the IR light with a specific polarization state; s603, the light scanning module guides the IR light to propagate to the diffraction grating along the IR light path; s604, the diffraction grating diffracts the IR light guided out by the light scanning module to the eyeball of the user; the S605 IR sensor module receives IR light reflected from the eye.

The foregoing is only a preferred embodiment of the present invention, and although the present invention has been disclosed in the preferred embodiments, it is not intended to limit the present invention. Those skilled in the art can make numerous possible variations and modifications to the present teachings, or modify equivalent embodiments to equivalent variations, without departing from the scope of the present teachings, using the methods and techniques disclosed above. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.

Claims

1. A compact eye tracking system, the system comprising:

2. The system of claim 1, wherein the polarization module comprises:

3. The system of claim 1, wherein the light scanning module comprises at least one MEMS scanning mirror.

4. The system of claim 1, wherein the diffraction grating is a bragg grating configured to be sensitive to left-handed circularly polarized light or right-handed circularly polarized light such that left-handed circularly polarized-1 order or right-handed circularly polarized +1 order diffracted light is diffracted when passing through the bragg grating.

5. The system of claim 1, wherein the diffraction grating is configured to be positioned proximate the optical waveguide.

6. The system of claim 1, wherein the diffraction grating is configurable to be out of contact with the optical waveguide.

7. The system of claim 6, wherein the IR light path does not pass through the optical waveguide.

8. The system of claim 7, further comprising a mirror configured to reflect IR light to the diffraction grating and the mirror allows visible light to pass through.

9. The system of claim 1, wherein one face of the diffraction grating has a light absorbing material.

10. A compact eye tracking method, the method comprising:

emitting IR light from a light source module;

the IR sensor module receives IR light reflected by the user's eye.