CN116482854A - Eye tracking using self-mixing interferometry - Google Patents

Abstract

This disclosure relates to "Eye Tracking Using Self-Mixing Interferometry". An eye-tracking device includes a head-mounted frame, an optical sensor subsystem mounted to the head-mounted frame, and a processor. The optical sensor subsystem includes a set of one or more SMI sensors. The processor is configured to operate the optical sensor subsystem to cause the set of one or more SMI sensors to emit a set of one or more light beams toward an eye of a user; receive a set of one or more SMI signals from the set of one or more SMI sensors; and use the set of one or more SMI signals to track movement of the eye.

Description

Eye tracking using self-mixing interferometry

This application is a divisional application of Chinese patent application 202211169329.1 with a filing date of September 22, 2022 and an invention title of "Eye Tracking Using Self-Mixing Interferometry".

Cross References to Related Applications

This application is a non-provisional application of U.S. Provisional Patent Application No. 63/247,188 filed September 22, 2021, and claims the benefit of said U.S. Provisional Patent Application under 35 U.S.C. 119(e), the contents of which are incorporated herein by reference.

technical field

The described embodiments relate generally to optical sensing, and more specifically, to tracking eye movement using optical sensors.

Background technique

Eye monitoring technology can be used to improve near-eye displays (eg, head-mounted displays (HMDs)), augmented reality (AR) systems, virtual reality (VR) systems, and the like. For example, gaze vector tracking, also known as gaze position tracking, can be used as input to display foveal rendering or human-computer interaction. Traditional eye monitoring techniques are camera-based or video-based and rely on active illumination of the eye, eye image acquisition, and extraction of eye features such as pupil center and corneal glint. The power consumption, form factor, computational cost, and latency of this eye-monitoring technology can be a significant burden on more user-friendly next-generation HMD, AR, and VR systems (eg, systems that are lighter weight, battery-powered, and more fully functional).

Contents of the invention

Embodiments of the systems, devices, methods, and devices described in this disclosure utilize one or more self-mixing interferometry (SMI) sensors to track eye movement. In some embodiments, an SMI sensor may be used alone or in combination with a camera to determine gaze vector or eye position.

In a first aspect, the present disclosure describes an eye tracking device. The eye-tracking device can include a head-mounted frame, an optical sensor subsystem mounted to the head-mounted frame, and a processor. The optical sensor subsystem can include a set of one or more SMI sensors. The processor can be configured to operate the optical sensor subsystem to cause the set of one or more SMI sensors to emit a set of one or more light beams toward an eye of the user; receive a set of one or more SMI signals from the set of one or more SMI sensors; and use the set of one or more SMI signals to track movement of the eye.

In a second aspect, this disclosure describes another eye-tracking device. The eye tracking device can include a set of one or more SMI sensors, a camera and a processor. The processor can be configured to cause the camera to acquire a set of images of a user's eye at a first frequency; cause a set of one or more SMI sensors to emit a set of one or more light beams toward the user's eye; sample a set of one or more SMI signals generated by the set of one or more SMI sensors at a second frequency greater than the first frequency; use at least one image of the set of images to determine a gaze vector of the eye;

In a third aspect, the present disclosure describes a method of tracking eye movement. The method can include operating an optical sensor subsystem such that a set of one or more SMI sensors in the optical sensor subsystem emits a set of one or more light beams toward an eye of a user; receiving a set of one or more SMI signals from the set of one or more SMI sensors; and tracking movement of the eye using the set of one or more SMI signals.

In addition to the exemplary aspects and embodiments described, further aspects and embodiments will be apparent by study of the following descriptions, with reference to the drawings.

Description of drawings

The present disclosure will be readily understood from the following detailed description taken in conjunction with the accompanying drawings, in which like reference numerals designate like structural elements, and in which:

Figure 1 shows an exemplary block diagram of an eye tracking device;

Figure 2A shows an exemplary graph of the tracked angular velocity of the eye;

Figure 2B shows an exemplary diagram of a tracked gaze of an eye;

Figure 3A shows a first exemplary eye-tracking device in which the optical sensing subsystem and processor are mounted to a pair of glasses;

Figure 3B shows a second exemplary eye-tracking device in which the optical sensing subsystem and processor are mounted to a VR headset;

4A shows a side view of an exemplary set of SMI sensors that may be mounted to a head-mounted frame and configured to emit light toward the eyes;

Figure 4B shows a front view of the eye and SMI sensor set shown in Figure 4A;

5 illustrates an exemplary front view of a first set of alternative SMI sensors that may be mounted to a head-mounted frame and configured to emit light toward the eyes;

6 illustrates an exemplary front view of a second set of alternative SMI sensors that may be mounted to the head-mounted frame and configured to emit light toward the eyes;

7 illustrates an exemplary side view of a third set of alternative SMI sensors that may be mounted to a head-mounted frame and configured to emit light toward the eyes;

Figure 8A shows an exemplary use of an SMI sensor in combination with a beam splitter;

Figure 8B shows an exemplary use of an SMI sensor in combination with a beam steering component;

Figure 9A shows a first exemplary integration of an SMI sensor with a display subsystem;

Figure 9B shows a second exemplary integration of an SMI sensor with a display subsystem;

Figure 10 illustrates an exemplary set of components that may be included in an optical sensor subsystem of an eye tracking device;

FIG. 11A illustrates a first exemplary method for tracking eye movement using a set of one or more SMI sensors;

11B illustrates a second exemplary method for tracking eye movement using a set of one or more SMI sensors in combination with a camera or other sensor; and

12A and 12B illustrate how a set of one or more SMI sensors can be used to map one or more surfaces or structures of the eye.

The use of cross-hatching or shading in the drawings is generally provided to clarify boundaries between adjacent elements and also to facilitate drawing legibility. Accordingly, neither the presence nor absence of cross-hatching or hatching indicates or indicates any preference or requirement for particular materials, material properties, element proportions, element dimensions, commonality to similarly illustrated elements, or any other feature, property, or characteristic of any element shown in the Figures.

Additionally, it should be understood that the proportions and dimensions (relative or absolute) of the various features and elements (and collections and groupings thereof) and the boundaries, spacings and positional relationships presented therebetween are provided in the drawings only to facilitate an understanding of the various embodiments described herein, and thus may not necessarily be presented or shown to scale and are not intended to indicate any preference or requirement over the illustrated embodiments to the exclusion of embodiments described in connection therewith.

Detailed ways

Reference will now be made in detail to the representative embodiments illustrated in the accompanying drawings. It should be understood that the following description is not intended to limit the embodiments to one preferred embodiment. On the contrary, it is intended to cover alternatives, modifications and equivalents, which may be included within the spirit and scope of the described embodiments as defined by the appended claims.

Most eye tracking systems are camera or image based and cannot track eye movement fast enough or with high enough accuracy using a reasonable amount of power. Lower power and lower cost systems with higher accuracy and sensitivity are desired.

Fast and accurate detection and classification of eye movements (such as distinguishing between smooth pursuit, saccades, gazes, nystagmus, and blinks) can be mission-critical, but can also be challenging for camera-based, video-based, or photodetector-based tracking systems—especially within tight power budgets. For example, fixations can be as short as tens of microseconds and as subtle as <0.25 degrees per second (deg/s) or <0.5 deg/s. Detection of such eye movement indicates the need for high resolution and high frame rate image acquisition and processing systems.

Previously, eye odometry based on low-power imaging has been proposed, where reduced resolution and higher frame rate image capture is fused with higher resolution and lower frame rate image capture for overall power savings (eg, compared to systems that only acquire higher resolution images at a higher frame rate). As an alternative, photodiode array-based eye trackers with lower latency and lower power consumption have been proposed, but have not yet proven useful in higher-resolution and higher-precision applications. Photodiode array-based eye trackers are therefore better suited for binary sensing applications, such as wake-up in mission-critical video-based eye trackers, or in near-eye display (or HMD) systems that rely on coarse gaze zone detection or gaze movement thresholds.

The following description refers to the use of SMI sensors alone or in combination with other sensors, such as cameras or other image-based sensors, to track eye movement. For the purposes of this description, an SMI sensor is considered to include a light emitter (e.g., a laser light source) and an SMI signal detector (e.g., a light detector, such as a photodiode, or an electrical detector, such as a circuit that measures the junction voltage or drive current of the light emitter). Each of the one or more SMI sensors may emit one or more fixed or scanning beams toward one or more structures of the user's eye (e.g., toward one or more of the iris, sclera, pupil, lens, limbus, eyelid, etc. of the eye). SMI sensors can be operated in accordance with operating safety regulations so as not to harm the user's eyes.

After emitting a beam, the SMI sensor can receive the retroreflected portion of the emitted light back into its resonant cavity. For good quality retroreflection, it may be useful to focus the beam emitted by the SMI sensor on the iris or sclera (or another diffusing structure) of the eye rather than on the cornea or pupil of the eye. The phase change of the retroreflected portion of the emitted light can mix with the phase of the light generated within the resonant cavity and can generate an SMI signal that can be amplified and detected. The amplified SMI signal, and in some cases, multiple SMI signals can be analyzed. The rotational movement of the eye can be retrieved and reconstructed by phase tracking the Doppler frequency from multiple orientations and positions of the user's eye.

Classification and quantification of user gaze behaviors such as smooth pursuit, saccades, fixations, nystagmus, and eye blinks can be identified using SMI sensors at high sampling rates to facilitate efficient, high-fidelity, digital content rendering of numbers, text, or images on near-eye display (or HMD) systems.

In some embodiments, SMI sensor data or determinations can be fused with absolute gaze direction sensing information (e.g., gaze vector sensing or gaze position sensing) acquired from a lower sampling rate gaze imaging system (e.g., a camera) to achieve absolute gaze tracking at much higher speed and with good accuracy. In contrast to imaging systems, which may include a million or more pixels, SMI sensor data may be obtained from one or a few (eg, two or three) SMI sensors. This can enable SMI sensors to generate SMI signals (or SMI sensor data) at much lower power consumption than imaging systems.

When the wavelength of light emitted by the SMI sensor is modulated, the SMI signal obtained from the SMI sensor can be used for absolute ranging of the surface, interface, and volume structures of the user's eye, on the order of about 100 μm resolution. This absolute distance measurement can provide an anchor for tracking the displacement of the eye contour during eye rotation.

When the wavelength of light emitted by one or more SMI sensors is modulated, and when the beam emitted by at least one SMI sensor is scanned and/or multiple beams are emitted, a displacement and/or velocity map (also known as a Doppler cloud), and/or a distance map (also known as a depth cloud) may be obtained or constructed. Additionally or alternatively, a Doppler cloud may also be obtained or constructed when the wavelength of light emitted by the one or more SMI sensors is not modulated. Doppler clouds can be of native high resolution, which can eg be at the μm or sub-μm level for individual frame displacements, or down to mm/s or deg/s levels for velocities. Single or multiple frames of a Doppler cloud can be considered a differential depth cloud. Additionally or alternatively, single or multiple frame measurements of the Doppler cloud can be processed in real-time to match predefined and/or locally calibrated difference maps or libraries and to extract eye-tracking information and/or position information (also known as pose information). Locally calibrated differential maps or libraries can be obtained including, but not limited to, cameras, depth clouds, etc. Using Doppler clouds alone or fused with depth clouds or other sensing modalities (eg, eye camera images, motion sensors, etc.) can provide an accurate and efficient way of tracking eye movement or position information.

These and other systems, devices, methods and apparatuses are described with reference to FIGS. 1-12B . However, those skilled in the art will readily appreciate that the detailed description given herein with respect to these figures is for illustrative purposes only and should not be construed as limiting.

Directional terms such as "top", "bottom", "upper", "lower", "front", "rear", "above", "under", "above", "below", "left", "right", etc. are used with reference to the orientation of some components in some of the figures described below. Because components in various embodiments can be positioned in a number of different orientations, directional terms are used for descriptive purposes only and are not limiting in any way. Directional terms are intended to be interpreted broadly, and thus should not be interpreted to exclude differently oriented components. Also, as used herein, the phrase "at least one of" following a series of items separating any of the items with the term "and" or "or" modifies the list as a whole and does not modify each member of the list. The phrase "at least one of" does not require selection of at least one of each of the items listed; rather, the phrase allows for meanings that include at least one of any of the items and/or at least one of any combination of items and/or at least one of each of the items. For example, the phrases "at least one of A, B, and C" or "at least one of A, B, or C" each refer to only A, only B, or only C; any combination of A, B, and C; and/or one or more of each of A, B, and C. Similarly, it will be understood that the order in which elements are presented herein, either in combination or separately, should not be construed to limit the present disclosure to only the order presented.

FIG. 1 shows an exemplary block diagram of an eye tracking device 100 . Eye tracking device 100 may include a head-mounted frame 102 , an optical sensor subsystem 104 mounted to head-mounted frame 102 , and a processor 106 . In some implementations, eye-tracking device 100 may also include one or more of display subsystem 108 , communication subsystem 110 (eg, wireless and/or wired communication subsystem), and distribution subsystem 112 . Processor 106, display subsystem 108, communications subsystem 110, and distribution subsystem 112 may be partially or fully mounted to, or in radio frequency or electrical communication with, one or more components mounted to head-mounted frame 102 (e.g., housed in a box or electronic device (e.g., a phone or a wearable device such as a watch) in radio-frequency (i.e., wireless) or electrical (e.g., wired) communication with one or more components mounted to head-mounted frame 102), or distributed among Between the headset frame 102 and a box or electronic device that is in radio frequency or electrical communication with one or more components mounted to the headset frame 102 . Optical sensor subsystem 104, processor 106, display subsystem 108, communication subsystem 110, and/or distribution subsystem 112 may communicate over one or more buses 116 or over the air (eg, wirelessly) using one or more communication protocols.

The headset 102 may take the form of a pair of glasses, a set of goggles, an augmented reality (AR) headset, a virtual reality (VR) headset, or other forms of the headset 102 .

Optical sensor subsystem 104 may include a set of one or more SMI sensors 114 . Each SMI sensor in a set of one or more SMI sensors may include a light emitter and a light detector. The light emitter may comprise one or more of a vertical cavity surface emitting laser (VCSEL), an edge emitting laser (EEL), a vertical external cavity surface emitting laser (VECSEL), a quantum dot laser (QDL), a quantum cascade laser (QCL), or a light emitting diode (LED) (e.g., an organic LED (OLED), a resonant cavity LED (RC-LED), a micro LED (mLED), a superluminescent LED (SLED), or an edge emitting LED), among others. In some cases, a light detector (or photodetector) may be positioned laterally adjacent to the light emitter (eg, mounted or formed on a substrate on which the light detector is mounted or formed). In other cases, photodetectors can be stacked above or below the photoemitters. For example, the photodetector may be a VCSEL, HCSEL, or EEL having a primary emission and a secondary emission, and the photodetector may be formed epitaxially in the same epitaxial stack as the photoemitter such that the photodetector receives some or all of the secondary emissions. In these latter embodiments, the light emitter and light detector can be formed similarly (e.g., both the light emitter and light detector can comprise a multiple quantum well (MQW) structure, but the light emitter can be forward biased and the light detector (e.g., a resonant cavity photodetector (RCPD) can be reverse biased). Alternatively, the light detector can be formed on a substrate, and the light emitter can be formed separately and mounted to the substrate, or relative to the light emitter, such that the light emitter's secondary light emission hits the light detector Alternatively, the light emitter can be formed on the substrate, and the light detector can be formed separately and mounted to the substrate, or relative to the light emitter, such that the secondary light emission of the light emitter strikes the light detector.

In some embodiments, optical sensor subsystem 104 may include a set of fixed or movable optical components (eg, one or more mirrors, gratings, filters, beam splitters, beam steering components, etc.). Optical sensor subsystem 104 may also include an image sensor (eg, a camera including an image sensor).

Processor 106 may include any electronic device capable of processing, receiving or transmitting data or instructions, whether such data or instructions are in the form of software or firmware or otherwise encoded. For example, processor 106 may include a microprocessor, a central processing unit (CPU), an application specific integrated circuit (ASIC), a digital signal processor (DSP), a controller, or a combination of such devices. As used herein, the term "processor" is intended to encompass a single processor or processing unit, multiple processors, multiple processing units, or other suitably configured computing elements.

In some embodiments, components of eye-tracking device 100 may be controlled by multiple processors. For example, selected components of eye-tracking device 100 (e.g., optical sensor subsystem 104) may be controlled by a first processor and other components of eye-tracking device 100 (e.g., display subsystem 108 and/or communication subsystem 110) may be controlled by a second processor, where the first processor and the second processor may or may not be in communication with each other.

In some embodiments, display subsystem 108 may include a display having one or more light emitting elements including, for example, LEDs, OLEDs, liquid crystal displays (LCDs), electroluminescent (EL) displays, or other types of display elements.

Communication subsystem 110 may enable eye-tracking device 100 to transmit or receive data from a user or another electronic device. Communication subsystem 110 may include a touch-sensing input surface, a crown, one or more microphones or speakers, or a wired or wireless (eg, radio frequency (RF) or optical) communication interface configured to transmit electronic, RF, or optical signals. Examples of wireless and wired communication interfaces include, but are not limited to, cellular, Wi-Fi, and Communication Interface.

Distribution subsystem 112 may be implemented with any collection of power sources and/or conductors capable of delivering energy to eye-tracking device 100 or components thereof. In some cases, distribution subsystem 112 may include one or more batteries or rechargeable batteries. Additionally or alternatively, distribution subsystem 112 may include a power connector or power cord that may be used to connect eye-tracking device 100 or components thereof to a remote power source, such as a wall outlet, a remote battery pack, or an electronic device to which eye-tracking device 100 is tethered.

Processor 106 may be configured to operate optical sensor subsystem 104 . Operating optical sensor subsystem 104 may include causing distribution subsystem 112 to provide power to optical sensor subsystem 104, providing control signals to a set of one or more SMI sensors 114, and/or providing control signals that electrically, electromechanically, or otherwise focus or adjust optical components of optical sensor subsystem 104. Operating the optical sensor subsystem 104 may cause the set of one or more SMI sensors 114 to emit a set of one or more light beams 118 toward the user's eye 120 . Processor 106 may also be configured to receive a set of one or more SMI signals from set of one or more SMI sensors 114 and track rotational movement of eye 120 using the set of one or more SMI signals.

In some cases, optical sensor subsystem 104 , processor 106 , display subsystem 108 , communication subsystem 110 , and/or distribution subsystem 112 may communicate via one or more buses, generally depicted as bus 116 .

In some cases, tracking the rotational movement of eye 120 may include estimating the angular velocity (or gaze movement) of eye 120 . In some cases, angular velocity may be tracked in each of three orthogonal directions (eg, in the x, y, and z directions). Tracking angular velocity in different directions may require scanning the one or more light beams emitted by the set of one or more SMI sensors 114, splitting the one or more light beams emitted by the one or more SMI sensors in the set of one or more SMI sensors 114, or configuring the set of one or more SMI sensors 114 to emit two or more light beams in different directions (and preferably in different orthogonal directions).

Tracking the rotational movement of the eye 120 may also include tracking the gaze (or gaze position) of the eye 120 .

In some embodiments, the processor 106 may be configured to classify the user's rotational eye movement. For example, rotational eye movement may be classified as at least one of smooth pursuit, saccade, fixation, nystagmus, or blink. Processor 106 may then cause display subsystem 108 to adjust the rendering of one or more images on the display in response to the classification of the rotational eye movement. In some embodiments, the processor 106 may be further configured to quantify the classified rotational eye movement (e.g., how fast or how much the user blinks, what is the angular velocity of the user's eye movement, or how often or how quickly the angular velocity of the user's eye 120 changes), and may further adjust the rendering of the one or more images in response to the quantified rotational eye movement.

In some embodiments, processor 106 may be configured to cause display subsystem 108 to change the state of the display in response to movement of eye 120 . Changing the state of the display may include changing what is displayed, but may also or alternatively include transitioning the display from a low power or off state to a higher power or on state, or alternatively, transitioning the display from a higher power or on state to a low power or off state.

In some embodiments, the display may include an array of display pixels mounted on a substrate, and the set of one or more SMI sensors 114 may include at least one SMI sensor mounted on the substrate, adjacent to, or within the array of display pixels. In other embodiments, the SMI sensor 114 may be provided separately from the display.

FIG. 2A shows an exemplary graph 200 of tracked angular velocity (or gaze movement) of an eye. For example, graph 200 shows tracking angular velocity in only two orthogonal directions. When the outer surface of the eye or another structure is modeled as a two-dimensional object, angular velocity can only be tracked in two dimensions. Alternatively, angular velocity can be more accurately tracked in three dimensions.

FIG. 2B shows an exemplary graph 210 of a tracked gaze (or gaze vector or gaze location) of an eye. For example, diagram 210 shows that gaze is tracked in only two orthogonal directions. Gaze can only be tracked in two dimensions when the outer surface of the eye or another structure is modeled as a two-dimensional object. Alternatively, gaze can be more accurately tracked in three dimensions.

SMI-based sensing may be particularly useful when a device or application needs to detect and/or classify subtle eye movements such as smooth pursuit, saccades, gazes, nystagmus, and blinks, since all of these eye movements can be detected and classified by tracking the angular velocity of the eye, and SMI-based sensing is generally faster, more accurate, and more energy-efficient (e.g., consumes less power) than video- or photodetector-based sensing when it comes to detecting changes in the angular velocity of the eye. Furthermore, with initial and periodic (low frequency) calibration of gaze position, high frequency gaze position vector tracking as shown in Figure 2B can be obtained from the integration of gaze velocity vector tracking as shown in Figure 2A.

FIG. 3A shows a first exemplary eye-tracking device in which an optical sensing subsystem 302 and a processor 304 are mounted to a pair of glasses 300 . For example, eyeglasses 300 (eg, a type of head-mounted frame) are shown including a first frame 308 and a second frame 310 , a bridge 312 connecting the first frame 308 to the second frame 310 , a first temple 314 connected to the first frame 308 , and a second temple 316 connected to the second frame 310 . In some embodiments, glasses 300 may include a heads-up display or function as AR glasses.

Each of the first frame 308 and the second frame 310 may hold a respective lens, such as the first lens 318 or the second lens 320 . The optics 318 , 320 may or may not magnify, focus, or otherwise alter the light passing through the optics 318 , 320 . For example, the lenses 318, 320 may correct the user's vision, block bright or harmful light, or simply provide a physical barrier through which light may pass with no or minimal adjustment. In some embodiments, first lens 318 and second lens 320 may be formed of glass or plastic. In some embodiments, first lens 318 and/or second lens 320 may serve as a display (e.g., a passive display screen) on which text, numbers, and/or images are projected by display subsystem 322, which may also be mounted to the pair of eyeglasses. Alternatively, first frame 308 and/or second frame 310 may hold a transparent or translucent display (e.g., light emitting diode (LED), organic LED (OLED), or other light emitting element operable by display subsystem 322 to display text, numbers, and/or images.

As another example, optical sensing subsystem 302 may be configured as described with reference to FIG. 1 and one or more of FIGS. 5-9B , and/or processor 304 may be configured to operate as described with reference to FIGS. 1 and 10-12B . One or more components of optical sensing subsystem 302 (e.g., SMI sensors, optics, cameras, etc.) may be mounted on one or more substrates attached to first frame 308, second frame 310, bridge 312, first temple 314, second temple 316, first lens 318, or second lens 320, or may be mounted directly to one or more of these components. Similarly, processor 304, display subsystem 322, communication subsystem 324, and/or distribution subsystem 326 may be mounted to one or more of these components. In some embodiments, optical sensor subsidiaries 302, processor 304, display subsystem 322, communication subsystem 324, and/or distribution system 326 can be installed in one or more components of glasses 300, installed in one or more parts of one or more parts installed in the wireless or electrical connection to the glasses 300 (e.g., in the user's phone or wearable device). The equipment connected to one or more components of the glasses 300 in the glasses 300 and the wireless or electricity.

Processor 304 , display subsystem 322 , communication subsystem 324 , and distribution subsystem 326 may be further configured or operative as described with reference to FIG. 1 .

FIG. 3B shows a second exemplary eye-tracking device in which an optical sensing subsystem 352 and a processor 354 are mounted to a virtual reality (VR) headset 350 . For example, a VR headset (one type of head-mounted frame) is shown including a VR module 358 that can be attached to a user's head by a strap 356 .

VR module 358 may include display subsystem 360 . Display subsystem 360 may include a display for displaying text, numbers and/or images.

As an example, optical sensing subsystem 352 may be configured as described with reference to FIG. 1 and one or more of FIGS. 5-9B , and/or processor 354 may be configured to operate as described with reference to FIGS. 1 and 10-12B . One or more components of optical sensing subsystem 352 (eg, SMI sensors, optics, cameras, etc.) may be mounted on one or more substrates attached to VR module 358 , or may be mounted directly to the housing of VR module 358 . Similarly, processor 354 , display subsystem 360 , communication subsystem 362 , and/or distribution subsystem 364 may be mounted to VR module 358 . In some embodiments, some or all of optical sensing subsystem 352, processor 354, display subsystem 360, communication subsystem 362, and/or distribution subsystem 364 may be installed within, or distributed between, VR module 358 and a device wirelessly or electrically connected to VR module 358 (e.g., in a user's phone or wearable device).

Processor 354 , display subsystem 360 , communication subsystem 362 , and distribution subsystem 364 may be further configured or operative as described with reference to FIG. 1 .

FIGS. 4A and 4B illustrate a set of exemplary SMI sensors 400 , 402 that may be mounted to a head-mounted frame, such as one of the head-mounted frames described with reference to FIGS. 1-3B . The set of SMI sensors 400, 402 may form part of an optical sensor subsystem, such as one of the optical subsystems described with reference to Figures 1 to 3B or elsewhere herein. The set of SMI sensors 400 , 402 may be configured to emit light toward an eye 404 . FIG. 4A shows a side view of an eye 404 and a set of SMI sensors 400 , 402 , and FIG. 4B shows a front view of an eye 404 and a set of SMI sensors 400 , 402 .

By way of example, the set of SMI sensors 400, 402 is shown to include the two SMI sensors in FIGS. 4A and 4B (eg, first SMI sensor 400 and second SMI sensor 402). In other embodiments, the set of SMI sensors 400, 402 may include more or fewer SMI sensors. The first SMI sensor 400 and the second SMI sensor 402 may emit respective light beams toward the eye 404 . In some embodiments, the light beams can be directed in different directions, which may or may not be orthogonal directions. A processor receiving the SMI signals generated by the SMI sensors 400, 402 may track the movement of the eye 404 in two dimensions (eg, the processor may track the angular velocity of the eye 404 in two dimensions) as the light beams are oriented in different directions. The light beams may strike the eye 404 at the same or different locations. For example, the light beam is shown hitting the eye 404 at the same location.

In some embodiments, light emitted by the SMI sensors 400, 402 may be directed or filtered by optics 406 or 408 (eg, one or more mirrors or beam steering components or other optical components).

FIG. 5 shows a front view of an alternative set of SMI sensors 500 , 502 , 504 that may be mounted to a head-mounted frame, such as one of the headsets described with reference to FIGS. 1-3B . The set of SMI sensors 500, 502, 504 may form part of an optical sensor subsystem, such as one of the optical subsystems described with reference to Figures 1 to 3B or elsewhere herein. The set of SMI sensors 500 , 502 , 504 may be configured to emit light toward an eye 506 .

In contrast to the set of SMI sensors described with reference to FIGS. 4A and 4B , the set of SMI sensors 500 , 502 , 504 shown in FIG. 5 includes three SMI sensors (eg, a first SMI sensor 500 , a second SMI sensor 502 , and a third SMI sensor 504 ). First SMI sensor 500 , second SMI sensor 502 , and third SMI sensor 504 may emit respective light beams toward eye 506 . In some embodiments, the light beams can be directed in different directions, which may or may not be orthogonal directions. A processor receiving the SMI signals generated by the SMI sensors 500, 502, 504 may track the movement of the eye 506 in three dimensions (e.g., the processor may track the angular velocity of the eye 506 in three dimensions) as the light beam is directed in different directions. The light beams may strike the eye 506 at the same or different locations. For example, the light beam is shown hitting the eye 506 at the same location.

FIG. 6 shows an exemplary front view of a second set of alternative SMI sensors 600 , 602 , 604 that may be mounted to a head-mounted frame, such as one of the headsets described with reference to FIGS. 1-3B . The set of SMI sensors 600, 602, 604 may form part of an optical sensor subsystem, such as one of the optical subsystems described with reference to Figures 1 to 3B or elsewhere herein. The set of SMI sensors 600, 602, 604 may be configured to emit light toward the eye.

The set of SMI sensors 600, 602, 604 shown in FIG. 6 includes three SMI sensors (eg, a first SMI sensor 600, a second SMI sensor 602, and a third SMI sensor 604). The first SMI sensor 600 , the second SMI sensor 602 and the third SMI sensor 604 may emit respective light beams toward the eye 606 . In some embodiments, the light beams can be directed in different directions, which may or may not be orthogonal directions. A processor receiving the SMI signals generated by the SMI sensors 600, 602, 604 may track the movement of the eye 606 in three dimensions (e.g., the processor may track the angular velocity of the eye 606 in three dimensions) as the light beam is directed in different directions. Compared to the set of SMI sensors described with reference to FIG. 5 , two of the beams hit the eye 606 at the same location (e.g., at a first location) and one of the beams hits the eye 606 at a different location (e.g., at a second location different from the first location). Alternatively, all beams may be directed to strike the eye 606 at the same location, or all beams may be directed to strike the eye 606 at different locations.

The optical sensor subsystem including the set of SMI sensors 600 , 602 , 604 may also include a camera 608 . Similar to the set of SMI sensors 600, 602, 604, a camera 608 may be mounted to the head mounted frame. Camera 608 may be positioned and/or oriented to capture images of eye 606 . The image may be of a portion or all of the eye 606 . A processor configured to operate the optical sensor subsystem may be configured to track the rotational movement of the eye 606 using images acquired by the camera 608 and SMI signals generated by the set of SMI sensors 600 , 602 , 604 . For example, the processor may use the camera 608 to acquire a set of images of the eye at a first frequency. The processor may also sample the set of one or more SMI signals at a second frequency greater than the first frequency. Image and SMI signal samples may be acquired/sampled in time overlapping fashion or in parallel at different times. In some cases, the processor may acquire one or more images; analyze the images to determine the position of the eye 606 relative to the SMI sensors 600, 602, 604 and/or beams of light emitted by the SMI sensors 600, 602, 604; and, if necessary, make adjustments to the optical sensor subsystem to ensure that the beams hit the desired structure of the eye 606. Adjustments to the optical sensor subsystem may include, for example, one or more of: adjusting beam steering components to steer one or more beams, addressing and causing a specific subset of SMI sensors 600, 602, 604 to emit light, etc. (see, for example, the description of FIGS. 7 and 8B ). Alternatively (e.g., as an alternative to relying on the camera 608), the SMI sensors 600, 602, 604 may be used to perform range measurements as the user moves their eyes 606, and the range measurements may be mapped to an eye model to determine whether the SMI sensors 600, 602, 604 are focusing on the desired eye structure. The eye model may be a generic eye model, or a user-specific generated eye model when the optical sensor subsystem operates in the training and eye model generation modes.

The optical sensor subsystem including the set of SMI sensors 600, 602, 604 may also include one or more light emitters 610, 612 capable of illuminating the eye 606 for taking an image of the eye 606 (or for other purposes). The light emitters 610, 612 may take the form of LEDs, lasers, or other light emitting elements of a display, or the like. The light emitters 610, 612 may emit visible light or non-visible light (eg, IR light), depending on the type of light that the camera 608 is configured to sense. The light emitters 610, 612 may be used to provide flood, sweep or spot illumination.

In some embodiments, the processor may determine or track the gaze vector of the eye 606 using the SMI signals generated by the set of SMI sensors 600 , 602 , 604 . In some embodiments, the processor may use one or more images acquired with the camera 608 to determine or track a gaze vector. In some embodiments, the processor may use one or more images acquired using the camera 608 in combination with the SMI signal to track the gaze vector. For example, one or more images may be used to determine a gaze vector, and then the SMI signal may be used to track eye 606 movement and update the gaze vector (or in other words, determine movement of the gaze vector).

FIG. 7 shows an exemplary side view of a third set of alternative SMI sensors 700 , 702 , 704 that may be mounted to a headset frame, such as one of the headset frames described with reference to FIGS. 1-3B . The set of SMI sensors 700, 702, 704 may form part of an optical sensor subsystem, such as one of the optical subsystems described with reference to Figures 1 to 3B or elsewhere herein. The set of SMI sensors 700 , 702 , 704 may be configured to emit light toward an eye 706 .

The set of SMI sensors 700, 702, 704 shown in FIG. 7 includes three SMI sensors (eg, a first SMI sensor 700, a second SMI sensor 702, and a third SMI sensor 704). First SMI sensor 700, second SMI sensor 702, and third SMI sensor 704 may emit respective beams toward eye 706, similar to how beams may be emitted by the set of SMI sensors described with reference to FIGS. 4A and 4B (or FIG. 5 or 6). However, not all SMI sensors 700, 702, 704 can emit beams at the same time. For example, second SMI sensor 702 and third SMI sensor 704 may be part of an addressable array of SMI sensors, which in some cases may include more than two SMI sensors. The SMI sensor array can be coupled to circuitry that can be used to address different SMI sensors or different subsets of SMI sensors in the SMI sensor array.

In some cases, light beams emitted by second SMI sensor 702 and third SMI sensor 704 (and in some cases, light beams emitted by other SMI sensors in the SMI sensor array) may be directed to a shared mirror, mirror set, mirror array, or one or more other optical components 708.

In some cases, the optical sensor subsystem including the set of SMI sensors may also include a camera 710 that may be used similarly to the cameras described with reference to FIG. 6 .

In some cases, the processor may selectively operate (eg, activate and deactivate) different SMI sensors 702, 704 in an array of SMI sensors (or different subsets of SMI sensors) for different purposes. For example, a processor may operate (or use) circuitry to activate a particular SMI sensor or a particular subset of SMI sensors with a particular focal point(s). In some cases, processor may cause camera 710 to acquire one or more images of eye 706 . The processor may then analyze the image to determine the gaze vector of the eye 706, and may activate one or more SMI sensors 702, 704 (in an array of SMI sensors) focused on a particular structure or region (or structures or regions) of the eye 706. The processor can also activate other SMI sensors, such as SMI sensor 700 .

FIG. 8A shows an exemplary use of an SMI sensor 800 in combination with a beam splitter 802 . SMI sensor 800 may be any of the SMI sensors described with reference to FIGS. 1-7 , or any of the SMI sensors described below. Beam splitter 802 may be positioned to split beam 804 emitted by the SMI sensor. The light beam may be split into multiple light beams 806, 808, 810 (eg, two, three or more light beams). In some cases, beam splitter 802 may be associated with one or more optics or beam steering components that redirect or turn multiple light beams 806 , 808 , 810 . In some cases, multiple light beams 806, 808, 810 may be redirected toward a shared focal point on (or in) the eye.

FIG. 8B shows an exemplary use of an SMI sensor 850 in combination with a beam steering component 852 . The SMI sensor 850 may be any of the SMI sensors described with reference to FIGS. 1-7 , or any of the SMI sensors described below. Beam steering component 852 may be positioned to steer beam 854 emitted by SMI sensor 850 . The processor may be configured to operate the beam steering component 852 and steer the beam 854 to different structures or regions of the eye. In some embodiments, the beam steering component 852 can include a beam focusing component or lens positioning mechanism that can be adjusted to change the focus of the beam along its axis. In some embodiments, the beam steering component 852 can be replaced with a beam focusing component.

FIG. 9A shows a first exemplary integration of an SMI sensor 900 with a display subsystem. SMI sensor 900 may be any of the SMI sensors described with reference to FIGS. 1-7 , or any of the SMI sensors described below. In some examples, the display subsystem may be the display subsystem described with reference to FIG. 1 , FIG. 3A , or FIG. 3B .

The display subsystem may include an array of display pixels 902 , 904 , 906 mounted or formed on a substrate 908 . For example, the display subsystem is shown as including blue pixel 902, green pixel 904, and red pixel 906, although the display subsystem may include multiple instances of each blue, green, and red pixel. In some cases, display pixels 902, 904, 906 may include LEDs or other types of light emitting elements.

SMI sensor 900 may be mounted on substrate 908 . For example, an SMI sensor 900 is shown mounted on a substrate 908 adjacent to an array of display pixels 902 , 904 , 906 . Alternatively, SMI sensor 900 may be mounted on substrate 908 within the array of display pixels 902, 904, 906 (ie, between display pixels). In some implementations, more than one SMI sensor 900 may be mounted on the substrate 908, with each SMI sensor 900 positioned adjacent to or within an array of display pixels 902, 904, 906. In some embodiments, the SMI sensor 900 can emit IR light. In other embodiments, SMI sensor 900 may emit visible light, ultraviolet light, or other types of light. In some implementations, one or more of display pixels 902, 904, 906 may operate as an SMI sensor.

Shared waveguide 910 may be positioned to direct beams emitted by display pixels 902, 904, 906 and SMI sensor 900 to beam steering component 912, such as a set of one or more mirrors movable by a microelectromechanical system (MEMS). In an alternative embodiment, a set of waveguides (eg, a set of optical fibers) may be used to direct light emitted by display pixels 902 , 904 , 906 and SMI sensor 900 to beam steering component 912 . The processor may operate display pixels 902, 904, 906 and beam steering component 912 to render text, numbers or images on the display. In some cases, the display may include one or more lenses of a pair of glasses, or a display within a VR headset.

Shared waveguide 910 (or set of waveguides) may receive a return portion of light emitted by SMI sensor 900, such as a portion of light reflected or scattered from an eye, and direct the return portion of the emitted light toward and into a resonant cavity of SMI sensor 900.

FIG. 9B shows a second exemplary integration of the SMI sensor 950 with the display subsystem. SMI sensor 950 may be any of the SMI sensors described with reference to FIGS. 1-7 , or any of the SMI sensors described below. In some examples, the display subsystem may be the display subsystem described with reference to FIG. 1 , FIG. 3A , or FIG. 3B .

The display subsystem may include an array of display pixels 952 , 954 , 956 and an SMI sensor 950 mounted or formed on a substrate 958 . Display pixels 952, 954, 956 and SMI sensor 950 may be configured as described with reference to FIG. 9A.

The waveguide 960 (or set of waveguides) may be positioned to direct light beams emitted by the display pixels 952, 954, 956 and the SMI sensor 950 to the further shared waveguide 962, or the distal portion of the shared waveguide 960 (or the distal portion of the set of waveguides) may be bent and light may be emitted out-coupled from the further shared waveguide 962, or out-coupled from the distal portion of the shared waveguide 962 or set of waveguides. The processor can operate display pixels 952, 954, 956 to project text, numbers or images on the display.

Returning portions of the light emitted by the SMI sensor 950 , such as a portion of light reflected or scattered from the eye, may pass through the waveguides 962 , 960 and be redirected toward and into the resonant cavity of the SMI sensor 950 .

FIG. 10 shows an exemplary set of components 1000 that may be included in an optical sensor subsystem of an eye tracking device. The set of components 1000 is typically divided between a subset of optical or optoelectronic components 1002, a subset of analog components 1004, a subset of digital components 1006, and a subset of system components 1008 (eg, processors and, in some cases, other control components).

A subset of optical or optoelectronic components 1002 may include a laser diode 1010 or another optical transmitter with a resonant cavity. Component 1002 may also include a photodetector 1012 (eg, a photodiode). Photodetector 1012 may be integrated into the same epitaxial stack as laser diode 1010 (e.g., above, below, or adjacent to laser diode 1010), or may be formed as a separate component stacked with or adjacent to laser diode 1010. Alternatively, photodetector 1012 may be replaced or supplemented by circuitry that measures the junction voltage or drive current of laser diode 1010 and electronically generates the SMI signal (ie, without a photosensitive element). Laser diode 1010 in combination with photodetector 1012 or alternative circuitry for generating the SMI signal may be referred to as an SMI sensor.

A subset of optical or optoelectronic components 1002 may also include module-level optics 1014 integrated with laser diode 1010 and/or photodetector 1012 , and/or system-level optics 1016 . Module-level optics 1014 and/or system-level optics 1016 may include, for example, optics, beam splitters, beam steering components, and the like. Module-level optics 1014 and/or system-level optics 1016 may determine where emitted light and return light (eg, light scattered from the eye) is directed.

A subset of analog components 1004 may include a digital-to-analog converter (DAC) 1018 and a current regulator 1020 for converting the drive current into the analog domain and providing it to the laser diode 1010 . Component 1004 may also include component 1022 for ensuring that laser diode 1010 is operating in accordance with operational safety specifications. Component 1004 may also include a transimpedance amplifier (TIA) and/or other amplifier 1024 for amplifying the SMI signal generated by photodetector 1012, and an analog-to-digital converter (ADC) for converting the amplified SMI signal into the digital domain. Component 1004 may also include means for correcting the SMI signal when the SMI signal is amplified or otherwise processed.

In some cases, a subset of analog components 1004 may interface with (eg, be multiplexed with) more than one subset of optical or optoelectronic components 1002 . For example, component 1004 may interface with two, three, or more SMI sensors.

A subset of digital components 1006 may include a scheduler 1026 for scheduling (eg, correlating) providing drive current to laser diode 1010 with providing digitized photocurrent obtained from photodetector 1012 to system component 1008 . Component 1006 may also include a drive current waveform generator 1028 that provides digital drive current to DAC 1018 . Component 1006 may also include a digital processing chain for processing the amplified and digitized output of photodetector 1012 . The digital processing chain may include, for example, a time domain signal preprocessing circuit 1030 , a fast Fourier transform (FFT) engine 1032 , a frequency domain signal preprocessing circuit 1034 , and a distance and/or velocity estimator 1036 . In some cases, some or all of components 1006 may be instantiated by a set of one or more processors.

A subset of system components 1008 may include, for example, a system-level scheduler 1038 that may schedule when an SMI sensor (or SMI sensors) or other components are used to track eye position (e.g., gaze vector) or movement (e.g., angular velocity). Component 1008 may also include other sensors, such as a camera 1040 or an inertial measurement unit (IMU) 1042 . In some cases, component 1008 (or a processor thereof) may track eye position or movement using one or more SMI signals acquired from one or more subsets of component 1002, one or more images acquired from cameras 1040, and/or other measurements (e.g., measurements acquired by IMU 1042). Components 1008 may also include sensor fusion applications 1044 and various other applications 1046 .

Various types of drive currents can be used to drive the laser diode 1010 . For example, laser diode 1010 may be driven with a DC current (eg, in a DC drive mode) for the purpose of performing Doppler analysis on the SMI signal generated by photodetector 1012 . Alternatively, the laser diode 1010 may be driven in a frequency modulated continuous waveform (FMCW) mode (eg, with a triangular wave drive current) when performing ranging. Alternatively, the laser diode 1010 may be driven in a harmonic drive mode (eg, with an IQ modulated drive current) when determining the relative displacement of the eye.

FIG. 11A illustrates a first exemplary method 1100 for tracking eye movement using a set of one or more SMI sensors 1104 . Method 1100 includes operating optical sensor subsystem 1102 such that set of one or more SMI sensors 1104 emits a set of one or more light beams toward the user's eyes. Optical sensor subsystem 1102 and SMI sensor 1104 may be configured similar to any of the optical sensor subsystems and SMI sensors described herein.

At 1106 , method 1100 can include receiving a set of one or more SMI signals from the set of one or more SMI sensors 1104 .

At 1108, method 1100 can include tracking the rotational movement of the eye using a set of one or more SMI signals. Operations at 1108 may include estimating (at 1110 ) linear and angular velocities of the eye, eg, using the SMI signal and Doppler interferometry. Operations at 1108 may also include (at 1112) estimating the range (or distance) to the eye, or (at 1114) estimating the surface quality (eg, surface texture) of the eye. The estimated extent or surface quality can be used not only to estimate the rotational movement of the eye, but also to determine (at 1116) the position of the eye, or the structure of the eye on which the set of one or more SMI sensors 1104 is focused.

At 1118 , method 1100 can include using the output of operation 1108 to determine gaze movement.

At 1120, method 1100 can include using the output of operation 1108 to identify a gaze arousal event (eg, the user opens their eyes, or the user looks in a particular direction, or the user performs a particular series of eye movements). In some cases, operations at 1120 or other operations may include identifying gaze sleep events (eg, the user closes their eyes, the user looks in a particular direction, or the user performs a particular series of eye movements), eye blink events, or other events.

At 1122, method 1100 can include performing an operation in response to the identified event of a particular type (eg, powering on or off a head mounted display or other device, answering a call, activating an application, adjusting a volume, etc.).

At 1124, method 1100 can include performing Doppler ranging to determine a change in position of the eye gaze vector. In some cases, Doppler ranging may be performed using an Extended Kalman Filter (EKF).

At 1126, method 1100 can include updating the gaze-to-head-mounted-display (gaze-HMD) vector (ie, determining how the eye gaze vector intersects the display, or how the eye gaze vector moves relative to the display).

At 1128, method 1100 can include causing a display subsystem of the HMD (or another display) to adjust rendering of text, numbers, or images on the display. In some cases, the adjustment may be in response to classifying eye movement (eg, classifying as smooth pursuit, saccade, fixation, nystagmus, or blink).

11B illustrates a second exemplary method 1150 for tracking the movement of an eye using the set of one or more SMI sensors 1104 described with reference to FIG. Camera 1152 may be configured similarly to other cameras described herein.

At 1158, method 1150 may include using camera 1152 to acquire a set of one or more images of the eye. In some embodiments, camera 1152 may acquire a set of images at a first frequency and may sample the SMI signal generated by SMI sensor 1104 at a second frequency (eg, a second frequency synchronized with the first frequency). The frequencies may be the same or different, but in some embodiments the second frequency may be greater than the first frequency. In this way, the camera 1152, which typically consumes more power and produces a greater amount of data, can be used to determine eye position or generate gaze vector data less frequently, and ensure that the SMI sensor is properly focused and produces good data; In some embodiments, images captured by camera 1152 may be used to generate, update or customize an eye model that may be used to direct or focus the beam emitted by the SMI sensor.

At 1160 , method 1150 can include estimating a gaze vector (or position) of the eye based on the image acquired by camera 1152 .

At 1162, method 1100 can include determining or updating a head-to-HMD vector (head-HMD vector). In other words, method 1100 can determine how the user's head is positioned relative to the display.

At 1164, method 1100 can include performing visual-Doppler odometry to determine a change in position of the eye gaze vector. In some cases, visual-Doppler ranging may be performed using an Extended Kalman Filter (EKF). In contrast to the Doppler odometry performed in method 1100, the visual-Doppler odometry performed at 1164 may utilize image-based position (or gaze vector) analysis of the eyes to fuse SMI-based movement (or position) analysis.

At 1126, method 1100 can include updating the gaze-to-head-mounted-display (gaze-HMD) vector (ie, determining how the eye gaze vector intersects the display, or how the eye gaze vector moves relative to the display).

At 1166, method 1100 may optionally use the output of IMU 1154 or outward-facing camera 1156 (ie, a camera that focuses on the environment around the user rather than the user's eyes) to perform inertial, video, or video-inertial odometry. The HMD-to-world (HMD-world) vector can then be determined or updated at 1168 using video-inertial odometry.

At 1170, method 1100 can include determining or updating a gaze-world vector. Such vectors can be used to augment the user's reality, eg, via a pair of glasses.

At 1128, method 1100 can include causing a display subsystem of the HMD (or another display) to adjust (eg, in an AR or VR environment) the rendering of text, numbers, or images on the display. In some cases, the adjustment may be in response to classifying eye movement (eg, classifying as smooth pursuit, saccade, fixation, nystagmus, or blink).

12A and 12B illustrate how one or more surfaces or structures of an eye 1200 can be mapped using a set of one or more SMI sensors. By way of example, a single SMI sensor 1202 is shown in FIGS. 12B and 12B . In some examples, SMI sensor 1202 may be replaced with multiple SMI sensors (eg, multiple discrete SMI sensors or an array of SMI sensors). The SMI sensor may be any of the SMI sensors described with reference to FIGS. 1 to 11B .

12A and 12B each show two side views of the same eye 1200 . The first side view in each figure (i.e., side views 1204 and 1206) shows a cross-section of eye 1200, and the second side view in each figure (i.e., side views 1208 and 1210) shows a computer-generated model of various eye structures identified by the processor after analyzing the SMI signal generated by the SMI sensor or multiple SMI signals generated by a plurality of different SMI sensors. Figure 12A shows the eye 1200 in a first position, and Figure 12B shows the eye 1200 in a second position.

In the case of a single SMI sensor 1202, the SMI sensor 1202 may be mounted to the head-mounted frame by means of a MEMS or other structure 1212 that enables the beam 1214 emitted by the SMI sensor 1202 to be scanned across the eye 1200, or the beam 1214 emitted by the SMI sensor 1202 may be received by a set of one or more optical elements that may be adjusted to scan the beam 1214 across the eye 1200. Alternatively, beam 1214 may be split using a beam splitter, and multiple beams may strike eye 1200 simultaneously or sequentially. Alternatively, the SMI sensor 1202 may be mounted to the head-mounted frame at a fixed location, and the user may be asked to move their eyes 1200 to different positions while the SMI sensor 1202 emits a beam.

A processor, such as any of the processors described herein, may receive the SMI signals generated by the set of one or more SMI sensors and use the SMI signals to determine a set of ranges for a set of points on or in the eye. Ranges may include absolute or relative ranges. The processor may then generate a map of at least one structure of the eye using the set of ranges. The graph may be a two-dimensional (2D) graph or a three-dimensional (3D) graph.

In some embodiments, the processor may be configured to use the map to identify a structure of the eye, or a boundary between a first structure of the eye and a second structure of the eye. Identified structures may include, for example, one or more of iris 1218, sclera 1220, pupil 1222, lens 1224, limbus 1226, eyelids, and the like.

In some embodiments, the processor may operate an optical sensor subsystem (eg, MEMS, one or more optical elements, a beam splitter, etc.) to direct one or more beams of light toward the identified structure of the eye. In some cases, the structure may be more diffuse than another structure of the eye.

In some embodiments, the processor may be configured to use the map to determine gaze vectors 1216 for the eyes. In some embodiments, the processor may also or alternatively be configured to obtain or construct a Doppler cloud using a set of one or more SMI signals. The one or more SMI signals correspond to simultaneously or sequentially projecting or emitting multiple beams of a set of one or more beams, and/or scanning at least one beam of a set of one or more beams. Doppler clouds can be obtained or constructed with or without VCSEL wavelength modulation. The processor may also or alternatively obtain or construct a deep cloud. Deep clouds can be obtained or constructed using only VCSEL wavelength modulation.

For example, when the wavelength of light emitted by one or more SMI sensors is modulated, and when the beam emitted by at least one SMI sensor is scanned and/or emits multiple beams, a Doppler cloud and/or depth cloud may be obtained or constructed. Additionally or alternatively, a Doppler cloud may be obtained or constructed when the wavelength of light emitted by the one or more SMI sensors is not modulated. As mentioned earlier, single or multiple frames of a Doppler cloud can be considered a differential depth cloud. Using measurements from single or multiple frames of the Doppler cloud processed in real time, predefined and/or locally calibrated difference maps or libraries can be matched, and/or eye tracking or position information can be extracted. As described herein, a locally calibrated differential map or library can be obtained including, but not limited to, cameras, depth clouds, and the like. Additionally, the use of Doppler clouds alone or fused with depth clouds or other sensing modalities (eg, eye camera images, motion sensors, etc.) can be used to provide an accurate and efficient way of tracking eye movement or position information.

The above description, for purposes of explanation, used specific nomenclature to provide a thorough understanding of the described embodiments. It will be apparent, however, to one of ordinary skill in the art, after reading this specification, that the described embodiments can be practiced without the specific details. Thus, the foregoing descriptions of specific embodiments described herein are presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the embodiments to the precise forms disclosed. It will be apparent to those of ordinary skill in the art in view of the present specification that many modifications and variations are possible in light of the above teachings.

As noted above, one aspect of the present technology may be the collection and use of data available from various sources, including biometric data (eg, the surface quality of a user's skin or fingerprints). This disclosure contemplates that, in some cases, this collected data may include personal information data that uniquely identifies or can be used to identify, locate, or contact a particular person. Such Personal Information Data may include, for example, biometric data (e.g., fingerprint data) and data linked thereto (e.g., demographic data, location-based data, phone numbers, email addresses, home addresses, data or records related to a user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information).

This disclosure recognizes that the use of such personal information data in the present technology may be used to benefit the user. For example, personal information data may be used to authenticate users to access their devices, or to collect performance metrics for user interactions with augmented or virtual worlds. In addition, this disclosure also contemplates other uses of personal information data that benefit users. For example, health and fitness data can be used to provide insights into a user's overall health, or can be used as positive feedback for individuals using technology to pursue health goals.

This disclosure envisages that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such Personal Information data will adhere to established privacy policies and/or privacy practices. Specifically, such entities shall implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as data collection and/or use changes. Personal information from users should be collected for the entity's lawful and reasonable uses and not shared or sold outside of those lawful uses. In addition, such collection/sharing should be done after receiving informed consent from users. In addition, such entities should consider taking any necessary steps to safeguard and secure access to such Personal Information Data and to ensure that others who have access to Personal Information Data comply with their privacy policies and procedures. In addition, such entities may subject themselves to third-party assessments to demonstrate compliance with widely accepted privacy policies and practices. In addition, policies and practices should be tailored to the specific types of personal data collected and/or accessed, and to applicable laws and standards including jurisdiction-specific considerations. For example, in the United States, the collection or acquisition of certain health data may be governed by federal and/or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be treated accordingly. Therefore, different privacy practices should be maintained in each country for different types of personal data.

Notwithstanding the foregoing, this disclosure also contemplates embodiments in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements and/or software elements may be provided to prevent or prevent access to such personal information data. For example, with respect to advertising delivery services, the technology of the present invention can be configured to allow users to choose to "opt in" or "opt out" to participate in the collection of personal information data at any time during or after registration for the service. As another example, a user may choose not to provide data to a targeted content delivery service. In yet another example, the user may choose to limit the length of time the data is maintained, or completely prohibit the development of a baseline profile for the user. In addition to providing "opt-in" and "opt-out" options, this disclosure contemplates providing notice related to access or use of personal information. For example, users can be notified that their personal information data will be accessed when downloading an application, and then reminded again just before personal information data is accessed by the application.

Furthermore, it is an object of this disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risk is minimized by limiting data collection and deleting data once it is no longer needed. Additionally, and when applicable, including in certain health-related applications, data de-identification may be used to protect user privacy. Where appropriate, de-identification may be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of data stored (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data across users), and/or other methods.

Thus, while this disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, this disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, the various embodiments of the technology of the present invention will not be unable to function normally due to the lack of all or part of such personal information data. For example, content may be selected and delivered to a user by inferring preferences based on non-personal information data or an absolute minimum amount of personal information, such as content requested by a device associated with the user, other non-personal information available to the content delivery service, or publicly available information.

Claims (27)

1. An eye tracking device comprising: head-mounted frame; an optical sensor subsystem mounted to the head-mounted frame and comprising, a set of one or more self-mixing interferometry (SMI) sensors; and a processor configured to: operating the optical sensor subsystem to cause the set of one or more SMI sensors to emit a set of one or more light beams toward the user's eyes; receiving a set of one or more SMI signals from the set of one or more SMI sensors; and Movement of the eye is tracked using the set of one or more SMI signals. 2. The eye tracking device of claim 1, wherein tracking the movement of the eye comprises estimating an angular velocity of the eye. 3. The eye tracking device of claim 1, wherein: the set of one or more SMI sensors includes at least two SMI sensors; and The tracked movement of the eye includes the tracked angular velocity of the eye in two dimensions. 4. The eye tracking device of claim 1, wherein: the set of one or more SMI sensors includes at least three SMI sensors; and The tracked movement of the eye includes the tracked angular velocity of the eye in three dimensions. 5. The eye-tracking device of claim 1, wherein the head-mounted frame comprises: a first mirror frame and a second mirror frame; a bridge connecting the first frame to the second frame; a first temple connected to the first mirror frame; and a second temple connected to the second mirror frame. 6. The eye tracking device of claim 1, further comprising: A display subsystem including a display mounted to the head-mounted frame. 7. The eye tracking device of claim 6, wherein: The processor is configured to: classifying the movement of the eye as at least one of blinking, smooth pursuit, saccade, fixation, or nystagmus; and The display subsystem is caused to adjust rendering of one or more images on the display in response to the classified movement of the eye. 8. The eye tracking device of claim 7, wherein: The processor is configured to: quantifying said classified movement of said eye; wherein, The rendering of the one or more images is further adjusted in response to the quantified movement of the eye. 9. The eye tracking device of claim 6, wherein: The processor is configured to: causing the display subsystem to change the state of the display in response to the movement of the eye. 10. The eye tracking device of claim 6, wherein: the display subsystem includes an array of display pixels mounted on a substrate; and The set of one or more SMI sensors includes at least one SMI sensor mounted on the substrate, adjacent to, or within the array of display pixels. 11. The eye tracking device of claim 1, wherein: The optical sensor subsystem includes, A beam splitter positioned to split the light beam emitted by the SMI sensors in the set of one or more SMI sensors. 12. The eye tracking device of claim 1, wherein: The optical sensor subsystem includes, waveguides positioned to direct light beams emitted by SMI sensors in the set of one or more SMI sensors to a plurality of outcouplings of the waveguides. 13. The eye tracking device of claim 1, wherein: The optical sensor subsystem includes, a beam steering component positioned to deflect a beam emitted by an SMI sensor in the set of one or more SMI sensors; and The processor is configured to operate the beam steering component and steer the beam. 14. The eye tracking device of claim 1, wherein: The optical sensor subsystem includes, optics positioned to focus light beams emitted by SMI sensors in the set of one or more SMI sensors; and a lens positioning mechanism; and The processor is configured to operate the lens positioning mechanism and focus the beam of light on a structure of the eye. 15. The eye tracking device of claim 1, wherein: The set of one or more SMI sensors includes, SMI sensor arrays; and circuitry configured to address different SMI sensors or different subsets of SMI sensors in the array of SMI sensors; and The processor is configured to selectively operate the different An SMI sensor or said different subset of SMI sensors. 16. The eye tracking device of claim 1, wherein: The processor is configured to: A set of ranges is determined for a set of points on or in the eye. 17. The eye tracking device of claim 16, wherein: The processor is configured to: A map of at least one structure of the eye is generated using the set of extents. 18. The eye tracking device of claim 17, wherein: The processor is configured to: identifying the structure of the eye using the map; and The optical sensor subsystem is operated to direct at least one beam of the set of one or more beams toward the identified structure of the eye. 19. The eye-tracking device of claim 16, wherein the set of ranges comprises a set of absolute ranges. 20. The eye-tracking device of claim 16, wherein the set of ranges comprises a set of relative ranges. 21. An eye tracking device comprising: a set of one or more self-mixing interferometry (SMI) sensors; camera; and a processor configured to: causing the camera to acquire a set of images of the user's eyes at a first frequency; causing the set of one or more SMI sensors to emit a set of one or more light beams toward the eye of the user; sampling the set of one or more SMI signals generated by the set of one or more SMI sensors at a second frequency greater than the first frequency; determining a gaze vector of the eye using at least one image in the set of images; and Movement of the eye is tracked using the set of one or more SMI signals. 22. The eye tracking device of claim 21, wherein: The processor is configured to: The gaze vector is updated using the tracked movement of the eye. 23. A method of tracking eye movement comprising: operating the optical sensor subsystem to cause a set of one or more self-mixing interferometry (SMI) sensors in the optical sensor subsystem to emit a set of one or more light beams toward the eye of the user; receiving a set of one or more SMI signals from the set of one or more SMI sensors; and The movement of the eye is tracked using the set of one or more SMI signals. 24. The method of claim 23, wherein using the set of one or more SMI signals to track the movement of the eye comprises: Angular velocity of the eye is estimated using the set of one or more SMI signals. 25. The method of claim 23, further comprising: at least one of the following: simultaneously projecting a plurality of beams of said set of one or more beams; or emitting the plurality of light beams sequentially; or scanning at least one beam of the set of one or more beams; wherein, Tracking said movement of said eye using said set of one or more SMI signals comprises: A Doppler cloud is constructed using the set of one or more SMI signals. 26. The method of claim 25, wherein using the set of one or more SMI signals to track the movement of the eye further comprises: Eye position information is extracted by matching one or more frames of the Doppler cloud to a difference map. 27. The method of claim 23, further comprising: modulating the emitted set of one or more light beams while, simultaneously projecting a plurality of beams of said set of one or more beams; or emitting the plurality of light beams sequentially; or scanning at least one beam of the set of one or more beams; wherein, Tracking said movement of said eye using said set of one or more SMI signals comprises: A depth cloud is constructed using the set of one or more SMI signals.