CN111316153A - Augmented reality display with waveguides configured to capture images of the eye and/or environment - Google Patents

Abstract

The head mounted display system may include a camera, at least one waveguide, at least one coupling optical element configured such that light is coupled into and directed within the waveguide, and at least one outcoupling element. At least one outcoupling element may be configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera. A camera may be positioned in the optical path relative to the at least one out-coupling optical element to receive at least a portion of the light that is coupled into the waveguide via a coupling element and guided therein and coupled by the out-coupling element Elements are coupled out of the waveguide so that images can be captured by the camera.

Description

Augmented reality display with waveguide configured to capture images of eye and/or environment

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is entitled "AUGMENTEDREALITY DISPLAY WITH EYEPIECE CONFIGURED TO CAPTURE IMAGES OF EYE ANDENVIRONMENT (with a waveguide configured to capture images of the eye and/or environment) under 35 U.S.C § 119(e)," filed on September 21, 2017. Augmented Reality Display)" of US Provisional Application No. 62/561645, the disclosure of which is incorporated herein by reference in its entirety.

technical field

The present disclosure relates to optical devices including augmented reality imaging and visualization systems.

Background technique

Modern computing and display technologies have facilitated the development of systems for so-called "virtual reality" or "augmented reality" experiences, in which digitally reproduced images or parts thereof are presented in such a way that they appear real or can be perceived as real to users. Virtual reality (or "VR") scenes typically involve the presentation of digital or virtual image information that is opaque to other actual real-world visual input; augmented reality (or "AR") scenes typically involve the presentation of digital or virtual image information as Enhancements to the visualization of the real world around the user. A mixed reality (or "MR") scene is an AR scene and typically involves virtual objects that are integrated into and respond to the natural world. For example, an MR scene may include AR image content that appears to be occluded by, or otherwise perceived to interact with, objects in the real world.

Referring to Figure 1, an augmented reality scene 10 is shown. The user of the AR technology sees a real-world park-like setting 20 featuring people, trees, buildings in the background, and a concrete platform 30 . The user also perceives that he/she "sees" "virtual content" such as a robot statue 40 standing on the real-world platform 30, and a flying cartoon-like avatar character 50 that appears to be an avatar of Bumblebee. These elements 50, 40 are "virtual" in that they do not exist in the real world. Because the human visual perception system is complex, and producing AR technology that facilitates a comfortable, natural, rich presentation of virtual image elements along with other virtual or real-world image elements is challenging.

The systems and methods disclosed herein address various challenges associated with AR and VR technologies.

Polarizing beam splitters can be used in display systems to direct polarized light to a light modulator, which then directs the light to a viewer. In general, there is a continuing need to reduce the size of display systems and, as such, there is also a continuing need to reduce the size of components of the display system, including components that utilize polarizing beam splitters.

SUMMARY OF THE INVENTION

Various implementations described herein include display systems configured to provide illumination and/or map image projections to the eye. Additionally or alternatively, the display system may image the eye and/or the environment.

In some embodiments, the head mounted display system is configured to project light onto a user's eyes to display augmented reality image content in the user's field of view. The head mounted display system may include a frame configured to be supported on a user's head. The display system may also include an image projector configured to project an image into the user's eye to display the image content in the user's field of view. The display system may include a camera, at least one waveguide, at least one coupling optical element configured such that light is coupled into and guided within the waveguide, and at least one out-coupling element. The at least one outcoupling element may be configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera. A camera may be placed in the optical path relative to the at least one outcoupling optical element to receive at least a portion of the light coupled into the waveguide via the coupling element and guided therein and through the outcoupling element This light is coupled out of the waveguide so that an image can be captured by the camera.

Description of drawings

Figure 1 shows a user view of augmented reality (AR) through an AR device.

Figure 2 shows an example of a wearable display system.

Figure 3 shows a conventional display system for simulating a three-dimensional image for a user.

4 illustrates aspects of a method of simulating a three-dimensional image using multiple depth planes.

5A-5C illustrate the relationship between the radius of curvature and the radius of focus.

Figure 6 shows an example of a waveguide stack for outputting image information to a user.

FIG. 7 shows an example of the outgoing beam output by the waveguide.

Figure 8 shows an example of a stacked waveguide assembly where each depth plane includes an image formed using multiple different component colors.

9A shows a cross-sectional side view of an example of a stack of waveguides, each stacked waveguide including an incoupling optical element. As discussed herein, the waveguide stack may include an eyepiece.

9B shows a perspective view of an example of the multiple stacked waveguides of FIG. 9A.

9C illustrates a top plan view of an example of the multiple stacked waveguides of FIGS. 9A and 9B.

10 schematically illustrates a cross-sectional side view of an example imaging system including an eyepiece, an image projector, a light source for illuminating the eye, and a camera for capturing an image of the eye.

Figure 11A schematically shows a light source for illuminating the eye and an image projector for injecting an image into the eye, both emitting light towards coupling optics on the waveguide of the eyepiece.

Figure 11B schematically shows projected light from a light source and from an image projector coupled into a waveguide.

Figure 11C schematically shows how the coupled light propagates through the waveguide by total internal reflection.

Figure 1 ID schematically shows the light from the light source and from the image projector being coupled out of the eyepiece.

Figure 1 IE schematically illustrates a waveguide and coupling optical element configured to propagate in-coupled light at least along the full dimension of the coupling optical element (eg, in the x-direction). Light entering the eye is shown as coming from an extended light source (eg, the area of the retina where the imaging light would capture).

Figure 12A is a cross-sectional view schematically showing light exiting the eye reflected from the retina and incident on the eyepiece.

Figure 12B schematically illustrates example light coupled into the waveguide of the eyepiece.

Figure 12C schematically illustrates collimated in-coupled light from the eye propagating through the waveguide towards the imaging device.

Figure 12D schematically illustrates in-coupled light from the eye propagating to one or more out-coupling optical elements.

Figure 12E schematically illustrates light from the eye being coupled out of the waveguide by outcoupling optics and directed to the camera so that an image of the eye (eg, retina) can be captured by the camera.

Figure 13A schematically illustrates how an imaging system can image various parts of the eye (eg, retina), which can enable the orientation of the eye to be determined and the eye position to be tracked.

Figure 13B shows a pattern of sequentially displayed gaze targets used to orient the eye in various different directions for imaging the retina. The resulting images correspond to different parts of the retina. For example, when the eye is oriented in various directions to view gaze targets that are positioned differently on the display, the image captured by the camera includes different parts of the retina. These images can be combined to form a larger map or composite image of the retina.

14A schematically shows a cross-sectional view of an imaging system including an eyepiece and a camera for collecting light from the environment in front of the eyepiece. Light from the environment is shown reflected or emitted from one or more physical objects in the environment. The collection of light from objects in the environment in front of the eyepiece can enable images of the environment to be captured.

Figure 14B schematically shows light from the environment coupled into the waveguide of the eyepiece by the coupling optical element.

Figure 14C schematically illustrates an imaging system that collects light from the environment using powered optics such as refractive optics (eg, a lens such as a wide field lens) in front of the eyepiece .

15A schematically illustrates an exemplary imaging system including polarization selective coupling-in optics for receiving light from an illumination source and coupling the light into a waveguide of an eyepiece. The eyepiece also includes a polarization selective light coupling element for coupling light out of the waveguide. A polarizer can be used to polarize light from the illumination source, and a half-wave retarder can be used to rotate the orientation of linearly polarized light so that the light is turned into the waveguide by polarization selective coupling-in optics.

Figure 15B schematically illustrates light from the eye (eg, light from the retina illuminated by infrared light from an illumination source) coupled back into the waveguide and directed to the camera for image capture.

Figure 16 schematically illustrates an imaging system configured to image the front of the eye (eg, the cornea). The imaging system includes an eyepiece as described above. The imaging system also includes a positive lens for collimating light collected from the front of the eye for coupling into the waveguide via the optical coupling element and propagation to the camera for image capture. The system also includes a negative lens to counteract the positive power introduced by the positive lens and prevent otherwise inverting the image of the environment in front of the eyepiece caused by the positive lens.

Figure 17 schematically illustrates another example imaging system configured to image the front of the eye (eg, the cornea). The imaging system includes a curved wavelength selective reflector for collimating light from the front of the eye for coupling into the waveguide via an optical coupling element and propagation to the camera for image capture. The wavelength selective reflector may operate in a reflective manner for infrared light reflected from the eye and in a transmissive manner for visible light from the environment in front of the user.

Figure 18 schematically illustrates an example imaging system that also includes a curved wavelength selective reflector for collimating light from the front of the eye for coupling into the waveguide via an optical coupling element and Propagated to the camera for image capture. Polarization selectivity can be employed to assist in controlling the path of light reflected from the eye. Illumination to the eye is provided via the waveguide, rather than multiple light sources positioned between the waveguide and the eye, as shown in FIG. 18 .

Figure 19 schematically shows an imaging system that includes a shutter to assist in the process for noise subtraction.

Figures 20A-20E schematically illustrate an alternative process for noise subtraction using wavelength modulation in conjunction with a curved wavelength selective reflector.

21 illustrates an example eyepiece that can be used to simultaneously project light into a user's eye to provide image content thereto and simultaneously receive image data of the user's eye or the environment in front of the user.

Figure 22 shows a cross-sectional side view of an example of a cholesteric liquid crystal diffraction grating (CLCG) with multiple uniform chiral structures.

23 shows an example of an imaging system that includes a forward-facing camera configured to image a wearer's eye using a cholesteric liquid crystal (CLC) off-axis mirror.

The drawings are provided to illustrate example embodiments, and are not intended to limit the scope of the present disclosure. The same reference numerals refer to the same parts throughout.

Detailed ways

Reference will now be made to the drawings, wherein like reference numerals refer to like parts throughout.

FIG. 2 shows an example of a wearable display system 60 . Display system 60 includes display 70 , and various mechanical and electronic modules and systems that support the functionality of display 70 . Display 70 may be coupled to a frame 80 that may be worn by a display system user or viewer 90 and configured to position display 70 in front of user 90's eyes. In some embodiments, the display 70 may be considered glasses. In some embodiments, speaker 100 is coupled to frame 80 and is configured to be positioned near the ear canal of user 90 (in some embodiments, another speaker (not shown) may optionally be positioned in the user's other ear) near the channel to provide stereo/shaping sound control). The display system may also include one or more microphones 110 or other means to detect sound. In some embodiments, the microphone is configured to allow the user to provide input or commands to the system 60 (eg, a selection of voice menu commands, natural language questions, etc.) and/or may allow communication with others (eg, with other similar display systems) user) audio communication. Microphones may also be configured as peripheral sensors to collect audio data (eg, sounds from the user and/or the environment). In some embodiments, the display system may also include peripheral sensors 120a, which may be separate from the frame 80 and attached to the body of the user 90 (eg, on the head, torso, extremities, etc. of the user 90). In some embodiments, peripheral sensor 120a may be configured to acquire data characterizing the physiological state of user 90 . For example, sensor 120a may be an electrode.

With continued reference to FIG. 2, the display 70 is operably coupled to a local data processing and module 140 through a communication link 130, such as through a wired lead or a wireless connection, which may be installed in various configurations, such as by Fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, embedded within a headset, or otherwise removably attached to the user 90 (eg, in a backpack-style configuration, in a belt-coupled connection configuration). Similarly, sensor 120a may be operably coupled to local processing and data module 140 through communication link 120b (eg, through a wired lead or wireless connection). Local processing and data module 140 may include a hardware processor and digital memory such as non-volatile memory (eg, flash memory or hard drive), both of which may be used to assist in processing, caching, and storing data. This data includes: a) data captured from sensors (eg, which may be operably coupled to the frame 80 or otherwise operably attached to the user 90), such as image capture devices (such as cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, radios, gyroscopes, and/or other sensors disclosed herein; and/or b) use of remote processing module 150 and/or remote data repository 160 (which includes and virtual content related data) acquired and/or processed data that may be communicated to the display 70 after such processing or retrieval. Local processing and data module 140 may be operably coupled to remote processing module 150 and remote data repository 160 by communication links 170, 180, such as via wired or wireless communication links, such that these remote modules 150, 160 are operatively connected to each other Coupled and available as a resource for the local processing and data module 140 . In some embodiments, the local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a radio, and/or a gyroscope. In some other embodiments, one or more of these sensors may be attached to frame 80 or may be a separate structure that communicates with local processing and data module 140 through wired or wireless communication paths.

With continued reference to FIG. 2, in some embodiments, remote processing module 150 may include one or more processors configured to analyze and process data and/or map image information. In some embodiments, the remote data repository 160 may include a digital data storage facility that may be available through the Internet or other network configuration in a "cloud" resource configuration. In some embodiments, remote data repository 160 may include one or more remote servers that provide local processing and data module 140 and/or remote processing module 150 with information, eg, information for generating augmented reality content . In some embodiments, all data is stored and all computations are performed in the local processing and data module, allowing fully autonomous use from remote modules.

Referring now to Figure 3, the perception of an image as "three-dimensional" or "3-D" can be accomplished by providing a slightly different representation of the image to each eye of the viewer. Figure 3 shows a conventional display system for simulating a three-dimensional image for a user. Two different images 190, 200 - one for each eye 210, 220 - are output to the user. The images 190, 200 are spaced a distance 230 from the eyes 210, 220 along an optical or z-axis parallel to the gaze of the viewer's eyes. The images 190, 200 are flat and the eyes 210, 220 can focus on the images by presenting a single accommodate state. Such 3D display systems rely on the human visual system to combine the images 190, 200 to provide depth perception and/or scale of the combined images.

However, it should be appreciated that the human visual system is more complex and more challenging to provide a true perception of depth. For example, many viewers of conventional "3-D" display systems find such systems uncomfortable or incapable of perceiving depth at all. Without being bound by theory, it is believed that a viewer of an object may perceive the object as "three-dimensional" due to a combination of vergence and adaptation. The vergence motion of the two eyes relative to each other (i.e., the rotation of the eyes so that the pupils move toward or away from each other to converge the eye gaze of the eyes to fixate on an object) is closely related to the focusing (or "adaptation") of the lens and pupil of the eye related. Under normal circumstances, changing the focus of the eye's lens, or adapting the eye to change focus when switching from one object to another at a different vergencereflex)” relationship and pupil dilation or constriction automatically results in matching changes that bring the vergence to the same distance. Also, under normal conditions, changes in vergence will trigger adaptive changes in lens shape and pupil size. As described herein, many stereoscopic or "3-D" display systems display a scene to each eye using slightly different renderings (and thus slightly different images) such that the human visual system perceives three-dimensional perspective. However, such systems are uncomfortable for many viewers because, among other things, such systems simply provide different representations of the scene, with the eye viewing all image information in a single state of adaptation, and violations of "adaptation" - Verge reflections" work. A display system that provides a better match between adaptation and vergence can result in a more realistic and comfortable three-dimensional image simulation.

4 illustrates aspects of a method of simulating a three-dimensional image using multiple depth planes. With continued reference to Figure 4, objects at different distances from the eyes 210, 220 on the z-axis are accommodated by the eyes 210, 220 to bring those objects in focus. The eyes 210, 220 exhibit specific adaptation states to bring objects into focus at different distances along the z-axis. Thus, it can be said that a particular adaptation state is associated with a particular one of the depth planes 240 having an associated focal length such that when the eye is in the adaptation state for that depth plane, objects in the particular depth plane or partial focus of the subject. In some embodiments, three-dimensional images may be simulated by providing different representations of the images for each of the eyes 210, 220, and may also be simulated by providing different representations of images corresponding to each of the depth planes image. Although shown separated for clarity of illustration, it should be understood that, for example, the fields of view of eyes 210, 220 may overlap as distance along the z-axis increases. Additionally, while shown flat for ease of illustration, it should be understood that the profile of the depth plane may be curved in physical space, for example, so that all features in the depth plane are in focus with the eye at a particular state of accommodation .

The distance between the object and the eye 210 or 220 can also change the amount of divergence of light from the object, as seen by the eye. 5A-5C illustrate the relationship between distance and ray divergence. The distance between the object and the eye 210 is represented by the order of decreasing distances R1, R2 and R3. As shown in Figures 5A-5C, the rays become more divergent as the distance to the object decreases. As the distance increases, the light becomes more collimated. In other words, it can be said that the light field produced by a point (an object or part of an object) has a spherical wavefront curvature that is a function of how far the point is from the user's eyes. The curvature increases as the distance between the object and the eye 210 decreases. Therefore, at different depth planes, the divergence of the rays is also different, and the divergence increases as the distance between the depth plane and the viewer's eye 210 decreases. Although only a single eye 210 is shown in FIGS. 5A-5C and other figures herein for clarity of illustration, it should be understood that discussions about eye 210 may apply to both eyes 210 and 220 of a viewer.

Without being bound by theory, it is believed that the human eye can generally interpret a limited number of depth planes to provide depth perception. Thus, by providing the eye with a different representation of an image corresponding to each of these limited number of depth planes, a highly confident simulation of perceived depth can be achieved. Different presentations can be individually focused by the viewer's eyes, helping to provide depth cues to the user based on eye adaptation and/or based on viewing different image features at different depth planes that are out of focus. It is required to focus on different image features of the scene located on different depth planes.

Figure 6 shows an example of a waveguide stack for outputting image information to a user. Display system 250 includes a waveguide stack or stacked waveguide assembly 260 that can be used to employ a plurality of waveguides 270, 280, 290, 300, 310 to provide three-dimensional perception to the eye/brain. In some embodiments, the display system 250 is the system 60 of FIG. 2 , of which FIG. 6 schematically illustrates portions of the system 60 in greater detail. For example, the waveguide assembly 260 may be part of the display 70 of FIG. 2 . It should be understood that in some embodiments, display system 250 may be considered a light field display. Additionally, the waveguide assembly 260 may also be referred to as an eyepiece.

With continued reference to Figure 6, the waveguide assembly 260 may further include a plurality of features 320, 330, 340, 350 located between the waveguides. In some embodiments, features 320, 330, 340, 350 may be one or more lenses. The waveguides 270, 280, 290, 300, 310 and/or the plurality of lenses 320, 330, 340, 350 may be configured to send image information to the eye with various levels of wavefront curvature or ray divergence. Each waveguide level can be associated with a particular depth plane and can be configured to output image information corresponding to that depth plane. The image injection device 360, 370, 380, 390, 400 can be used as a light source for the waveguides and can be used to inject image information into the waveguides 270, 280, 290, 300, 310, each of which can be is configured to distribute incident light through each respective waveguide for output to the eye 210 . Light exits the output surfaces 410, 420, 430, 440, 450 of the image injection devices 360, 370, 380, 390, 400 and is injected into the corresponding input surfaces 460, 470, 480, 490, 500. In some embodiments, each of the input surfaces 460, 470, 480, 490, 500 may be an edge of the respective waveguide, or may be a portion of the major surface of the respective waveguide (ie, directly facing the world 510 or the viewer's eye). 210 of the waveguide surfaces). In some embodiments, a single beam (eg, a collimated beam) can be injected into each waveguide to output clones oriented toward the eye 210 at a particular angle (and amount of divergence) corresponding to the depth plane associated with the particular waveguide The entire field of view of the collimated beam. In some embodiments, a single one of the image injection devices 360, 370, 380, 390, 400 may be associated with multiple (eg, three) of the waveguides 182, 184, 186, 188, 190 and will Light is injected into a plurality (eg, three) of the waveguides 270 , 280 , 290 , 300 , 310 .

In some embodiments, the image injection devices 360, 370, 380, 390, 400 are separate displays, each display generating image information for injection into the respective waveguides 270, 280, 290, 300, 310, respectively. In some other embodiments, the image injection device 360, 370, 380, 390, 400 is the output of a single multiplexed display, which may feed the image injection device 360, eg, via one or more light pipes (such as fiber optic cables) , 370 , 380 , 390 , 400 each of the image injection devices uses a tube to deliver image information. It will be appreciated that the image information provided by the image injection devices 360, 370, 380, 390, 400 may include light of different wavelengths or colors (eg, as discussed herein, different component colors).

In some embodiments, the light injected into the waveguides 270, 280, 290, 300, 310 is provided by a light projector system 520 including a light module 540, which may include light emitters, such as light emitting diodes (LEDs). Light from light module 540 may be directed to light modulator 530 and modified by light modulator 530 (eg, a spatial light modulator) via optical beam splitter 550 . The light modulator 540 may be configured to vary the perceived intensity of light injected into the waveguides 270 , 280 , 290 , 300 , 310 . Examples of spatial light modulators include liquid crystal displays (LCDs), including liquid crystal on silicon (LCOS) displays. It should be understood that the image injection devices 360, 370, 380, 390, 400 are shown schematically and that in some embodiments these image injection devices may represent different light paths and locations in a common projection system that is Configured to output light into an associated one of the waveguides 270 , 280 , 290 , 300 , 310 .

In some embodiments, display system 250 may be a scanning fiber display that includes one or more scanning fibers configured to scan in various patterns (eg, raster scan, helical scan, Lissajous ( Lissajous patterns, etc.) project light into one or more waveguides 270 , 280 , 290 , 300 , 310 and ultimately into the eye 210 of the viewer. In some embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may schematically represent a single scanning fiber or a bundle of scanning fibers configured to inject light into in one or more waveguides 270 , 280 , 290 , 300 , 310 . In some other embodiments, the illustrated image injection devices 360, 370, 380, 390, 400 may schematically represent a plurality of scanning fibers or bundles of scanning fibers, each of the plurality of scanning fibers or bundles of scanning fibers Each of the are configured to inject light into an associated one of the waveguides 270 , 280 , 290 , 300 , 310 . It should be appreciated that one or more optical fibers may be configured to transmit light from the optical module 540 to the one or more waveguides 270 , 280 , 290 , 300 , 310 . It will be appreciated that one or more intermediate optical structures may be provided between the one or more scanning fibers and the one or more waveguides 270, 280, 290, 300, 310, for example to redirect light emerging from the scanning fibers to a or multiple waveguides 270 , 280 , 290 , 300 , 310 .

The controller 560 controls the operation of one or more of the stacked waveguide assemblies 260 , including the operation of the image injection devices 360 , 370 , 380 , 390 , 400 , the light source 540 and the light modulator 530 . In some embodiments, controller 560 is part of local data processing module 140 . Controller 560 includes programming (eg, instructions in a non-transitory medium) that adjusts the timing and provision of image information to waveguides 270, 280, 290, 300, 310, eg, according to any of the various aspects disclosed herein. In some embodiments, the controller may be a single monolithic device, or a distributed system connected by wired or wireless communication channels. In some embodiments, controller 560 may be part of processing module 140 or 150 (FIG. 2).

With continued reference to Figure 6, the waveguides 270, 280, 290, 300, 310 may be configured to propagate light within each respective waveguide by total internal reflection (TIR). The waveguides 270, 280, 290, 300, 310 may each be planar or have other shapes (eg, curved) with top and bottom major surfaces and edges extending between these top and bottom major surfaces . In the configuration shown, the waveguides 270, 280, 290, 300, 310 may each include out-coupling optical elements 570, 580, 590, 600, 610 configured to propagate within each respective waveguide by The light is redirected out of the waveguide and extracted out of the waveguide to output image information to the eye 210 . The extracted light may also be referred to as out-coupled light, and the out-coupled optical element light may also be referred to as light extraction optics. The extracted light beam may be output by the waveguide at a position where the light propagating in the waveguide impinges on the light extraction optical element. The outcoupling optical elements 570, 580, 590, 600, 610 may, for example, comprise gratings of diffractive optical features, as discussed further herein. Although illustrated as being disposed at the bottom major surfaces of the waveguides 270 , 280 , 290 , 300 , 310 for ease of description and clarity of drawing, in some embodiments the outcoupling optical elements 570 , 580 , 590 , 600 , 610 may be disposed at the top and/or bottom major surfaces, and/or may be disposed directly in the volume of the waveguides 270, 280, 290, 300, 310, as discussed further herein. In some embodiments, outcoupling optical elements 570 , 580 , 590 , 600 , 610 may be formed in layers of material attached to the transparent substrate to form waveguides 270 , 280 , 290 , 300 , 310 . In some other embodiments, the waveguides 270, 280, 290, 300, 310 can be a single piece of material, and the outcoupling optical elements 570, 580, 590, 600, 610 can be formed on the surface of the piece of material and/or the inside the sheet material.

With continued reference to Figure 6, as discussed herein, each waveguide 270, 280, 290, 300, 310 is configured to output light to form an image corresponding to a particular depth plane. For example, the waveguide 270 closest to the eye may be configured to deliver collimated light (which is injected into such waveguide 270 ) to the eye 210 . Collimated light can represent the optical infinity focal plane. The next upstream waveguide 280 may be configured to send collimated light passing through the first lens 350 (eg, a negative lens) before it can reach the eye 210; such a first lens 350 may be configured to produce a slightly convex Wavefront curvature such that the eye/brain interprets light from the next ascending waveguide 280 as coming from a first focal plane closer inward toward the eye 210 from optical infinity. Similarly, the third upstream waveguide 290 has its output light passing through the first lens 350 and the second lens 340 before reaching the eye 210; the combined optical power of the first lens 350 and the second lens 340 can be configured To create another incremental wavefront curvature so that the eye/brain interprets light from the third waveguide 290 as coming from a second focal plane that is farther from optical infinity than light from the next upstream waveguide 280 Closer inward toward the person.

The other waveguide layers 300, 310 and lenses 330, 320 are similarly configured, with the highest waveguide 310 in the stack sending its output through all lenses between it and the eye for the aggregate power representing the focal plane closest to the person ( aggregate focal power). To compensate for the stack of lenses 320, 330, 340, 350 when viewing/interpreting light from the world 510 on the other side of the stacked waveguide assembly 260, a compensation lens layer 620 may be provided at the top of the stack to compensate for the lens stack below 320, 330, 340, 350 focal power. This configuration provides as many perceptual focal planes as there are waveguide/lens pairings available. The outcoupling optics of the waveguide and the focusing aspects of the lens may be static (ie, not dynamic or electroactive). In some alternative embodiments, either or both may be dynamic using electroactive features.

In some embodiments, two or more of the waveguides 270, 280, 290, 300, 310 may have the same associated depth plane. For example, multiple waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same depth plane, or multiple subsets of waveguides 270, 280, 290, 300, 310 may be configured to output images set to the same An image of multiple depth planes, one for each depth plane. This can provide advantages for forming tiled images to provide an extended field of view at those depth planes.

With continued reference to Figure 6, the outcoupling optical elements 570, 580, 590, 600, 610 can be configured to redirect light out of their respective waveguides and have the appropriate amount of divergence for the particular depth plane output associated with that waveguide or a collimated amount of this light. As a result, waveguides with different associated depth planes can have different configurations of out-coupling optical elements 570, 580, 590, 600, 610 that output light with different amounts of divergence depending on the associated depth plane . In some embodiments, the light extraction optical elements 570, 580, 590, 600, 610 may be volumes or surface features that may be configured to output light at specific angles. For example, the light extraction optical elements 570, 580, 590, 600, 610 may be volume holograms, surface holograms and/or diffraction gratings. In some embodiments, features 320, 330, 340, 350 may not be lenses; rather, they may simply be spacers (eg, cladding and/or structures used to form air gaps).

In some embodiments, outcoupling optical elements 570, 580, 590, 600, 610 are diffractive features that form a diffractive pattern, or "diffractive optical elements" (also referred to herein as "DOEs"). Preferably, the DOE has sufficiently low diffraction efficiency that only a portion of the light beam is deflected toward the eye 210 through each intersection of the DOE, while the remainder continues to travel through the waveguide via TIR. The light carrying the image information is thus split into multiple correlated exit beams that exit the waveguide at multiple locations, and the result is a fairly uniform pattern of exit emission towards the eye 210 for this particular collimated beam that bounces within the waveguide .

In some embodiments, one or more DOEs may be switchable between an "on" state where they are actively diffracting and an "off" state where they are not diffracting significantly. For example, a switchable DOE can include a polymer dispersed liquid crystal layer in which the droplets contain a diffraction pattern in a host medium, and the index of refraction of the droplets can be switched to substantially match the index of refraction of the host material (in this case, The pattern does not appreciably diffract incident light), or the droplets can be switched to an index of refraction that does not match that of the host medium (in which case the pattern actively diffracts incident light).

In some embodiments, a camera assembly 630 (eg, a digital camera, including visible and infrared cameras) may be provided to capture images of the eye 210 and/or tissue surrounding the eye 210, eg, to detect user input and/or monitor the user's physiological state. As used herein, a camera can be any image capture device. In some embodiments, camera assembly 630 can include an image capture device and a light source to project light (eg, infrared light) to the eye, which can then be reflected by the eye and detected by the image capture device. In some embodiments, camera assembly 630 may be attached to frame 80 ( FIG. 2 ) and may be in electrical communication with processing modules 140 and/or 150 , which may process image information from camera assembly 630 . In some embodiments, one camera assembly 630 may be used for each eye to monitor each eye separately.

Referring now to FIG. 7, an example of an exit beam output by the waveguide is shown. One waveguide is shown, but it should be understood that other waveguides in waveguide assembly 260 (FIG. 6) may function similarly, wherein waveguide assembly 260 includes multiple waveguides. Light 640 is injected into waveguide 270 at input surface 460 of waveguide 270 and propagates within waveguide 270 by TIR. At the point at which light 640 impinges on DOE 570, a portion of the light exits the waveguide as outgoing beam 650. The exit beam 650 is shown as being substantially parallel, but as discussed herein, depending on the depth plane associated with the waveguide 270, the exit beam 650 may also be redirected at an angle (eg, to form a diverging exit beam) to Spread to eye 210. It should be understood that a substantially parallel outgoing beam may be indicative of a waveguide having out-coupling optics that couple the light out to form what appears to be disposed at a large distance (eg, optical infinity) from the eye 210 image on the depth plane. Other waveguides or other sets of out-coupling optics may output a more divergent outgoing beam pattern, which would require the eye 210 to adapt at a closer distance to focus it on the retina and would be interpreted by the brain as coming from a beam that is closer to the eye 210 than optical infinity. distance light.

In some embodiments, a full-color image may be formed at each depth plane by superimposing images of each component color (eg, three or more component colors). Figure 8 shows an example of a stacked waveguide assembly where each depth plane includes an image formed using multiple different component colors. The illustrated embodiment shows depth planes 240a-240f, although greater or lesser depths are contemplated. Each depth plane may have three or more component color images associated with it, including: a first image G of a first color; a second image R of a second color; and a third image of a third color image B. For the optical power (dpt) after the letters G, R and B, the different depth planes are represented in the figures by different numbers. As an example, the number following each of these letters represents the optical power (1/m), or the inverse distance of the depth plane from the viewer, and each box in the figure represents a separate component color image. In some embodiments, to account for differences in the eye's focusing of different wavelengths of light, the precise placement of the depth planes for the different component colors may vary. For example, different component color images for a given depth plane may be placed on the depth plane corresponding to different distances from the user. Such an arrangement may increase visual acuity and user comfort and/or may reduce chromatic aberration.

In some embodiments, the light of each component color may be output by a single dedicated waveguide, and thus, each depth plane may have multiple waveguides associated with it. In such an embodiment, each box in the figure including the letter G, R or B may be understood to represent a separate waveguide, and each depth plane may provide three waveguides, wherein each depth plane provides three component colors image. Although the waveguides associated with each depth plane are shown in this figure adjacent to each other for ease of description, it should be understood that in a physical setup, the waveguides could all be arranged in a stack with one waveguide per layer. In some other embodiments, multiple component colors may be output by the same waveguide, such that, for example, only a single waveguide may be provided per depth plane.

With continued reference to Figure 8, in some embodiments, G is green, R is red, and B is blue. In some other embodiments, other colors associated with other wavelengths of light (including magenta and cyan) may be used in addition to, or in place of, red, green, or blue or more.

It should be understood that references to light of a given color throughout this disclosure will be understood to include light of one or more wavelengths within the range of wavelengths that a viewer perceives to have light of the given color. For example, red light can include one or more wavelengths of light in the range of about 620-780 nm, green light can include one or more wavelengths of light in the range of about 492-577 nm, and blue light can include light of one or more wavelengths in the range of about 435- One or more wavelengths of light in the range of 493nm.

In some embodiments, light source 540 (FIG. 6) may be configured to emit light at one or more wavelengths (eg, infrared and/or ultraviolet wavelengths) outside the visual perception range of a viewer. In addition, coupling in, outcoupling, and other light redirecting structures of the waveguides of the display 250 can be configured to direct the light out of the display and emit it toward the user's eye 210, eg, for imaging and/or user stimulation applications.

Referring now to FIG. 9A, in some embodiments, it may be necessary to redirect light impinging on the waveguide to couple the light into the waveguide. The light can be redirected and coupled into its corresponding waveguide using coupling optics. FIG. 9A shows a cross-sectional side view of an example of a plurality of stacked waveguides or group of stacked waveguides 660, each stacked waveguide including an in-coupling optical element. Each waveguide may be configured to output one or more different wavelengths of light, or one or more different wavelength ranges of light. It should be understood that stack 660 may correspond to stack 260 (FIG. 6) and that the resulting stack of waveguides 660 may correspond to a portion of multiple waveguides 270, 280, 290, 300, 310, except from image injection devices 360, 370, One or more of 380, 390, 400 is injected into the waveguide from a location where the light needs to be redirected for incoupling.

The illustrated stacked waveguide set 660 includes waveguides 670 , 680 and 690 . Each waveguide includes an associated in-coupling optical element (which may also be referred to as a light input region on the waveguide), eg, in-coupling optical element 700 disposed on a major surface (eg, upper major surface) of waveguide 670, In-coupling optical element 710 disposed on a major surface (eg, upper major surface) of waveguide 680 , and in-coupling optical element 720 disposed on a major surface (eg, upper major surface) of waveguide 690 . In some embodiments, one or more of the in-coupling optical elements 700, 710, 720 may be disposed on the bottom major surface of the respective waveguide 670, 680, 690 (especially if the one or more in-coupling optical elements are in the case of reflective, deflecting optics). As shown, the in-coupling optical elements 700, 710, 720 may be disposed on the upper major surface of their respective waveguides 670, 680, 690 (or on top of the next lower waveguide), particularly on those in-coupling optical elements In the case of transmissive, deflecting optics. In some embodiments, the in-coupling optical elements 700 , 710 , 720 may be disposed in the body of the respective waveguides 670 , 680 , 690 . In some embodiments, as discussed herein, the coupling optical elements 700, 710, 720 are wavelength selective such that they selectively redirect one or more wavelengths of light while transmitting other wavelengths of light. Although shown on one side or corner of their respective waveguides 670, 680, 690, it should be understood that in some embodiments, the in-coupling optical elements 700, 710, 720 may be disposed on their respective waveguides 670, 680, 690 in other areas.

As shown, the in-coupling optical elements 700, 710, 720 may be laterally offset from each other. In some embodiments, each in-coupling optical element can be shifted such that it receives light without the light passing through the other in-coupling optical element. For example, each in-coupling optical element 700, 710, 720 may be configured to receive light from a different image injection device 360, 370, 380, 390, and 400 as shown in FIG. 710 , 720 are separated (eg, laterally spaced apart) such that it receives substantially no light from other coupled-in optical elements 700 , 710 , 720 .

Each waveguide also includes associated light distributing elements, eg, light distributing element 730 disposed on a major surface (eg, top major surface) of waveguide 670, surface) and light distributing element 750 disposed on a major surface (eg, the top major surface) of the waveguide 690. In some other embodiments, the light distributing elements 730, 740, 750 may be disposed on the bottom major surfaces of the associated waveguides 670, 680, 690, respectively. In some other embodiments, the light distributing elements 730, 740, 750 may be disposed on the top and bottom major surfaces of the associated waveguides 670, 680, 690, respectively; or the light distributing elements 730, 740, 750 may be disposed on different The top and bottom major surfaces of the associated waveguides 670, 680, 690 are on different ones.

The waveguides 670, 680, 690 may be spaced and separated by, for example, layers of gas, liquid and/or solid material. For example, as shown, layer 760a may separate waveguides 670 and 680; and layer 760b may separate waveguides 680 and 690. In some embodiments, layers 760a and 760b are formed of a low index of refraction material (ie, a material having a lower index of refraction than the material forming the immediately adjacent ones of the waveguides 670, 680, 690). Preferably, the refractive index of the material forming the layers 760a, 760b is 0.05 or more, or 0.10 or less less than the refractive index of the material forming the waveguides 670, 680, 690. Advantageously, the low index layers 760a, 760b may serve as claddings that facilitate total internal reflection (TIR) of light passing through the waveguides 670, 680, 690 (eg, TIR between the top and bottom major surfaces of each waveguide) (clad) layer. In some embodiments, layers 760a, 760b are formed of air. Although not shown, it should be understood that the top and bottom of the illustrated waveguide group 660 may include immediately adjacent cladding layers.

Preferably, for ease of manufacture and other considerations, the waveguides 670, 680, 690 are formed of similar or identical materials, and the layers 760a, 760b are formed of similar or identical materials. In some embodiments, the materials forming the waveguides 670, 680, 690 may be different between one or more of the waveguides, and/or the materials forming the layers 760a, 760b may be different, while still maintaining the various Refractive index relationship.

With continued reference to FIG. 9A , light rays 770 , 780 , 790 are incident on waveguide group 660 . It will be appreciated that the light rays 770, 780, 790 may be injected into the waveguides 670, 680, 690 by one or more of the image injection devices 360, 370, 380, 390, 400 (FIG. 6).

In some embodiments, the light rays 770, 780, 790 have different properties, eg, different wavelengths or different wavelength ranges, which may correspond to different colors. The in-coupling optical elements 700 , 122 , 720 each deflect incident light such that the light propagates through a respective one of the waveguides 670 , 680 , 690 by TIR. In some embodiments, the in-coupling optical elements 700, 710, 720 each selectively deflect one or more specific wavelengths of light while transmitting other wavelengths to the underlying waveguide and associated in-coupling optical elements.

For example, in-coupling optical element 700 may be configured to deflect light 770 having a first wavelength or range of wavelengths while transmitting light 780 and 790 having different second and third wavelengths or ranges of wavelengths, respectively. The transmitted light 780 impinges on and is deflected by the in-coupling optical element 710, which is configured to deflect light having a second wavelength or wavelength range. Light ray 790 is deflected by in-coupling optical element 720 configured to selectively deflect light having a third wavelength or range of wavelengths.

With continued reference to Figure 9A, the deflected rays 770, 780, 790 are deflected such that they propagate through the respective waveguides 670, 680, 690; that is, the coupling optical elements 700, 710, 720 of each waveguide deflect the light to the respective into the waveguides 670, 680, 690 to couple light into the corresponding waveguides. Light rays 770, 780, 790 are deflected at an angle that propagates the light through the respective waveguides 670, 680, 690 via TIR. Light rays 770, 780, 790 propagate via TIR through the respective waveguides 670, 680, 690 until impinging on the respective light distributing elements 730, 740, 750 of the waveguides.

Referring now to FIG. 9B, a perspective view of an example of the multiple stacked waveguides of FIG. 9A is shown. As described above, in-coupled light rays 770, 780, 790 are deflected by in-coupled optical elements 700, 710, 720, respectively, and then propagate by TIR within waveguides 670, 680, 690, respectively. Then, the light rays 770, 780, 790 impinge on the light distribution elements 730, 740, 750, respectively. Light distributing elements 730, 740, 750 deflect light rays 770, 780, 790 so that they propagate towards outcoupling optical elements 800, 810, 820, respectively.

In some embodiments, the light distributing elements 730, 740, 750 are orthogonal pupil expanders (OPEs). In some embodiments, the OPE deflects or distributes light to the out-coupling optical elements 800, 810, 820, and in some embodiments, the OPE also increases the beam or spot of the light as it travels to the out-coupling optical element size. In some embodiments, the light distributing elements 730 , 740 , 750 may be omitted, and the in-coupling optical elements 700 , 710 , 720 may be configured to deflect light directly to the out-coupling optical elements 800 , 810 , 820 . For example, referring to Figure 9A, light distributing elements 730, 740, 750 may be replaced by outcoupling optical elements 800, 810, 820, respectively. In some embodiments, the outcoupling optical elements 800, 810, 820 are exit pupils (EP) or exit pupil expanders (EPE), which direct light in the viewer's eye 210 (FIG. 7). It will be appreciated that the OPE may be configured to increase the size of the eyebox in at least one axis, and the EPE may increase the eyebox in, for example, an axis orthogonal to the axis of the OPE. For example, each OPE can be configured to redirect a portion of the light impinging on the OPE to the EPE of the same waveguide, while allowing the remainder of the light to continue propagating along the waveguide. Upon hitting the OPE again, another portion of the remaining light is redirected to the EPE, and the remainder of this remaining light continues to propagate further down the waveguide, and so on. Similarly, upon impinging on the EPE, a portion of the impinging light is directed out of the waveguide towards the user, and the remainder of the light continues to propagate through the waveguide until it strikes the EP again, at which point another portion of the impinging light is directed out the waveguide, and so on. Thus, whenever a portion of the light is redirected by an OPE or EPE, the coupled-in light of a single beam can be "duplicated", resulting in a field of cloned beams, as shown in Figure 6. In some embodiments, the OPE and/or EPE may be configured to modify the size of the beam.

9A and 9B, in some embodiments, waveguide set 660 includes waveguides 670, 680, 690; coupling optical elements 700, 710, 720; light distributing elements (eg, OPEs) 730, 740, 750; and Outcoupling optics (eg, EP) 800, 810, 820 for each component color. The waveguides 670, 680, 690 may be stacked with an air gap/cladding between each. In-coupling optics 700, 710, 720 redirect or deflect incident light (with different coupling-in optics that receive light of different wavelengths) into their waveguides. The light then propagates at an angle, which will result in TIR within the respective waveguides 670, 680, 690. In the example shown, light ray 770 (eg, blue light) is deflected by first in-coupling optical element 700, then continues to jump to the waveguide, interacts with light distributing element (eg, OPE) 730 in the manner described above, and then interacts with Outcoupling optical elements (eg, EPs) 800 interact. Light rays 780 and 790 (eg, green light and red light, respectively) will pass through waveguide 670 , where light ray 780 strikes and is deflected by in-coupling optical element 710 . Light ray 780 then hops via TIR to waveguide 680 , continuing to light distributing element (eg, OPE) 740 to reach it and then to outcoupling optical element (eg, EP) 810 . Finally, light 790 (eg, red light) passes through the waveguide 690 to impinge on the light-in-coupling optical element 720 of the waveguide 690 . The light in-coupling optical element 720 deflects the light 790 so that the light travels by TIR to the light distributing element (eg, OPE) 750 and then to the out-coupling optical element (eg, EP) 820 by TIR. The out-coupling optical element 820 then finally couples the light 790 out to the viewer, who also receives the out-coupled light from the other waveguides 670,680.

9C illustrates a top plan view of an example of the multiple stacked waveguides of FIGS. 9A and 9B. As shown, the waveguides 670, 680, 690 and the light distributing elements 730, 740, 750 and associated outcoupling optical elements 800, 810, 820 associated with each waveguide may be vertically aligned. However, as discussed herein, the in-coupling optical elements 700, 710, 720 are not vertically aligned; instead, the in-coupling optical elements are preferably non-overlapping (eg, laterally spaced as seen in top view). As discussed further herein, this non-overlapping spatial arrangement facilitates one-to-one injection of light from different sources into different waveguides, allowing a particular light source to be uniquely coupled to a particular waveguide. In some embodiments, arrangements comprising non-overlapping spatially separated in-coupling optical elements may be referred to as shifted pupil systems, and in-coupling optical elements within these arrangements may correspond to sub-pupils.

Eye Imaging and Environmental Imaging

As described above, the head mounted display can be used to provide image content to the user that is integrated with, combined with and/or superimposed on the view of the world in front of the wearer. Such head mounted display systems may be configured to project light into the user's eyes to form augmented reality image content, as well as transmit light from the environment in front of the user to the user. The head mounted display system may include one or more cameras for imaging the environment and/or the user's eyes. An outward-facing camera can be used to directly image the environment, for example, to determine where to place augmented reality image content relative to objects in the environment. For example, imaging the environment can provide the location of the table so that the head mounted display can render images of people standing near the table rather than standing on or in the table. Inward facing cameras can be used to shape the eye directly, eg for eye tracking. Disclosed herein are examples of head mounted display systems and/or imaging systems that may be configured to also image the eye and/or the environment. In some designs, the system does not require inward facing and/or outward facing cameras to directly image the eye and/or the environment, respectively. Such a system may employ one or more cameras configured to receive light from the eye/environment via the eyepiece, such as one or more waveguides in the eyepiece in optical communication with the one or more cameras. From the light collected by the one or more waveguides, one or more cameras can generate an image of the eye or the environment in front of the user. Using waveguides to collect light for imaging the eye and/or the environment can potentially reduce the form factor of the head mounted display, making the head mounted display potentially more compact and/or aesthetically desirable.

FIG. 10 shows an example imaging system 900 configured to image the eye integrated with an eyepiece 950 that may be used in a head mounted display. Eyepieces 950, which may be positioned in front of the user's eye 210, may be used to inject image content into and image the eye. FIG. 10 shows one eyepiece 950 positioned in front of one eye 210 . Various head mounted display systems such as shown in FIG. 2 may include pairs of eyepieces 950 and associated components disposed in front of respective left and right eyes 210 . A single waveguide 940 is shown in Figure 10, but the waveguide 940 may include one, two, three, four, six, seven, eight or more waveguides (eg, one or more waveguide stacks).

Imaging system 900 may include a light source or illumination source 960 that illuminates the eye to facilitate image capture, an eyepiece 950 including a waveguide 940 configured to propagate light therein, and/or an imaging device 920 such as a camera for image capture. Also shown is an image projector 930 for producing an image that can be injected into the eye via eyepiece 950. Eyepiece 950 may include one or more waveguides 940 configured to transmit light from illumination source 960 and/or image projector 930 to the eye and from the eye to camera 920 . The eyepiece 950 may further include one or more coupling optical elements 944 for coupling light out of the waveguide 940 and into the eye for illuminating the eye and for image injection, and/or for coupling light out of the eye and coupling the optical into the waveguide for image capture. Eyepiece 950 may additionally include one or more incoupling optical elements 942 for coupling light from illumination source 960 and/or image projector 930 into waveguide 940 and one or more coupling optics out of the waveguide One or more out-coupling optical elements 952 to camera 920 .

The eyepiece 950 may be provided on a frame that is wearable on the head. The eyepiece 950 may be positioned in front of the eye 210 . Eyepiece 950 may have an inner or nasal side closer to the wearer's nose and an opposite outer or temporal side closer to the temple and away from the wearer's nose. In FIG. 10, coupling optical element 944 is medial or nasal relative to coupling in optical element 942 and coupling out optical element 952, which are lateral or temporal to coupling optical element 944. Illumination source 960 is also more medial or nasal relative to image projector 930 (or image projector is lateral or temporal than illumination source 930). However, the relative positions can be different. For example, in some designs, illumination source 960 may be more lateral or temporal than image projector 930 .

The waveguide 940 may comprise a sheet or layer having two major surfaces (front and rear) with the largest surface area and disposed opposite each other. When the user wears the head mounted display, the forward surface may be further away from the user's eyes 210 (closer to the environment in front of the wearer) and the rearward surface may be closer to the user's eyes (and away from the environment in front of the wearer). The waveguide 940 may comprise a transparent material (eg, glass, plastic) having an index of refraction greater than 1.0 such that light may be guided therein by total internal reflection between major surfaces. Elements with the same number may have the same function for one or more of the embodiments described herein.

Coupling optics 944 for coupling light from the waveguide 940 to the eye 210 and/or from the waveguide to the eye may be disposed on or in the waveguide 940 . As shown in FIG. 10, a coupling optical element 944 may be disposed in the optical path between the user's eye 210 and the waveguide 940 such that light coupled from the waveguide 940 via the coupling optical element 944 may be incident on the user's eye 210 (eg, to illuminate the eye and/or for image injection). Coupling optics 944 may include a plurality of turning features configured to turn light directed within the waveguide out of the waveguide, or to turn light incident on coupling optics 944 to the waveguide at an angle is guided in it by total internal reflection. Coupling optics 944 and turning features can be physically engaged with waveguide 940 . For example, coupling optical element 944 may include a holographic or diffractive optical element (eg, a surface relief grating) that is patterned (eg, etched) in or on waveguide 940 . The coupling optical element 944 may comprise a layer disposed on the waveguide 940 or may be formed in the waveguide 940 . For example, volume holographic or other diffractive optical elements can be formed by changing the refractive index of the material comprising the waveguide or layers disposed thereon. Thus, the coupling optical element 944 may be provided in the volume of the waveguide 940 or as a layer thereon.

Depending on the design, the coupling optical element 944 may be transmissive or reflective, and may operate in a transmissive or reflective manner. For example, coupling optical elements 944 may include transmissive or reflective diffractive optical elements (eg, gratings) or holographic optical elements that operate in transmissive or reflective fashion, respectively, to, for example, redirect light transmitted therethrough or reflected therefrom. Coupling optical elements 944 may include polarizing optical elements, such as polarization selective turning elements (eg, polarizers). The polarization selective turning element may include one or more polarization gratings, diffractive optical elements, and/or holographic optical elements, and may include liquid crystal structures, such as liquid crystal polarization gratings. Coupling optics 944 may be configured to direct light from image projector 930 and/or light source 960 directed within waveguide 940 by total internal reflection (TIR) at an angle less than (eg, more normal to) the critical angle. The user's eye 210 so as to emerge from the waveguide to the eye. Additionally or alternatively, the coupling optical element 944 may be configured to couple light from the eye 210 into the waveguide 940 at an angle greater than (eg, less normal to) the critical angle to be absorbed therein by total internal reflection. Navigate to camera 920.

As shown in FIG. 10 , coupling optics 942 for coupling light from illumination source 960 and/or image projector 930 into waveguide 940 may be disposed on or in waveguide 940 . The in-coupling optical element 942 may be disposed in the optical path between the light source 960 and the waveguide 940 such that light coupled from the light source 960 via the in-coupling optical element 942 is guided within the waveguide 940 . The in-coupling optical element 942 may include, for example, a plurality of turning features configured to angularly turn light incident thereon into the waveguide to be guided in the waveguide by total internal reflection. The coupling optical element 942 may comprise a liquid crystal structure, such as a liquid crystal polarization grating. Additionally or alternatively, the coupling optical element 942 may comprise a blazed grating. In-coupling optical element 942 may comprise a layer disposed on waveguide 940, or may be formed on or in waveguide 940 (eg, patterned), or may be otherwise fabricated therein. For example, surface holographic or diffractive optical elements (eg, surface relief gratings) can be fabricated by patterning (eg, etching) the surface of the waveguide or layers thereon. Volume holographic or diffractive optical elements can also be formed by changing the refractive index of the material comprising the waveguide or layers thereon. Thus, the in-coupling optical element 942 may be provided in the volume of the waveguide 940 or in a layer thereon. Depending on the design, the coupling optics 942 may be transmissive or reflective, and may operate in a transmissive or reflective manner. For example, the coupling optical element 942 may comprise a transmissive or reflective diffractive optical element (eg, a grating) or a holographic optical element that operates in transmissive or reflective fashion, respectively, eg, to redirect light transmitted therethrough or reflected therefrom.

The coupling-in optical element 942 may comprise a reflective optical element (eg, a mirror). For example, the coupling-in optical element 942 may include an off-axis reflector. Additionally or alternatively, coupling optical element 942 and/or coupling optical element 944 may include polarizing optical elements, such as polarization selective turning elements (eg, polarizers). The polarization selective turning element may include one or more polarization gratings, diffractive optical elements, and/or holographic optical elements, and may include liquid crystal structures, such as liquid crystal polarization gratings. For example, one or both of in-coupling optical element 942 and/or coupling optical element 944 may include a liquid crystal polarization grating (LCPG). LCPG can potentially provide efficient diffraction at broad wavelengths. Thus, LCPG can be used to couple into optical element 942 and/or to couple optical element 944 . LCPG can be polarization dependent. LCPG or other types of liquid crystal gratings, diffractive optical elements or optical elements may include a pattern or arrangement of liquid crystal molecules configured to provide one or more functions, such as turning light into or out of a waveguide. Accordingly, the coupling-in optical element 942 and/or the coupling optical element 944 may comprise polarization gratings. Additionally or alternatively, the coupling optical element 942 and/or the coupling optical element 944 may comprise liquid crystal, and thus in some implementations one or both may be a liquid crystal grating or a liquid crystal diffractive optical element. Additionally or alternatively, one or both of coupling optical element 942 and/or coupling optical element 944 may comprise a blazed grating. In some designs, the coupling optical element 942 includes a liquid crystal reflector, such as a cholesteric liquid crystal reflective lens (eg, a reflective liquid crystal diffractive lens, a Bragg reflector structure, a reflective liquid crystal diffraction grating, etc.). Some non-limiting examples of liquid crystal gratings, liquid crystal polarization gratings, and other liquid crystal optical elements are discussed in the following published application, which is hereby incorporated by reference in its entirety for all purposes: filed November 16, 2017 U.S. Publication No. 2018/0143438 entitled "MULTILAYER LIQUID CRYSTAL DIFFRACTIVEGRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES"; Nov. 16, 2017 U.S. Publication No. 2018/0143485, titled "SPATIALLY VARIABLE LIQUID CRYSTAL DIRRRACTION GRATINGS (Spatially Variable Liquid Crystal Diffraction Gratings)"; Grating Waveguide Optical Multiplexer)" US Publication No. 2018/0143509; US Publication No. entitled "DISPLAY SYSTEM WITH VARIABLEPOWER REFLECTOR", filed Feb. 22, 2018 2018/0239147; U.S. Publication No. 2018/0239177, filed Feb. 22, 2018, titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ONPOLARIZATION CONVERSION"; and 2017 US Publication No. 2018/0164627, entitled "DIFFRACTIVE DEVICES BASED ONCHOLESTERIC LIQUID CRYSTAL," filed Dec. 7. However, the design of the in-coupling optical element 942 and/or the coupling optical element 944 is not limited to these, and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarization gratings. More information on examples of cholesteric liquid crystal structures such as reflectors can also be found below in the section entitled "Cholesteric Liquid Crystal Mirrors". As mentioned above, other liquid crystal optical elements as well as other non-liquid crystal optical elements can be used. Accordingly, many types of coupling optical elements (eg, coupling optical element 942 and/or coupling optical element 944), diffractive optical elements, gratings, polarization gratings, etc. may be used, typically those described herein as well as other types of gratings, diffractive Optical elements, liquid crystal elements and optical elements. In various implementations, coupling optics 942 may be configured to couple light from image projector 930 and/or light source 960 into the waveguide at an angle greater than the critical angle to be absorbed within waveguide 940 by total internal reflection Guide to eye to user's eye 210 .

Waveguide 940 may include one or more waveguides. In some implementations, the one or more waveguides 940 comprise a stack of waveguides. For example, in some designs, different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eyes. For example, a first waveguide or set of waveguides may be configured to output light that is collimated or having a first divergence as projected from a first depth, and a second waveguide or set of waveguides may be configured to output divergent (not collimated) or light at a second divergence (greater than the first) as if projected from a second depth closer than the first. In some designs, different waveguides may be configured to output light with different correlated colors. For example, the first waveguide may be configured to output red light, the second waveguide may be configured to output green light, and the third waveguide may be configured to output blue light. The fourth waveguide may be configured to output and/or input infrared light.

The outcoupling optical element 952 for coupling light from the waveguide 940 to the camera 920, such as shown in FIG. 10, may include, for example, a plurality of turning features configured to turn the light incident thereon at an angle such that The light is not guided within the waveguide and turned out of the waveguide to the camera. The outcoupling optical element 952 may be disposed within the waveguide 940 or may be patterned (eg, etched) in or on a surface (eg, major surface) of the waveguide 940 . For example, surface holographic or diffractive optical elements (eg, surface relief gratings) can be fabricated by patterning (eg, etching) the surface of a waveguide or a layer thereon. Volume holographic or diffractive optical elements can also be formed by changing the refractive index of the material comprising the waveguide or layers disposed thereon. Depending on the design, the outcoupling optical element 952 may be transmissive or reflective, and may operate in a transmissive or reflective manner. For example, the outcoupling optical element 952 may include, for example, a transmissive or reflective diffractive optical element (eg, a grating) or a holographic optical element that operates in transmission or reflection, respectively, eg, to divert light transmitted therethrough or reflected therefrom .

The outcoupling optical element 942 may comprise a reflective optical element (eg, a mirror). For example, outcoupling optical element 952 may include an off-axis reflector. In some designs, outcoupling optical element 952 may comprise a polarizing optical element, such as a polarization selective turning element (eg, a polarizer). Thus, the polarization selective turning element may include one or more polarization gratings, diffractive optical elements and/or holographic optical elements, and may include liquid crystal structures such as liquid crystal polarization gratings. In some implementations, for example, the outcoupling optical element 952 may comprise a liquid crystal polarization grating (LCPG). LCPG can potentially provide efficient diffraction at broad wavelengths. Likewise, LCPG can be used to couple out optical element 952 . LCPG can be polarization dependent. An LCPG or other type of liquid crystal grating may include a pattern or arrangement of liquid crystal molecules configured to provide one or more functions, such as turning light into or out of a waveguide. Accordingly, the outcoupling optical element 952 may comprise a polarization grating. Additionally or alternatively, the outcoupling optical element 952 may comprise liquid crystal, and thus, in some implementations, may be a liquid crystal grating or other liquid crystal optical element such as a liquid crystal diffractive optical element. Additionally or alternatively, the outcoupling optical element 952 may comprise a blazed grating. In some designs, the outcoupling optical element 952 includes a liquid crystal reflector, such as a cholesteric liquid crystal reflective lens (eg, a reflective liquid crystal diffractive lens, a Bragg reflector structure, a reflective liquid crystal diffraction grating, etc.). Some non-limiting examples of liquid crystal gratings, liquid crystal polarization gratings, and other liquid crystal optical elements are discussed in the following published application, which is hereby incorporated by reference in its entirety for all purposes: filed November 16, 2017 U.S. Publication No. 2018/0143438 entitled "MULTILAYER LIQUID CRYSTALDIFFRACTIVE GRATINGS FOR REDIRECTING LIGHT OF WIDE INCIDENT ANGLE RANGES"; Nov. 16, 2017 U.S. Publication No. 2018/0143485, filed Nov. 16, 2017, titled "SPATIALLY VARIABLE LIQUID CRYSTAL DIRRRACTION GRATINGS (Spatially Variable Liquid Crystal Diffraction Gratings)"; U.S. Publication No. 2018/0143509 to Cross Grating Waveguide Optical Multiplexer); U.S. Publication No. 2018/0143509 titled "DISPLAY SYSTEM WITHVARIABLE POWER REFLECTOR" filed Feb. 22, 2018 .2018/0239147; US Publication No. 2018/0239177, filed Feb. 22, 2018, titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASEDON POLARIZATION CONVERSION"; and 2017 US Publication No. 2018/0164627, entitled "DIFFRACTIVE DEVICES BASED ONCHOLESTERIC LIQUID CRYSTAL," filed Dec. 7, 2018. However, the design of the outcoupling optical element 952 is not limited to these, and may include other types of optical elements, diffractive optical elements, liquid crystal optical elements, liquid crystal gratings, and liquid crystal polarization gratings. More information on examples of cholesteric liquid crystal structures such as reflectors can also be found below in the section entitled "Cholesteric Liquid Crystal Mirrors". As mentioned above, other liquid crystal optical elements as well as other non-liquid crystal optical elements can be used. Accordingly, many types of coupling optical elements (eg, outcoupling optical element 952), diffractive optical elements, gratings, polarization gratings, etc. may be used, typically those described herein as well as other types of gratings, diffractive optical elements, liquid crystal elements or Optical element. As described above, outcoupling optical element 952 may be configured to redirect light guided within waveguide 940 at an angle less than the critical angle so as not to be guided within the waveguide by total internal reflection but exit to camera 920 .

In various designs, the coupling optical element 944 may be transparent in the visible spectrum so that the user can see the environment in front of the user through the coupling optical element 944 and the eyepiece 950 . For example, if the in-coupling optical element 942 is used to receive light from the image projector 930 and/or if the illumination source 960 is configured to output visible light for illuminating the eye 210 with visible light, the in-coupling optical element 942 may also enable in the visible light spectrum light turns. In some embodiments, for example, if illumination source 960 is configured to output infrared light to illuminate eye 210 with infrared light, coupling optics 942 are configured to divert the infrared light. In some designs, such as that shown in FIG. 10 , the in-coupling optical element 942 may be more medial or nasal than the out-coupling optical element 952 . However, in other designs, the in-coupling optical element 942 may be located more laterally or temporally than the out-coupling optical element 952 . In some implementations, such as that shown in FIG. 10 , the out-coupling optical element 952 may be adjacent to the in-coupling optical element 942 although non-adjacent positioning is possible.

As shown in FIG. 10, the illumination source 960 may be positioned on the same side of the eyepiece 950 as the eye 210 (eg, rearward or proximal). (The proximal end may refer to the side closest to the eye 210.) Alternatively, the illumination source 960 may be positioned on the opposite side of the eye 210 (eg, anterior or distal). Illumination source 960 may be configured to direct light into at least one of the major surfaces of waveguide 940 via in-coupling optical element 942 . Light source 960 may be configured to emit invisible light (eg, infrared). Light source 960 may include one or more LEDs. The LEDs may include infrared LEDs. Light source 960 may be configured to emit coherent light. In some designs, light source 960 includes a laser (eg, an infrared laser). In some designs, light source 960 emits pulsed light. For example, camera 920 may be configured to capture images periodically. Therefore, the illumination source 960 can be pulsed to coincide with the time period during which the camera is acquiring images. The intensity output from the illumination source 960 may be reduced when the camera is not acquiring an image. By concentrating the total energy of the irradiation in a short period of time, an increased signal-to-noise ratio can be obtained without exposing the eye 210 to unsafe intensity levels. In some cases, for example, camera 920 captures an image every 30 milliseconds, and the exposure time of the camera is a few milliseconds. Illumination source 960 may be configured to output pulses with similar periods and durations to match the pulses of camera 920 .

In some implementations, different light sources with different wavelengths are alternately pulsed to provide different wavelengths of illumination at different times, as described below.

In-coupling optical element 942 may, for example, be in direct optical communication with illumination source 960 and/or image projector 930 to direct light from said image projector 930 and/or light source 960 therein. For example, light emitted by light source 960 may be incident on in-coupling optical element 942 prior to optically interacting with coupling optical element 944 and/or coupling optical element 952 .

As shown in Figures 11A-11E, light 902 projected from image projector 930 may form an image on the retina. Image projector 930 may include light sources, modulators, and/or projection optics. The light source for image projector 930 may include one or more LEDs, lasers, or other light sources, and may include one or more visible light sources. The modulators may include spatial light modulators, such as liquid crystal spatial light modulators. Such spatial light modulators may be configured, for example, to modulate the intensity of light at different spatial locations. Projection optics may include one or more lenses. Other types of image projectors 930 capable of projecting and/or forming images may be employed. For example, image projector 930 may include a scanning fiber.

Image projector 930 and coupling optics 942 may be in direct optical communication with each other. Image projector 930 may, for example, be aligned with coupling optical element 942 into which light from image projector 930 is directed. In some cases, image projectors 930 are positioned adjacent to corresponding coupling optics 942 and/or waveguides 940 . Image projector 930 may also be disposed in an optical path including coupling optical element 942 , coupling optical element 944 , and eye 210 .

Image projector 930 may be a separate element compared to illumination source 960, as shown in Figure 10 and Figures 11A-11E. However, in some cases, image projector 930 may be used as the illumination source. For example, in addition to injecting images into eye 210, image projector 930 may be used to direct visible and/or infrared light into the eye to illuminate the eye for image capture. Alternatively, however, one or more separate light sources 960 may be used to illuminate the eye 210 for image capture.

The light emitted by the illumination source 960 may include light in a specific wavelength range, such as invisible light. Illumination source 960 may be configured to project invisible (eg, infrared) light onto/into eye 210 to image one or more portions of eye 210 (eg, cornea, retina). In certain example implementations, light source 960 may be configured to emit light in a range of approximately 850 nm to 940 nm. Light source 960 may be configured to emit light extending over a wavelength range of at least about 20 nm. Other ranges are also possible. The wavelength range of emission can be 5 nm, 10 nm, 15 nm, 50 nm, 75 nm, 100 nm, 150 nm, 200 nm, or any range between any of these values. Light source 960 may be configured to emit light of broadband wavelengths across any range, such as within the infrared spectrum.

Imaging device 920, which may include a camera, may include a detector array and possibly imaging optics. The detector array may include, for example, a CCD or CMOS detector array, and the imaging optics may include one or more lenses. One or more lenses may have a positive optical power and associated focal length. In some designs, camera 920 is focused at infinity. For example, the optics may have a focal length f, and the detector array may be positioned at a distance away from the optics corresponding to the focal length, such that objects at large distances are imaged onto the detector array. Similarly, collimated light from the eye or object in the environment will be focused on the detector array to form an image of the eye or object thereon.

The imaging device 920 may be disposed on the opposite side of the waveguide 940 from the illumination source 960 and/or the eye 210 . In some designs, imaging device 920 may be positioned on the same side of waveguide 940 as illumination source 960 and/or eye 210 . As shown in FIG. 10, the imaging device 920 may be positioned near the outer or temporal edge of the eyepiece 950, although other locations are possible.

11A-11E illustrate the operation of the example imaging system 900 of FIG. 10 . FIG. 11A shows illumination source 960 emitting light 902 toward coupling optical element 942 on waveguide 940 . As shown, light 902 may be directed to eyepiece 950 generally at normal incidence, although other angles are possible. In some designs, light source 960 is configured to emit collimated light into eyepiece 950 . As shown in FIG. 11B , illumination light 902 may be coupled into waveguide 940 via in-coupling optical element 942 . In some designs, the in-coupling optical element 942 includes a diffractive optical element (eg, a grating, a holographic element) on which light incident on is diffracted at an angle greater than the critical angle of the waveguide, such that the in-coupled light 904 passes through the entire Internal reflection (TIR) is directed within eyepiece 950 . In some designs, in-coupling optical element 942 may be configured to direct light toward coupling optical element 944 . The coupling-in optical element 942 may be polarization selective. For example, the incoupling optical element 942 may include a polarization selective turning element, such as a polarization grating, eg, a liquid crystal polarization grating. FIG. 11C shows how the coupled light 904 propagates through the waveguide 940 by TIR.

FIG. 11D shows an exemplary imaging system 900 with light coupled out of the eyepiece 950 . As in-coupled light 904 propagates through waveguide 940 , some of the light may be incident on coupling optical element 944 . The coupling optics 944 may be configured to couple the in-coupled light 904 out of the eyepiece 950 and toward the user's eye 210 . Coupling optics 944 may be configured to couple light as collimated light toward eye 210 . The coupling optical element 944 can be tuned to have light of a specific wavelength range. For example, coupling optics 944 may be configured to couple infrared light (eg, between approximately 700 nm and 15000 nm) out of waveguide 940 . In some designs, coupling optics 944 may be configured to couple multiple wavelengths of light out of eyepiece 950 . For example, coupling optical element 944 can be tuned for both infrared and visible light. Coupling optics 944 may also be configured to couple light into waveguide 940, as described more fully below.

Coupling optics 944 may be configured to add one or more dimensions to the eye box for the user. For example, one or more dimensions may be measured along a first axis (eg, the x-axis). Eyepiece 950 may further include an orthogonal pupil expander (OPE). The OPE may have at least one light redirecting element disposed on or in the waveguide (eg, on one of the major surfaces), or the OPE may be disposed within the waveguide 940 . The OPE may include similar or identical features to those described above with respect to the light distributing elements 730 , 740 , 750 . In some implementations, the light redirecting elements may comprise diffractive optical elements. The OPE may be configured to increase the size of the eyebox along a second axis (eg, the y-axis) orthogonal to the first axis.

Figure 1 ID shows some of the light exiting the eyepiece 950 towards the user's eye 210. In some designs, coupling optical element 944 is configured such that coupled light 904 incident on coupling optical element 944 at portions of the coupling optical element along the first axis (eg, parallel to the x-axis) are Each portion of the shaft coupling optical element 944 exits the eyepiece 950 . This may provide the user with light for projecting an image or illuminating the eye with respect to different eye positions or positions.

As shown in FIGS. 11D-11E , coupling optics 944 may be configured to couple in-coupled light 904 out of eyepiece 950 as collimated light. The light may also be directed approximately normal to the major surfaces of eyepiece 950 and/or waveguide 940 . Collimated light can be directed into the eye and focused by the eye (eg, the cornea and natural lens of the eye) onto the retina. This light 908 incident on the retina may provide illumination to image the retina and/or provide image content to the eye. Some of this light 908 may, for example, be reflected or scattered off the retina, exit the eye and provide a retinal image to be captured. Light source 960 may be an extended light source such that light will illuminate an area of the retina.

FIGS. 12A-12E illustrate how the imaging system 900 of FIGS. 11A-11E may additionally or alternatively be used for image collection of the eye 210 . FIG. 12A shows light 910 reflecting off the retina as it exits the eye 210. As shown, light 910 scattered or reflected from the retina, passing through the eye's natural lens, the pupil in the eye, and the cornea can be collimated. The light may also be incident on eyepiece 950 at normal incidence (eg, at right angles to the major surfaces of waveguide 940 and/or coupling optical element 944). Coupling optical element 944 may be configured to couple light 910 reflected from the retina into waveguide 940 .

FIG. 12B shows an exemplary imaging system 900 with light coupled into the eyepiece 950 . Coupling optical elements 944 may include turning features such as diffractive optical elements or other structures that redirect light at angles greater than the critical angle to guide light within waveguide 940 . Coupling optics 944 may be configured to direct in-coupled light 914 generally toward light source 960 and/or imaging device 920 . Coupling optics 944 may be configured to couple out of waveguide 940 less than a fraction of the light propagating towards camera 920 . For example, a partially reflective element (eg, a semi-transparent mirror) may be provided on or in the waveguide 940 such that a portion of the in-coupled light 914 continues to propagate in the waveguide 940 by total internal reflection, while reducing the amount of the in-coupled light 914 along the Leakage from the waveguide 940 of the portion of the waveguide 940 in which the coupling optical element 944 is disposed. The fraction of light that is not leaked out can be any fraction between 0 and 1. For example, the fraction may be 0.90, where 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is held within the waveguide at each reflection of the light. Other portions are also possible (eg 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80 or a range between any of these values). Such partially reflective elements can be similarly used in the implementations described below.

As shown in FIG. 12C , the collimated in-coupled light 914 may continue to propagate through the waveguide 940 towards the imaging device 920 . FIG. 12D shows how some of the in-coupled light 914 continues to propagate until incident on one or more out-coupled optical elements 952 . To reduce the amount of leakage of the in-coupled light 914 from the in-coupling optical element 942, the in-coupling optical element 942 may be configured to couple very little of the light propagating toward the camera 920 out of the waveguide. For example, a partially reflective element (eg, a semi-transparent mirror) may be disposed on or in the waveguide 940 such that a portion of the in-coupled light 914 continues to propagate within the waveguide 940 through total internal reflection, while reducing the amount of the in-coupled light 914 along the Leakage from the waveguide 940 of the portion of the waveguide 940 in which the in-coupling optical element 942 is disposed. The fraction of light that is not leaked out can be any fraction between 0 and 1. For example, the fraction may be 0.90, where 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is held within the waveguide at each reflection of the light. Other portions are also possible (eg 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80 or a range between any of these values). Such partially reflective elements can be similarly used in the implementations described below.

As shown in FIG. 12E , outcoupling optical element 952 may be configured to couple light guided in waveguide 940 out of waveguide 940 and into imaging device 920 . As a result, light propagating in waveguide 940 incident on outcoupling element 952 is redirected so that it exits waveguide 940, eg, from a major surface of waveguide 940 (eg, the front or rear side of waveguide 940) is ejected and directed to the imaging device 920 . The outcoupling optical element 952 may be configured to direct the light 926 to exit the waveguide 940 perpendicular (eg, normal to) the major surface of the waveguide 940 . In some designs, outcoupling optical element 952 is configured to direct collimated light 924 onto imaging device 920 at normal incidence to the photosensitive portion of imaging device 920 . As mentioned above, the camera 920 can be infinity focused, eg, the imaging optics can be configured to focus collimated light onto the detector array.

Accordingly, the waveguide 940 may be configured to guide light coupled from the user's eye 210 into the waveguide 940 for reception by the imaging device 920 (eg, a camera) in order to capture an image of at least a portion of the user's eye 210 . The same waveguide 940 can be configured to direct light coupled from the image projector 930 such that the light from the image projector 930 can be directed to the user's eye 210 such that the image from the image projector 930 is in the user's field of view. In some implementations, the same waveguide is configured to guide light coupled from the illumination source 960 such that the light from the illumination source can be directed to the user's eye 210 to illuminate the eye so that an image of the eye can be captured by the camera 920.

In some implementations, the same coupling optics 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 for reception by the imaging device 920 and (ii) enable light from the image projector 930 Light is coupled out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. In some implementations, the same coupling optics 944 can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 so that the light from the illumination source can illuminate the eye.

In other designs, different waveguides may be used and/or different coupling optical elements 944 may be used. For example, in some designs, the first waveguide 940 may be configured to guide light coupled from the user's eye 210 for receipt by the camera 920 to capture an image of at least a portion of the user's eye 210, and the second waveguide may be configured to guide The light is coupled from the image projector 930 so that the light from the image projector 930 can be directed to the user's eye 210 . The first waveguide and the second waveguide may be stacked on top of each other. Additionally or alternatively, another waveguide may be configured to guide light coupled from the illumination source 960 such that the light from the illumination source may be directed to the user's eye 210 to illuminate the eye.

Additionally, in some implementations, the first coupling optical element 944 can be configured to (i) couple light from the user's eye 210 into the waveguide 940 for reception by the imaging device 920 , and (ii) to couple light from the image projector 930 The light is coupled out of the waveguide 940 to the user's eye 210 to project the image content into the user's field of view. Additionally or alternatively, another coupling optical element can be configured to couple light from the illumination source 960 out of the waveguide to the user's eye 210 so that the light from the illumination source can illuminate the eye.

In some designs, coupling optical element 944 may include multiple diffractive optical elements (DOEs). For example, the first DOE may be configured to couple light from the user's eye 210 into the waveguide 940 for reception by the imaging device 920 . The second DOE may be configured to couple light from the image projector 930 out of the waveguide 940 to the user's eye 210 to project image content into the user's field of view. Optionally, the third DOE may be configured to couple light from the light source 960 out of the waveguide 940 to the user's eye 210 to illuminate the eye. The first and second (and possibly third) DOEs may be stacked, such that in some implementations, light from the environment in front of the user passes through the first DOE, then is incident on the second DOE, and then on the third DOE. DOE and incident on the user's eye. However, the order can be different.

In some designs, the first DOE and the second DOE are integrated in a single element or volume of waveguide 940 . In some implementations, for example, both the first DOE and the second DOE overlap each other within the waveguide 2102 (eg, occupy the same or approximately the same volume). For example, the first and second DOEs may be recorded in the same medium.

As described above, image capture of the eye, eg, the retina, can facilitate eye tracking. For example, FIG. 13A shows an imaging system 900 configured to image various parts of the eye 210 (eg, the retina) at different times, eg, when the eye is in different positions. Stages A and B may refer to images of the eye 210 during different orientations of the eye. Figure 13A shows the imaging of the eye 210 and its results during both Phase A and Phase B imaging.

In some implementations, light emission 928 (eg, from illumination source 960 as described above or from one or more illumination sources configured and/or positioned differently) can be used to obtain one or more images of retina 962, As shown in Figure 13A. The image of retina 962 may include one or more regions 964 , 966 imaged during different orientations of eye 210 . FIG. 13A shows two regions 964 , 966 of the image of the retina 962 . For example, the region 964 of the retina imaged in stage A may be imaged while the eye 210 is directed at an angle normal to the waveguide 940 . While the eye 210 is oriented at an acute angle to the waveguide 940, image data for the region 966 of the retina imaged in stage B may be obtained. Using one or more orientations of eye 210 during one or more phases of imaging, a composite image or map of retina 962 may be obtained. Processing electronics or processors such as data module 140 (see FIG. 2 ) may be used to find overlapping image data between two adjacent regions. Using the overlapping regional image data, a composite image of the retina 962 can be determined. A larger size (eg full size) composite image or map of the user's retina may be stored.

As described herein, a head mounted display may be used to map the retina of a user's eye based on the direction in which the user's eye is directed. To provide realistic and intuitive interaction with objects in the user's environment using eye gaze and/or to identify the wearer of the head-mounted display device, the head-mounted display system may incorporate the uniqueness and likelihood of the user's eye characteristics using retinal mapping Other conditions that have some effect on eye measurements. For example, images may be identified based on the location of blood vessels in the corresponding retinal images.

Retina mapping may involve a process that enables a computing device to learn how to associate a user's eye gaze (eg, as identified in a retinal image) with a gaze point in 2D or 3D space. Eye gaze may be associated with a single point in 2D or 3D space. Eye gaze can also be associated with multiple points in space that can describe the movement of a virtual object (eg, a series of points, the position of a moving image).

The head mounted display system can determine the user's eye gaze based on the retinal image. The head mounted display system may obtain retinal images using sensors (eg, an eye camera such as imaging device 920). The head mounted display system may image one or both eyes of the user as the user changes his or her eye gaze (eg, as the user looks around to follow a moving or shifting calibration target or gaze target). To map the user's retina, the head mounted display system may present virtual objects, such as gaze objects, for viewing by the user. The virtual target may be associated with one or more known gaze points in 2D or 3D space. When a user gazes at a target, the head-mounted display system can acquire retinal images and associate the images with the gaze point. The head mounted display system may calculate and/or generate a mapping matrix based on the association of the respective retinal images and gaze points associated with the target.

The retinal mapping results can reflect the uniqueness in each person's eye. For example, a head mounted display system can generate a mapping matrix customized for one or both eyes of a particular individual. For example, a user may have different amounts of eye movement or eye gaze in response to a particular target. Additionally or alternatively, the user may have different positions, sizes, shapes and/or orientations of blood vessels in the retina. As a result, by generating calibration results specific to an individual user, the head mounted display system may allow for more accurate user interaction with eye gaze and/or may allow identification of a specific user.

Thus, when a user puts on the head mounted display device, the system can detect whether the user is a previous user or a new user. A confusion matrix can be computed in which the scores for a particular eye gaze image stored in system memory are compared to the corresponding image for the current user. The confusion matrix may include comparison scores for multiple eye gazes and associated retinal images. Based on the comparison scores, the system may be able to make a determination as to the identity of the user (eg, whether the user is an individual associated with the stored retinal image or composite map) and/or make a confidence in the determination. The confidence level may include, for example, an identity coefficient. Stored images (eg, composite images or maps) can be compared with images obtained later (referred to as instant or real-time images obtained with respect to the current user). If the system detects that the user is a new user, the system can provide an alert, or other action can be taken.

The system may apply filtering, such as digital filtering or map image processing, to the image of the retina captured by the camera. Such filtering or imaging processing can, for example, enhance features that can be used for identification, stitching, combining synthetic images, eye tracking, and the like. Such filtering or mapping may include edge enhancement. Such filters may include, for example, Frangi filters, although other types of filters may be used. Such filters or processing (eg, edge enhancement or Frangi filters) can be used to enhance and/or detect image features such as blood vessels or tubular structures or fibers in retinal images.

Figure 13B shows a pattern of gaze targets displayed sequentially that can be used in a retinal mapping process. These virtual targets to which the user's eyes are to direct their gaze can result in redirecting the eye gaze to various different directions during which the retina can be imaged. The resulting images associated with different gaze directions correspond to different parts of the retina. As mentioned above, the image captured by the camera includes different parts of the retina as the eye gazes in different directions to view the fixation target at different locations on the display. These images can be combined to form larger retinal maps or composite images.

FIG. 13B shows virtual objects at sixteen different positions in the user's field of view (FOV) 1200 . In various implementations, the virtual object will be presented at a given location at a given time. During the time when the virtual object is presented to the user at that particular location, one or more retinal images will be obtained. The image or images may be associated with the target location and/or the corresponding gaze direction. More or fewer target locations can be used. In the example shown in Figure 13B, sixteen target positions 1202a 1202p are displayed. More or fewer target locations can be used. The target location can also be different. Targets may appear in different order in different locations. For example, the target may move in a raster pattern from the left side of the user's field of view to the user's field of view, back from right to left, and then from left to right, thereby lowering the position of the target in the field of view, with each lateral channel spanning the field of view field. However, other patterns and methods are also possible. Likewise, targets can be rendered identically or differently in different locations. For example, the rendered objects can be of different sizes, shapes, colors, etc. The targets may be rendered to the user sequentially during the eye tracking calibration process. For example, as described above, the head mounted display system may render the target in a serpentine pattern. For example, target 1202a could be followed by 1202b, then 1202c, then 1202d, then 1202h, then 1202g, and so on. Other patterns are also possible. For example, targets can be displayed in a more random or non-sequential pattern. In some embodiments, a single target is displayed to the user, and the target moves around the user's field of view (eg, passes or temporarily stops at locations 1202a-1202p during movement of the target). The head-mounted display system can acquire images of the user's retina while the user is looking at these objects. For example, the head mounted display system may acquire a first image when the user is looking at a target at a first location 1202a, a second image may be acquired when the user is looking at a target at a second location 1202b, and when the user is looking at a target at a second location 1202b A third image may be acquired while the user is looking at the object at the third location 1202c, and so on. The wearable system may associate the first image with the first location 1202a, the second image with the second location 1202b, the third image with the third location 1202c, and so on. Adjacent images can be stitched together in the database to create full or partial retinal maps. For example, two images can be stitched together with appropriate registration using features or portions of features common to multiple images (eg, blood vessels or portions thereof). In various implementations, adjacent target locations will produce overlapping images that can be aligned and stitched together. For example, target location 1202a and target location 1202b and target location 1202b and target location 1202c may produce overlapping and adjacent retinal images that can be stitched to each other. Thus, many different retinal images can be obtained with different eye gazes, thereby combining a larger image of the retina (eg, a composite image or map).

As described above, eye tracking can be performed using synthetic retinal images or maps. For example, after the object is no longer displayed, the user may move the eye gaze as the user looks at a different real object located in front of the user and the head mounted display or augmented reality (virtual) image content displayed by the head mounted display. One or more retinal images may be obtained at these times. The terms "instantaneous" or "real-time" images may be used herein to describe these images obtained after calibration, which may be used for eye tracking (or other purposes, such as obtaining biological data). These "instantaneous" or "live" images may correspond to a portion of a synthetic retinal image or map. The system may be configured to substantially match the "instantaneous" or "real-time" retinal image to a portion of the composite retinal image or retinal map. Such matching may be based on features or portions of features (vessels or portions thereof) common to a portion of the "instantaneous" or "real-time" retinal image as well as the composite retinal image or map. Based on where the "instantaneous" or "real-time" retinal image coincides with the portion of the synthetic retinal image or map, the gaze direction can be inferred. Different gaze directions will produce retinal images corresponding to different parts of the retinal map. Therefore, identifying the location of the "instantaneous" or "real-time" retinal image on the synthetic retinal image or map will provide information about the user's gaze direction. Such or similar methods can be used for eye tracking, eg, tracking eye movement and changes in eye gaze. As described above, edge enhancement, edge detection, or other digital filtering and/or processing may be used to enhance and/or correlate features of different images with a synthetic retinal image or retinal map.

In various implementations after completing an initial calibration process in which a virtual target or gaze target is displayed (eg, at multiple locations) to combine the composite retinal image or map, the composite retinal image or map may still be refined. For example, when additional retinal images are obtained, the additional images may be used to further refine or improve the synthesized retinal image or map. Thus, when additional "instantaneous" or "real-time" retinal images are obtained, eg, for the purpose of providing eye tracking, the "instantaneous" or "real-time" retinal images may be used to further refine or improve the synthetic retinal image or map. As the user continues to look at various locations in the display (with or without calibration targets), the retinal composite image or map can be further refined using additional images acquired after the initial calibration in which the virtual or gaze target is displayed. Thus, the quality of the synthesized retinal image or map can be increased.

Additional non-limiting examples of how eye tracking is accomplished and/or how a synthetic retinal image or map can be produced and the retinal images used are in a submission titled "EYE IMAGE COLLECTION" on January 17, 2017 It is described in US Publication No. 2017/0205875, the disclosure of which is hereby incorporated by reference in its entirety.

Thus, as described above, larger portions of the retina may be recorded and mapped by obtaining retinal and/or other images of the eye using an imaging system such as that described herein, and such images may facilitate eye tracking. For example, the image of the eye 210 shown in Figure 13A can be captured when the eye is in an arbitrary position. Processing electronics or processors (eg, the same or different from those described above for forming the composite image) can then combine the captured image of the user's retina in real-time with the stored composite or larger size (eg, full size) image A comparison is made to track eye movements. A given image of the user's retina captured in real time may reveal a specific part of the user's retina. As described above, by comparing such a captured image to a stored image of the user mapping a larger portion of the user's retina, the system can determine which portion of the user's retina is shown in the captured image, and thus can determine which portion of the user's retina is to be produced. Eye position/orientation. See Figure 13A, which shows images of two different retinal portions produced when the eye is in two different positions and/or orientations. Thus, the position and/or orientation of the eye can be determined by capturing different images of the retina and determining which part of the retina is visible. This determination can be performed even if a composite image is not formed but multiple images of the retina for different eye positions/orientations are recorded and stored in a database. When a future image of the retina is obtained, the image can be compared to images in a stored image database to determine which image in the database is similar to the most recently obtained image of the eye. Matching the most recent image to one or more of the images in the database with an associated position and/or orientation may enable the orientation and/or position of the newer image to be determined. Other eye tracking methods can be used based on images captured using the designs described herein.

Retinal images can also be used for other purposes, as described herein. For example, the retinal image can be used to verify that the user is the same user for which the synthetic retinal image or map was obtained. Retinal images obtained while the user is wearing the head-mounted display system (eg, during a calibration process and/or later use) can be compared with previously acquired synthetic retinal images or maps that are stored (eg, the day before or when previously activated). head-mounted display) for comparison. If the recently acquired retinal image does not match a portion of the synthetic retinal image or map, it may be concluded that the current user is different from the previous user (eg, for which a synthetic virtual image or map was created). Such methods can be used for security, for example, to verify that the current user of the head mounted display device is the owner or typical user of the device. Thus, biological data obtained via retinal imaging can be used for security purposes.

Retinal imaging can also be used to collect biological data to monitor a user's health. Medically relevant data can be obtained from retinal images. Such medical data may be useful for monitoring a user's health.

Although this article discusses various applications of eye imaging in the context of retinal imaging, such as eye tracking, collection of biological data for health monitoring and safety, other parts of the user (eg, the user's eye) can be imaged for these and other purposes.

Although it is described above that the eyepiece 950 can be used to facilitate imaging of the eye, the eyepiece can also be used to image the world in front of the user. For example, FIGS. 14A-14B illustrate an example imaging system 900 that may be used to image a portion of an environment and/or objects in a portion of the environment located in front of a user. The imaging system 900 used may be a system similar to that described with respect to FIGS. 11A-11E and/or with respect to FIGS. 12A-12E , except that light from the environment towards the eyepiece and the user is collected by the eyepiece 950 . For example, FIG. 14A shows light 970 from an environment that is reflected and/or emitted by one or more physical objects 972 in the environment and directed toward the user and the eyepiece 950 . As shown, light 970 from the environment may be approximately collimated (eg, at infinity), for example because physical objects 972 in the environment may be located at a sufficient distance from imaging system 900 to reach imaging system 900 is collimated or approximately collimated. In some implementations, imaging system 900 may be configured to image the environment and/or objects in the environment without using any optical elements (eg, lenses, mirrors) with optical power in imaging system 900 .

The imaging system 900 shown in Figures 14A and 14B is similar to the imaging system described above. The imaging system includes an eyepiece 950 that includes one or more waveguides 940 that include coupling optics 944 configured to convert light from an image projector 930 (not shown) Guided into the eye 210 to form an image therein. The one or more waveguides may include multiple waveguides (eg, stacks of waveguides) configured to couple in/out multiple corresponding colors/wavelengths. Each waveguide in the waveguide stack can be configured to guide light of a particular color (eg, red, green, blue). For example, the most distal waveguide (eg, stack of waveguides) may be configured for visible light (eg, red, blue, green) such that the waveguide is configured to couple in and out the same wavelength of visible light. Additionally or alternatively, waveguides configured to couple in and out invisible (eg, infrared) light may be positioned near the eye 210 . Such multiple waveguides corresponding to waveguide 940 may be used in any of the other implementations described herein. Imaging system 900 may also include an imaging device (eg, camera) 920 and an out-coupling optical element 952 configured to divert light reflected from eye 210 propagating within waveguide 940 to the camera. In Figures 14A and 14B, the illumination source 960 has been excluded as it may not be needed to image the environment in front of the user. However, an illumination source (eg, light source 960 described above) may be used in some designs.

Eyepiece 950, waveguide 940, coupling optics 944, outcoupling optics 952, and camera 920 may be the same as or similar to those described above. For example, coupling optical element 944 can be physically engaged with waveguide 940 . For example, coupling optics 944 and/or decoupling optics 952 may be placed in the optical path between the environment in front of eyepiece 950 and camera 920 such that light from the environment is coupled into waveguide 940 via coupling optics 944 and via the coupling The outgoing optics are coupled out of the waveguide to be incident on the camera 210 (eg, to form an image of at least a portion of the environment). Coupling optics 944 may include a plurality of turning features configured to turn light directed within the waveguide out of the waveguide or to turn light incident on coupling optics 944 into the waveguide at an angle to pass through the entire waveguide. Internal reflection is guided in it. The outcoupling optical element 952 may include a plurality of turning features configured to turn the light (from the environment) guided within the waveguide at an angle such that the light is not guided within the waveguide by total internal reflection but is guided out towards the camera. Coupling optical element 944 , outcoupling optical element 952 , and their associated turning features may be physically engaged with waveguide 940 . For example, coupling optical element 944 and/or outcoupling optical element 952 may include one or more holographic or diffractive optical elements (eg, surface relief gratings) patterned (eg, etched) in or on waveguide 940 . Coupling optical element 944 and/or outcoupling optical element 952 may include layers disposed on waveguide 940 or may be formed in waveguide 940 . For example, volume holographic or diffractive optical elements can be formed by changing the refractive index of the material comprising the waveguide or layers disposed thereon. Thus, the coupling optical element 944 and/or the outcoupling optical element 952 may be disposed in the volume of the waveguide 940 or a layer disposed thereon. Depending on the design, the coupling optics 944 and/or the decoupling optics 952 may be transmissive or reflective, and may operate in a transmissive or reflective manner. For example, coupling optical element 944 and/or decoupling optical element 952 may comprise transmissive or reflective diffractive optical elements (eg, gratings) or holographic optical elements that operate in transmissive or reflective fashion, respectively, eg, so that light or holographic optical elements transmitted therethrough Light reflected from it is turned. Coupling optical element 944 and/or outcoupling optical element 952 may include polarizing optical elements, such as polarization selective turning elements (eg, polarizers). The polarization selective turning element may include one or more polarization gratings, diffractive optical elements, and/or holographic optical elements, and may include liquid crystal structures, such as liquid crystal polarization gratings. In some implementations, the reflective optical element may include a reflector (eg, a mirror). Other elements such as waveguide 940 may also be similar to those described above.

Figure 14B illustrates the operation of the imaging system 900 shown in Figure 14A. Light 970 from the environment is coupled into waveguide 940 through coupling optical element 944 . Coupling optics 944 may be configured to turn the collimated light at an angle greater than the critical angle of waveguide 940 such that at least a portion of the collimated light is directed within the waveguide towards camera 920 by total internal reflection. The outcoupling optical element 952 may be configured to receive at least a portion of the light from the environment in front of the user, which is coupled into and guided within the waveguide 940 via the coupling optical element 944 . The out-coupling optics 952 can be configured to couple in-coupled light from the waveguide 940 to the camera 920 so that an image of the environment can be captured by the camera 920 . The environment image may be passed to processing electronics (eg, one or more processors), such as data module 140 (see Figure 2). The data module 140 may be configured to render a modified image of the environment in an augmented reality context. Processing electronics may communicate with camera 920 via wired or wireless electronic signals. Additionally or alternatively, processing electronics may communicate with camera 920 using one or more remote receivers. Processing electronics may reside remotely (eg, cloud computing devices, remote servers, etc.).

Thus, the imaging system 900 may be used to directly image the environment, which may be useful for a number of reasons. For example, imaging the environment can be used to determine where to place augmented reality image content relative to objects in the environment. For example, imaging the environment can provide the location of the table so that the head mounted display can render images of people standing near the table rather than on or in the table. The imaging system 900 described for imaging the environment may also be used to image the eye 210, such as described with respect to Figures 10, 11A-11E and/or 12A-12E.

It may be desirable to use imaging system 900 to image a wide field of view of the environment. Figure 14C schematically illustrates an imaging system 900 for collecting light from the environment using a powered optical element or lens (eg, refractive optical element 980 (eg, a wide field lens)) located in front of the eyepiece. Refractive optical element 980 may have positive optical power. A refractive optical element 980 (eg, a positive lens) focuses the collimated light 970 from the environment towards the waveguide 940 . Other types of lenses than those shown in Figure 14C may be used. Transmitted light (not shown) may pass through a powered optical element or lens, such as refractive optical element 990 (eg, a negative lens) configured to have an equal and opposite negative optical power to refractive optical element 980 . Negative lens 990 may have similar or the same optical power as positive lens 980 to compensate or cancel the optical power of the positive lens or a portion thereof. In this way, light from the environment (eg, the distal end of waveguide 940) can pass through negative lens 990, eyepiece 950, and positive lens 980 with substantially no net change in optical power introduced into the eye by these two lenses. Negative lens 990 may be configured to compensate or cancel the power of positive lens 980 such that the user will not experience the power of the positive lens when viewing the environment in front of eyepiece 950. Negative lens 990 will also counteract the effect of positive lens 980 to invert the image of objects in the environment in front of the wearer. Although some of the light rays converge, some of the light 970 from the environment may be coupled into the waveguide 940 through the coupling optical element 944 . The in-coupled light incident on the out-coupling optical element 952 can exit the waveguide 940 .

Implementations (eg, the implementations described in Figures 14A-14C) may be used outside of an augmented reality environment. For example, imaging system 900 configured to image the environment is intended to be implemented in wearable devices such as glasses (including afocal glasses) or bifocals. Such imaging system 900 may not require image projector 930 and/or light source 960. Additionally or alternatively, such imaging systems 900 may not require coupling optics configured for corresponding image projectors 930 and/or light sources 960 .

It may be advantageous to implement an imaging system 900 for imaging an environment on a viewing screen (eg, television screen, computer screen) such as a handheld device (eg, cell phone, tablet). Imaging system 900 can improve video chat capabilities. For example, a viewer who sees a chat partner looking at the screen may appear to be directly at the viewer. This will be possible because the light captured by imaging system 900 will be captured in the same area that the user is looking at (eg, as opposed to the viewing screen, but with light captured by a separate outward-facing camera at a different location) light).

In implementations in which imaging system 900 of FIG. 14C is also used to image eye 210 , light source 960 and/or image projector 930 may be configured to inject light into waveguide 940 . Since light reflected from the eye that is coupled into the waveguide will pass through refractive optical element 990 (eg, a negative lens), a positive-power refractive optical element may be provided between light source 960 and/or image projector 930 and waveguide 940 . The positive lens may be configured to compensate or cancel any optical power provided by the refractive optical element 990 before the in-coupled light from the light source and/or light projector is incident on the eye 210 . Other types of lenses than those shown in FIG. 14C may be used as the optical element 990 . Alternatively or additionally, processing electronics in communication with the light source and/or image projector may be configured to alter the image sufficient to present an undistorted image to the user after the light has passed through the refractive optical element 990 . In some designs, the corresponding in-coupling optics, out-coupling optics, and/or coupling optics may be configured to operate on non-collimated light (eg, diverging, converging light).

In various implementations, the same waveguide 940 can be used to (i) propagate light from the eyepiece 950 and the environment in front of the user to the camera 940, and (ii) propagate light from the image projector 930 to the eye 210 at The image content is formed therein. Using the same waveguide 940 may simplify the system and/or the eyepiece, and may make the system and/or eyepiece more compact, potentially providing a reduced form factor. For other reasons, it may also be advantageous to reduce the thickness of the eyepiece 950 by reducing the number of waveguides 940 . Lower cost and a more simplified manufacturing process may be some of these advantages.

Also in various designs, the same or different imaging systems may be used in the same head mounted display to propagate light from the eye to camera 940 by way of waveguides in eyepiece 950, eg, as described above. Such a system may also use an eyepiece to transmit light from the illumination source to the eye 210 to illuminate the eye. In some designs, the eyepiece may additionally be used to propagate light from image projector 930 to eye 210 to form image content therein. Using eyepieces to help image the environment and image the eye (and possibly illuminate the eye) can simplify and/or make the system more compact, potentially providing a reduced form factor.

Additionally, in some implementations, the same waveguide 940 can be used to (i) propagate light from the environment in front of the eyepiece 950 to the camera 940, and (ii) propagate light from the eye 210 to the camera to capture an image of the eye . The same waveguides can be used to propagate light from image projector 930 to eye 210 to form image content therein and/or to propagate light from illumination source 960 to eye 210 to illuminate the eye for image capture. Using the same waveguide 940 may simplify the system and/or the eyepiece, and may make the system and/or eyepiece more compact, potentially providing a reduced form factor. For other reasons, it may also be advantageous to reduce the thickness of the eyepiece 950 by reducing the number of waveguides 940 . Lower cost and a more simplified manufacturing process may be some of these advantages.

Similarly, in addition to coupling light from the environment into the waveguide 940, the same coupling optics 944 may be configured to direct light from the image projector 930 to the eye 210 to form image content therein and/or to Light from the eye is directed into waveguide 940 to be directed to camera 920 therein. Additionally or alternatively, the same coupling optics 944 may be configured to couple light from the illumination source 960 directed within the waveguide 940 out of the waveguide 940 to the user's eye 210 .

As described above, one or more of coupling optical element 944, in-coupling optical element 942, or out-coupling optical element 952 may include polarization selective coupling elements. Thus, in various designs, light input into eyepiece 950 or waveguide 940 is polarized so as to be properly acted upon by the polarization selective turning element.

Thus, in some embodiments, the illumination source 960 comprises a polarized light source with an appropriate polarization to be properly acted upon by the polarization selective coupling/steering element.

One or more polarization-specific filters and polarization-modifying elements may be included in various imaging systems 900 , such as those in which image projector 930 and/or light source 960 are placed directly opposite each other through waveguide 940 . For example, in configurations in which the polarization-sensitive elements are aligned on opposite sides of the waveguide 940 at the same side location, the polarization-sensitive elements may help reduce directional light emission into the imaging device 920 and/or reduce the light emission of the imaging device 920. saturation. 15A-15B illustrate such a configuration. The light source 960 shown in FIG. 15A may be configured to direct light through a polarization-specific filter 982 such as a polarizer (eg, a linear polarizer) and/or through a polarization configured to alter the polarization state of incident light Modification element 986 (eg polarization rotator). A retarder such as a half-wave retarder can, for example, rotate the linear polarization. Thus, a properly oriented half-wave retarder or half-wave plate can rotate s-polarized light to p-polarized light and vice versa. Thus, in various implementations, polarization-specific filters 982 and/or polarization-modifying elements 986 are disposed in the optical path between the light source 960 and the in-coupling optical element 942 in order to provide the in-coupling optical element with appropriately oriented polarization. In some implementations, imaging system 900 does not include polarization modifying elements, but instead includes appropriately oriented polarization filters, such as polarizers.

Light emitted by light source 960 may pass through the arrangement of optical elements in a particular order. For example, as shown in FIG. 15A, light from light source 960 may first pass through a polarization-specific filter 982 (eg, a polarizer) and then through a polarization-modifying element 986 (eg, a rotator). After the light has passed through the polarization modifying element 986, the light can be incident on the in-coupling optical element 942, which can direct the light into the waveguide 940 to be guided therein.

For example, light source 960 may be configured to emit light of mixed polarizations (eg, s-polarization and p-polarization). The polarization-specific filter 982 may be configured to transmit only light in a first polarization state (eg, p-polarization). As the light continues, the polarization modification element 986 can be configured to change the polarization state of the light (eg, from p-polarization to s-polarization). The coupling optical element may be configured to convert the s-polarized light to an angle greater than the critical angle of the waveguide such that the s-polarized light is guided within the waveguide. When the coupled light 904 propagates through the waveguide 940, the coupled light 904 may be polarized substantially with the second polarization (s-polarization). The coupling optical element 944 may be configured to turn only light of the second polarization state (s-polarization). Coupling optics 944 may be configured to couple in-coupled light 904 out of waveguide 940 and to eye 210 to provide illumination for image capture.

To prevent direct illumination (eg, saturation) of imaging device 920, polarization modifying element 958 and/or polarization-specific filter 984 may be disposed in or on waveguide 940 such that only a specific polarization state (eg, p-polarization ) light can pass through the polarization specific filter 984 and reach the imaging device 920. Polarization modifying element 958 (eg, a half-wave plate) may be configured to change the state of polarization (eg, from s-polarization to p-polarization). The polarization specific filter 984 may be configured to transmit only light of a specific polarization (eg, p-polarized light) therethrough. In this way, light passing through polarization specific filter 982 will not be configured to transmit directly through polarization specific filter 984 . In any of the above implementations such as those in Figures 10, 11A-11E, and 12A-12E (eg, where image projector 930 and/or light source 960 are on the same optical axis, as shown in Figure 15A), polarization-specific Filter 982, polarization modifying element 986, coupling-in optical element 942, polarization modifying element 958, and/or polarization-specific filter 984 may be implemented according to the design of FIG. 15A. Polarization specific filter 984 may be a transflective polarizer (eg, a polarizer beam splitter) configured to transmit light of a first polarization and redirect or reflect light of a second polarization different from the first polarization.

A partially reflective element (eg, a semi-transparent mirror) may be included to divert the in-coupled light 904 to the imaging device 920 . A partially reflective element may be disposed between the in-coupling optical element 942 and the polarization altering element 986 such that a portion of the in-coupled light 914 is reflected towards the imaging device 920 while reducing leakage of the in-coupled light 914 from the waveguide 940 . The fraction of light that is not leaked can be any fraction between 0 and 1. For example, the fraction may be 0.90, where 90% of the light propagating through the waveguide 940 along the coupling optical element 944 is retained within the waveguide 940 at each reflection of the light. Other portions are also possible (eg, ranges between 0.10, 0.20, 0.30, 0.40, 0.50, 0.60, 0.70, 0.80, or any of these values).

Figure 15B shows the propagation of light reflected or scattered off the retina. Some of the light 910 incident on the coupling optical element 944 with the second polarization (s-polarization) reflected by the retina is turned by the coupling optical element 944 at an angle greater than the critical angle of the waveguide 940 and thus can be guided therein. Some of the light may not be coupled into waveguide 940 and will be transmitted therethrough as uncoupled light 912 . In-coupled light 904 may propagate through waveguide 940 and towards the camera.

Other implementations may benefit from the use of polarization selective elements near the light source and camera. For example, various systems may be configured to provide illumination with a first polarization and capture images through a camera using light with a different polarization. For example, when imaging the retina, this configuration can be used to reduce unwanted reflections, such as from the cornea. The reflection from the cornea will be specular. Thus, if light of a first polarization is incident on the cornea, the light reflected from the cornea will maintain that first polarization. Instead, the retina is diffuse. If light of the first polarization is incident on the retina, the light reflected from the retina does not merely maintain the first polarization. Diffuse reflections are more likely to result in unpolarized light. Therefore, there will be a second polarization different from the first polarization in the reflected light. Likewise, by illuminating with a first polarization and imaging with a second, different polarization, the retina can be imaged with reduced glare from the cornea.

Thus, in various implementations, polarization-specific filters 982, 984 may be used together to reduce unwanted reflected light from the eye 210 (eg, from the cornea). For example, unwanted light, glare, or flashes may reflect off the cornea, which may saturate the image captured by imaging device 920. Light reflected from the cornea may be specular and maintain its polarization. In contrast, light reflected from the retina may be more diffusely reflected and may be less uniformly polarized. Likewise, combinations of polarizers can be used to remove some or most of the unwanted reflected light. Initially, polarized light may be used to illuminate the eye 210 . In some designs, a polarized illumination source (eg, light source 960) may be used. Additionally or alternatively, a first polarizer (eg, polarization specific filter 982) may be located at the beginning of the light path of the illumination source to provide the initial polarization of the light. A second polarizer (eg, polarization-specific filter 984 ) can be positioned in the optical path before the light enters imaging device 920 . The second polarizer may be rotated 90° from the first polarizer (eg, polarizers 982, 984 may be "crossed"). As a result, the eye will be illuminated in the first polarization with some of the light of the first polarization reflected from the cornea. This light will not pass through the polarizer 984 near the camera. However, the light reflected from the retina will include the second polarization. Likewise, light diffusely reflected from the retina will pass through the polarization 984 near the camera and will enable the camera to capture images of the retina. Thus, in such a configuration, unwanted light received from the eye (eg, from the cornea) that may enter the imaging device 920 may be reduced or eliminated. Other configurations are also possible. For example, polarization selective in-coupling optics 942 for coupling light from light source 960 into waveguide 940 and polarization selective out-coupling optics for coupling light out of the waveguide to camera 920 may be employed to have different Polarization Sensitive Properties. For example, polarization selective in-coupling optics can selectively divert light from an illumination source having a first polarization into the waveguide, while out-coupling optics can selectively divert light having a second, different polarization from the waveguide to the camera . The effect may again be to reduce or remove unwanted light received from the eye (eg, from the cornea) prior to entering the imaging device 920.

Various imaging systems 900 are discussed herein that can use eyepieces 950 to collect light to image the retina. However, imaging system 900 may be configured to image other parts of the eye, such as the front of the eye. FIG. 16 shows how imaging system 900 may be used to image the front of eye 210 (eg, the cornea). Imaging system 900 may include one or more elements of the example imaging system 900 described above. Additionally, the exemplary imaging system 900 may include one or more powered optical elements or lenses, eg, powered refractive optical elements 980, 990 having optical power. For example, a positive power lens or positive lens 980 may be positioned proximal to (eg, closer to eye 210 ) of eyepiece 950 between eye 210 and the eyepiece. A negative power lens or negative lens 990 may be positioned on the far side of the eyepiece 950 between the eyepiece and the environment in front of the user. One or both of the lenses 980, 990 may be variable focus elements (eg, zoom lenses) and/or may include liquid crystal elements. In some designs, one or both of lenses 980, 990 comprise Fresnel lenses. The lenses 980, 990 may incorporate liquid crystals to create a Fresnel lens function. Such functionality may allow variable focusing of one or both of the lenses 980, 990. In some designs, one or more of the lenses 980 , 990 may be integrated with the eyepiece 950 and/or fabricated (eg, formed) on or in the eyepiece 950 .

In various embodiments, the coupling optical element 944 is configured to divert the collimated light reflected from the eye 210 into the optical waveguide to be guided therein. Accordingly, positive lens 980 may be configured to collimate light reflected from eye 210, such as the front of the eye (eg, the cornea). Thus, positive lens 980 may have a focal length equal to or substantially equal to the distance of the lens from the portion of eye 210 to be imaged (eg, the cornea).

Negative lens 990 may have similar or the same power as positive lens 980 to compensate or cancel the power of the positive lens. In this way, light from the environment (eg, the distal end of waveguide 940) can pass through negative lens 990, eyepiece 950, and positive lens 980 with substantially no net change in optical power introduced by these two lenses. As such, the negative lens 990 may be configured to cancel or cancel the power of the positive lens 980 such that the user will not experience the power of the positive lens when viewing the environment in front of the eyepiece 950. Negative lens 990 will also counteract the effect of positive lens 980 to invert the image of objects in the environment in front of the wearer.

Figure 16 shows light 928 incident on the cornea scattered therefrom. Imaging system 900 may be configured to capture this light 988 reflected from the cornea. For example, positive lens 980 may collect and collimate a portion of light 988 scattered from the cornea. This light 988, collimated by the positive lens 980, is incident on the coupling optical element 944, which is configured to divert the collimated light into the waveguide 940 at an angle greater than the critical angle of the waveguide such that the light is absorbed therein by TIR. guide. Coupling optical element 944, outcoupling optical element 952, and/or waveguide 940 may be as described above. The resulting outcoupling light 906 may be extracted from the waveguide 940 by outcoupling optics 952 to a camera (not shown).

Figure 16 shows light 928, such as collimated light, which may come from eyepiece 950 as described above. Illumination source 960 can couple light into waveguide 940, and coupling element 944 can couple light from illumination source 960 out of the waveguide. Coupling element 944 may be configured to couple light out of waveguide 940 as collimated light. This light strikes and scatters from the front of the eye (eg, the cornea). This scattered light 988 may be collected by positive lens 980 and imaging system 900 to form an image of the front of eye 210, as described above. Also as described above, the illumination 928 directed onto the eye 210 may be invisible (eg, infrared) light.

FIG. 16 also shows an alternative arrangement for illuminating the eye 210 . In some designs, one or more light sources 934, such as LEDs or emitters, may be positioned relative to the eye 210 to direct light thereon rather than through the waveguide 940 and to the eye 210 via TIR. In some implementations, eyepiece 950 or waveguide 940 is not in the optical path between one or more light sources 934 and eye 210 . In some designs, multiple such light sources 934 may be arranged in a pattern near and/or around the eye (eg, a circular or annular pattern). In some designs, the pattern of light sources 934 may define an illumination axis that is parallel (eg, coaxial) to the optical axis of one or more lenses 980 , 990 . The one or more light sources 934 may be similar to the one or more light sources 960 described above and may be pulsed, for example. Similarly, the one or more light sources 934 may include infrared light sources such as infrared LEDs or another type of invisible light. Optionally, the one or more light sources may include visible light sources that emit visible light. Alternatively, one or more light sources may emit visible and invisible light (eg, infrared light).

FIG. 17 illustrates another example imaging system 900 configured to image a portion of an eye 210 (eg, the front of the eye (eg, the cornea)). In contrast to the transmissive optical element (lens) 980 shown in Figure 16, the imaging system 900 shown in Figure 17 employs a reflective optical element 996 that is configured to collimate light from the eye. A reflective optical element will have less aberration than a transmissive optical element, and chromatic aberration generally does not apply to a reflective optical element, such as the reflector 996 shown in FIG. 17 . Thus, by using reflective surfaces to collect light from the eye 210, less aberrations (eg, chromatic aberration) may be introduced into the captured image of the eye.

FIG. 17 shows, for example, an imaging system 900 that includes a curved transmissive optical element 996 with a wavelength-dependent reflective coating 998 . A curved transmissive optical element 996 may be provided at the distal end of the waveguide 940 (on the ambient side of the eyepiece 950). Thus, a curved transmissive optical element 996 may be positioned between the environment in front of the wearer and the waveguide 940 and/or the coupling optical element 944 . Similarly, waveguide 940 and/or coupling optical element 944 may be disposed between curved transmissive element 996 and eye 210 .

The wavelength-dependent reflective coating 998 may be configured to reflect light of a particular wavelength or range of wavelengths. In some implementations, for example, wavelength-dependent reflective coating 998 can be configured to reflect invisible light (eg, infrared light) within a range of wavelengths, while wavelength-dependent reflective coating 998 can be configured to transmit visible light. In some cases, a wavelength-dependent reflective coating 998 may be disposed on the surface of the curved transmissive optical element 996 .

As mentioned above, in various designs, coupling optical element 944 is configured to divert collimated light reflected from eye 210 into waveguide 940 to be guided therein. Accordingly, reflective optical element 996 may be configured to collimate light reflected from eye 210, such as the front of the eye (eg, the cornea). Accordingly, the curved reflective optical element 996 may have a positive optical power for light incident on the proximal side of the curved reflective optical element 996 reflected from the wavelength-dependent reflective coating 998 . In particular, in various designs, reflective optical element 994 may have a focal length that is equal to or substantially equal to the distance from reflective optical element 996 to the portion of eye 210 to be imaged (eg, cornea, iris, etc.). An example value of the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is between 4 cm and 6 cm. In some designs, the focal length is about 5cm. The focal length may be within any range formed by any of these values or may be outside such ranges in different designs.

In various implementations, a reflective optical element 996 is disposed distal to the eyepiece 950 in front of the eyepiece. Thus, a reflective optical element 996 is positioned between the eyepiece 950 and the environment in front of the user. Similarly, eyepiece 950 is positioned between reflective optical element 996 and eye 210 .

The curved transmissive optical element 996 may have a curved reflective surface with any shape of curvature. In some implementations, the surface is rotationally symmetric. In some implementations, the surface may be spherical or non-spherical (eg, parabolic). Non-rotationally symmetric shapes are also possible. However, in various designs, the reflective surface has a positive optical power. Reflective optical element 996 may include, for example, concave mirrors for at least certain wavelengths and/or polarizations.

The curved transmissive optical element 996 can be configured to have negligible power in transmission. Likewise, the curved transmissive optical element 996 can be configured to transmit light without introducing convergence or divergence. In one example, the curved transmissive optical element 996 may have substantially the same curvature of the inner radius as the outer radius. Thin optical elements 996 may, for example, reduce optical aberrations for light transmitted therethrough, may be lighter and/or more compact.

In various designs, the reflective optical element 996 includes a material that transmits visible light so that the user can see the environment in front of the wearer. In some cases, to enhance transmission, the curved transmissive optical element 996 may be coated with an anti-reflective coating on the outer surface (eg, the distal surface). The anti-reflective coating may be configured to reduce reflection of visible light such as, for example, red, green and/or blue light. However, reflective optical element 996 may be configured to reflect a portion of the light scattered from eye 210 to form an image of the eye. Thus, the reflective optical element 996 may operate differently with respect to different lights. For example, reflective optical element 996 may operate differently for different wavelengths. Reflective optical element 996 may be configured to reflect infrared light and transmit visible light.

As described above, one or more light sources 934 may be configured to illuminate eye 210 with infrared light. As shown schematically in FIG. 17, the resulting light 988 reflected from the eye 210 (eg, the cornea) may diverge. The curved transmissive optical element 996 may be configured to receive light 988 reflected from the eye (eg, cornea, iris). The wavelength dependent reflective coating 998 may be configured to reflect light 988 reflected from the eye because the wavelength illumination used to illuminate the eye is the same wavelength (eg, 850 nm) reflected by the reflective coating on the curved transmissive optical element 996 . For example, the eye may be illuminated with infrared light (eg, 850 nm), and the curved transmissive optical element 996 may be configured to reflect infrared light (eg, 850 nm) and pass visible light. The shape of the curved transmissive optical element 996 can also be configured to collimate the light 988 reflected from the eye and reflect the light to the coupling optical element 944 that steers the collimated light into the waveguide 940 for use therein Booted by TIR.

In FIG. 17, as in some other designs, one or more light sources 934, such as LEDs or emitters, may be positioned relative to the eye 210 to direct light thereon, rather than via TIR through the waveguide 940 and to the eye 210 on. In some implementations, the eyepiece 950 or waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210 . In some designs, multiple such light sources 934 may be arranged in a pattern near and/or around the eye (eg, a circular or annular pattern). In some designs, the pattern of light sources 934 may define an illumination axis that is parallel (eg, coaxial) to the optical axis of one or more lenses 980 , 990 . The one or more light sources 934 may be similar to the one or more light sources 960 described above and may be pulsed, for example. Similarly, the one or more light sources 934 may include infrared light sources such as infrared LEDs or another type of invisible light. However, other types of light sources can be used.

FIG. 18 illustrates another example imaging system 900 configured to image a portion of an eye 210 (eg, the front of the eye (eg, the cornea)). In the implementation shown in Figure 18, polarization selection is employed to help control the path of light reflected from the eye. In particular, in various designs, coupling optical element 944 is polarization selective. For example, light having a first polarization is transmitted through coupling optical element 944, while light having a second, different polarization is diverted through coupling optical element 944 into waveguide 940 to be coupled therein by TIR. Thus, in various implementations, the eye 210 is illuminated with polarized light, or a polarizer (not shown) is positioned between the eye and the waveguide 940 so that light from the eye incident on the waveguide is polarized. For example, the emitter 934 may emit polarized light, or a polarizer may be positioned in front of the emitter 934 such that the eye 210 is illuminated with polarized light. Thus, in various designs, the polarization of polarized light incident and/or reflected from eye 210 received by light coupling element 944 may be the first polarization such that the light is directed to reflector 996 .

Likewise, in various implementations, coupling optical element 944 (and/or outcoupling optical element 952 ) is configured to transmit transmission having a first polarization state (eg, a first linear, circular, or elliptical polarization state (eg, p- polarized, left-handed circularly or elliptically polarized, etc.), and divert light having a second polarization state (eg, a second linear, circular, or elliptical (eg, s-polarized, right-handed circularly, or elliptically polarized, etc.)) to the In and/or out of the waveguide. In some implementations, eye illuminator 934 may emit only or primarily a first polarization (eg, p-polarization) or further include a polarization-modifying element (eg, a polarization modifying element (eg, p-polarization) configured to transmit only light in a first polarization state (eg, p-polarization) , polarizer). Additionally, coupling optical element 944 and/or outcoupling optical element 952 may be configured to divert light of a second polarization (eg, s-polarization) into and/or out of the waveguide, respectively.

Similar to the imaging system 900 shown in FIG. 17 , the imaging system 900 of the curved reflector 998 shown in FIG. 17 includes a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998 . The wavelength-dependent reflective coating 998 may be configured to reflect light of a particular wavelength or range of wavelengths. In some implementations, for example, wavelength-dependent reflective coating 998 can be configured to reflect invisible light (eg, infrared light) within a range of wavelengths, while wavelength-dependent reflective coating 998 can be configured to transmit visible light. In some cases, a wavelength-dependent reflective coating 998 may be disposed on the surface of the curved transmissive optical element 996 .

In various implementations, a curved transmissive optical element 996 is disposed distal to the eyepiece 950 in front of the eyepiece. Thus, a reflective optical element 996 is positioned between the eyepiece 950 and the environment in front of the user. Similarly, eyepiece 950 is positioned between reflective optical element 996 and eye 210 .

Thus, light having a first polarization (eg, p-polarization) from eye 210 is incident on coupling optical element 944 and passes therethrough to curved transmissive optical element 996 . Imaging system 900 also includes polarization modifying optical elements 978, such as retarders (eg, quarter-wave retarders). The retarder 978 is transmissive and imparts a quarter wavelength retardation to light transmitted therethrough. The light is incident on and reflected from the curved transmissive optical element 996 . The wavelength dependent reflective coating 998 may be configured to reflect wavelengths of light reflected from the eye. Thus, the light is reflected from the curved surface of the curved transmissive optical element 996 and is collimated. This collimated light passes through retarder 978 again, imparting another quarter wavelength retardation to the light transmitted therethrough. The retardation (eg, a fully retarded wave) introduced on the two lights passing through the retarder rotates the polarization. Thus, the first polarization (eg, p-polarization) transmitted through polarization selective coupling optical element 944 on the first pass is converted to a second polarization (s-polarization) and turned into waveguide 940 to be guided by TIR to the Camera 920. As mentioned above, in various designs, coupling optical element 944 is configured to divert collimated light reflected from eye 210 into waveguide 940 to be guided therein. Accordingly, reflective optical element 996 may be configured to collimate light reflected from eye 210, such as the front of the eye (eg, the cornea). Thus, the curved reflective optical element 996 may have a positive optical power. In particular, in various designs, reflective optical element 994 may have a focal length equal to or substantially equal to a focal length from reflective optical element 996 to the portion of eye 210 to be imaged (eg, cornea, iris, etc.). An example value of the focal length may be, for example, 2 cm to 8 cm. In some implementations, the focal length is between 4 cm and 6 cm. In some designs, the focal length is about 5cm.

In various designs, reflective optical element 996 may include a curved surface configured to reflect light. In some cases, the curved surface can be spherical or rotationally symmetric. Reflective optical element 996 may include concave mirrors for at least some wavelengths and/or polarizations.

In various designs, the reflective optical element 996 includes a material that transmits visible light so that the user can see the environment in front of the wearer. Accordingly, the wavelength-dependent reflective coating 998 disposed on the surface of the curved transmissive optical element 996 may transmit visible light or at least certain wavelengths of visible light. The curved transmissive optical element 996 may also be coated with an anti-reflective coating on the outer surface (eg, the distal surface). The anti-reflective coating can be configured to reduce the reflection of red, green and/or blue light. However, reflective optical element 994 may be configured to reflect a portion of the light scattered from eye 210 to form an image of the eye. Thus, the reflective optical element 996 may operate differently with respect to different lights. For example, reflective optical element 996 may operate differently with respect to light of different polarization states (and/or wavelengths). Reflective optical element 996 may be configured to transmit visible light and reflect infrared light.

As shown in FIG. 17 , one or more light sources 934 , such as the LEDs or emitters in FIG. 18 , may be positioned relative to the eye 210 to direct light thereon, rather than through the waveguide 940 and onto the eye 210 via TIR. Thus, in some implementations, the eyepiece 950 or waveguide 940 is not in the optical path between the one or more light sources 934 and the eye 210 . In some designs, multiple such light sources 934 may be arranged in a pattern near and/or around the eye (eg, a circular or annular pattern). The one or more light sources 934 may be similar to the one or more light sources 960 described above, and may be pulsed, for example. Similarly, the one or more light sources 934 may include infrared light sources such as infrared LEDs or another type of invisible light. In particular, in various implementations, light source 934 may emit light reflected by wavelength-dependent reflective coating 998 and/or curved transmissive optical element 996 . However, other types of light sources can be used.

Although the polarization selective coupling optical element 944 is configured to be polarization selective depending on the type of linear polarization incident thereon, other polarization selective coupling optical elements may respond to other types of polarization states, such as different types of circular or elliptical polarization ) is polarization selective. The polarization selective coupling optical element 944 may, for example, be configured such that a first polarization, such as a first circular or elliptical polarization (eg, left-handed or LHP polarization), is transmitted through the polarization selective coupling optical element 944 such that, for example, a second circular or elliptical polarization A second polarization of polarization (eg, right-handed polarization or RHP) is turned into the optical waveguide and vice versa. Such polarization selective coupling optical element 944 may comprise liquid crystal such as cholesteric liquid crystal. Some examples of liquid crystal optics are discussed in the following sections: Section titled "Cholesteric Liquid Crystal Mirror"; filed December 7, 2017 titled "DIFFRACTIVE DEVICES BASED ONCHOLESTERIC LIQUID CRYSTAL (based on Diffractive Devices for Cholesteric Liquid Crystals)" U.S. Publication No. 2018/0164627; U.S. Publication No. entitled "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR", filed Feb. 22, 2018 .2018/0239147; US Publication No. 2018/0239177, filed Feb. 22, 2018, titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION"; in full For purposes, the entire contents of which are incorporated herein by reference.

A polarization modifying element or retarder, such as a circular polarizer, may be disposed between the eye and the polarization selective coupling optical element 944 to convert light reflected from the eye to a first polarization (eg, LHP). The LHP light will pass through the polarization selective coupling optical element 944, reflect from the reflector 998, change the polarization to RHP, and be diverted by the polarization selective coupling optical element 944 into the waveguide to reach the camera.

In some implementations, the reflector 996 may be polarization selective in its reflectivity such that only light of a particular polarization state is reflected and/or light of a different polarization state is transmitted. Such optical elements may include liquid crystals, such as cholesteric liquid crystals. Examples of such optical elements are discussed in the following sections: Section entitled "Cholesteric Liquid CrystalMirror"; Submission on December 7, 2017 entitled "DIFFRACTIVE DEVICESBASED ON CHOLESTERIC LIQUID CRYSTAL (Cholesteric Liquid CrystalMirror) Diffractive Devices of Steroidal Liquid Crystals)" US Publication No. 2018/0164627; US Publication No. 2018, entitled "DISPLAY SYSTEM WITH VARIABLE POWERREFLECTOR", filed Feb. 22, 2018 /0239147; US Publication No. 2018/0239177, filed Feb. 22, 2018, titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ON POLARIZATION CONVERSION"; for all purposes, by Reference is incorporated herein in its entirety. Such optical elements may reflect light in a first polarization state, such as a first circular or elliptical polarization state (left-handed circular or elliptical polarization) and transmit light in a second circular or elliptical polarization state (eg, right-handed circular or elliptical polarization) polarization) of the second polarization state, and vice versa. In some embodiments, the liquid crystal is disposed on the curved surface of the reflector 996 such that upon reflection, the reflector has an optical power, such as a positive optical power. In various other implementations, the liquid crystal optical elements may be flat or planar. For example, liquid crystals can be disposed on a flat or planar substrate or layer. Although flat, optical power can be included in the liquid crystal optical element. Such elements may be referred to as cholesteric liquid crystal reflective lenses. Thus, light from the eye can be collimated and reflected to the coupling optical element 998. For example, the reflector may reflect light in a first polarization state (eg, left-handed circular or elliptical) and transmit light in a second polarization state (eg, right-handed circular or elliptical). Thus, the eye 210 is illuminated with left-handed circularly polarized light, or light reflected from the eye is transmitted through a polarizer (eg, a circular or elliptical polarizer) that transmits light having a first polarization (eg, a left-handed circular or elliptical polarizer) polarized light). Coupling optics 944 can also be polarization selective and can transmit LHP light and turn RHP light into the waveguide. LHP light from the eye passes through coupling optics 944 . This transmitted LHP light is also incident on and reflected from the wavelength selective liquid crystal reflector 996 . In some designs, the wavelength selective liquid crystal reflector 996 converts a first polarization state (eg, LHP) to a second polarization state (eg, RHP) upon reflection. The light of the second polarization state (eg, RHP) is directed to coupling optics 944 which diverts the light of the second polarization state (RHP) into the waveguide 940 and to the camera 920 .

In some designs, the coupling optical element 944 does not include a liquid crystal grating, but instead, for example, a surface relief diffraction grating or a holographic grating. As mentioned above, these coupling optical elements 944 that do not include cholesteric liquid crystals may also include volume diffractive or holographic optical elements or gratings.

Thus, light scattered from the eye is reflected back to the waveguide 940 by the reflective optical element 996 for coupling into the waveguide through the coupling element 944. Conversely, however, a portion of the unpolarized light from the environment in front of the wearer corresponding to the second polarization state (eg, RHP) will be transmitted through reflective optical element 996 . Thus, the wearer can see the object through the reflective optical element 996 .

However, in various designs, the reflective optical element 996 will have negligible power in transmission. For example, reflective optical element 996 may have curved surfaces of the same curvature on both sides of the optical element such that the optical element has a negligible total power for light transmitted therethrough.

As mentioned above, in various implementations, the reflective optical element 996 includes a cholesteric liquid crystal reflective lens, a cholesteric liquid crystal reflective element, such as those discussed in the following sections: entitled "Cholesteric Liquid Crystal Mirror" )"; filed Dec. 7, 2017 titled "DIFFRACTIVE DEVICES BASED ONCHOLESTERIC LIQUID CRYSTAL" U.S. Publication No. 2018/0164627; Title filed Feb. 22, 2018 U.S. Publication No. 2018/0239147 for "DISPLAY SYSTEM WITH VARIABLE POWER REFLECTOR"; filed Feb. 22, 2018 and titled "VARIABLE-FOCUS VIRTUAL IMAGE DEVICES BASED ONPOLARIZATION CONVERSION ( Polarization Conversion Based Variable Focus Virtual Image Device)" US Publication No. 2018/0239177; the entire contents of which are incorporated herein by reference for all purposes. Such optical elements may operate with respect to a particular wavelength or wavelength range. Therefore, light such as infrared light reflected from the eye can be acted upon by the cholesteric liquid crystal reflective element. However, light outside this wavelength range, such as visible light from the environment, can pass through the cholesteric liquid crystal reflective element without being operated by the cholesteric liquid crystal reflective element. Thus, the cholesteric liquid crystal reflective element has a power that is negligible for the visible light from the environment passing therethrough.

As described above, in some implementations, illumination source 960 couples light into waveguide 940 from which the light is redirected to illuminate eye 210 . In such an embodiment, the coupling optical element 944 may be polarization selective. For example, coupling optical element 944 may transmit a first polarization (p polarization) and transmit a second polarization (s polarization).

Thus, if light from illumination source 906 propagates through waveguide 940 and is turned by coupling optical element 944, the illumination will be s-polarized. A polarization-modifying optical element (eg, a quarter-wave retarder) may be disposed between the waveguide 940 and the eye 210 to cause rotation of polarized light reflected from the eye. Light from light source 960 reflected from eye 210 will pass through the quarter wave retarder twice, and as a result, the s-polarized light exiting the waveguide by coupling element 944 to illuminate the eye will be converted to p-polarized light.

This p-polarized light will be transmitted through coupling optical element 944 and the waveguide and incident on reflective optical element 996 .

Imaging system 900 may further include a second polarization modifying element 978, which may include, for example, a retarder or a waveplate as described above. The retarder may comprise, for example, a quarter-wave retarder. A second polarization modifying element 978 may be disposed at the distal end of the waveguide 940 between the waveguide and the reflector 996 . A second polarization modifying element 978 may also be disposed between the coupling element light 944 and the reflector 996 . Light from eye 210 (p-polarized) transmitted through coupling element 944 is converted to circular polarization through second polarization modifying element 978 . If reflector 996 reflects circularly polarized light, the light will be reflected back to waveguide 940 after passing through polarization modifying element 978 again. Two passes through the polarization modifying element (eg, quarter wave retarder) 978 will convert the light to s-polarized light, which will be diverted by the coupling element 944 into the waveguide to be guided therein and to the camera (not shown).

As shown in FIG. 18, the light 988 reflected from the eye 210 is diverging. This light is incident on a reflector 996 that is curved or has positive optical power, and can thus be collimated. Coupling optical element 944 configured to steer collimation into waveguide 940 will thus direct this collimated light from curved reflective optical element 996 towards imaging device 920 (not shown). Thus, light reflected from the eye 210 collimated by the curved reflective optical element 996 is coupled into the waveguide 940 and directed therein towards the outcoupling optical element 952 . Outcoupling optics 952 may be configured to direct light out of eyepiece 950 and to a camera (not shown).

The configuration of the imaging system can have various variations. Different types of reflectors 996 and coupling elements 944 may be employed. The reflector 996 and coupling element 944 may be configured to operate with linearly polarized light or circularly or elliptically polarized light, for example. As discussed, reflector 996 has optical power. The reflector 996 and coupling element 944 may comprise cholesteric liquid crystal grating reflectors and/or lenses with no optical power. A polarization modifying element 978, such as a retarder, may be included between the coupling element 944 and the reflector and/or between the coupling element 944 and the eye. In some embodiments, a polarizer, such as a circular polarizer or a linear polarizer, may be positioned between the eye and the coupling element 944 . If, for example, unpolarized light is reflected from the eye, a polarizer (eg, a circular polarizer or a linear polarizer) may be disposed between the eye and the coupling element 944 . In some such cases, coupling element 944 is polarization selective.

In configurations such as those shown in Figures 17 and 18, where light reflected from the eye passes through the waveguide 940 to a curved reflective optical element 996 to be collimated and redirected back to the waveguide, introducing background noise. This background noise is created by light initially passing through the coupling optics 944 from the eye. As described above, coupling optics 944 may be configured to divert the collimated light into waveguide 940 to be directed therein and to camera 920 where the image is formed. However, the coupling optical element 944 will deflect some of the non-collimated light incident thereon. Thus, upon initially passing through coupling optical element 944 and waveguide 940 to curved reflective optical element 996, some of the non-collimated (divergent) light reflected from the eye will be coupled into the waveguide through coupling optical element 944 and will Background noise contributes to the image of the eye formed by camera 920 . This noise will be superimposed on the image formed by the collimated light retroreflected by the curved reflective optical element 996 , which is coupled into the waveguide by the coupling optical element 944 and guided therein and to the camera 920 .

In some designs, this noise can be subtracted from the image. The process of subtracting noise from the signal may involve (a) measuring the amount of light (referred to as N) that is redirected by coupling optics 944 and reaches camera 920 as it initially passes through coupling optics 944 to the curved reflective optics 996 and (b) Measure the total signal at camera 920 as light passes through coupling optics 944 and waveguide 940 to curved reflective optics 996 , is collimated and reflected back to the coupling optics, and is turned to camera 920 . This total signal will also include some noise N, since the non-collimated light reflected from the eye will have passed through the coupling optical element 944 to the curved reflective optical element 996, so some of the non-collimated light will be diverted by the coupling optical element 944 to camera 920. If the noise N can be measured separately from the total signal T containing the noise superimposed on the eye image, then the noise N can be subtracted from the total signal T as shown in the following equation:

I=T–N

where I represents the image with the noise component N removed.

The above two measurements (a) and (b) can be obtained in various ways. For example, as shown in FIG. 19 , a shutter 936 may be provided between the curved reflective optical element 996 and the waveguide 940 and coupling optical element 944 . The shutter 936 may be configured to block light when the shutter is in the first state and transmit light when the shutter is in the second state. The shutter 936 may comprise, for example, a liquid crystal shutter.

Thus, the noise component N can be measured when the shutter 936 is in the first state where light reflected from the eye 210 is incident on the coupling optical element 944 and through it towards the curved reflective optical element 996, however, can be prevented by the closed shutter The noise component N reaches the curved reflective optical element. As noted above, some of the light reflected from the eye 210 , although primarily uncollimated, does couple into the coupling optics 944 and is turned into the waveguide and directed therein to the camera 920 . As mentioned above, the formation of this light image does not contribute but generates background noise. The camera 920 may record this noise N when the shutter 936 is closed.

When the shutter 936 is in the second state with the shutter open, the total signal T including both the noise N and the image can be measured. The light reflected from the eye 210 is again incident on the coupling optical element 944 . Some of this light reflected from the eye 210 , although largely uncollimated, is coupled into the coupling optics 944 and turned into the waveguide where it is directed to the camera 920 . However, most of this light reflected from eye 210 passes through coupling optical element 944 , through open shutter 936 and to curved reflective optical element 996 . Curved reflective optics 996 collimates at least a portion of the light and reflects it back to coupling optics 944, which turns the collimated light into waveguide 920 and directs it to camera 920 to form the eye 210 images. Camera 920 may capture images of eye 210 .

Processing electronics (eg, processing electronics 140 ) in communication with camera 920 may receive noise component N measured when shutter 936 is in the first closed state and total signal T measured when shutter 936 is in the second open state and may Subtract the two (T-N). In this way, noise N, which is caused by non-collimated light reflected from eye 210 that is coupled into coupling optical element 944 when it first passes through coupling optical element 944, can be subtracted from the total image signal T. contributed. Processing electronics may communicate with camera 920 via wired electronic signals. Additionally or alternatively, processing electronics may communicate with camera 920 using one or more remote receivers. Processing electronics may reside remotely (eg, cloud computing devices, remote servers, etc.).

The measurements of (a) and (b) may be performed in other ways to obtain N and T and subtract N from T. For example, if the curved reflective optical element 996 is wavelength selective, as shown in Figure 18, the eye can be illuminated with different wavelengths of light at different times. For example, to perform measurement (a) and quantify the noise N, the eye may be illuminated with wavelengths that are not reflected by the reflective optical element 996 that is not curved. However, in order to perform measurement (b) and quantify the total signal T, the eye can be illuminated with wavelengths reflected by the curved reflective optical element 996 . The noise N can then be subtracted from the total T, as described above (eg, T-N).

20-20E illustrate an example imaging system 900 configured to measure and subtract the noise component N using wavelength modulation, as described above. The imaging system 900 in Figures 20A-20E includes a wavelength selective curved transmissive optical element 996 (eg, as described above with reference to Figures 17 and 18). For example, the curved transmissive optical element 996 has a wavelength-dependent reflective coating 998 on its curved surface. Imaging system 900 may also include one or more light sources or illumination sources (not shown) configured to illuminate eye 210 . One or more light sources may be configured to emit infrared light. However, one or more light sources may be configured to emit different colors or wavelengths of light at different times. This wavelength modulation may enable N to be measured individually in order to subtract N from the total signal T.

In various implementations, for example, the one or more illumination sources 960, 934 may be configured to emit one or more wavelengths λ _reflected by the curved reflective optical element in a first state, and in a second state The one or more wavelengths λ that are emitted unreflected _{are unreflected} . In the second state, emission does not exceed a negligible amount of wavelength λ _reflected , which wavelength is reflected by the curved reflective optical element. Similarly, in the first state, no more than a negligible amount of unreflected wavelength λ _{is emitted unreflected} .

In some examples, the _reflection wavelength λreflection may be between approximately 800 nm and 950 nm. The reflection wavelength λ _reflection can be between about 835 nm and 915 nm. The reflection wavelength [lambda] _reflection may be between about 840 nm and 870 nm. In some designs, the reflected wavelength λ _reflects about 850 nm. Light emission 928 from one or more light sources 960 may illuminate the eye.

As shown in FIG. 20B , unreflected light 988 having wavelength λ _{that is not reflected} by the curved reflective optical element 944 (and no more than a negligible amount of light λ _reflected by the curved optical element) reflects off a portion of the eye 210 (eg, cornea). Because the light includes wavelengths λ _{that are not reflected} by the curved reflective optical element 944, ray 916 is shown propagating through the curved reflective optical element 996 to the environment in front of the user.

Although the light 988 incident on the coupling optical element 944 is not collimated, the coupling optical element couples at least some of the light 914 into the waveguide 940 to be directed to the camera 920 . Thus, camera 920 can capture an image corresponding to noise component N (image #1) resulting from non-collimated light that was optically coupled upon initially passing through to curved reflective optical element 996 Element 944 turns. This image (image #1) is background noise and indeed is not an image of a recognizable eye. Processing electronics 140 is shown receiving this first image (image #1).

In Figures 20C-20E, an illumination source (not shown) emits one or more wavelengths λ _reflected by the curved reflective optical element, and _{no more than a negligible amount of wavelength λ is} unreflected and unreflected. The wavelength λ _reflection may be, for example, 850 nm.

As shown in FIG. 20C, some of the light 988 reflected from the eye 210 incident on the coupling optical element 944 on the first pass through the coupling optical element 944 is coupled into the waveguide 940 by the coupling optical element 944 (FIG. 20B). shown in ) and directed towards the camera 920. Additionally, the curved transmissive optical element 996 that selectively reflects light reflected at wavelength λ _reflects and collimates the uncoupled light 918 reflected from the eye 210 incident on the curved transmissive optical element. As shown in FIG. 20E , coupling optical element 944 turns and couples this collimated reflected light into waveguide 940 and toward camera 920 . Figure 20E shows the two components reaching the camera 920, light 988 reflected from the eye 210 incident on the coupling optics 944 (which is coupled into the waveguide 940 by the coupling optics) on the first pass through the coupling optics 944 , and light reflected and collimated by the curved transmissive optical element 996 coupled into the waveguide by the coupling optical element. Camera 920 may capture an image corresponding to this total image component T (image #2). Processing electronics 140 is shown receiving this second image (image #2).

As mentioned above, the processing electronics can subtract the noise T-N from the image. In this example, image #1 can be subtracted from image #2. Accordingly, processing electronics 140 may be configured to modify the second image based on the first image. However, other methods are also possible. For example, processing electronics 140 may be configured to create a new image that represents a version of the second image with reduced optical noise. Implementations for subtracting noise from an image can be used in the implementations described above. For example, the implementations shown in FIGS. 10, 11A-11E, and/or 12A-12E may include a shutter 936 and/or a curved transmissive optical element 996 having a wavelength-dependent reflective coating 998 that is Configured to selectively reflect uncoupled light 912 and direct the light to imaging device 920 .

As described above, Image #1 is obtained when light with one or more wavelengths λ unreflected _{that is not reflected} by the reflective optical element is illuminated, and no more than a negligible amount of wavelength λ _reflection is reflected. Image #2 is obtained when light having one or more wavelengths λ _reflected by the curved reflective optical element is illuminated, and _{no more than a negligible amount of wavelength λ is} unreflected and unreflected. Accordingly, one or more illumination sources 960, 934 may be configured to modulate wavelengths. For example, in some designs, the one or more illumination sources 960, 934 may include a first illumination source configured to output one or more wavelengths λ _not reflected by the curved reflective optical element _reflection , and no more than a negligible amount of wavelength λ _reflection is reflected. The one or more illumination sources may further include a second illumination source configured to output one or more wavelengths λ _reflected by the curved reflective optical element and not exceed a negligible amount of wavelength λ _Unreflected Unreflected. The intensities of the first and second illumination sources can be alternately increased and decreased, turned on and off, attenuated and not attenuated, passed and blocked to provide modulation of the wavelength of light that illuminates the eye. For example, during the first time interval, the first illumination source may be blocked while the second illumination source is not. During a subsequent second time interval, the second illumination source may be blocked while the first illumination source is not. This process can be repeated to provide modulation of the wavelength of light that strikes the eye. In other designs, the wavelength of the light source can be tuned and detuned to move the wavelength back and forth between λ _{- reflected} and λ-unreflected. Other arrangements are possible.

As described above, imaging system 900 may be included in a head mounted display, such as an augmented reality display, which also provides the ability to image the eye by collecting light with eyepiece 950. Such an imaging system 900 can be used for eye tracking. Multiple images of the retina of the eye or the front of the eye may be obtained. Movement and/or repositioning of the eye can be determined from these images to track the position and/or orientation of the eye. These imaging systems can also be used for biological imaging and/or for identifying users. For example, images of the user's eyes (eg, retina or iris) may be obtained and recorded. Subsequent images of the wearer's eye (eg, retina or iris) may be obtained at a later time. The two images can be compared to determine whether the wearer in the subsequent instance is the wearer in the first instance. However, other uses of the imaging system are possible.

Although the illumination system may be described above as being waveguide based and comprising one or more waveguides, other types of light turning optical elements may be employed in place of the waveguides. Such light turning optics may include turning features to direct light from the light turning optics, eg, onto a spatial light modulator. Thus, in any of the examples described herein, as well as in any of the examples below, any reference to a waveguide may be replaced with a light-turning optical element in place of a waveguide. Such light redirecting optical elements may include, for example, polarizing beam splitters, such as polarizing beam splitting prisms.

As mentioned above, the systems described herein may allow for the collection of biometric data and/or biometric identification. For example, the eye or a portion thereof (eg, the retina) can be imaged to provide such biometric data and/or biometric identification. When a user (presumably the same user) wears the head mounted display system, images of the eye, such as the retina, may be obtained at different times. Collections of such images may be recorded, for example, in a database. These images can be analyzed to collect biological data. Such biometric data may be useful for monitoring a user's health or medical condition. Various medical parameters can be monitored by imaging the patient (eg, the patient's eye (eg, retina)). Medical parameters can be recorded and compared to subsequent measurements taken while the user wears the head-mounted display system.

Additionally, if a person starts wearing a head-mounted display system and captures images of the user's eyes that do not match the images stored in the database, it can be concluded that the person currently wearing the head-mounted display system is similar to previous users different. This is useful when determining whether the target user is wearing a headset or the headset is being worn by a new user. Such features may allow for certain medical, safety, and/or ease-of-use applications or functions. For example, the head mounted display may be configured to identify the wearer based on characteristics of the wearer's eyes. For example, the system may be configured to determine an individual based on the wearer's retina (eg, blood vessels), cornea, or other ocular characteristics. For example, in some implementations, a series of indicia can be determined for a particular wearer. Based on the marked series, the system may be able to determine that a previous user is wearing a headset, or alternatively, that another user is wearing a headset. Markers may include the shape or center of the user's cornea, the configuration of blood vessels in the user's retina, the intensity and/or location of reflections of light from the cornea, the shape of aspects of the eye, and/or any other biomarker. In some implementations, a confusion matrix can be determined. As discussed above in the discussion of developing retinal mapping, eg, using virtual/gaze targets located at various locations (see, eg, FIG. 13B ), the system may have user viewing or eye poses in a set of predetermined directions and develop a A feature matrix of the eye or part of the eye (eg, cornea, retina, etc.) associated with each orientation or eye pose. Using such a matrix, the system can determine the identity of an individual. Other methods are possible.

Similarly, as described above, various configurations of the system are possible. For example, FIG. 21 shows an example eyepiece 900 that can be used to simultaneously project light into a user's eye while imaging the user's eye. The illustrated eyepiece 900 includes a coupling optical element 2104 , a light distributing element 2108 and a light consolidating element 2116 , and an outcoupling optical element 2120 on the opposite side of the coupling optical element 2112 . Each of these optical elements may be disposed within or on the waveguide 2102. The waveguide 2102 may correspond, for example, to one of the waveguides 670, 680, 690 described herein (eg, see Figures 9A-9C). In-coupling optical element 2104 may correspond to one of coupling optical elements 700, 710, 720 described herein and/or to in-coupling optical element 942 (see, eg, FIG. 10) and may be configured to convert an image from a projector Content is injected into the waveguide and/or illumination from light source 960 is injected. The light distributing element 2108 may correspond to one of the light distributing elements 730, 740, 750 described herein (see, eg, FIGS. 9A-9C and may be used to distribute light in a given direction and to The light of 2104 is redirected to coupling optical element 2112. Coupling optical element 2112 may correspond to coupling optical element 944 described herein (eg, see Figure 10). In some designs, coupling optical element 2112 includes reference herein to outcoupling optical elements 800 , 810 , 820 (see FIGS. 9A-9C ). Light combining element 2116 may be configured to couple the lateral spatial extent of light received from coupling optical element 2112 and redirect the light to outcoupling optical element 2120 . The out-coupling optical element 2120 may correspond to the out-coupling optical element 952 described herein (eg, see FIG. 10 ).

Coupling optics 2104 may be disposed within or on waveguide 2102 to receive light such as from a projector (eg, image projector 930) and/or an illuminator (eg, light source 960). Light may be delivered to the associated light distributing optical element 2108 via the waveguide 2102 . Any of the in-coupling optical element 2104, light distributing optical element 2108, or coupling optical element 2112 may be disposed on a major surface of the waveguide (eg, on the top or bottom surface) or within the waveguide. Similarly, any one or combination of light combining element 2116 and/or outcoupling optical element 2120 may be disposed on or within a major surface (eg, the top surface or both major surfaces) of waveguide 2102.

Coupling optical element 2112 can receive light from light distributing element 2108 (eg, via TIR) and expand the light to enter the user's eye. Thus, the coupling optical element 2112 may be positioned in front of the user's eyes and project image content therein. Additionally or alternatively, coupling optics 2112 may be configured to provide illumination light onto and/or into the user's eye.

Light reflected from the eye (eg, illumination light from an illumination source) may be reflected and captured by coupling optics 2112. Thus, in some embodiments, coupling optics 2112 may be used to couple light received from light distributing element 2108 out and light received from the eye into waveguide 2102.

In some embodiments, coupling optical element 2112 may include one or more diffractive optical elements (DOEs), such that coupling optical element 2112 has dual functionality. The first DOE (eg, grating, holographic area) can also be configured to couple light out, and the second DOE can be configured to couple reflected light from the eye into the waveguide 2102 . In some embodiments, the first and second DOEs are superimposed within the waveguide 2102 (eg, occupy the same or approximately the same volume).

Optionally, in some embodiments, coupling optical element 2112 includes at least two DOEs stacked on top of or in front of another DOE. For example, referring to FIG. 21, the first DOE of the coupling optical element 2112 may be positioned above, while the second diffractive element may be positioned below the first DOE. In other implementations, the order of each DOE can be reversed.

Cholesteric liquid crystal mirror

Some liquid crystals are in phases called chiral or cholesteric phases. In the cholesteric phase, liquid crystals can exhibit molecular twisting along axes perpendicular to the director, where the molecular axis is parallel to the director. As described herein, a cholesteric liquid crystal (CLC) layer includes a plurality of liquid crystal molecules in a cholesteric phase, the plurality of liquid crystal molecules extending, for example, in a direction perpendicular to the direction of the director (eg, a layer depth direction) and extending along the A rotation direction such as clockwise or counterclockwise is in turn rotated or twisted. The director of the liquid crystal molecules in the chiral structure can be characterized as a helix with a pitch (p), which corresponds to the length in the depth direction of the layer, which corresponds to the liquid crystal molecules in the chiral structure passing through one complete pass along the first rotational direction. The net rotation angle of the rotation. In other words, the helical pitch refers to the distance at which the liquid crystal molecules undergo a complete 360° twist. Liquid crystals exhibiting chirality can also be described as having a twist angle or rotation angle (φ), which for example can refer to the relative azimuthal rotation between successive liquid crystal molecules in the direction of the layer normal, and can have a net twist angle or net rotation angle, the net twist angle or net rotation angle may, for example, refer to the relative azimuthal rotation between the uppermost liquid crystal molecule and the lowermost liquid crystal molecule across a specified length (eg, the length of a chiral structure or the thickness of a liquid crystal layer) . As used herein, a chiral structure refers to a plurality of liquid crystal molecules in a cholesteric phase extending in a direction, eg, perpendicular to the direction of the director (eg, the layer depth direction) and in a direction such as clockwise or Continuous rotation or twist in a counter-clockwise direction of rotation. On the one hand, the director of a liquid crystal molecule in a chiral structure can be characterized as a helix with a helical pitch.

22 shows a cross-sectional side view of a cholesteric liquid crystal (CLC) layer 1004 including a plurality of uniform chiral structures, according to an embodiment. In the CLC layer 1004, adjacent chiral structures in the lateral direction (eg, the x-direction) have liquid crystal molecules aligned similarly. In the illustrated embodiment, the chiral structures 1012-1, 1012-2, ... 1012-i are similarly configured such that liquid crystal molecules of different chiral structures located at about the same depth, eg, are closest to the light The liquid crystal molecules of the incident surface 1004S have the same rotation angle, successive rotation angles of successive liquid crystal molecules located at approximately the same depth, and a net rotation angle of the liquid crystal molecules of each chiral structure.

CLC 1004 includes a CLC layer 1008 that includes liquid crystal molecules arranged as a plurality of chiral structures 1012-1, 1012-2, ... 1012-i, wherein each chiral structure includes a plurality of liquid crystal molecules, where i is any greater than A suitable integer of 2. In operation, when incident light having a combination of a left-handed circularly polarized beam and a right-handed circularly polarized beam is incident on the surface 1004S of the CLC layer 1008 by Bragg reflection, light having one of the circular polarization handedness is blocked by the CLC layer 1004 is reflected, while light with opposite polarization handedness is transmitted through the CLC layer 1008 without substantial interference. As described herein and throughout this disclosure, handedness is defined as viewing along the direction of propagation. According to an embodiment, when the polarization directions or the handedness of the polarizations of the light beams 1016-L, 1016-R are matched such that they have the same rotation direction as the liquid crystal molecules of the chiral structures 1012-1, 1012-2, ... 1012-i , reflecting incident light. As shown, incident on surface 1004S is light beam 1016-L with left-handed circular polarization and light beam 1016-R with right-handed circular polarization. In the illustrated embodiment, the liquid crystal molecules of the chiral structures 1012-1, 1012-2, . . . 1012-i are continuously in a clockwise direction (ie, positive x-direction), which is the same direction of rotation as beam 1016-R with right-handed circular polarization. As a result, light beam 1016-R with right-handed circular polarization is substantially reflected, while light beam 1016-L with left-handed circular polarization is substantially transmitted through CLC layer 1004.

As described above, the CLC layer can be configured as a Bragg reflector by matching the handedness of the polarization of incident elliptically or circularly polarized light with the rotation direction of the liquid crystal molecules of the chiral structure of the CLC layer. Furthermore, one or more CLC layers with different pitches can be configured as wavelength selective Bragg reflectors with high bandwidth. Based on the concepts described herein with respect to various embodiments, the CLC layer may be configured as an off-axis or on-axis mirror configured to selectively reflect a first wavelength range, eg, infrared wavelengths (eg, near-infrared), while transmitting eg Another wavelength range of visible wavelengths.

FIG. 23 shows an example of an eye tracking system 2300 employing a cholesteric liquid crystal reflector (CLCR), such as a cholesteric liquid crystal reflector (CLCR), such as configured to be directed to a viewer's eye 302, according to various embodiments Imaging with the wavelength-selective CLCR 1150. Unlike the CLC layer 1004 described above with respect to FIG. 22, the chiral structures adjacent in the lateral direction (eg, the x-direction) in the wavelength selective CLCR 1150 have liquid crystal molecules arranged differently. That is, the chiral structures are configured such that liquid crystal molecules of different chiral structures (eg, liquid crystal molecules closest to the light incident surface 1004S) located at about the same depth have different rotation angles. As a result, light incident on the CLCR 1150 is reflected at an angle (θ _R ) relative to the layer depth direction, as described further below in the context of the eye tracking system 2300 .

Eye tracking can be a useful feature in interactive vision or control systems, including wearable display systems described elsewhere in this specification for virtual/augmented/mixed reality display applications as well as other applications . In order to achieve effective eye tracking, it may be desirable to obtain a low-angle image of the eye 302, which may in turn require placing the eye tracking camera 702b near the center of the viewer's eye. However, such a position of the camera 702b may interfere with the user's viewing. Alternatively, the eye tracking camera 702b may be positioned at a lower position or side. However, this position of the camera may increase the difficulty of obtaining robust and accurate eye tracking due to the capture of the eye image at a steeper angle. By configuring CLCR 1150 to selectively reflect infrared (IR) light 2308 (eg, having a wavelength of 850 nm) from eye 302 while transmitting visible light 2304 from the world, camera 702b can be positioned away from the user's viewing while capturing normal or low viewing angles eye image. Such a configuration does not interfere with the viewing of the user, as visible light is not reflected. As shown, the same CLCR 1150 can also be configured as an IR illumination source 2320 by reflecting IR light from an IR source such as an IR LED into the eye 302. The low viewing angle of the IR illuminator can result in less occlusion, such as from eyelashes, and this configuration allows for more robust detection of specular reflections, a feature that may be useful in modern eye tracking systems.

Still referring to FIG. 23, according to various embodiments, the CLCR 1150 includes one or more cholesteric liquid crystal (CLC) layers, each cholesteric liquid crystal (CLC) layer including a plurality of chiral structures, wherein each chiral structure A plurality of liquid crystal molecules extending in the layer depth direction (eg, z direction) and continuously rotating in the first rotation direction are included, as described above. The arrangement of the liquid crystal molecules of the chiral structure is periodically varied in a lateral direction perpendicular to the depth direction of the layers, such that the one or more CLC layers are configured to substantially Bragg reflect a first incident light having a first wavelength (λ ₁ ) At the same time the second incident light having the second wavelength (λ ₂ ) is substantially transmitted. As described above, when viewed in the layer depth direction, each of the one or more CLC layers is configured to substantially Bragg reflect an elliptically or circularly polarized first having a polarization handedness matching the first rotational direction and second incident light while being configured to substantially transmit elliptically or circularly polarized first and second incident light having a polarization handedness opposite to the first rotational direction. According to an embodiment, the arrangement of the liquid crystal molecules periodically varying in the lateral direction is arranged to have a period in the lateral direction such that the ratio between the first wavelength and the period is between about 0.5 and about 2.0. According to an embodiment, the first wavelength is in the near infrared range between about 600 nm and about 1.4 μm, eg about 850 nm, and the second wavelength is in the visible range with one or more colors as described elsewhere in the specification . According to various embodiments, the liquid crystal molecules of the chiral structure are pretilted with respect to the direction of the normal to the layer depth direction. As configured, the one or more CLC layers are configured such that the first incident light is reflected at an angle (θ _R ) relative to the layer depth direction (z-direction) that exceeds about 50° relative to the layer depth direction, about 60°, about 70° or about 80°.

So configured, the wavelength selective CLCR 1150 includes one or more cholesteric liquid crystal (CLC) layers, each cholesteric liquid crystal layer including a plurality of liquid crystal molecules extending in a layer depth direction and continuously rotating in a first rotational direction , wherein the arrangement of the chiral-structured liquid crystal molecules changes periodically in a lateral direction perpendicular to the layer depth direction, such that one or more CLC layers are configured to substantially Bragg reflect a first wavelength having a first wavelength such as an IR wavelength The incident light simultaneously substantially transmits second incident light having a second wavelength, eg, a visible wavelength.

Similar liquid crystal layers and structures can be used for the reflector 996 and coating 998 described above in connection with Figures 17-20E. Coating 998 may, for example, comprise a liquid crystal coating, and in certain implementations may be wavelength and/or polarization selective. However, other types of coatings 998 and reflectors 996 may be employed.

In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes can be made therein without departing from the broader spirit and scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Indeed, it should be appreciated that the systems and methods of the present disclosure each have several innovative aspects, no single one of which is solely responsible or required for the desired properties disclosed herein. The various features and processes described above may be used independently of one another or may be used in combination in various ways. All possible combinations and subcombinations are intended to fall within the scope of this disclosure.

Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination. Furthermore, although features may be described above as being performed in certain combinations, even if initially so exemplified, in some cases one or more features from an exemplified combination can be deleted from the combination, and all Exemplary combinations may involve subcombinations or variations of subcombinations. No single feature or group of features is required or integral to every embodiment.

Conditional terms used herein, such as (among other things) "could," "could," "may," "may," "eg," etc. are generally intended to mean that some embodiments include and other embodiments do not include a certain certain features, elements and/or steps unless otherwise specifically stated or understood from the context. Thus, such conditionals are generally not intended to imply that features, elements, and/or steps are in any way necessary for one or more embodiments, or that one or more embodiments necessarily include use with or without a programmer Logic to determine whether these features, elements and/or steps are included or to be performed in any particular embodiment upon input or prompting. The terms "comprising", "comprising", "having" and the like are synonymous and are used inclusively in an open-ended fashion and do not exclude additional elements, features, acts, operations, and the like. Furthermore, the term "or" is used in its inclusive sense (rather than its exclusive sense), thus, when used, for example, to concatenate lists of elements, the term "or" means one, some or all of the elements of the list. In addition, the articles "a," "an," and "said" as used in this application and the appended examples should be construed to mean "one or more" or "at least one" unless specifically stated otherwise. Similarly, although operations may be depicted in the figures in a particular order, it should be recognized that such operations need not be performed in the particular order, or in a sequential order, or performing all illustrated operations, to achieve desirable results. Furthermore, the figures may schematically depict one or more example processes in flowchart form. However, other operations not shown can be incorporated into the illustratively shown example methods and processes. For example, one or more additional operations can be performed before, after, concurrently with, or during any of the illustrated operations. Additionally, in other implementations, operations may be rearranged or reordered. In some cases, multitasking and parallel processing can be advantageous. Furthermore, the separation of the various system components described in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the program components and systems can generally be integrated together in a single software product or Packaged into multiple software products. In addition, other embodiments are within the scope of the following examples. In some cases, the actions recited in the examples can be performed in a different order and still achieve desirable results.

Thus, the present disclosure is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with this disclosure, the principles and novel features disclosed herein. Various example systems and methods are provided below.

Example

Example Part I

1. A head-mounted display system configured to project light onto a user's eyes to display augmented reality image content in the user's field of view, the head-mounted display system comprising:

a frame configured to be supported on the user's head;

an image projector configured to project an image into the user's eye to display image content in the user's field of view;

camera;

at least one waveguide;

at least one coupling optical element configured such that light is coupled into and guided within the waveguide; and

at least one outcoupling element configured to couple light guided in the waveguide out of the waveguide and direct the light to the camera,

wherein the camera is disposed in an optical path relative to the at least one out-coupling optical element to receive at least a portion of the light, the at least a portion of the light being coupled to the waveguide via the coupling element is guided in and out of the waveguide by the out-coupling element, enabling images to be captured by the camera.

2. The system of example 1, wherein the at least one coupling optical element is configured such that light from an environment in front of the user wearing the head mounted display system is coupled into the at least one waveguide and is guided therein so that an image of the environment can be captured by the camera.

3. The system of any of the preceding examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of the user wearing the head mounted display system is coupled into and guided within the at least one waveguide such that an image of the eye can be captured by the camera.

4. The system of any of the preceding examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of the user wearing the head mounted display system is coupled into and guided within the waveguide such that an image of the eye can be captured by the camera, the system is configured to image the front of the eye.

5. The system of any of the preceding examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of the user wearing the head mounted display system is coupled into and guided within the waveguide such that an image of the eye can be captured by the camera, the system is configured to image the corneal surface of the eye.

6. The system of any of the preceding examples, wherein the at least one coupling optical element is configured such that light reflected from the eyes of the user wearing the head mounted display system is coupled into and guided within the waveguide such that an image of the eye can be captured by the camera, the system is configured to image the retina of the user's eye.

7. The system of any of the above examples, further comprising an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eye to the user's field of view displaying augmented reality image content, at least a portion of the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head mounted display such that the transparent portion will come from the Light of the environment in front of the user is transmitted to the user's eyes to provide a view of the environment in front of the user.

8. The system of any one of examples 7-8, wherein the eyepiece is configured to receive light from the image projector and direct the light into the user's eye to be directed toward the user. The user's field of view displays augmented reality image content.

9. The system of any of examples 7-9, wherein the eyepiece includes the at least one waveguide.

10. The system of example 7, wherein the image projector is configured to direct light into the edge of the eyepiece.

11. The system of example 9 or 10, wherein the image projector is configured to direct light into an edge of the at least one waveguide.

12. The system of any of the preceding examples, further comprising at least one coupling-in optical element configured to couple light from the image projector into the at least one waveguide for guiding light from the image projector. light from the image projector to provide the image content to the user's eyes.

13. The system of any of the preceding examples, wherein the at least one coupling optical element is further configured to direct light from the image projector guided in the waveguide from the at least one waveguide is coupled out so that the image content can be viewed by the user's eyes.

14. The system of any of the preceding examples, wherein the same coupling optical element is configured to couple light from the image projector guided within the waveguide out of the waveguide enabling Image content is viewed by the user's eyes, and light is coupled into the at least one waveguide to be directed therein to the camera.

15. The system of any of the preceding examples, further comprising at least one image content out-coupling optical element configured to divert light from the image projector guided within the waveguide from the at least one A waveguide is coupled out so that the image content can be viewed by the user's eyes.

16. The system of any of the preceding examples, wherein the at least one coupling optical element faces the eye of the user wearing the head-mounted imaging system to receive light from the eye .

17. The system of any of the preceding examples, wherein the at least one coupling optical element is configured such that light from the environment in front of the user wearing the head mounted imaging system is coupled into and guided in the at least one waveguide such that an image of the environment can be captured by the camera.

18. The system of any of the preceding examples, wherein the at least one coupling optical element faces the environment in front of the user wearing the head-mounted imaging system to receive signals from the environment. Light.

19. The system of example 15, wherein the at least one image content out-coupling optical element is configured to couple light from the image projector guided in the waveguide out of the at least one waveguide, and the at least one coupling optical element is configured such that light is coupled into the waveguide and directed therein to the camera, wherein the at least one image content outcoupling optical element and the at least one coupling optical element are each other overlay.

20. The system of example 15, wherein the at least one image content out-coupling optical element is configured to couple light from the image projector directed within the waveguide out of the at least one waveguide, and the at least one coupling optical element is configured such that light is coupled into the waveguide and directed therein to the camera, wherein the at least one image content outcoupling optical element and the at least one coupling optical element stacked on top of each other.

21. The system of example 15, wherein the at least one image content out-coupling optical element is configured to couple light from the image projector guided within the waveguide out of the at least one waveguide, and the at least one coupling optical element is configured such that light is coupled into the waveguide and directed therein to the camera, wherein the at least one image content outcoupling optical element and the at least one coupling optical element are integrated in the same diffractive optical element.

22. The system of example 15, wherein the at least one coupling optical element is configured such that light is coupled into the first waveguide and directed therein to the camera, and the at least one image content is coupled out An optical element is configured to couple light from the image projector directed within the second waveguide out of the second waveguide.

23. The system according to any of the preceding examples, wherein the at least one coupling optical element is configured such that light is coupled into the first waveguide and directed therein to the camera, and the image projector is configured to couple light into the second waveguide to provide image content to the eye.

24. The system of any of the preceding examples, wherein the image projector includes a light source, a modulator, and projection optics.

25. The system of any of the preceding examples, wherein the image projector comprises a scanning fiber.

26. The system of any of examples 24 or 25, wherein the modulator comprises a light modulator.

27. The system of example 26, wherein the light modulator comprises a spatial light modulator.

28. The system of any of the preceding examples, wherein the camera includes a detector array and imaging optics.

29. The system of example 28, wherein the imaging optics are configured to focus collimated light onto the detector array.

30. The system of any of the preceding examples, wherein the at least one waveguide comprises a visible light transparent material having an index of refraction sufficient to guide light in the waveguide by total internal reflection.

31. The system of any of the preceding examples, wherein the at least one waveguide comprises a stack of waveguides.

32. The system of example 31, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eye.

33. The system of example 31 or 32, wherein different waveguides in the waveguide stack are configured to output light having different colors.

34. The system of any one of examples 31, 32 or 33, wherein the different waveguides comprise first, second and third waveguides, the system being configured such that the first waveguide is used for red light, The second waveguide is for green light, and the third is for blue light.

35. The system of any of examples 12 to 34, wherein the coupling optical element comprises a diffractive optical element or a reflector.

36. The system of any one of examples 12 to 34, wherein the coupling optical element comprises a diffractive optical element.

37. The system of any preceding example, wherein the coupling optical element comprises a diffractive optical element.

38. The system of any preceding example, wherein the coupling optical element comprises liquid crystal.

39. The system of any preceding example, wherein the coupling optical element comprises a liquid crystal polarization grating.

40. The system of any preceding example, wherein the outcoupling optical element comprises a diffractive optical element.

41. The system of any of the preceding examples, wherein the outcoupling optical element comprises liquid crystal.

42. The system of any of the preceding examples, wherein the outcoupling optical element comprises a liquid crystal polarization grating.

43. The system of any preceding example, wherein the coupling element is configured to increase the size of the eyebox along at least one axis.

44. The system of example 43, further comprising an orthogonal pupil expander comprising at least one light redirecting element in or on the at least one waveguide, the at least one light redirecting element being configured to The size of the eyebox is increased along an axis orthogonal to the at least one axis.

45. The system of example 44, wherein the at least one light redirecting element comprises a diffractive optical element.

46. The system of any of the preceding examples, wherein the same coupling element is configured to (a) couple light into the at least one waveguide to be received by the camera, and (b) to couple light from Light from the image projector is coupled out of the at least one waveguide to the user's eye.

47. The system of any one of the preceding examples, wherein the same coupling element is configured to (a) couple light from the environment into the at least one waveguide to be received by the camera, and (b) Light from the image projector is coupled out of the at least one waveguide to the user's eye.

48. The system of any of the preceding examples, wherein the same coupling element is configured to (a) couple light from the eye into the at least one waveguide for reception by the camera, and (b) coupling light from the image projector out of the at least one waveguide to the user's eye.

49. The system of any one of the preceding examples, further comprising a reflective surface having an optical power, the reflective surface being configured to receive light reflected from a user's eye through the eyepiece and guide the light Back to the eyepiece.

50. The system of example 49, wherein the at least one coupling element is configured such that light from a user's eye that passes through at least one waveguide and is reflected from the reflective surface back to at least one waveguide is coupled to the and guided in at least one waveguide.

51. The system of any one of examples 49 to 50, wherein a camera is disposed in an optical path relative to the at least one out-coupling optical element to receive at least a portion of the light from the user's eye, the light At least a portion of it is reflected from the reflective surface and coupled into the waveguide via the coupling element and guided therein and out of the waveguide via the outcoupling element.

52. The system of any one of examples 49 to 51, wherein the reflective surface reflects infrared light but transmits visible light.

53. The system of any of examples 49 to 52, wherein the reflective surface is curved.

54. The system of any of examples 49 to 53, wherein the reflective surface is provided on the curved optical element.

55. The system of any one of examples 49 to 54, wherein the reflective surface is provided on a concave mirror.

56. The system of any one of examples 49 to 55, wherein the reflective surface has positive power with respect to reflection and negligible power with respect to transmission.

57. The system of any of examples 49 to 56, wherein the reflective surface is configured to collimate light from the user's eyes.

58. The system of any of examples 49 to 57, wherein the reflective surface is configured to collimate light from the retina of the user's eye.

59. The system of any one of examples 49 to 58, wherein the reflective surface is configured to collimate light from the front region of the user's eye.

60. The system of any one of examples 49 to 59, wherein the reflective surface is configured to collimate light from the cornea of the user's eye.

61. The system of any one of examples 49 to 60, wherein the reflective surface is formed on the curved optical element and includes an infrared reflective coating.

62. The system of example 61, wherein the curved optical element has negligible power with respect to light transmitted therethrough.

63. The system of example 61 or 62, wherein the curved optical element has first and second curved surfaces on opposite sides of the curved optical element, the first and second curved surfaces have the same curvature.

64. The system of any one of examples 49 to 63, further comprising a retarder disposed relative to the reflective surface and the coupling optical element to pass through the at least one waveguide and reflect from the reflective surface back to the at least one waveguide and Polarization rotation of the light coupled to the optical element.

65. The system of any preceding example, wherein at least one coupling element comprises a polarization selective steering element.

66. The system of any preceding example, wherein at least one coupling element comprises a polarization grating.

67. The system of any one of the preceding examples, wherein the at least one coupling element is configured to turn light guided within the at least one waveguide out of the waveguide as collimated light directed toward the user's eye.

68. The system of any of the preceding examples, wherein at least one coupling element is configured to divert collimated light from a reflective surface into at least one waveguide.

69. The system of any of the preceding examples, wherein at least one out-coupling element comprises an off-axis reflector.

70. The system of any of the preceding examples, wherein at least one outcoupling element comprises a polarization selective turning element.

71. The system of any of the preceding examples, wherein at least one outcoupling element comprises a polarization grating.

72. The system of any of the preceding examples, wherein at least one outcoupling element comprises liquid crystal.

73. The system of any of the preceding examples, wherein at least one outcoupling element comprises a liquid crystal polarization grating.

74. The system of any of the preceding examples, further comprising a circular polarizer.

75. The system of any of the preceding examples, wherein the coupling element comprises a polarization selective turning element.

76. The system of any preceding example, wherein the coupling element comprises a polarization grating.

77. The system of any preceding example, wherein the at least one coupling-in element comprises a diffractive optical element.

78. The system of any of the preceding examples, wherein at least one coupling element comprises a diffraction grating.

79. The system of any of the preceding examples, wherein the coupling element comprises an off-axis reflector.

80. The system of any one of examples 49 to 79, wherein the reflective surface comprises a liquid crystal reflector.

81. The system of any one of examples 49 to 80, wherein the reflective surface comprises a cholesteric liquid crystal reflective lens.

82. The system of any one of the preceding examples, wherein the same waveguide (a) will be coupled from a user's eye to guide light into the at least one waveguide to be received by the camera in order to capture the user an image of at least a portion of the eye, and (b) directing light coupled from the image projector such that light from the projector can be directed to the user's eye to cause the light from the image projector to The image is in the user's field of view.

83. The system of any one of the preceding examples, wherein the same coupling element (a) couples light from the user's eye into the at least one waveguide to be received by the camera, and (b) coupling light from the image projector out of the at least one waveguide to the user's eye.

84. The system of any one of examples 49 to 83, further comprising electronics configured to cause the camera to capture the first image when light reflected from the reflective surface is blocked.

85. The system of example 84, wherein the electronics are configured to cause the camera to capture the second image when light reflected from the reflective surface is not blocked.

86. The system of example 85, wherein the electronics are configured to modify the second image using the first image.

87. The system of example 85 or 86, wherein the electronics are configured to subtract from the second image based on the first image.

88. The system of any of the preceding examples, wherein the system is configured to perform eye tracking based on images of the eyes.

89. The system of example 88, wherein performing eye tracking based on the image of the eye comprises storing an image of a retina of the eye.

90. The system of any one of the preceding examples, wherein the system is configured to:

using the camera to obtain an image of a portion of the retina of the eye;

comparing one or more stored images of the retina to images of the portion of the retina; and

The user's gaze is determined based on a comparison of the one or more stored images to images of the portion of the retina obtained from the camera.

91. The system of example 90, wherein determining the user's gaze comprises determining which portion of the retina corresponds to the image of the portion of the retina.

92. The system of any of examples 90 to 91, wherein determining the user's gaze comprises determining the orientation of the eyes.

93. The system of any of the preceding examples, wherein the system is configured to obtain biometric data based on one or more images of a user's eye obtained with the camera.

94. The system of any of the preceding examples, wherein the system is configured to identify a user via biosensing based on one or more images of the eye obtained with the camera.

95. The system of any one of the preceding examples, wherein the system is configured to provide illumination of a first polarization and to preferentially by means of the camera using light of a second polarization different from the first polarization Capture images.

96. The system of any one of the preceding examples, wherein the system is configured to illuminate the user's eye with light of a first polarization and to illuminate the eye of the user with light of a second polarization different from the first polarization by means of The camera preferentially captures images of the user's eyes.

97. The system of example 95 or 96, wherein the first and second polarizations are orthogonal.

98. The system of any of the preceding examples, further comprising a light source arranged to provide illumination for capturing images using the camera.

99. The system of any of the preceding examples, further comprising a light source arranged to illuminate a user's eyes.

100. The system of example 98 or 99, wherein the light source comprises one or more infrared light sources.

101. The system of any one of examples 98 to 100, wherein the light source comprises one or more infrared light emitting diodes (LEDs).

102. The system of any of examples 98 to 101, wherein the light source is pulsed.

103. The system of any one of examples 98 to 102, further comprising an off-axis reflector arranged to receive light from the light source and illuminate the user's eye with the light.

104. The system of any of examples 98 to 103, wherein the light source is configured to input light into a waveguide to provide the illumination.

105. The system of any one of examples 98 to 104, wherein the light source is configured to input light into a waveguide disposed relative to the eye to provide illumination to the eye.

106. The system of example 104 or 105, further comprising an illumination coupling optical element configured to couple light from the light source into the waveguide.

107. The system of any one of examples 98 to 103, wherein the light source is configured to input light into the at least one waveguide to provide illumination.

108. The system of example 107, further comprising an illumination coupling optical element configured to couple light from the light source into the at least one waveguide to provide illumination.

109. The system of any of examples 98 to 103, wherein the light source is configured to input light into the same waveguide as used to project image content to the user's eye.

110. The system of any one of examples 98 to 104, wherein the light source is configured to provide illumination to a user's eye to input light into the same waveguide as used to guide the light to the camera.

111. The system of any one of examples 98 to 105, wherein the light source is configured to input light into the same waveguide as used to direct light from the user's eye to the camera.

112. The system of any one of examples 109 to 111, further comprising an illumination coupling optical element configured to couple light from the light source into the waveguide.

113. The system of any one of examples 106, 108, or 112, wherein the illumination coupling optical element is polarization selective, coupling light having a first polarization.

114. The system of examples 98 to 113, wherein the light source is a polarized light source configured to output polarized light having a first polarization.

115. The system of any one of examples 98 to 114, wherein the light source is configured to direct polarized light having a first polarization onto the eye.

116. The system of examples 98 to 115, further comprising an illumination polarizer having a first polarization disposed in the optical path between the light source and the eye to polarize light directed to the eye.

117. The system of example 116, wherein an illumination polarizer is disposed in the optical path between the light source and the waveguide and is configured to provide illumination.

118. The system of any one of examples 98 to 117, further comprising an image acquisition polarizer on the optical path between the eye and the camera.

119. The system of example 118, wherein the image acquisition polarizer is located at the proximal end of the camera.

120. The system of example 118 or 119, wherein the image acquisition polarizer is disposed between (a) the at least one waveguide configured to direct light to the camera and (b) the camera in the light path.

121. The system of any one of examples 118 to 120, wherein the image acquisition polarizer reduces the amount of the first polarized light reaching the camera.

122. The system of examples 118 to 121, wherein the image acquisition polarizer comprises a polarizer configured to selectively couple light with a second polarization different from the first polarization to the camera .

123. The system of any of the above examples, further comprising at least one light combining element disposed in the optical path between the at least one coupling element and the at least one outcoupling optical element to reduce The lateral spatial extent of light from the at least one coupling element before reaching the at least one outcoupling optical element.

124. The system of any of the preceding examples, wherein the at least one light combining element comprises a diffractive optical element.

125. The system of any of the preceding examples, wherein the at least one light combining element comprises a holographic or diffraction grating.

126. The system of any one of the preceding examples, wherein the at least one waveguide comprises a material that is transparent to infrared light having an index of refraction sufficient to guide infrared light in the waveguide by total internal reflection.

127. The system of any of the preceding examples, wherein the at least one coupling optical element comprises an exit pupil expander.

128. The system of any of the preceding examples, wherein the system includes optical power to increase the collimation of light reflected from the eye coupled into the waveguide to be directed to the camera.

129. The system of any of the preceding examples, wherein the system includes optical power to increase collimation of light reflected from the front of the eye coupled into the waveguide to be directed to the camera.

130. The system of any of the preceding examples, wherein the system includes optical power to increase collimation of light reflected from the cornea of the eye coupled into the waveguide to be directed to the camera.

131. The system of any one of examples 128 to 130, wherein the optical power comprises a positive optical power.

132. The system of any of examples 128 to 131, wherein the optical power is provided by a lens.

133. The system of any one of examples 88 to 132, wherein the one or more stored images of the retina of the eye comprises a composite image of the retina of the eye generated using multiple images of different parts of the retina of the eye .

134. The system of any one of examples 88 to 133, wherein the composite image of the retina comprises a plurality of images of the retina stitched together.

135. The system of any one of examples 88 to 134, wherein the plurality of images of the retina stitched together comprise images obtained when a gaze target is displayed in the user's field of view at respective locations.

136. The system of any one of examples 88 to 135, wherein the one or more stored images of the retina comprise images obtained when a gaze target is displayed in the user's field of view at respective locations.

137. The system of any one of examples 88 to 136, wherein the system is further configured to update the composite image using the obtained image of the portion of the retina of the eye.

138. The system of any one of examples 88 to 137, wherein updating the composite image of the retina using the obtained image of the portion of the retina comprises stitching the obtained image into the composite image with the obtained image. The part of the retina shown in the image corresponds to the part in the part.

139. The system of any one of examples 88 to 138, wherein the system is further configured to apply a digital filter to the obtained image of the portion in the retina of the eye to obtain an image of the portion in the retina. filtered image.

140. The system of example 139, wherein the system is further configured to compare one or more stored images of the retina to filtered images of portions of the retina.

141. The system of any one of examples 139 to 140, wherein the digital filter comprises a Frangian filter.

142. The system of any one of examples 88 to 139, wherein the system is configured to apply edge enhancement to the obtained image of the portion in the retina.

143. The system of any of the preceding examples, wherein the system is configured to perform user identification verification using an image of the retina.

144. The system of any one of the preceding examples, wherein the system is configured to:

using the camera to obtain an image of the portion in the retina of the eye;

One or more stored images of the retina are compared to images of the portion of the retina.

145. The system of example 144, wherein the one or more stored images of the retina of the eye comprises a composite image of the retina of the eye generated using multiple images of different portions of the retina of the eye.

146. The system of any one of examples 144 to 145, wherein the composite image of the retina comprises a plurality of images of the retina stitched together.

147. The system of any one of examples 144 to 146, wherein the plurality of images of the retina stitched together comprise images obtained when a gaze target is displayed in the user's field of view at respective locations.

148. The system of any one of examples 144 to 146, wherein the one or more stored images of the retina comprise images obtained when a gaze target is displayed in the user's field of view at respective locations.

149. The system of any of examples 144 to 148, wherein the system is further configured to update the composite image using the obtained image of the portion of the retina of the eye.

150. The system of any one of examples 144 to 149, wherein updating the composite image of the retina using the obtained image of the portion of the retina comprises stitching the obtained image into the composite image with the obtained image. The part of the retina shown in the image corresponds to the part in the part.

151. The system of any one of examples 144 to 150, wherein the system is further configured to apply a digital filter to the obtained image of the portion in the retina of the eye to obtain an image of the portion in the retina. filtered image.

152. The system of example 151, wherein the system is further configured to compare one or more stored images of the retina to filtered images of portions of the retina.

153. The system of any of examples 144 to 152, wherein the digital filter comprises a Frangian filter.

154. The system of any one of examples 144 to 153, wherein the system is configured to apply edge enhancement to the obtained image of the portion in the retina.

Example Part II

1. A head-mounted display system configured to project light onto a user's eyes to display augmented reality image content in the user's field of view, and configured to view in front of a user wearing the head-mounted display system for imaging at least a portion of an environment, the head-mounted display system includes:

a frame configured to be supported on the user's head;

an image projector configured to project an image;

camera;

an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eye to display augmented reality image content to the user's field of view, at least a portion of the eyepiece is transparent and is The head mounted display is positioned at a position in front of the user's eyes when the user is wearing the head mounted display such that the transparent portion transmits light from the environment in front of the user to the user's eyes to provide the A view of the environment in front of the user, the eyepiece includes:

(a) at least one waveguide;

(b) at least one coupling optical element configured to couple light from the image projector into the at least one waveguide for directing light from the image projector therein;

(c) at least one coupling optical element configured to couple light directed in the waveguide from the image projector out of the waveguide and direct the light to a user's eye; and

(d) at least one outcoupling element configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera,

wherein an image projector is arranged in an optical path relative to the at least one coupling optical element to couple light from the image projector into the waveguide to be guided therein so as to pass through the at least one a coupling element couples the light out of the waveguide to the user's eye such that the image from the projector is within the user's field of view,

wherein the coupling element is configured such that light from the environment in front of the user wearing the head mounted display is coupled into and guided within the waveguide,

wherein a camera is arranged in an optical path relative to the at least one outcoupling optical element to receive at least a portion of the light from the environment in front of the user, the at least a portion of the light being coupled to the waveguide via a coupling element is guided in and out of the waveguide by the out-coupling element so that images can be captured by the camera, and

wherein the same waveguide (a) guides light coupled to the waveguide from the environment for reception by the camera to capture an image of at least a portion of the environment in front of the user, and (b) guides light from the projector The coupled light causes light from the projector to be directed to the user's eyes such that the image from the projector is within the user's field of view.

2. The system of example 1, wherein the image projector includes a light source, a modulator, and projection optics.

3. The system of example 1 or 2, wherein the image projector comprises a scanning fiber.

4. The system of example 2 or 3, wherein the modulator comprises a light modulator.

5. The system of example 4, wherein the light modulator comprises a spatial light modulator.

6. The system of any of the preceding examples, wherein the camera includes a detector array and imaging optics.

7. The system of example 6, wherein the imaging optics are configured to focus collimated light onto the detector array.

8. The system of any of the preceding examples, wherein the at least one waveguide comprises a material that is transparent to visible light, the material having an index of refraction sufficient to guide light in the waveguide by total internal reflection.

9. The system of any of the preceding examples, wherein the at least one waveguide comprises a stack of waveguides.

10. The system of example 9, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eye.

11. The system of example 9 or 10, wherein different waveguides in the waveguide stack are configured to output light having different colors.

12. The system of any one of examples 9, 10 or 11, wherein the different waveguides comprise first, second and third waveguides, the system being configured such that the first waveguide is used for red light, The second waveguide is for green light, and the third is for blue light.

13. The system of any of the preceding examples, wherein the coupling optical element comprises a diffractive optical element or a reflector.

14. The system of any of the preceding examples, wherein the coupling optical element comprises a diffractive optical element.

15. The system of any of the preceding examples, wherein the outcoupling optical element comprises a diffractive optical element.

16. The system of any preceding example, wherein the coupling element is configured to increase the size of the eyebox along at least one axis.

17. The system of example 16, further comprising an orthogonal pupil expander comprising at least one light redirecting element in or on the at least one waveguide, the at least one light redirecting element configured to The size of the eyebox is increased along an axis orthogonal to the at least one axis.

18. The system of example 17, wherein the at least one light redirecting element comprises a diffractive optical element.

19. The system of any preceding example, wherein the same coupling element is configured to (a) couple light from the environment into the at least one waveguide for reception by the camera, and (b) Light from the image projector is coupled out of the at least one waveguide to the user's eye.

20. The system of any preceding example, wherein the same coupling element is configured to (a) couple light from the environment into the at least one waveguide for reception by the camera, and (b) Light from the image projector is coupled out of the at least one waveguide to the user's eye.

21. The system of example 20, wherein the at least one coupling element is configured such that light from the user's eye passing through the eyepiece and reflected from the reflective surface back to the eyepiece is coupled into the waveguide and is which is guided.

22. The system of any one of examples 20 to 21, wherein a camera is disposed in the light path relative to the at least one out-coupling optical element to receive at least a portion of the light from the user's eye, the light At least a portion of it is reflected from the reflective surface and coupled into the waveguide via the coupling element and guided therein and out of the waveguide via the outcoupling element.

23. The system of any one of examples 20 to 22, wherein the reflective surface reflects infrared light but transmits visible light.

24. The system of any of examples 20 to 23, wherein the reflective surface is curved.

25. The system of any of examples 20 to 24, wherein the reflective surface is provided on the curved optical element.

26. The system of any one of examples 20 to 25, wherein the reflective surface is provided on a concave mirror.

27. The system of any one of examples 20 to 26, wherein the reflective surface has positive optical power with respect to reflection and negligible power with respect to transmission.

28. The system of any one of examples 20 to 27, wherein the reflective surface is configured to collimate light from the user's eyes.

29. The system of any one of examples 20 to 28, wherein the reflective surface is configured to collimate light from the retina of the user's eye.

30. The system of any one of examples 20 to 29, wherein the reflective surface is configured to collimate light from the front region of the user's eye.

31. The system of any one of examples 20 to 30, wherein the reflective surface is configured to collimate light from the cornea of the user's eye.

32. The system of any one of examples 20 to 31, wherein the reflective surface is formed on a curved optical element having an infrared reflective coating on the reflective surface.

33. The system of example 33, wherein the curved optical element has negligible power with respect to light transmitted therethrough.

34. The system of example 32 or 33, wherein the curved optical element has first and second curved surfaces on opposite sides of the curved optical element, the first and second curved surfaces have the same curvature.

35. The system of any one of the above examples, further comprising a retarder disposed relative to the reflective surface and the coupling optical element to allow light passing through the eyepiece and reflected back from the reflective surface to the eyepiece and the coupling optical element. Polarization rotation.

36. The system of any preceding example, wherein at least one coupling element comprises a polarization selective steering element.

37. The system of any preceding example, wherein at least one coupling element comprises a polarization grating.

38. The system of any one of the preceding examples, wherein the at least one coupling element is configured to turn light guided within the at least one waveguide out of the waveguide as collimated light directed toward the user's eye.

39. The system of any of the preceding examples, wherein the at least one coupling element is configured to divert collimated light from the reflective surface into the at least one waveguide.

40. The system of any of the preceding examples, wherein at least one out-coupling element comprises an off-axis reflector.

41. The system of any of the preceding examples, wherein at least one outcoupling element comprises a polarization selective turning element.

42. The system of any of the preceding examples, wherein at least one outcoupling element comprises a polarization grating.

43. The system of any of the preceding examples, further comprising a circular polarizer.

44. The system of any of the preceding examples, wherein the coupling element comprises a polarization selective turning element.

45. The system of any preceding example, wherein the coupling element comprises a polarization grating.

46. The system of any preceding example, wherein the coupling element comprises an off-axis reflector.

47. The system of any of examples 20 to 34, wherein the reflective surface comprises a liquid crystal reflector.

48. The system of any one of examples 20 to 34 or 37, wherein the reflective surface comprises a cholesteric liquid crystal reflective lens.

49. The system of any of the preceding examples, wherein the same waveguide (a) will be coupled from a user's eye to guide light into the at least one waveguide to be received by the camera in order to capture the user an image of at least a portion of the eye, and (b) directing light coupled from the image projector such that light from the projector can be directed to the user's eye to cause the light from the image projector to The image is in the user's field of view.

50. The system of any one of the preceding examples, wherein the same coupling element (a) couples light from the user's eye into the at least one waveguide to be received by the camera, and (b) coupling light from the image projector out of the at least one waveguide to the user's eye.

51. The system of any of the preceding examples, further comprising electronics configured to cause the camera to capture a first image when light reflected from the reflective surface is blocked.

52. The system of example 51, wherein the electronics are configured to cause the camera to capture the second image when light reflected from the reflective surface is not blocked.

53. The system of example 52, wherein the electronics are configured to modify the second image using the first image.

54. The system of example 53, wherein the electronics are configured to subtract from the second image based on the first image.

55. The system of any of the preceding examples, wherein the system is configured to perform eye tracking based on the image of the eye.

56. The system of example 55, wherein performing eye tracking based on the image of the eye comprises storing an image of a retina of the eye.

57. The system of any one of the preceding examples, wherein the system is configured to:

storing an image of the retina of the eye;

capturing an image of a portion of the retina of the eye;

comparing the stored image of the retina with the image of the portion of the retina; and

The user's gaze is determined based on a comparison of the stored image with the image of the portion in the retina.

58. The system of example 57, wherein determining the user's gaze comprises determining which portion of the retina corresponds to the image of the portion of the retina.

59. The system of any one of examples 57 to 58, wherein determining the user's gaze comprises determining the orientation of the eyes.

60. The system of any of the preceding examples, further comprising a light source arranged to illuminate a user's eyes.

61. The system of example 60, wherein the light sources comprise one or more infrared light sources configured to direct infrared light to a user's eyes.

62. The system of example 60 or 61, wherein the light source comprises one or more infrared light emitting diodes (LEDs).

63. The system of any one of examples 60 to 62, wherein the light source is pulsed.

64. The system of any preceding example, further comprising an off-axis reflector arranged to receive light from the light source and illuminate the user's eye with the light.

65. A head-mounted imaging system configured to image at least a portion of an environment in front of a user wearing the head-mounted imaging system, the head-mounted imaging system comprising:

a frame configured to be supported on the user's head;

camera;

an eyepiece disposed on the frame, at least a portion of the eyepiece being transparent and disposed in front of the user's eyes when the user wears the head-mounted imaging system, such that the transparent portion transmitting light from the environment in front of the user to the user's eyes to provide a view of the environment in front of the user, the eyepiece comprising:

(a) at least one waveguide;

(b) at least one in-coupling optical element configured such that light from the environment in front of a user wearing the head-mounted imaging system is coupled into and directed within the waveguide; and

(c) at least one outcoupling element configured to couple light guided within the waveguide out of the waveguide and direct the light to the camera,

wherein a camera is arranged in an optical path relative to the at least one outcoupling optical element to receive at least a portion of the light from the environment in front of the user, the at least a portion of the light being coupled to the waveguide via a coupling element is guided in and out of the waveguide by the out-coupling element so that images can be captured by the camera.

66. The system of example 65, wherein the camera includes a detector array and imaging optics.

67. The system of example 66, wherein the imaging optics are configured to focus collimated light onto the detector array.

68. The system of any one of examples 65 to 67, wherein the at least one waveguide comprises a material that is transparent to visible light, the material having an index of refraction sufficient to guide light in the waveguide by total internal reflection.

69. The system of any one of examples 65 to 68, wherein the at least one waveguide comprises a stack of waveguides.

70. The system of example 69, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eye.

71. The system of example 69 or 70, wherein different waveguides in the waveguide stack are configured to output light having different colors.

72. The system of any one of examples 69 to 71, wherein the different waveguides comprise first, second and third waveguides, the system being configured such that the first waveguide is used for red light, the second The waveguide is for green light, and the third is for blue light.

73. The system of any of examples 65 to 72, wherein the coupling optical element comprises a diffractive optical element.

74. The system of any of examples 65 to 73, wherein the outcoupling optical element comprises a diffractive optical element.

75. The system of any one of examples 65 to 74, wherein the coupling element is configured to increase the size of the eyebox along at least one axis.

76. The system of example 75, further comprising an orthogonal pupil expander comprising at least one light redirecting element in or on the at least one waveguide, the at least one light redirecting element being configured to The size of the eyebox is increased along an axis orthogonal to the at least one axis.

77. The system of example 76, wherein the at least one light redirecting element comprises a diffractive optical element.

Example part III

1. A head-mounted display system configured to project light onto a user's eyes to display augmented reality image content in the user's field of view, and configured to display augmented reality image content to a user wearing the head-mounted display system imaging at least a portion of the eye, the head-mounted display system includes:

a frame configured to be supported on the user's head;

an image projector configured to project an image;

camera;

an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eyes to display augmented reality image content to the user's field of view, at least a portion of the eyepiece is transparent and is The head mounted display is positioned at a position in front of the user's eyes when the user is wearing the head mounted display such that the transparent portion transmits light from the environment in front of the user to the user's eyes to provide the A view of the environment in front of the user, the eyepiece includes:

(a) at least one waveguide;

(b) at least one coupling optical element configured to couple light from the image projector into the at least one waveguide for directing light from the image projector therein;

(c) at least one coupling optical element configured to couple light directed in the waveguide from the image projector out of the waveguide and direct the light to a user's eye; and

(d) at least one outcoupling element configured to couple the light guided within the waveguide out of the waveguide and direct the light to the camera; and

a reflective surface having an optical power configured to receive light reflected from a user's eye through the eyepiece and direct the light back to the eyepiece;

wherein an image projector is arranged in an optical path relative to the at least one coupling optical element to couple light from the image projector into the waveguide to be guided therein so as to pass through the at least one coupling optical element a coupling element couples the light out of the waveguide to the user's eye so that the image from the projector is within the user's field of view,

wherein the at least one coupling element is configured such that light from the user's eye that passes through the eyepiece and is reflected from the reflective surface back to the eyepiece is coupled into and guided within the waveguide, and

wherein a camera is positioned in the optical path relative to the at least one out-coupling optical element to receive at least a portion of the light from the user's eye, the at least a portion of the light being reflected from the reflective surface and coupled into the waveguide via the coupling element and is guided therein and coupled out of the waveguide via the outcoupling element.

2. The system of example 1, further comprising a light source arranged to illuminate the user's eyes.

3. The system of example 2, wherein the light sources comprise one or more infrared light sources configured to direct infrared light to the user's eyes.

4. The system of example 2 or 3, wherein the light source comprises one or more infrared light emitting diodes (LEDs).

5. The system of any of examples 2 to 4, wherein the light source is pulsed.

6. The system of any of the preceding examples, further comprising an off-axis reflector arranged to receive light from the light source and illuminate the eye with the light.

7. The system of any of the preceding examples, wherein the reflective surface reflects infrared light but transmits visible light.

8. The system of any of the preceding examples, wherein the reflective surface is curved.

9. The system of any of the preceding examples, wherein the reflective surface is provided on the curved optical element.

10. The system of any of the preceding examples, wherein the reflective surface is provided on a concave mirror.

11. The system of any of the preceding examples, wherein the reflective surface has positive power with respect to reflection and negligible power with respect to transmission.

12. The system of any of the preceding examples, wherein the reflective surface is configured to collimate light from the user's eyes.

13. The system of any of the preceding examples, wherein the reflective surface is configured to collimate light from the retina of the user's eye.

14. The system of any of the preceding examples, wherein the reflective surface is configured to collimate light from the front region of the user's eye.

15. The system of any of the preceding examples, wherein the reflective surface is configured to collimate light from the cornea of the user's eye.

16. The system of any of the preceding examples, wherein a reflective surface is formed on a curved optical element having an infrared reflective coating on the reflective surface.

17. The system of example 9 or 16, wherein the curved optical element has negligible power with respect to light transmitted therethrough.

18. The system of any of examples 9 or 16 or 17, wherein the curved optical element has a first curved surface and a second curved surface on opposite sides of the curved optical element, the first curved The surface and the second curved surface have the same curvature.

19. The system of any one of the above examples, further comprising a retarder positioned relative to the reflective surface and the coupling optical element to allow light passing through the eyepiece and reflected back from the reflective surface to the eyepiece and the coupling optical element. Polarization rotation.

20. The system of any of the preceding examples, wherein at least one coupling element comprises a polarization selective steering element.

21. The system of any of the preceding examples, wherein at least one coupling element comprises a polarization grating.

22. The system of any of the preceding examples, wherein the at least one coupling element is configured to divert light guided within the at least one waveguide out of the waveguide as collimated light directed toward the user's eye.

23. The system of any preceding example, wherein the at least one coupling element is configured to divert collimated light from the reflective surface into the at least one waveguide.

24. The system of any of the preceding examples, wherein at least one out-coupling element comprises an off-axis reflector.

25. The system of any of the preceding examples, wherein at least one outcoupling element comprises a polarization selective turning element.

26. The system of any preceding example, wherein at least one outcoupling element comprises a polarization grating.

27. The system of any of the preceding examples, further comprising a circular polarizer.

28. The system of any of the preceding examples, wherein the coupling element comprises a polarization selective steering element.

29. The system of any preceding example, wherein the coupling element comprises a polarization grating.

30. The system of any of the preceding examples, wherein the coupling element comprises an off-axis reflector.

31. The system of any of the preceding examples, wherein the reflective surface comprises a liquid crystal reflector.

32. The system of any of the preceding examples, wherein the reflective surface comprises a cholesteric liquid crystal reflective lens.

33. The system of any of the preceding examples, wherein the image projector includes a light source, a modulator, and projection optics.

34. The system of any of the preceding examples, wherein the image projector comprises a scanning fiber.

35. The system of any of the preceding examples, wherein the modulator comprises a light modulator.

36. The system of example 34, wherein the light modulator comprises a spatial light modulator.

37. The system of any of the preceding examples, wherein the camera includes a detector array and imaging optics.

38. The system of example 36, wherein the imaging optics are configured to focus the collimated light onto a detector array.

39. The system of any of the preceding examples, wherein the at least one waveguide comprises a visible light transparent material having an index of refraction sufficient to guide light in the waveguide by total internal reflection.

40. The system of any of the preceding examples, wherein the at least one waveguide comprises a stack of waveguides.

41. The system of example 40, wherein different waveguides in the waveguide stack are configured to output light with different wavefront divergences, as if projected from different distances from the user's eye.

42. The system of example 40 or 41, wherein different waveguides in the waveguide stack are configured to output light having different colors.

43. The system of any one of examples 40, 41 or 42, wherein different waveguides comprise first, second and third waveguides, the system being configured such that the first waveguide is used for red light , the second waveguide is for green light, and the third waveguide is for blue light.

44. The system of any of the preceding examples, wherein the coupling optical element comprises a diffractive optical element or a reflector.

45. The system of any preceding example, wherein the coupling optical element comprises a diffractive optical element.

46. The system of any preceding example, wherein the outcoupling optical element comprises a diffractive optical element.

47. The system of any preceding example, wherein the coupling element is configured to increase the size of the eyebox along at least one axis.

48. The system of example 47, further comprising an orthogonal pupil expander comprising at least one light redirecting element in or on the at least one waveguide, the at least one light redirecting element being configured The size of the eyebox is increased along an axis orthogonal to the at least one axis.

49. The system of example 48, wherein the at least one light redirecting element comprises a diffractive optical element.

50. The system of any of the preceding examples, wherein the same waveguide (a) will be coupled from a user's eye to guide light into the at least one waveguide to be received by the camera in order to capture the user an image of at least a portion of the eye, and (b) directing light coupled from the image projector such that light from the projector can be directed to the user's eye to cause the light from the image projector to The image is in the user's field of view.

51. The system of any of the preceding examples, wherein the same coupling element (a) couples light from the user's eye into the at least one waveguide to be received by the camera, and (b) coupling light from the image projector out of the at least one waveguide to the user's eye.

52. The system of any of the preceding examples, further comprising electronics configured to cause the camera to capture a first image when light reflected from the reflective surface is blocked.

53. The system of example 52, wherein the electronics are configured to cause the camera to capture the second image when light reflected from the reflective surface is not blocked.

54. The system of example 53, wherein the electronics are configured to modify the second image using the first image.

55. The system of example 54, wherein the electronics are configured to subtract from the second image based on the first image.

56. The system of any of the preceding examples, wherein the system is configured to perform eye tracking based on the image of the eye.

57. The system of example 56, wherein performing eye tracking based on the image of the eye comprises storing an image of a retina of the eye.

58. The system of any one of the preceding examples, wherein the system is configured to:

storing an image of the retina of the eye;

capturing an image of a portion of the retina of the eye;

comparing the stored image of the retina with the image of the portion of the retina; and

The user's gaze is determined based on a comparison of the stored image with the image of the portion in the retina.

59. The system of example 58, wherein determining the user's gaze comprises determining which portion of the retina corresponds to the image of the portion of the retina.

60. The system of any one of examples 58 to 59, wherein determining the user's gaze comprises determining the orientation of the eyes.

61. The system of any of the preceding examples, wherein the coupling element is configured such that light from an environment in front of a user wearing a head mounted display is coupled into and guided in the waveguide .

62. The system of any one of the preceding examples, wherein a camera is positioned in an optical path relative to the at least one out-coupling optical element to receive at least a portion of the light from the environment in front of the user, the light Said at least a portion of is coupled into and guided in the waveguide via a coupling element, and is coupled out of the waveguide via the out-coupling element to enable an image of the environment to be captured by the camera.

63. The system of any one of the preceding examples, wherein the same waveguide (a) guides light coupled to the waveguide from the environment to be received by the camera in order to capture the environment in front of the user and (b) directing light coupled from the projector such that light from the projector is directed to the user's eye such that the image from the projector is located at the within the user's field of view.

64. The system of any of the preceding examples, wherein the same coupling element (a) couples light from the environment into the at least one waveguide to be received by the camera, and (b) Light from the image projector is coupled out of the at least one waveguide to the user's eye.

65. A head-mounted display system configured to project light to a user's eyes to display augmented reality image content in the user's field of view, and configured to display augmented reality image content to a user wearing the head-mounted display system imaging at least a portion of the eye, the head-mounted display system includes:

a frame configured to be supported on the user's head;

an image projector configured to project an image;

camera;

an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eye to display augmented reality image content to the user's field of view, at least a portion of the eyepiece is transparent and is The head mounted display is positioned at a position in front of the user's eyes when the user is wearing the head mounted display such that the transparent portion transmits light from the environment in front of the user to the user's eyes to provide the A view of the environment in front of the user, the eyepiece includes:

(a) at least one waveguide;

(b) at least one coupling optical element configured to couple light from the image projector into the at least one waveguide for directing light from the image projector therein;

(c) at least one coupling optical element configured to couple light directed in the waveguide from the image projector out of the waveguide and direct the light to a user's eye; and

(d) at least one outcoupling element configured to couple the light guided within the waveguide out of the waveguide and direct the light to the camera; and

a positive lens having a positive optical power disposed in the optical path between the user's eye and the eyepiece such that light reflected from the user's eye is projected through the lens to the eyepiece; and

a negative lens having a negative optical power positioned on the other side of the eyepiece with the positive lens to offset the optical power of the positive lens for light from the environment in front of the user,

wherein an image projector is arranged in an optical path relative to the at least one coupling optical element to couple light from the image projector into the waveguide to be guided therein so as to pass through the at least one a coupling element couples the light out of the waveguide to the user's eye such that the image from the image projector is within the user's field of view,

wherein the at least one coupling element is configured such that light from the user's eye passing through the lens to the eyepiece is coupled into and guided within the waveguide, and

wherein the camera is positioned in the optical path relative to the at least one out-coupling optical element to receive at least a portion of the light from the user's eye, the at least a portion of the light being reflected from the reflective surface and coupled into the waveguide via the coupling element and is guided therein and out of the waveguide via the outcoupling element.

66. The system of example 65, wherein the positive lens comprises a Fresnel lens.

67. The system of example 65 or 66, wherein the positive lens is configured to collimate light from an anterior region of the user's eye.

68. The system of any of examples 65, 66 or 67 above, wherein the positive lens is configured to collimate light from the cornea of the user's eye.

69. The system of any one of examples 65 to 68, wherein the system is configured to perform eye tracking based on the image of the eye.

70. The system of any one of examples 65 to 69, further comprising a light source arranged to illuminate the user's eyes.

71. The system of example 70, wherein the light sources comprise one or more infrared light sources configured to direct infrared light to a user's eye.

72. The system of example 70 or 71, wherein the light source comprises one or more infrared light emitting diodes (LEDs).

73. The system of any of the preceding examples, wherein the system is configured to identify a user via biosensing based on the image of the eye.

Claims (20)

1. A head-mounted display system configured to project light onto a user's eyes to display augmented reality image content in the user's field of view, the head-mounted display system comprising: a frame configured to be supported on the user's head; an image projector configured to project an image into the user's eye to display image content in the user's field of view; camera; at least one waveguide; at least one coupling optical element configured such that light is coupled into and guided within the waveguide; and at least one outcoupling element configured to couple light guided in the waveguide out of the waveguide and direct the light to the camera, wherein the camera is disposed in an optical path relative to the at least one out-coupling optical element to receive at least a portion of the light, the at least a portion of the light being coupled to the waveguide via the coupling element is guided in and out of the waveguide by the out-coupling element, enabling images to be captured by the camera. 2. The system of claim 1, wherein the at least one coupling optical element is configured such that light from an environment in front of the user wearing the head mounted display system is coupled to the at least one waveguide and being guided therein so that an image of the environment can be captured by the camera. 3. The system of claim 1, wherein the at least one coupling optical element is configured such that light reflected from the eyes of the user wearing the head mounted display system is coupled to the at least one coupling optical element is guided in and within a waveguide so that an image of the eye can be captured by the camera. 4. The system of claim 1, wherein the at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head mounted display system is coupled to the waveguide and guided therein such that an image of the eye can be captured by the camera, the system is configured to image the front of the eye. 5. The system of claim 1, wherein the at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head mounted display system is coupled to the waveguide and guided therein such that an image of the eye can be captured by the camera, the system is configured to image the corneal surface of the eye. 6. The system of claim 1, wherein the at least one coupling optical element is configured such that light reflected from the eye of the user wearing the head mounted display system is coupled to the waveguide and guided therein such that an image of the eye can be captured by the camera, the system is configured to image the retina of the user's eye. 7. The system of claim 1, further comprising an eyepiece disposed on the frame, the eyepiece configured to direct light into the user's eyes to display augmented reality images to the user's field of view content, at least a portion of the eyepiece is transparent and positioned in front of the user's eyes when the user is wearing the head mounted display, such that the transparent portion will come from the environment in front of the user of light is transmitted to the user's eyes to provide a view of the environment in front of the user. 8. The system of claim 7, wherein the eyepiece is configured to receive light from the image projector and direct the light into the user's eye for display to the user's field of view Augmented reality image content. 9. The system of claim 7, wherein the eyepiece includes the at least one waveguide. 10. The system of claim 7, wherein the image projector is configured to direct light into the edge of the eyepiece. 11. The system of claim 9, wherein the image projector is configured to direct light into an edge of the at least one waveguide. 12. The system of claim 1, further comprising at least one coupling optical element configured to couple light from the image projector into the at least one waveguide for directing from the image projector of light to provide the image content to the user's eyes. 13. The system of claim 1, wherein the at least one coupling optical element is further configured to couple light from the image projector guided in the waveguide out of the at least one waveguide such that The image content can be viewed by the user's eyes. 14. The system of claim 1, wherein the same coupling optical element is configured to couple light from the image projector guided within the waveguide out of the waveguide so that it can be used by the user The eye views the image content and couples light into the at least one waveguide to be directed therein to the camera. 15. The system of claim 1, further comprising at least one image content out-coupling optical element configured to couple light from the image projector directed within the waveguide out of the at least one waveguide , so that the image content can be viewed by the user's eyes. 16. The system of claim 1, wherein the at least one coupling optical element faces the eye of the user wearing the head mounted imaging system to receive light from the eye. 17. The system of claim 1, wherein the at least one coupling optical element is configured such that light from the environment in front of the user wearing the head mounted imaging system is coupled to the at least one coupling optical element is guided in and within a waveguide so that an image of the environment can be captured by the camera. 18. The system of claim 1, wherein the at least one coupling optical element faces the environment in front of the user wearing the head mounted imaging system to receive light from the environment. 19. The system of claim 15, wherein the at least one image content out-coupling optical element is configured to couple light from the image projector guided in the waveguide out of the at least one waveguide , and the at least one coupling optical element is configured such that light is coupled into the waveguide and directed therein to the camera, wherein the at least one image content outcoupling optical element and the at least one coupling optical element on top of each other. 20. The system of claim 15, wherein the at least one image content out-coupling optical element is configured to couple light from the image projector guided within the waveguide out of the at least one waveguide , and the at least one coupling optical element is configured such that light is coupled into the waveguide and guided therein to the camera, wherein the at least one image content outcoupling optical element and the at least one coupling optical element Elements are stacked on top of each other.

Images