TWI769480B - Eye tracking structure, electronic device and smart glasses - Google Patents

Abstract

An embodiment of the present disclosure provides an eye tracking structure. The eye tracking structure includes a transparent substrate, a plurality of infrared micro LEDs arranged in an array on the substrate, a plurality of micro optical sensing elements arranged in an array on the substrate, and a micro integrated circuit. The infrared micro LEDs and the micro optical sensing elements are electrically connected to the micro integrated circuit. The infrared micro LEDs are used to emit infrared light to a user's eyeball, and the micro optical sensing elements are used to sense the infrared light reflected by the user's eyeball and collect an image of the user's eyeball. The integrated circuit is used to determine a rotation position of the user's eyeball according to the image of the user's eyeball. Another embodiment of the present disclosure provides an electronic device and smart glasses.

Description

Eye tracking structure, electronic device and smart glasses

The present invention relates to the field of display technology, and in particular, to an eye tracking structure, an electronic device and smart glasses.

Eye tracking technology is a technology that uses mechanical, electronic, optical and other detection methods to obtain the user's current "gazing direction". With the rapid development of computer vision, artificial intelligence technology and digital technology, eye tracking technology has become a hot research field and has a wide range of applications in the field of human-computer interaction, such as virtual reality, augmented reality, vehicle assisted driving, user Plural fields such as experience, cognitive impairment diagnosis, etc.

The inventor's research found that the existing eye-tracking structures generally have the defect of low light transmittance. Therefore, there is a need to provide an eye tracking structure with high transmittance.

One aspect of the present invention provides an eye tracking structure, comprising: a transparent substrate; a plurality of infrared micro LEDs, an array of which is arranged on the substrate; a plurality of micro-optical sensing elements, arranged in an array on the substrate; and a micro-integrated circuit, disposed on the substrate, the infrared micro-LED and the micro-optical sensing element are electrically connected to the micro-integrated circuit; Wherein, the infrared micro LED is used to emit infrared light to the eyeball of the user, and the micro optical sensing element is used to sense the infrared light reflected by the eyeball of the user and collect the image of the eyeball of the user. The micro-integrated circuit is used for determining the rotational position of the user's eyeball according to the image of the user's eyeball.

Another aspect of the present invention provides an electronic device, which includes a body and an eye-tracking structure disposed on the body, wherein the eye-tracking structure is the above-mentioned eye-tracking structure.

Another aspect of the present invention provides smart glasses, which include a frame and a lens disposed on the frame, wherein the above-mentioned eye tracking structure is mounted on the lens.

In the eye-tracking structure, the electronic device applying the eye-tracking structure, and the smart glasses applying the eye-tracking structure, the infrared micro-LEDs, the micro-optical sensing elements, and the micro-integrated circuits disposed on the transparent substrate are all micro-scale Yes, the light penetration rate of the eye tracking structure is very high.

10, 20, 30, 40: Eye Tracking Architecture

11: Substrate

12: Infrared Micro LED

13: Miniature Optical Sensing Elements

14: Micro integrated circuits

15: Pixel Micro LED

152: Red Micro LED

154: Green Micro LED

156: Blue Micro LED

16: Leads

17: Pixel unit

18: Drive circuit

100: Electronics

50: Ontology

52: Frames

522: Frame

524: temples

526: Connector

54: Lenses

FIG. 1 is a schematic plan view of an eye tracking structure according to a first embodiment of the present invention.

FIG. 2 is a schematic structural diagram of an eye tracking structure according to a second embodiment of the present invention.

FIG. 3 is a schematic diagram of the arrangement of multiple pixel units of the eye tracking structure according to the third embodiment of the present invention.

FIG. 4 is a schematic diagram of the arrangement of multiple pixel units of the eye tracking structure according to the fourth embodiment of the present invention.

FIG. 5 is a schematic perspective view of an electronic device according to an embodiment of the present invention.

FIG. 1 is a schematic plan view of an eye tracking structure 10 according to a first embodiment of the present invention. As shown in FIG. 1 , the eye tracking structure 10 includes a transparent substrate 11 , a plurality of infrared miniature light emitting diodes (LEDs) 12 arranged in an array on the substrate 11 , A plurality of micro optical sensing elements 13 and a micro integrated circuit (micro IC or μ IC) 14 disposed on the substrate 11 are provided. The infrared micro LED 12 and the micro optical sensing element 13 are electrically connected to the micro integrated circuit 14 .

The infrared micro-LED 12 is used to emit infrared light to the user's eyeball. The micro-optical sensing element 13 is used for sensing the infrared light reflected back by the user's eyeball and collecting the image of the user's eyeball. The micro-integrated circuit 14 is used for determining the position of the user's eyeball rotation according to the image of the user's eyeball, so as to realize the eye tracking function.

In the eye-tracking structure 10 , the infrared micro-LED 12 , the micro-optical sensing element 13 , and the micro-integrated circuit 14 are all micron-level, and the light transmittance of the eye-tracking structure 10 is very high. In one embodiment, the micro integrated circuit 14 can determine the user's eye movement, such as eye movement, gaze, smooth tracking, blinking, etc., according to the position of the user's eyeball movement.

In one embodiment, an area of the user's eye tracked by the eye tracking structure 10 is defined as the target area. Wherein, the target area includes some or all of the cornea. The target area may also include some or all of the eye.

In one embodiment, the target area has a diameter of at least 12 mm (the diameter of an adult cornea is approximately 11.5 mm). In other embodiments, the target area may be significantly larger, eg, greater than or equal to 15 mm.

In one embodiment, the target area may have any shape, such as circular, rectangular, square, oval, and the like.

In one embodiment, the target area includes a boundary area between the sclera of the eye and the cornea of the eye, the boundary area may be defined by where the cornea transitions to the sclera.

In one embodiment, the infrared micro LEDs 12 emit infrared light into the target area. The miniature optical sensing element 13 captures an image of the infrared light reflected back from the target area. The micro integrated circuit 14 determines the position of the user's eyeball rotation based on the image of the infrared light reflected back from the target area. Therein, the infrared light reflected back from the target area includes, for example, reflections of infrared light from parts of the eye (eg, the cornea, iris, and/or sclera) in the target area. In one embodiment, the size of the infrared micro LEDs 12 ranges from 1 micron to 100 microns. The size of the miniature optical sensing element 13 ranges from 1 micrometer to 100 micrometers.

In one embodiment, the miniature optical sensing element 13 is a Complementary Metal Oxide Semiconductor (CMOS) element. The micro-integrated circuit 14 can control the micro-optical sensing element 13 and the infrared micro-LED 12 via micro-transfer. The micro integrated circuit 14 may include CMOS performance and embedded memory (eg, non-volatile memory).

In one embodiment, the infrared micro LED 12 and the micro optical sensing element 13 are electrically connected to the micro integrated circuit 14 by wires (not shown). Wherein, the lead wire is a transparent conductive material, such as indium tin oxide (Indium Tin Oxide, ITO).

In one embodiment, the material of the substrate 11 is transparent glass or transparent plastic. Wherein, when the eye tracking structure 10 is installed on the lens of the smart glasses, the material of the substrate 11 is transparent glass with certain bending properties or transparent plastic with certain bending properties.

In one embodiment, the eye tracking structure 10 can be applied to virtual reality (VR), augmented reality (AR), and mixed reality (MR) products.

FIG. 2 is a schematic structural diagram of an eye tracking structure 20 according to a second embodiment of the present invention. The difference between the eye-tracking structure 20 of the second embodiment and the eye-tracking structure 10 of the first embodiment is that the eye-tracking structure 20 further includes a plurality of pixel micro LEDs 15 arranged on the substrate 11 in an array. The infrared micro LEDs 12 and the micro optical sensing elements 13 are spaced apart from the plurality of pixel micro LEDs 15 . The pixel micro-LED 15 is electrically connected to the micro-integrated circuit 14 , and performs image display under the control of the micro-integrated circuit 14 . That is, the eye tracking structure 20 has both the eye tracking function and the transparent display function.

In one embodiment, the size of the pixel micro-LED 15 ranges from 1 μm to 100 μm, which has the advantages of high brightness, low power consumption, high reliability, and short response time.

In one embodiment, the infrared micro-LED 12 , the micro-optical sensing element 13 , and the pixel micro-LED 15 are electrically connected to the micro-integrated circuit 14 through wires 16 , respectively. The lead 16 is made of transparent conductive material, such as indium tin oxide (Indium Tin Oxide, ITO).

In one embodiment, the eye tracking structure 20 may include a plurality of the micro integrated circuits 14 . Each micro integrated circuit 14 controls an infrared micro LED 12 , a micro optical sensing element 13 and a plurality of pixel micro LEDs 15 .

In one embodiment, the pixel micro-LEDs 15 include a plurality of the pixel micro-LEDs 15 that emit light of different colors, eg, red micro-LEDs 152 that emit red light, and green micro-LEDs 154 that emit green light. And the blue micro LED 156 that emits blue light, by mixing the three primary colors of red, green and blue, the eye tracking structure 20 emits white light to realize the display function.

In the eye tracking structure 20, each micro integrated circuit 14 controls eight red micro LEDs 152, eight green micro LEDs 154 and eight blue micro LEDs 156, but not limited thereto.

In one embodiment, the display of the pixel micro-LEDs 15 can be either actively driven or passively driven.

FIG. 3 is a schematic diagram of the arrangement of the plurality of pixel units 17 of the eye tracking structure 30 according to the third embodiment of the present invention. The difference between the eye tracking structure 30 of the third embodiment and the eye tracking structure 20 of the second embodiment lies in the arrangement of the pixel micro LEDs 15 , the infrared micro LEDs 12 and the micro optical sensing elements 13 .

In the eye tracking structure 30 , the eye tracking structure 30 defines a plurality of pixel units 17 . Each of the pixel units 17 includes a plurality of the pixel micro-LEDs 15 that emit light of different colors, one of the infrared micro-LEDs 12 and one of the micro-optical sensing elements 13 . The pixel micro LEDs 15 in each of the pixel units 17 include a red micro LED 152 , a green micro LED 154 and a blue micro LED 156 .

In other embodiments, the arrangement of the pixel unit 17 and the infrared micro LED 12 , the pixel unit 17 and the micro optical sensing element 13 do not necessarily have to be in a one-to-one correspondence.

As shown in FIG. 3 , the plurality of pixel units 17 are arranged in a matrix. The red micro LEDs 152 , the green micro LEDs 154 , the blue micro LEDs 156 and the infrared micro LEDs 12 in each pixel unit 17 are sequentially arranged in a row. The arrangement of the plurality of pixel units 17 is that one row is all the micro-optical sensing elements 13, and the other row is one of the infrared micro-LEDs 12, one of the red micro-LEDs 152, one of the green micro-LEDs 154, and one of the micro-LEDs 154. blue miniature The LEDs 156 are arranged alternately and periodically. In other embodiments, the arrangement of the plurality of pixel micro-LEDs 15 and the infrared micro-LEDs 12 that emit light of different colors in each pixel unit 17 is not limited to that shown in FIG. Guaranteed to line up.

In another embodiment, the red micro LEDs 152 , the green micro LEDs 154 , the blue micro LEDs 156 and the infrared micro LEDs 12 in each pixel unit 17 are sequentially arranged in a row. The arrangement of the plurality of pixel units 17 is that one row is all the micro optical sensing elements 13, and the other row is one of the red micro-LED 152, one of the green micro-LED 154, one of the blue micro-LED 156, and one of the micro LEDs. The infrared micro LEDs 12 are arranged alternately and periodically. Similarly, the arrangement of the plurality of pixel micro-LEDs 15 and the infrared micro-LEDs 12 in each pixel unit 17 that emit light of different colors is not limited to this, and can also be adjusted as needed, as long as they are arranged in a row .

FIG. 4 is a schematic diagram of the arrangement of the plural pixel units 17 of the eye tracking structure 40 according to the fourth embodiment of the present invention. The difference between the eye tracking structure 40 of the fourth embodiment and the eye tracking structure 30 of the third embodiment is that in each pixel unit 17 , the arrangement of the pixel micro LEDs 15 , the infrared micro LEDs 12 and the micro optical sensing elements 13 is different.

As shown in FIG. 4 , in the eye tracking structure 40 , a plurality of the pixel units 17 are arranged in a matrix. The red micro LEDs 152 , the green micro LEDs 154 , the blue micro LEDs 156 and the infrared micro LEDs 12 in each pixel unit 17 are arranged in a 2×2 matrix. Wherein, one red micro LED 152 and one infrared micro LED 12 are arranged in one row of each pixel unit 17 , and one blue micro LED 156 and one green micro LED 154 are arranged in the other row. The arrangement of the plurality of pixel units 17 is that one row is all the micro optical sensing elements 13 , and the other row is all the 2×2 matrix.

In one embodiment, the red micro-LED 152 , the green micro-LED 154 , the blue micro-LED 156 and the infrared micro-LED 12 included in the 2×2 matrix share a driving circuit 18 , and are connected to the driving circuit 18 . Driven to emit light.

In other embodiments, the arrangement of the plurality of pixel micro LEDs 15 and the infrared micro LEDs 12 that emit light of different colors is not limited to that shown in FIG. . In addition, in other embodiments, each pixel unit 17 is not limited to include pixel micro-LEDs 15 of the above three colors, and may also include pixel micro-LEDs 15 that emit light of other colors.

FIG. 5 is a three-dimensional schematic diagram of an electronic device 100 according to an embodiment of the present invention. The electronic device 100 includes a body 50 and an eye-tracking structure 10 ( 20 , 30 , 40 ) disposed on the body 50 .

The electronic device 100 shown in FIG. 5 is smart glasses. The body 50 includes a frame 52 and a lens 54 disposed on the frame 52 . The mirror frame 52 includes a mirror frame 522 , a mirror leg 524 and a connecting portion 526 . The temple 524 is connected to the frame 522 by the connecting portion 526 and can be folded relative to the frame 522 .

In one embodiment, the smart glasses include the eye tracking structure 10 mounted on the lens 54 , and the smart glasses have an eye tracking function. In the eye-tracking structure 10, the substrate 11 is transparent, the infrared micro-LED 12 and the micro-optical sensing element 13 are micron-scale, and the arrays are arranged on the substrate 11, so that the eye-tracking structure 10 has high light transmittance, When the eye tracking structure 10 is installed on the lens 54, the lens 54 can be kept transparent without blocking the user's sight.

In another embodiment, the smart glasses include the eye tracking structure 20 ( 30 , 40 ) mounted on the lens 54 , and the smart glasses have dual functions of eye tracking and transparent display. When the pixel micro LED 15 emits light, the smart glasses realize the function of transparent display, and when the infrared micro LED 12 and the micro optical sensing element 13 are used together, the smart glasses realize the eye tracking function.

In the eye-tracking structure 20 ( 30 , 40 ), the pixel micro-LED 15 , the micro-optical sensing element 13 and the infrared micro-LED 12 are integrated on the same substrate 11 , so that the dual functions of eye-tracking and transparent display can be combined. Thereby, it is avoided to additionally provide a display layered with the eye tracking structure in the smart glasses, and the thickness of the lens 54 is reduced.

Similarly, in the eye tracking structure 20 ( 30 , 40 ), the substrate 11 is transparent, the pixel micro-LEDs 15 , the infrared micro-LEDs 12 , and the micro-optical sensing elements 13 are micron-scale, and the arrays are arranged on the substrate 11 . The eye tracking structure 20 (30, 40) has high light transmittance, and when the eye tracking structure 20 (30, 40) is installed on the lens 54, the lens 54 can be kept transparent without blocking the user Sight.

In other embodiments, the electronic device 100 may also be a non-contact smart terminal, which performs preset operations by tracking the user's eyeballs. For example, the electronic device 100 can be a mobile phone, and can display a specific screen by tracking the movement of the user's eyeballs.

The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced. without departing from the spirit and scope of the technical solutions of the present invention.

10: Eye Tracking Architecture

11: Substrate

12: Infrared Micro LED

13: Miniature Optical Sensing Elements

14: Micro integrated circuits

Claims (11)

An eye-tracking structure, which is improved by comprising: a transparent substrate; a plurality of infrared micro LEDs, an array of which is arranged on the substrate; a plurality of micro-optical sensing elements, an array of which is arranged on the substrate; and a micro-integrated circuit, Disposed on the substrate, the infrared micro-LED and the micro-optical sensing element are electrically connected to the micro-integrated circuit; wherein, the infrared micro-LED is used for emitting infrared light to the user's eyeball, and the micro-LED is The optical sensing element is used for sensing the infrared light reflected back by the user's eyeball and collecting the image of the user's eyeball, and the micro integrated circuit is used for determining according to the image of the user's eyeball The position where the user's eyeballs are turned. The eye-tracking structure of claim 1, wherein an area of the user's eye tracked by the eye-tracking structure is defined as a target area, and the target area includes a boundary area between the sclera of the eye and the cornea of the eye , the boundary area is defined by the position where the cornea of the eye transitions to the sclera of the eye; the infrared micro LED is used to emit the infrared light into the target area, and the micro optical sensing element is used to capturing an image of the infrared light reflected back from the target area, the micro integrated circuit for determining the position of the user's eye roll based on the image of the infrared light reflected back from the target area . The eye tracking structure according to claim 2, further comprising a plurality of pixel micro-LEDs arranged in an array on the substrate, the pixel micro-LEDs are electrically connected to the micro integrated circuit, and are connected to the micro-integrated circuit. Image display is performed under the control of the integrated circuit. The eye-tracking structure of claim 3, wherein the eye-tracking structure defines a plurality of pixel units, each of the pixel units includes a plurality of the pixel micro-LEDs that emit light of different colors, one of the infrared Micro LED and one of the micro optical sensing elements. The eye-tracking structure of claim 4, wherein the pixel micro-LEDs in each of the pixel units include red-emitting red micro-LEDs, green-emitting green micro-LEDs, and blue-emitting blue micro-LEDs . The eye-tracking structure according to claim 5, wherein the red micro-LED, the green micro-LED, the blue micro-LED and the infrared micro-LED in each pixel unit are in a row or group Arranged in a row. The eye-tracking structure according to claim 6, wherein the plurality of pixel units are arranged in a matrix, and the arrangement of the plurality of pixel units is such that one row is all the micro-optical sensing elements, and the other row is all the micro-optical sensing elements. The red micro-LEDs, the green micro-LEDs, the blue micro-LEDs, and the infrared micro-LEDs are alternately and periodically arranged. The eye tracking structure according to claim 5, wherein the red micro-LED, the green micro-LED, the blue micro-LED and the infrared micro-LED in each pixel unit form a 2×2 Matrix arrangement. The eye-tracking structure according to claim 8, wherein the plurality of pixel units are arranged in a matrix, and the arrangement of the plurality of pixel units is such that all the micro-optical sensing elements in one row and all the micro-optical sensing elements in the other row are arranged in a matrix. The 2×2 matrix described above. An electronic device includes a body and an eye-tracking structure disposed on the body, wherein the eye-tracking structure is the eye-tracking structure according to any one of claim 1 to 9. A kind of smart glasses includes a frame and a lens arranged on the frame, wherein the eye tracking structure according to any one of claims 1 to 9 is mounted on the lens.

Images