WO2007101690A1

WO2007101690A1 - Eye determination apparatus

Info

Publication number: WO2007101690A1
Application number: PCT/EP2007/002017
Authority: WO
Inventors: Bengt REHNSTRÖM
Original assignee: Tobii Technology AB
Current assignee: Tobii AB
Priority date: 2006-03-09
Filing date: 2007-03-08
Publication date: 2007-09-13
Anticipated expiration: 2008-09-09

Abstract

Eye detection apparatus intended for use together with e.g. a computer monitor (7) and for detecting the presence, position and/or gaze direction of a person looking on the monitor includes illuminators (2' 2') and a light detector (3) placed at an edge of the monitor. A single optical filter plate (6) is arranged at the same edge for filtering light to be sensed by the light detector and light emitted by at least one of the illuminators. The filter plate (6) hides the at least one illuminator (2' 2') and the light detector (3) from being directly observed by the person to be studied. The light detector (3) and the at least one illuminator (2' 2') are positioned relative to one another, relative to the single optical filter plate (6), relative to the refractive properties of the filter plate (6) in respect of the light emitted from the at least one illuminator (2', 2') and an angle of the emitted light that first order back reflections in a front surface of the filter plate (6) facing the person are prevented from reaching the light detector.

Description

EYE DETERMINATION APPARATUS

TECHNICAL FIELD

The present invention relates to eye determination apparatus for detecting eyes, in particular for tracking eyes and/or gaze directions of a person.

BACKGROUND

The concept of eye tracking is well known in the art, and a number of different techniques have been developed for accomplishing automatic eye and gaze tracking. In the area of remote, non-obtrusive eye tracking, the most commonly used designs are based on pupil center corneal reflection (PCCR) methods. The basic idea behind these methods is to use at least one light source and to capture a series of images of the eye using a camera. In each image the reflection of the light source, the glint, in the cornea and the pupil is identified. A vector defined from the glint to the center of the pupil is then used to estimate the gaze direction of the eye. Furthermore, within the PCCR-eye-tracking field there exist two main strategies to identify the pupil in the images mentioned above. The light source may be positioned as close as possible to the camera's optical axis. This results in that a part of the retina of the eye illuminated by the light source reflects light back into the camera, and hence the pupil appears bright in the recorded images. Tracking methods based on this strategy are therefore categorized as bright pupil (BP) PCCR methods. Alternatively, the light source can be positioned at some distance from the optical axis of the camera. As a result, substantially no light from the light source will be reflected via the retina into the camera, and the pupil appears dark in the registered images. Eye tracking methods based on this strategy are therefore categorized as dark pupil (DP) PCCR methods.

Whether a BP- or DP-PCCR method is preferable depends on inter alia the ambient light conditions, the subject's age and gender because these factors influence the pupil area. Moreover, the BP response is highly influenced by the ethnicity of the person whose eyes are being tracked. For instance, it has been found that Hispanics generally have a very strong BP response and Caucasians have a somewhat weaker BP response, however still fair enough. Nevertheless, Asians in many cases have an inadequate BP response. Hence, in order to ensure a reliable eye tracking, a combination of BP- and DP-PCCR tracking methods is often desirable.

Monitoring or tracking eye movements and gaze points can be used in many different contexts, such as for evaluating the object at which a person is looking and for evaluating the respective person. This can be of particular interest in e.g. evaluation of Internet home pages, advertising and advertisements and in educating pilots in simulator surroundings where information about the position of the pilot's eyes may be required. It can also be used in interactive applications, such as for assisting a physically disabled person to use a computer and in advanced computer games. Also, the detection of eyes can be used for deciding the presence of persons such as the number of persons present in front of some object of interest.

When doing eye determination, apparatus that draws as little attention as possible is almost always preferable. A goal e.g. in eye tracking studies is usually that the eye tracker should have as little impact as possible on the outcome of the study. Thus an eye tracker should preferably be as non- obtrusive as possible. It would be ideal if a person sitting in front of the eye determination apparatus did not even know that she/he is being eye tracked.

Remote eye tracking systems have historically been fairly obtrusive. It has been apparent to the person in front of the eye tracker that she/he is being eye tracked. One such system based on the bright pupil technique is disclosed in U.S. patent No. 4,836,670. Most of the remote eye tracking systems intended to allow head movements of the subject have been based on a camera together with a motor allowing it to pan and tilt. One such system is the ASL Model R6.

Eye determination systems such as for remote eye tracking are generally based on one or more illuminators emitting light at wavelengths in the range from 800 to 1500 nm, and one or more light detectors, generally a camera having a lens. In most eye tracking products the cameras as well as the illuminators are visible to the subject sitting in front of the eye tracker, see e.g. published International patent applications Nos. WO 2004/043599 and 2005/124521 . The company Tobii Technology AB has a product, Tobii 1750, in which the illuminators as well as the detector are placed behind optical filters. In particular, the detector is placed behind its own optical filter, this filter having a high optical quality.

The Tobii 1750 is the most non-obtrusive eye tracker currently available on the market. However, the number of filters as well as the required high optical quality of the filter in front of the detector renders it expensive to manufacture the product. The number of filters and the shape thereof results in that many customers, or persons-on-test, focus on the round filter and ask if this is where the camera is positioned. Moreover, the need for a high optical quality of the filter in front of the detector places restrictions on the shape of the filter, since these kinds of filters are generally only available in round or rectangular shapes.

SUMMARY

It is an object of the invention to provide an efficient eye determination apparatus, which is even more unobtrusive in terms of the registration means being noticeable to the user/test subject than in the existing products.

According to the invention, the object is achieved by an eye determination apparatus which includes at least one light source, a light detector and a control unit connected to the light detector, wherein a single optical filter plate is arranged to filter light to be sensed by the light detector and light emitted by the at least one light source. The single optical filter plate covers the at least one light source and the light detector, thereby hiding the at least one light source and the light detector from being directly observed by a person staying in the neighborhood of the eye determination apparatus or a person looking at the eye determination apparatus. Furthermore, the light detector and the at least one light source are positioned relative to one another, relative to the single optical filter plate, relative to the refractive properties of the filter plate in respect of the light emitted from the at least one light source and an angle α of the emitted light that first order back reflections in a front surface of the filter plate facing the person are prevented from reaching the light detector.

The proposed apparatus is usable in many eye determination and eye tracking installations, also in eye detecting equipment that does not allow free head movement, such as head mounted eye tracking systems or remote eye trackers that require that the subject is sitting with a chin rest. The latter situation is of course impossible to make non-obtrusive. The subject will know that he is being studied, but he will not necessarily know that or when he is being eye tracked.

Additional objects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objects and advantages of the invention may be realized and obtained by means of the methods, processes, instrumentalities and combinations particularly pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

While the novel features of the invention are set forth with particularly in the appended claims, a complete understanding of the invention, both as to organization and content, and of the above and other features thereof may be gained from and the invention will be better appreciated from a consideration of the following detailed description of non-limiting embodiments presented herein below with reference to the accompanying drawings, in which:

Fig. 1 is a schematic picture from the front of an embodiment of eye tracking apparatus for detecting and tracking eyes and/or gaze angles,

Fig. 2 is a schematic of the electrical connection of the illumination and the light detector of eye tracking apparatus,

Fig. 3 is a front view of a filter plate behind which a light detector and an illuminating unit are placed, Fig. 4 is a view similar to Fig. 3 in which the light detector and the illuminating unit are placed at some distance of each other,

Fig. 5 is a view similar to Fig. 3 in which two illuminators are placed behind the filter plate,

Fig. 6 is a view similar to Fig. 3 in which the filter plate has a different shape, Fig. 7 is a schematic of an installation including a computer monitor for detecting and tracking eyes and/or gaze angles,

Figs. 8 and 9 are graphs illustrating different characteristics of a filter and typical characteristics of a light emitting device and a light detector used together with the filter, and Fig. 10 is a schematic detail view illustrating a distance critical to reduce back reflections of light issued from a light emitting device.

DETAILED DESCRIPTION

In Fig. 1 an eye determination device 1 including a light emitting device 2, also called light source or illuminator, and a light detector 3 for capturing images is illustrated. A person, schematically indicated at 4, can stay in front of the eye determination device and then the person's eyes can be detected. The light emitting device and the detector are connected to a control unit 5, e.g. an electronic microprocessor or microcomputer. The control unit performs the activation of the light emitting device and the light detector, and also makes an evaluation of the signal received from the light detector. Specifically, the evaluation can include determining whether there is any eye in images captured by the light detector, to determine the number of eyes if any, to determine the geometrical position of an eye, the distance of an eye to the eye determination device, the gaze direction of an eye, etc. The light emitting device 2 may e.g. comprise one or more LEDs emitting near infrared light.

The light emitting device 2 and the light detector 3 are both placed behind a single plate 6 of an optical filtering or light filtering material. The filtering material can be a suitable polymer. The light emitting device is positioned to avoid back reflection of light, i.e. light that is reflected in the outer filter surface to be received by the light detector. When performing eye tracking based on the dark pupil technique back reflection is generally not any problem. When using bright pupil eye tracking it is much more difficult to avoid back reflections. Measures that can be taken to avoid back reflections include using a filter that has as high a transmittance as possible for light from the illuminator, placing the illuminator at a small distance from the optical axis of the light detector instead of as close as possible, using an illuminator having as small a viewing angle as possible, using a filter that is as thin as possible and placing the filter as close to the illuminator and the detector as possible. The common filter 6 has the purpose of blocking ambient light from reaching the light detector 3. Ambient light refers to all light that does not originate or is not derived from the illuminators of the eye determination apparatus. If too much ambient light would be sensed by the light detector, it may degrade the functionality of the eye determination apparatus and is therefore not desirable. At the same time the single filter makes the components incorporated in the eye determination device almost invisible to a subject or person staying in front of it.

The optical filter plate 6 may suitably be designed to have a relatively high transmittance of light at the wavelengths of the light emitting device 2, e.g. a transmittance higher than 80 %. According to the discussion above, it may also be suitably designed to have a fairly low transmittance, e.g. a transmittance smaller than 20 %, for light having wavelengths different from those of light issued by the light emitting device, in particular for emitted light of such different wavelengths to which the light detector is also sensitive. If the light emitting device emits in the near infrared spectrum, the filter may suitably block most of the light visible to the human eye.

Figs. 8 and 9 are graphs illustrating a typical detector response, the emission intensity of a typical illuminator and transmission characteristics of filter plates having different designs such as made from two different materials. The filter design used for obtaining the filter characteristic graph drawn in Fig. 8 makes the filter work as a low-pass filter allowing only light to pass having wavelengths close to, corresponding to and longer than those of light emitted by the illuminator. Such a filter designs of low-pass type may in many cases be less costly than band-pass type filters, the latter case illustrated in Fig. 9 where the filter used instead works as a band-pass filter allowing only light to pass having wavelengths close to and corresponding to those of light emitted by the illuminator.

In Figs. 3 - 6 some embodiments of the filter plate 6 arranged at an illumi- nator and a light detector are illustrated that can be suitable for use in eye detection apparatus. Hence, in Fig. 3 the light emitting device 2' is placed as close as possible to the optical axis of the light detector 3 and e.g. as shown coaxially with said optical axis. This arrangement can be particularly useful for eye detection using the bright pupil effect for determining e.g. the position of a person's eye or the direction in which a person is looking. In Fig. 4 the light emitting device 2" is placed at some distance from the optical axis of the light detector 3, this arrangement is especially useful for eye detection using the dark pupil effect. The embodiment shown in Fig. 5 is a combination of the arrangements in Figs. 3 and 4 including both an illuminator positioned close to the optical axis of the light detector and an illuminator placed at some distance of the light detector.

Fig. 10 is a highly schematic view seen from the side of a portion of the filter plate 6 and the detector 3 and an illuminator 2 both placed directly behind the common filter plate 6, where the illuminator is placed relatively close to the detector. It is observed that for such a positioning of the detector 3 and illuminator 2, the risk is very small that back reflections of light emitted by the illuminator 2 and reflected by the rear surface of the filter 6 (i.e. the surface adjacent to the illuminator 2 and detector 3) reaches the detector 3 provided that an angle α of the emitted light is relatively narrow. However, even if the angle α is relatively narrow back reflections in the front surface of the filter 6 (i.e. the opposite surface of the filter 6 facing the person to be studied) may in some cases reach the detector 3.

Nevertheless, according to the invention, the detector 3 and illuminator 2 are positioned relative to one another, relative to the common filter plate 6, relative to its refractive properties in respect of the light emitted from the illuminator 2 and the angle α that back reflections in the front surface are prevented from reaching the detector 3 while allowing a relatively small distance x between the illuminator 2 and the detector 3. Specifically, no back reflections of the first order will be sensed by the detector 3, if the thickness of the filter plate 6 is d and the distance x between adjacent edges of the light emitting surface of the illuminator 2 and the active detecting surface of the light detector 3 fulfils the condition x > 2d tan(α/2). It is worth underlining that Fig. 10 is a highly schematic illustration of the proposed filter, detector and illuminator arrangement and how undesired effects of back reflections in the front surface can be avoided. For example, if at least one of the detector 3 and the illuminator 2 is positioned at a distance away from the filter plate 6, this influences the above-mentioned condition. Nevertheless, for typical filter plate 6 thicknesses and distances between the illuminator 2 and the filter plate 6 respective between the detector 3 and the filter plate 6, x can be a relatively small distance. Furthermore, to ensure that a desired angle α is attained, and at the same time allow more flexibility regarding the positioning of the illuminator 2 and the detector 3 relative to the filter plate 6, a so-called shader (or a divider element) may be inserted between the illuminator 2 and the detector 3. The shader is arranged such that it contacts the filter plate 6 along a line and thus screens off the light emitted from the illuminator 2.

Provided that the filter 6 has a sufficiently high transmittance for the light emitted by the illuminator 2, only the reflections of the first order need to be taken into account. However, these reflections may indeed be problematic. Namely, only a comparatively small fraction of the light from the illuminator 2 reaches the intended target in form of the person's 4 eye, since the amount of light energy per unit target area decreases with the square of the distance to that area. Moreover, the target (i.e. the eye) causes a very small amount of light to be reflected back towards the detector 3. This is because the cornea is spherical and the iris reflects light diffusely. Consequently, the useful light returning to the detector 3 has an energy level whose magnitude is several factors of ten lower than the magnitude of the light energy emitted from the illuminator 2. Due to the applicable distance relationships, also any first order back reflections in the front surface of the filter plate 6 will have an energy level being several factors of ten higher than the energy level of the useful reflected light. Therefore, if any such back reflections reach the detector 3, this light will disturb the detector 3 and thus render detection of light reflected in the person's 4 eye impossible, or at least problematic.

Another condition that can be used is that the illuminator, for the bright pupil case, should be positioned so close to the optical axis of the light detector that then angle between a line from the illuminator to the intended user position and a line from the same intended user position to the light detector is smaller than some suitable chosen small angle such as 4.5°.

The filter plate 6 may e.g. be made from a polymeric material which e.g. by milling can be given any suitable shape, e.g. rectangular as illustrated in Figs. 1 and 3 - 5 or be tapering with a narrower portion at the light emitting device 2" placed at some distance as seen in Fig. 6. Such a suitable shape, e.g. adapted to shape of the remaining portion of the eye determination device, can assist in hiding the active parts of the eye determination device. All these arrangements can be used to detect the presence of eyes and/or the place at which the eye or eyes of a person is located and/or the direction in which a person is looking. A polymer filter can often be produced in any suitable shape at a low cost. The number of filters that are needed for a product can be reduced by using the single filter plate described herein, this making the product less complex and hence making it less expensive to manufacture the product. An industrial designer working on new products is also given larger possibilities to make an appealing product design using a single filter plate while at the same time hiding the eye tracking functionality from the end user. Typically, low-cost polymer filters are less capable of filtering out ambient light in the visible range of the spectrum, and thus prevent such undesired light (i.e. noise) from reaching the detector 3, than today's state-of-the-art special purpose detector filters. However, this imperfection can be compensated for by pulsating the light emitted from the illuminator 2. Thereby, the average energy level of the emitted light can be maintained while the exposure time of the detector 3 can be reduced. As a result, the overall noise level becomes moderate.

In Fig. 7 an installation for determining the point on a monitor 7 at which a user of a computer 8 is looking/gazing is shown. The determination is perfor- med by tracking the eyes of the user and in particular by determining the gaze angles of the eyes and the position of the eyes in three dimensions in front of the monitor. The installation includes the monitor 8, two identical light sources 2" mounted at a common edge, such as the bottom of the monitor 7, e.g. at the bottom corners, a photosensor or light detector 3 such as a CMOS camera placed at the center of the bottom edge of the monitor and a light source 2' placed close to the optical axis of the photosensor. The light sensitive surface of the light detector may as illustrated have a rectangular shape. Each of the light sources 2', 2" may as above comprise one or more LEDs emitting near infrared light. The control unit performs as above the required control, the evaluation associated therewith and calculations, in particular controlling the switching-on and off of the light sources, and processing images captured by the light detector 3. The control unit can e.g. be a combination of a microcontroller 5' built in the monitor 7 which communicates over a USB interface with a control module 5" such as a software program running in the computer 8. The optical filter plate 6 is placed at said common edge, the bottom edge of the monitor 7, covering the light emitting devices 2¹, 2" and the light detector 3.

While specific embodiments of the invention have been illustrated and desc- ribed herein, it is realized that numerous other embodiments may be envisaged and that numerous additional advantages, modifications and changes will readily occur to those skilled in the art without departing from the spirit and scope of the invention. Therefore, the invention in its broader aspects is not limited to the specific details, representative devices and illustrated examples shown and described herein. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. It is therefore to be understood that the appended claims are intended to cover all such modifications and changes as fall within a true spirit and scope of the invention. Numerous other embodiments may be envisaged without departing from the spirit and scope of the invention.

Claims

1. An eye determination apparatus including at least one light source, a light detector and a control unit connected to the light detector, characterized in that the apparatus comprises a single optical filter plate arranged to filter light to be sensed by the light detector and light emitted by the at least one light source, the single optical filter plate covering the at least one light source and the light detector thereby hiding the at least one light source and the light detector from being directly observed by a person staying in the neighborhood of the eye deter- mination apparatus or a person looking at the eye determination apparatus, and the light detector and the at least one light source are positioned relative to one another, relative to the single optical filter plate, relative to the refractive properties of the filter plate in respect of the light emitted from the at least one light source and an angle of the emitted light that first order back reflections in a front surface of the filter plate facing the person are prevented from reaching the light detector.

2. The eye determination apparatus according to claim 1 , wherein the single optical filter plate has a relatively high transmittance, in particular a transmittance higher than 80 %, for light at wavelengths emitted by the at least one light source and has a relatively low transmittance, in particular a transmittance lower than 20 %, for most of or for at least a significant portion of light having wavelengths to which the light detector is sensitive but that are different from the wavelengths of light emitted by the at least one light source.

3. The eye determination apparatus according to any one of the claims 1 or 2, wherein: the at least one light source emits light having wavelengths larger than 800 nm, and the single optical filter plate has a high transmittance, in particular a transmittance higher than 80 %, for light of wavelengths emitted by the at least one light source and has a relatively low transmittance, in particular a transmittance lower than 20 %, for most of the visible light.

4. The eye determination apparatus according to any one of the preceding claims, wherein the at least one light source includes a light source positioned close to or at the optical axis of the light detector.

5 5. The eye determination apparatus according to any one of the preceding claims, wherein the at least one light source includes a light source positioned so close to the optical axis of the light detector that an angle between a line from this light source to the intended user position and a line from the intended user position to the light detector is smaller than 4.5°.

io 6. The eye determination apparatus according to any one of the preceding claims, wherein the at least one light source includes a light source arranged coaxially with an optical axis of the light detector.

7. The eye determination apparatus according to any one of the preceding claims, wherein the at least one light source includes a light source located

15 at a distance from the optical axis of the light detector, the distance in particular being larger or significantly larger than the largest dimension of the photosensitive area of the light detector.

8. The eye determination apparatus according to any one of the preceding claims, wherein the apparatus is arranged to be used with a computer

20 monitor.

9. The eye determination apparatus according to claim 8, wherein the at least one light source, the light detector and the optical filter plate are all arranged to be located at a common edge of the computer monitor, in particular below the active screen area of the monitor.

25 10. The eye determination apparatus according to claim 9, wherein the at least one light source includes at least two light sources, the single optical filter plate is arranged to cover the at least two light sources.