DE19719695A1 - Method and device for measuring the attention of living beings within a set of living beings - Google Patents

Abstract

The invention relates to a method and device for detecting the attention of living organisms, specially human beings, within a plurality of analog living organisms. To this end, light is emitted towards the group. The light reflected by the living organism is detected in the form of an image (40). The image (40) is analyzed in such a way that the light that is reflected by the fundus oculi of the living organisms is filtered and recognized as an image point (53). The image points (53) are counted.

Description

The invention relates to a method for measuring the attentiveness of living things within a set of living things, with the features:

• a) sending out a light to the crowd of living beings;
• b) Detecting the light reflected by the living being in the form of a picture.

The invention further relates to a device for measuring the attention of living beings within a set of living beings with:

• a) Means for emitting a light on the crowd the living being;
• b) means for detecting the reflections of the living beings light in the form of an image.

It is known e.g. B. in the advertising industry, what attention z. B. an advertisement, such as a Advertising poster, generated within a crowd that the Advertising display can see, for example, on an advertising Poster passed.

This usually happens because the crowd how many is observed and estimated by observers People within the crowd look at the advertising turn presentation. You can also watch a video for this purpose Set up the camera and turn on the image captured by the camera look at another place or draw on and finally it is also possible to photograph the crowd in question graph and the captured photos afterwards to evaluate speaking.

These known methods and devices, which in essence either from a personal observation or from a Photography / videography exist, but are very complex in the implementation and / or evaluation.

On the other hand, there is a need for a way the attention of living beings, especially humans,  to be able to measure reliably. This is not only interesting for the advertising industry but also for TV stations, who want to determine how much of the viewers and for how long sees a particular television program. To determine this so-called ratings are traditionally measured devices used in certain households be, with these households a representative cross cut through the population.

The people living in the household are now instructed on a keypad of the device by pressing a button to enter which person within what time duration which program is watching on TV.

It is known that this acquisition of ratings with some uncertainties. Not only is uncertain whether the people living in the household concerned Operate the device in question reliably, too it is also reliable whether a person in the household, which is their capacity as a viewer of a particular broadcast signaled by pressing the button assigned to it, then this program is actually followed closely or is actually doing another job, for example Reads newspaper.

There is therefore a need to be attentive of the members of this household can.

In a method of the type mentioned at the outset, this object is achieved according to the invention by the following features:

• c) Analyzing the image so that from eyes backgrounds of living beings reflected light filtered out and recognized as a pixel; and
• d) counting the pixels.

In the case of a device of the type mentioned in the introduction, the object on which the invention is based is achieved by:

• c) means for analyzing the image such that of Reflecting behind the eyes of living things Light is filtered out and recognized as a pixel; and
• d) means for counting the pixels.

The object underlying the invention is based on this Way completely solved.

By detecting the reflections from the eye backgrounds tiertes light is first ensured that only those people whose eyes are open are grasped and with the eye open in a predetermined direction look. It can therefore reliably all in this way People who are not aware are excluded if they neither have their eyes closed or in any other than look the direction of interest.

In a preferred development of the invention Process is emitted as light infrared light.

This measure has the advantage that with one for the measurement sufficient light intensity can be worked without  this is annoying or even harmful for the observed living being is because infrared light in the invisible spectrum of the Light and its amplitude are medically acceptable values can be set.

In a corresponding manner, it is expedient that the reflected Capture light through an infrared filter.

This measure has the advantage that the ambient light is wide can be filtered out, this filter effect naturally the better the less infrared components the surrounding light has.

In a further preferred embodiment of the Invention According to the method, the light is emitted in a pulsed manner.

This measure has the advantage that with a relative low average light intensity can be worked a correspondingly high level during the pulse duration is adjustable.

The pulse duration is preferably of the order of magnitude a few ms, especially in the order of 5 ms.

A method is particularly preferred in which the image as Video image is captured. This measure has the advantage that the image is in a common waveform, so that a subsequent image processing is particularly simple.

In a preferred development of this embodiment for example, multiple videos will be consecutive pictures taken with a given repetition rate.  

This measure has the advantage that on the one hand it is longer timely observation of living things is possible, on the other hand but by separately evaluating the individual video images higher reliability can be achieved.

The repetition frequency is preferably 25 Hz, but can for example, also be 30 Hz.

In the context of this exemplary embodiment, it is further preferred if to record consecutive video images different light sources can be used.

This measure has the advantage that the quantity or group of Living beings from different light sources different angles can be illuminated, so that the Mes solutions become even more reliable.

It is particularly preferred to take successive video images alternately in at least two different memories read. It is particularly preferred that the one Analyze stored image while that for image recorded at least the second memory becomes.

This measure has the advantage that with a relatively high re frame rate of the video images can be worked, others but for the evaluation of one image each is available twice as long compared to a procedure in which the images are strictly serial would work.  

Open especially when using several parallel memories there is the possibility of a tremor-free display a video monitor, especially on an LCD screen.

This measure has the advantage that the observed group of living things also by optically viewing a video image can be operated using a LCD screen also opens up the possibility of ver drive with portable devices.

In a preferred embodiment of the invention The number of pixels is represented alphanumerically posed.

This measure has the advantage that an immediate out evaluation of the measured values is possible.

The number of pixels can preferably be in the Video image can be displayed.

This has the advantage that the measurement immediately matches the video image be assigned and z. B. by printing out the video image can be documented with the displayed numbers.

In order to suppress artifacts when carrying out the method according to the invention, which arise, for example, by reflection of the measuring light on well-reflecting surfaces (jewelry, watch glasses and the like), several preferred method variants are provided:
In a first variant, the pixels are only taken into account if they occur in pairs.

This variant takes into account the fact that with a careful observation both eyes in the same direction are aligned so that the pixels are forcibly in pairs must occur.

In another variant, the pixels are only then taken into account when they are a predetermined distance are spaced.

This measure takes into account the fact that z. B. at People the pupil distance is largely always the same and is about 65 mm.

Suppression of artifacts is also possible lich that be only such paired pixels are taken into account, the connecting line of which is essentially runs horizontally.

This variant takes into account the fact that an attentive sam observing person usually upright holds.

A further suppression of artifacts is possible Lich that only those pixels are taken into account that in similar to a predetermined extent of a circular shape.

This variant takes advantage of the fact that the Pupil is circular and that reflect from the fundus The light thus appears as a circular point. You can also use this version a Ver Apply refinement by using only such pixels takes into account whose diameter in a predetermined Diameter range. This diameter range can  For example, by a certain number of pixels in the captured Video image.

In another variant of the suppression of artifacts the image points similar to a circular shape are only then takes into account if their brightness is a predetermined one Measure of the average brightness of the surrounding background of the picture stands out.

This process variant can be used above all if to perform the measurement of a strong infrared light ver is used, whose images clearly differ from the surrounding Stand out background.

Finally, suppression of artifacts is beneficial still possible because the image similar to a circular shape points are only taken into account if the pixel of another, substantially circular area is.

This method variant takes advantage of the fact that the pupil of an eye from the also circular iris is surrounded, which also with correspondingly accurate measurement is recognizable in the video image.

It has already been mentioned that the invention be particularly can preferably be used in the advertising industry, so that preferred embodiments of the device according to the invention device are preferably designed to be portable.

Finally, a device according to the invention can be preferred instruct in operative connection with one on a television set orderly detection device to determine the on  switching rates are in operative connection, or the device is attached to an advertising object.

Further advantages result from the description of the beige added drawing.

It is understood that the above and the following standing features to be explained not only in the ever because specified combination, but also in other combinations nations or alone can be used without the To leave the scope of the present invention.

Embodiments of the invention are shown in the drawing and are described in more detail in the following description explained. Show it:

Fig. 1 shows a highly schematic beam path to explain the acquisition technology underlying the invention;

Fig. 2 is an extremely schematic schematic Dar position of a video camera, as can be used in the context of the present invention;

FIGS. 3 and 4 are illustrations of a person who is evaluated in a case of non-attentive and attentive in the case other than (Fig. 4) head (Fig. 3);

Figure 5 is a means of the inventive method to image taken of a group of people who are attentive partially within the meaning of the invention and partly non-attention.

Fig. 6 is an extremely schematic representation for explaining a possibility of suppressing artifacts;

FIG. 7 shows a further illustration, likewise for illustrating options for suppressing artifacts; FIG.

Fig. 8 is an extremely schematic block diagram of an embodiment of a device according to the invention; and

Fig. 9 is a highly schematic perspective view of a use case of the present invention.

In Figs. 1 and 2 10 denotes a camera, in particular a video camera. The camera 10 comprises a housing 11 and a tube 12 placed at the front. A conventional lens 13 is located at the front end of the tube 12 . An infrared filter 14 is preferably arranged in front of the lens 13 .

The lens 13 is surrounded by a ring 15 which holds a plurality of light emitting diodes, in particular infrared light emitting diodes 16 . Within the housing 11 of the camera 10 there is also an image recording element, in particular a CCD element 17 with a schematically indicated connection element 18 .

As a camera 10 z. For example, a conventional CCD video camera can be used, such as that sold by the company DBS Digitale Bildverarbeitung, Bremen, under the type designation DMK 2002. Such a camera has a good sensitivity in the near infrared (700 to 900 mm) without infrared filter.

The lens 13 can be a conventional wide-angle lens of 28 mm focal length in order to be able to monitor a reasonably large field of view.

Commercially available high-power infrared light-emitting diodes can be used as the light-emitting diodes 16 , such as those used for. B. are available from Conrad Electronic under the type designation 185809-99. The light emitting diodes 16 can be pulsed synchronously via a corresponding circuit from the serial interface of a computer used for evaluation, the pulse duration preferably corresponding to 5 ms lighting duration per video image. The light energy is then sufficient to carry out the desired measurements at a distance of 5 to 10 m. The visible light is faded out by the filter 14 , which is permeable to infrared between 700 and 900 nm.

The camera 10 is operated so that it can be used with a portable computer housing including a motherboard, video card and LCD display. The video images are preferably displayed at a refresh rate of 25 Hz. In this way, the weight of the overall arrangement can be kept at about 6 kg. The CCD element 17 is selected so that the pixel resolution of the video image is approximately 795 (H) × 596 (V) pixels.

In the right half of FIG. 1, 20 is an eye, in particular a human eye. The eye 20 has a lens 21 and an eye fundus 22 , which is essentially formed by the retina.

The light 25 emitted by the infrared light-emitting diodes 16 reaches through the lens 21 through to the fundus 22 and is diffusely reflected there, as indicated at 26 . The light 27 reflected by the lens 21 now passes through the lens 13 of the camera 10 and reaches the CCD element 17 .

In FIGS. 3 and 4 is shown how this measurement in a person 30 a and 30 b impacts that once "inattentive" in the sense of the present invention and is "attention" in the other case.

Fig. 3 shows the case of the inattentive person 30 a. The person 30 a looks in Fig. 3 to the left, as indicated by a letter L and an arrow 32 . The pupils 31 a are therefore directed to the right in the view of FIG. 3. Thus, no emitted light 25 reaches the fundus 22 because the optical axis in the viewing direction according to FIG. 3 differs from the optical axis 28 according to FIG. 1. The image of the eyes of the person 30 a recognizable in the video image is therefore inconspicuous.

If, however, the person 30 b facing the camera 10 as shown in FIG. 4, then the case illustrated in FIG. 1 by way of the beam path provides a. The light 27 reflected by the background 22 of the eye now passes fully into the camera 10 and its CCD element 17 , so that the pupils 31 b are clearly recognizable as light spots.

This effect is known from the usual photography, in the case of people illuminated with a flash with red eyes because the flash light in the open eye falls and from the well-circulated network skin is reflected in the back of the eye. This effect will  in poor lighting conditions even more so that the pupil is wide open in these lighting conditions.

In Fig. 5 the case is now shown that not only a single person but rather a group of people is measured at the same time with the explanatory technique.

A total of nine people can be seen schematically in FIG. 5, some of them looking in the direction of the camera (attentively) or looking in another direction (inattentive).

The video scene 40 reproduced in FIG. 5 shows a total of five people 41 , 42 , 43 , 44 , 45 who are attentive in this sense and four people 46 , 47 , 48 , 49 who are inattentive in this sense. In addition, three artifacts 50 , 51 , 52 can be seen.

As can be seen from FIG. 5, the eyes (or respectively the iris) of the attentive persons 41 to 45 are depicted as clear image points 53 , while this is not the case for the non-attentive persons 46 to 49 .

In addition, the video scene 40 shows a first artifact 50 in the person 49 because the infrared light was reflected on a hair clip. In person 47 , an artifact 51 is shown by reflection on a belt buckle, while a third artifact 52 is of unknown origin.

In order to be able to reliably suppress these artifacts 50 to 52 , various procedures are conceivable.

Fig. 6 shows, for example, that two pixels 53 , which correspond to the pupil pair of two eyes, must be arranged at least approximately at a predetermined distance, namely the usual pupil distance of people d _A , which is approximately 65 mm.

In addition, the connecting line between the two pixels 53 should run approximately horizontally, since strange people normally hold their heads upright. This is indicated in Fig. 6 with an angle α, which should be approximately 0 ° to the horizontal.

Fig. 7 indicates further options for suppressing artifacts.

As already stated above, the CCD element 17 of the camera 10 preferably has a resolution of 795 × 596 pixels in the horizontal or vertical. The pixels are designated 55 in FIG. 7.

In order to be able to reliably detect an artifact, ie a non-circular pixel, z. B. a minimum pupil diameter of 4 to 5 pixels, ie picture elements, is recommended. With 5 pixels pupil diameter and an average pupil diameter of 5 mm, 1 pixel in the video image corresponds to approximately 1 mm. The field of view of the video scene 40 can therefore be approximately 8 m in size.

You can now search the video scene 40 for circular image points that are at least 4 pixels in diameter. You can also check whether their brightness is higher than the average background brightness.

Finally, one can also evaluate whether the pixel 53 recognized as the pupil 31 is surrounded by another circle, namely the iris 56 of the eye, as shown in FIG. 7. Of course, the iris 56 does not have to be shown in its full extent, since it can be partially covered by the eyes lids above and below.

In the block diagram shown in FIG. 8, the camera 10 with its already explained elements 11 to 18 can be seen at the top.

A first signal line 58 leads from the camera 10 to a storage unit 60 . The storage unit 60 comprises a first memory 61 and a second memory 62 . A second signal line 63 and a third signal line 64 lead from the first memory 61 and the second memory 62 to a control unit 65 . The control unit 65 has a fourth control line 66 on the output side, which also serves as a clock line. The control unit 65 preferably comprises an alphanumeric display 68 .

A fifth signal line 67 leads from the control unit 65 to a display unit 70 . The display unit 70 has an LCD screen 71 .

The video image recorded by the CCD element 17 is alternatively fed to the first memory 61 via the first signal line 58 or to the second memory 62 with the subsequent video image. If e.g. B. a first video image was read into the first memory 61 , this is immediately transmitted to the control unit 65 via the second signal line 63 , while the subsequent image is read into the second memory 62 . In the control unit 65 , the image read from the first memory 61 is evaluated in the sense already mentioned by searching the video scene 40 ( FIG. 5) of the image for corresponding pixels 53 while the artifacts 50 to 52 are suppressed. The result of this counting of the pixels is then shown in the display 68 . It is understood that the number of pixels is divided by two in order to deduce the number of attentive persons from the number of pixels.

At the same time, the video scene 40 is displayed on the LCD screen 71 of the display unit 70 .

While the video image read in the second memory 62 is being evaluated, the video image that follows in turn is read into the first memory 61 , etc.

In order to coordinate and synchronize these operations, the fourth control line 66 serving as the clock line is provided. A clock signal sent out by the control unit 65 not only controls the alternating control of the memories 61 and 62 , it also supplies the CCD element 17 with the necessary synchronization pulses and finally also ignites the light-emitting diodes 16 which , for example, during the recording of each video image 5 ms can be switched on.

The required image processing algorithms, which are preferably written in the programming language Borland C ++ for Windows 95, run in the control unit 65 to find the pixels 53 . This allows detection and counting of the pixels 53 in the video scene 40 in real time, ie with the refresh rate of z. B. 25 Hz. The number of detected pixels 53 is written together with the time of the recording in a file and can be displayed at any time as a graphic.

Finally, Fig. 9 shows a preferred application of the invention.

A conventional television set 80 can be seen , on which a detection device 85, which is also known per se, is located. The device 85 is used to record ratings. It is provided with a keypad 86 , with which the members of a predetermined household "register" or "deregister" for a particular television program. In this way it is recorded which members of the family watch which television programs, at what times and over what periods.

This arrangement, which is known per se, is supplemented according to the invention by the camera 10 , which is located on the detection device 85 or is integrated therein. The rest of the evaluation electronics, which can also be located within the arrangement according to FIG. 9 or away therefrom, is not shown for the sake of simplicity.

If the detection device 85 is combined with a camera 10 and the value arrangement according to the present application, it is not only possible to determine in this way whether the number of people indicated in the keypad 86 is actually assembled in front of the television set, but much more additionally by capturing and evaluating a video scene 40 similar to FIG. 5, capturing how many of the people present watch the television program in question or indulge in another activity or even fall asleep.

Claims (27)

1. A method for measuring the attention of living beings within a set of living beings with the features:

• a) emitting a light ( 25 ) on the amount of living beings;
• b) capturing the light reflected by the living being in the form of an image (40);
  characterized by the other features:
• c) analyzing the image (40) in such a way that light ( 27 ) reflected from the background ( 22 ) of the living being is filtered out and recognized as the image point ( 53 ); and
• d) counting the pixels ( 53 ).

2. The method according to claim 1, characterized in that infrared light is emitted as light ( 25 ). 3. The method according to claim 2, characterized in that the reflected light is detected by an infrared filter ( 14 ). 4. The method according to one or more of claims 1 to 3, characterized in that the light ( 25 ) is emitted in a pulsed manner. 5. The method according to claim 4, characterized in that the light is pulsed with a pulse duration of a few ms becomes. 6. The method according to claim 5, characterized in that the pulse duration is about 5 ms. 7. The method according to one or more of claims 1 to 6, characterized in that the image ( 40 ) is detected as a video image. 8. The method according to claim 7, characterized in that successively several video images in front given repetition frequency. 9. The method according to claim 8, characterized in that the repetition frequency is about 25 Hz. 10. The method according to claim 7 or 8, characterized records that for recording successively in time Video images of different light sources are used the. 11. The method according to claim 10, characterized in that the different light sources at different Locations are positioned. 12. The method according to one or more of claims 8 to 11, characterized in that successive video images are read alternately in at least two different memories ( 61 , 62 ). 13. The method according to claim 12, characterized in that the image stored in the one memory ( 61 ) is analyzed, while the image intended for the at least second memory ( 62 ) is recorded. 14. The method according to one or more of claims 7 to 13, characterized in that the video image is displayed on an LCD screen ( 71 ). 15. The method according to one or more of claims 1 to 14, characterized in that the number of pixels ( 53 ) is represented alphanumerically ( 68 ). 16. The method according to claim 15 and one or more of the Claims 7 to 14, characterized in that the An number is shown in the video image. 17. The method according to one or more of claims 1 to 16, characterized in that to suppress artifacts, the pixels ( 53 ) are only taken into account if they occur in pairs. 18. The method according to claim 17, characterized in that the paired pixels ( 53 ) are only taken into account if they were spaced apart by a predetermined distance (dA). 19. The method according to claim 17 or 18, characterized in that the paired pixels ( 53 ) are only taken into account if their Ver connecting line ( 1 ) runs substantially horizontally (α) ver. 20. The method according to one or more of claims 1 to 19, characterized in that to suppress artifacts, the pixels ( 53 ) are only taken into account if they resemble a circular shape to a predetermined extent. 21. The method according to claim 20, characterized in that the pixels similar to a circular shape ( 53 ) are only taken into account if their diameter is in a predetermined diameter range. 22. The method according to claim 20 or 21, characterized in that the similar to a circular image points ( 53 ) are only taken into account if their brightness stands out by a predetermined amount from the average brightness of the surrounding background of the image ( 40 ). 23. The method according to one or more of claims 20 to 22, characterized in that the pixels similar to a circular shape ( 53 ) are only taken into account if the pixel is surrounded by a further, essentially circular region. 24. Device for measuring the attention of living beings within a set of living beings with:

• a) means for emitting a light ( 25 ) on the amount of living beings;
• b) means for detecting the light reflected by the living being in the form of an image ( 40 ), characterized by:
• c) means for analyzing the image ( 40 ) in such a way that light ( 27 ) filtered out from the background of the eye ( 22 ) of the living being and detected as a pixel ( 53 ); and
• d) means for number of pixels ( 53 ).

25. The device according to claim 24, characterized in that it is designed to be portable. 26. The apparatus of claim 24 or 25, characterized in that it is in operative connection with a on a television ( 80 ) arranged detection device ( 85 ) for determining the ratings. 27. The apparatus of claim 24 or 25, characterized records that it is attached to an advertising object.