JPH09175224A - Pupil measuring device - Google Patents

Abstract

(57) [Abstract] [Purpose] A single image processing system can be used for multiple observers, and at least one of the eyeblink, pupil diameter, line-of-sight position, and movement amount of each observer. Provided is a pupil measuring device capable of surely measuring one. An image pickup means for photographing a face portion of an observer, an image extraction portion for extracting a pupil portion of the face portion from a video signal photographed by the image pickup means, and a pupil portion extracted by the image extraction portion Further, a coloring processing unit for coloring each person to be observed, and an image calculation unit for extracting and detecting at least one of a pupil diameter, a line-of-sight position and a movement amount, and a blink of each color from the colored pupil portion. It is equipped with.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a blink of an observer photographed by a plurality of image pickup means (TV cameras, etc.),
The present invention relates to a pupil measuring device for surely measuring pupil information such as a pupil diameter, a line-of-sight position, and a movement amount.

[0002]

2. Description of the Related Art Conventionally, a face image of a person to be observed (an actor on the stage, a singer on the stage, or an observer who is watching images such as television and movies) is photographed by an image pickup means such as a television camera, When detecting pupil information such as the eyeblink, pupil diameter, line-of-sight position, and amount of movement of the observed person, the face portion or one eye of the observed person is greatly zoomed in to effectively use the resolution of the imaging means. I am shooting. In other words, one imaging means targets only one observer, and only the face portion or one eye of the target observer is enlarged and photographed.

[0003]

In the conventional system,
Since the face information or one eye of the observed person is magnified and photographed to measure the pupil information, one image processing system can determine the blink, pupil diameter, line-of-sight position, and movement amount of one person. It was the best to measure. For this reason, when measuring the pupil information of a plurality of observed persons, image processing systems corresponding to the number of observed persons are required, which causes a significant cost increase.

Further, when detecting the positions of the pupils of a plurality of observers in the related art, the pupils separated by the infrared camera (imaging means) have only high brightness, so that the pupils of a plurality of observers are detected. A technique is used in which the angle of view of the infrared camera is enlarged so that the positions do not overlap each other, or images are taken, or separate infrared cameras and image processing systems are used. However, in the former case, there is a problem that the resolution of the pupil part is lowered and the detection accuracy is deteriorated, and in the latter case, the hardware price is required as much as the number of the observed persons as the number of the observed persons who simultaneously obtain the data increases.

The present invention has been made under the circumstances as described above, and an object of the present invention is to enable a single image processing system to deal with a plurality of observers, and moreover, to each observer. Another object of the present invention is to provide a pupil measuring device capable of surely measuring at least one of the eye blink, the pupil diameter, the line-of-sight position, and the movement amount.

[0006]

SUMMARY OF THE INVENTION The present invention relates to a pupil measuring apparatus for measuring the pupils and the like of actors on the stage, singers on the stage, and observers who are viewing images such as television and movies.
The above object of the present invention is to provide an image pickup means for picking up a face portion of a person to be observed, an image extracting portion for extracting a pupil portion of the face portion from a video signal picked up by the image pickup means, and an image extracting portion for extraction. At least one of the pupil diameter, the line-of-sight position and the movement amount, and the blink of each eye is extracted and detected for each color from the colored pupil portion. It is achieved by providing an image calculation unit.

[0007]

DESCRIPTION OF THE PREFERRED EMBODIMENTS In the present invention, a face portion of an observed person is imaged by an image pickup means, a pupil portion is extracted from the face portion by an image extraction section, and the extracted pupil portion is colored by a coloring processing section. Different coloring is performed for each color, and at least one of the pupil diameter, the line-of-sight position, and the amount of movement is extracted and detected for each color from the colored pupil. That is, according to the present invention, the pupil part of each person is extracted from the face image of the person photographed by the image pickup means such as a plurality of television cameras, and the pupil part of each face image is colored with a different color and superimposed. However, by utilizing the fact that the color of each colored part is no longer covered by the eyelid at the time of blinking and that each colored part moves when the line of sight moves, one image processing system The blink, pupil diameter, and line-of-sight position can be detected individually and with high accuracy.

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an embodiment of the present invention, in which n
Infrared LEDs 11 to 1n and n infrared cameras 21
2n is used to measure the pupil information of the n observers 31 to 3n. The video signals V1 to Vn from the infrared cameras 21 to 2n are input to the image extracting units 41 to 4n, and the pupil information E1 to En of the pupil portion extracted by the image extracting units 41 to 4n is the coloring processing unit 51 to. It is input to 5n and colored in different colors. Coloring processing unit 51
The coloring signals C1 to Cn colored with 5n to 5n are input to the calculating unit 6, and the calculating unit 6 blinks each of the observed persons 31 to 3n.
The pupil diameter and the line-of-sight movement amount are detected and output as pupil information ES. The infrared LEDs 11 to 1n are paired with the infrared cameras 21 to 2n, respectively.
Irradiates the observed persons 31 to 3n with infrared rays, and the reflected light is received by the infrared cameras 21 to 2n to capture an image.

In such a configuration, the infrared camera 21
2n is the observed person 31 as shown in FIG.
The face parts of 3 to 3n are imaged, and the video signals V1 to Vn thereof are input to the image extracting units 41 to 4n. In this example, the entire face is taken up and imaged, but only one eye part may be further taken up and imaged. Since the retina of the fundus absorbs visible light and reflects infrared light, as shown in FIGS. 2A and 2B, only the pupil portion of the video signal V is imaged with extremely high brightness. The brightness of the pupil part is so large that it cannot be compared with the reflected light amount of the skin part of the surrounding face, and the image extracting parts 41 to 41
In 4n, the pupil portion is binarized and extracted by setting an appropriate threshold value as shown in FIG. The pupil information E1 to En extracted by the image extracting units 41 to 4n are input to the coloring processing units 51 to 5n, respectively, and the pupil portions are colored in different colors of n colors. The colored signals C1 to Cn colored in the pupil portion are input to the calculation unit 6 and combined,
The pupil portion is tracked for each color, the size is calculated, and the like, and the nictation, pupil diameter, and line-of-sight movement amount of the viewers 31 to 3n are output as pupil information ES. Since the colors of the observed persons 31 to 3n are different, individual calculation and detection are possible.

FIG. 3 shows an example in which the pupil images of the left and right eyes of four people are colored in different colors and combined, and 11R-11L,
12R-12L, 13R-13L, and 14R-14L are the left and right eyes of four persons, respectively, and are colored in four different colors, R indicates the right eye, and L indicates the left eye.

By the way, a blink occurs simultaneously in the left and right eyes, and only one of the eyes needs to be measured. Usually, only one eye is required for the pupil diameter and the amount of movement of the line of sight, but especially when a stereoscopic image is observed. Is useful for making binocular measurements. Figure 4 shows the principle of blink detection.
Although time elapses as in (B) → (C), the reflected light of the pupil disappears at the time of blinking, and this disappearance can be detected as a blink. Further, FIG. 5 shows the principle of detecting the pupil diameter, and (A) to (D) show each scanning line of the pupil imaging portion of one screen, that is, the luminance distribution in the sub-scanning direction. Since the pupil is generally partially covered by the upper eyelid, FIG.
Although it is rare to be detected as a true circle as in (A), since the imaged portion is a part of a circle, the width gradually increases for each scanning line and then decreases. FIG. 5 shows the distribution, generally the largest length part of it, ie FIG.
“D” in (C) is defined as the pupil diameter. FIG. 6 shows the principle of detecting the line-of-sight movement. For example, the movement of the maximum length (d) portion of FIG. 5 (C) is taken as an example. 6 (A)-
(D) shows the luminance distribution in the field direction of this maximum length portion, and the broken line shows the position of its center of gravity. Therefore, by detecting the movement of the center of gravity of the luminance distribution, a1 → a2 →
It is possible to detect the line-of-sight movement and the movement amount such as a3 → a4. These measurements are performed in parallel or repeatedly for n observers. Regarding the pupil diameter, in order to further improve the accuracy, it is also possible to reproduce the true circle from the distribution of the arcs shown in FIG. 4 and obtain the diameter by calculation.

FIG. 7 is a flowchart showing the operation of the present invention, corresponding to FIG.
The video signals V1, V2, ..., Vn of 2, ..., 2n are input to the image extracting units 41, 42 ,. In the image extraction units 41, 42, ..., 4n, first, the binarization circuit 41
, 41n are binarized, and the binarized information is cut out by a preset threshold value (see FIG. 2), and the pupil part is detected. The pupil information E1, E2, ...
, 5n are input and colored into n colors, and the respective colored signals C1, C2, ..., Cn are input to the calculation unit 6. The computing unit 6 first synthesizes the colored signals C1 to Cn (see FIG. 3).
After that, the color is extracted, the presence or absence of color is determined for each color to detect the blink, the horizontal maximum length calculation is performed to detect the pupil diameter, and the center of gravity shift calculation is performed to calculate the eye movement amount. To detect.

In the above description, each observer is colored with a different color, but it is also possible to distinguish by the density (brightness) or by the dot percentage or screen. Further, the infrared camera may be any image pickup means capable of detecting the pupil of the observed person.

[0015]

EXAMPLE FIG. 8 shows a wiring diagram of an actual device example.
Eight infrared cameras 21 to 21 for measuring a human observer
28 is installed, and a monochrome video signal which is a video signal from each of the infrared cameras 21 to 28 is input to the coloring circuit device 70. The coloring circuit device 70 has a display unit 71 for displaying the images picked up by the infrared cameras 21 to 28, and the displayed image can be switched by a selection switch 72, and the threshold value for pupil detection is a level. Setting switch 7
3 can be adjusted and set for each color. The coloring signal from the coloring circuit device 70 is input to the image processing device 80, the image processing device 80 performs operations such as composition and color extraction, and outputs pupil information. Further, FIG. 9 shows a line of sight (roughly (A) in the figure) showing a rough movement of the left and right pupils with respect to four observers, a pupil diameter of one eye ((B) in the same figure), and a blink of one eye ( (C) of the figure shows a measurement example, and for all the observers at the bottom, the line of sight is a combination of the observers 1 to 4,
The pupil diameter is averaged, and the blink is obtained by addition.

[0016]

EFFECTS OF THE INVENTION By using the present invention, the pupil portions of each observer are colored in different colors and combined so that even if the pupil portions of a plurality of observers overlap or are displayed on the same monitor, Since human eyes are displayed in different colors, they can be observed at the same time. Since the separation is easy and the pupil part can be photographed using the entire angle of view of the camera, the detection accuracy increases dramatically, and it is possible to separate each observer by color, so the hardware price is lower than the conventional one. It can be held for one person. By applying the present invention, any or all of the blinks, pupil diameters, line-of-sight positions, and movement amounts of a plurality of observers can be detected in real time.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an example of the present invention.

FIG. 2 is a diagram for explaining a principle of detecting a pupil portion according to the present invention.

FIG. 3 is a diagram showing an example of combining binarized and colored pupil images.

FIG. 4 is a diagram for explaining a blink detection principle.

FIG. 5 is a diagram for explaining the principle of pupil diameter detection.

FIG. 6 is a diagram for explaining the principle of detection of eye movement.

FIG. 7 is an operation diagram showing a flowchart of an operation example of the present invention.

FIG. 8 is a connection diagram of a specific device example of the present invention.

FIG. 9 is a diagram showing a specific information distribution according to an embodiment of the present invention.

[Explanation of symbols]

 11-1n Infrared LED 21-2n Infrared camera 31-3n Observed person 41-4n Image extraction part 51-5n Coloring processing part 6 Operation part 411-41n Binarization circuit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shigezumi Kuwashima 3-12-12 Kitasenzuku, Ota-ku, Tokyo Inside Applied Measurement Laboratory, Inc.

Claims (1)

[Claims] 1. An image pickup means for picking up a face portion of an observer, an image extracting portion for extracting a pupil portion of the face portion from a video signal picked up by the image pickup means, and an image extracting portion for extracting the pupil portion of the face portion. A coloring processing unit for coloring the pupil portion for each of the observers, and an image calculation for extracting and detecting at least one of the pupil diameter, the line-of-sight position and the movement amount, and the blink of each color from the colored pupil portion. And a pupil measuring device.