JPH06148508A - Eye gaze detector - Google Patents

Abstract

(57) [Abstract] [Purpose] Excludes the influence of an optical image of extreme brightness due to external light, etc., and accurately extracts a Purkinje image. In the Purkinje image extracting means 1, rising and falling slopes of signals of the Purkinje image candidates stored in the Purkinje image candidate storing means are detected, and a Purkinje image is selected based on the result. The Purkinje image candidate stored in the Purkinje image candidate storage means is discriminated from the positional relationship between the pupils and the Purkinje image is selected based on the result, or the Purkinje image candidate storage means stores it in the Purkinje image candidate storage means. The number of pixels in the two-dimensional direction of the signal of each Purkinje image candidate is detected, Purkinje image is selected based on this result, or the optical image center of each Purkinje image candidate stored in the Purkinje image candidate storage means Purkinje image selecting means is provided for selecting a Purkinje image based on the intervals of the light projecting means which are substantially equal to each other.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement of a visual axis detecting device used in a video camera or the like.

[0002]

2. Description of the Related Art A conventional method for calculating the position of a Purkinje image (corneal reflection image) is disclosed in JP-A-61-272552. FIG. 9 shows this Japanese Patent Laid-Open No. 61-17255.
2 shows a conventional example described in No. 2.

When a light emitting element (not shown) is turned on by a drive circuit with an alternating current, the position of an image of the Purkinje effect is measured by a quadrant element or a semiconductor position detecting element (PSD) 31 and a circuit 40, and an A / D not shown It is digitized by the converter and input to the control circuit.

[0004]

However, the above-mentioned conventional device has the following problems.

That is, conventionally, the center of gravity of a Purkinje image is obtained by calculating the center of gravity of an optical image formed on a two-dimensional semiconductor element (two-dimensional PSD) by a digital circuit. Therefore, when there is a signal similar to the Purkinje image, such as the observer's tears or the ghost of the glasses used by the observer, or when there is an optical image with extremely high brightness due to external light or the like. Due to the adverse effect, it was difficult to accurately determine the position of the Purkinje image.

(Object of the Invention) An object of the present invention is to provide a line-of-sight detection apparatus capable of accurately extracting a Purkinje image by eliminating the influence of an optical image of extreme brightness due to external light or the like.

[0007]

SUMMARY OF THE INVENTION According to the present invention, a Purkinje image extracting unit detects a light image having a brightness of a certain value or more from a signal from a light receiving unit and stores the light image as a Purkinje image candidate. An image candidate storage means is provided, and
In the Purkinje image extracting means, a Purkinje image candidate storage means for detecting a light image having a brightness of a certain value or more from the signal from the two-dimensional light receiving means and storing the light image as a Purkinje image candidate is provided, and the light quantity is equal to or more than a certain value. An optical image having brightness is selected and stored as a Purkinje image candidate.

According to the present invention, the Purkinje image extracting means detects the rising and falling slopes of the respective signals of the Purkinje image candidates stored in the Purkinje image candidate storing means, and based on this result, the Purkinje image is detected. The Purkinje image selecting means for selecting is selected, and the Purkinje image is selected depending on whether or not the rising and falling slopes of the respective signals of the Purkinje image candidates are steep.

Further, according to the present invention, in the Purkinje image extracting means, the positional relationship between the Purkinje image candidates stored in the Purkinje image candidate storing means and the pupil is discriminated, and the Purkinje image is selected based on the result. Image selection means is provided to determine the positional relationship between the Purkinje image candidate and the pupil (center), and a closer one is selected as the Purkinje image.

According to the present invention, the number of pixels in the two-dimensional direction of each signal of the Purkinje image candidates stored in the Purkinje image candidate storage unit is detected in the Purkinje image extraction unit, and the Purkinje image is detected based on this result. Purkinje image selection means for selecting the, to detect the number of pixels in the two-dimensional direction of each signal of the Purkinje image candidates, those larger than the predetermined number of pixels are excluded due to ghosts of glasses, etc. I try to select the Purkinje statue based on the one.

Further, according to the present invention, the Purkinje image extracting means has a distance between the light image centers of the Purkinje image candidates stored in the Purkinje image candidate storing means, which are substantially equal to the arrangement direction of the light projecting means. A Purkinje image selecting means for selecting a Purkinje image is provided, and the Purkinje image is selected based on the light image center of each Purkinje image candidate, which is approximately equal to the arrangement direction of the light projecting means within a predetermined interval. There is.

Further, according to the present invention, based on the number of Purkinje image extracting means substantially equal to the light image center of each Purkinje image candidate stored in the Purkinje image candidate storing means in the arrangement direction of the light projecting means. A Purkinje image selecting means for selecting a Purkinje image is provided, and the Purkinje image is selected based on the number of the light image centers of the respective Purkinje image candidates that are substantially equal to the arrangement direction of the light projecting means equal to the number of the light projecting means. I am trying to do it.

[0013]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on the illustrated embodiments.

FIG. 1 is a block diagram showing the schematic arrangement of a camera equipped with a line-of-sight detection apparatus according to the first embodiment of the present invention.

In FIG. 1, 1 is an MPU (micro processing unit), 2 is a memory, and 3 is a CCD described later.
And a driver circuit for driving an IRED (infrared light emitting diode), 4 is a CC which is a two-dimensional image sensor
D and 5 are IRED groups composed of a plurality of IREDs that project infrared light that is insensitive to the eyes of an observer, and are arranged in pairs in the horizontal and vertical directions at a predetermined interval. Reference numeral 6 is a lens drive unit for performing AF (autofocus), 7 is an aperture drive unit, and 8 is a shutter unit.

FIG. 2 is a flow chart showing the main operation of the camera provided with the visual axis detection device having the above-mentioned structure.

When a line-of-sight detection request is made by turning on the switch SW1 interlocking with the first stroke operation of the release button (not shown) of the camera, the MPU 1 makes a step 102
The routine for detecting the line of sight from is entered.

In step 102, initialization processing of variables used for calculation is performed, and in the next step 103, accumulation time is set (step 103). This is done in consideration of the presence / absence of glasses and the intensity of external light, and at the same time, IR
The IRED pair is also selected to be turned on from within the ED group 5. Then, the process proceeds to the storage control operation after step 104.

First, in step 104, the CCD
The driver circuit 3 gives an instruction for performing the charge clearing operation of No. 4
Do against. Upon receiving this instruction, the driver circuit 3 performs a clear operation to erase the charges remaining in the memory zone of the CCD 4, the charge transfer line, and the like. Then step 1
05, the I selected in step 103 above
At the same time as transmitting the IRED selection signal to the driver circuit 3 to turn on the RED pair, the accumulation signal is set to the high level to start accumulation by the CCD 4, and when the above set accumulation time has elapsed, the accumulation signal is set to the low level and accumulation is performed. To finish. The IRED pair selected in synchronization with this accumulation is turned on.

Next, the MPU 1 proceeds to the operations after step 106 for performing the processing of extracting the optical image block (Purkinje image candidate) and the pupil edge candidate.

First, in step 106, the image signal for one line of the CCD 4 is sequentially read through the driver circuit 3, A / D converted, and the value is stored in the memory 2. Then, in the next step 107, the optical image block (Purkinje image candidate) and the pupil edge candidate are extracted using this data. This processing is performed for the number of lines of the CCD 4.

The "extraction of the optical image block (Purkinje image candidate)" performed in step 107 will be described with reference to the flowchart of FIG.

The MPU 1 sets 1 in step 201.
Read line data. And next step 2
In 02 and 203, first, an optical image block having a brightness equal to or higher than a certain fixed value is extracted. Then, it is determined whether or not the rising and falling conditions of the optical image block are satisfied. That is, it is determined whether or not the brightness has a certain value or more and the slopes at both ends are steep to some extent. Note that only the optical image block width, for example, the number of pixels having a luminance of a certain value or more and a certain value or more may be regarded as valid.

Now, let the brightness of the i-th pixel in a line be d
When [i] is set, the following condition (1) d [i] ≧ Const1 AND d [i] −d [i−a] ≧ Const2 (2) d [j] ≧ Const1 AND d [j] −d [J−a] ≧ Const2 However, if Const1, Const2, and a are constants, and j is a variable of i + 1 or more, this is recognized as an optical image block (Purkinje image candidate), and in step 204, i is a rising point. The coordinate, j is stored as the falling point coordinate.

When this extraction operation is sequentially performed from the left side (smaller pixel number), the rising point coordinate i adopts the point which first satisfies the above condition (1), but the falling point coordinate j. The last point that satisfies the condition (2) above is adopted. In addition, since it is naturally considered that a plurality of optical image blocks exist in one line, this extraction calculation process is performed for all the pixels considered to be effective in that line.

When it is determined in step 205 that the optical image block has been extracted before that,
In the next step 206, the rising point that was just stored,
It is determined whether the falling point is the same as the optical image block. That is, if the condition of [the right boundary of the extracted light image block ≦ the average of the right and left ends of the current light image block [(i + j) / 2] ≦ the left boundary of the extracted light image block] is satisfied, the same light image is obtained. It is regarded as a block, and the process proceeds to step 207. At this step,
The values of the rising edge of the optical image block (the right boundary of the optical image block), the falling edge of the optical image block (the left boundary of the optical image block), and the lower boundary of the optical image block (the vertical optical image block width) Update. The coordinates of the upper, lower, left, and right boundaries of the n-th extracted optical image block are Bny1, Bny2, and Bn, respectively.
When x1 and Bnx2 are set, Bnx1> i, Bnx1 = i Bnx2 <j, Bnx2 = j Bny2 = Bny2 + 1, and the boundary values are updated.

If the conditions are not satisfied,
When it is judged that a new optical image block has been generated, step 208
In this step, the coordinates of the upper, lower, left and right boundaries of the optical image block are stored as follows.

Bnx1 = i Bnx2 = j Bny1 = (the number of read lines) Bny2 = (the number of read lines) Further, when the optical image block is branched, or two optical image blocks merge in the middle. As a countermeasure against this, it may be possible to duplicate the optical image block and perform absorption processing.

When the optical image block is branched, the optical image block is duplicated.

For example, in the case of branching as shown in FIG. 4A (see optical image block), two optical image blocks shown in FIG. Duplicate the overlapping part of. In the line L2, two light image blocks are extracted, and both of these two light image blocks are the light image blocks that have been extracted by that time (the light image blocks of the overlapping portion of, in FIG. 4B). Are considered the same. Therefore, in order to generate the two optical image blocks of FIG. 4B, the processing is continued by adding the overlapping portion to both.

That is, the processing is performed as if the optical image block existed from the beginning, and in line L2, the rising point of the optical image block (the right boundary of the optical image block) and the falling point of the optical image block ( The values of the left boundary of the light image block) and the lower boundary of the light image block (light image block width in the vertical direction) are updated as follows. The coordinates of the upper, lower, left, and right boundaries of the overlapping portion are Bny1, Bny2, and
When Bnx1 and Bnx2 are set, Bnx1> i, Bnx1 = i Bnx2 <j, Bnx2 = j Bny2 = Bny2 + 1, and the boundary values are updated. The values of the respective boundaries are updated in the same manner also on and after the line L2.

As a result of the above, in the example of FIG.
The left and lower boundary values are updated in the optical image block, and the right and lower boundary values are updated in the optical image block.

When two optical image blocks merge,
Absorbs the light image block.

For example, in the case of merging as shown in FIG. 5, the optical image block extracted in the line L2 is the line L.
The two optical image blocks extracted up to 1 are integrated into one optical image block by absorbing them. That is, in the line L2, the rising point of the optical image block (right side boundary of the optical image block), the falling point of the optical image block (left side boundary of the optical image block), and the lower side boundary (vertical direction) of the optical image block. The value of the optical image block width of is updated as follows.

The coordinates of the upper, lower, left, and right boundaries of the optical image block are Bay1, Bay2, Bax1, Bax2, and the coordinates of the upper, lower, left, and right boundaries of the optical image block are Bby1, Bb, respectively.
When y2, Bbx1 and Bbx2 are set, the coordinates Bny1, Bny2 and Bnx of the upper, lower, left and right boundaries of the new optical image block are set.
1, Bnx2 is: if Bax1> i, then Bnx1 = i if Bax1 ≦ i, then if Bnx1 = Bax1 Bbx2 <j, then if Bnx2 = j Bbx2 ≧ j, then if Bnx2 = Bbx2 Byy = By2 <Bby1, if Bnx2 = By2b2by2 By1 <Bby1; = Bby2 + 1 and the value of each boundary is updated. After line L2,
The absorbed optical image block is subjected to the same process as the normal optical image block extraction process to update the boundary value of the optical image block.

Either one or both of the duplication and absorption treatments may be performed.

When it is determined in step 209 that the operation for one line is completed, the process proceeds to step 210, and pupil edge extraction processing is performed.

Returning to FIG. 2 again, when it is determined in step 108 that this process has been completed for all lines, the process proceeds to step 109 and the "Purkinje image selection" process is started.

In the first processing here, the block widths of the respective optical image blocks in the horizontal and vertical directions are examined, and only those having a width of a constant Const3 or less are adopted. Objects having a larger block width than this are optical images other than the Purkinje image, such as ghosts of eyeglasses and reflections from eyelids illuminated by external light, and are therefore excluded.

In the second process, the vertical coordinate of the optical image block adopted in the first process is examined. If a certain light image block and a certain light image block overlap or are in contact with each other in the vertical direction, it is determined that the two light image blocks have substantially the same vertical coordinates. That is, the upper, lower, left, and right boundary positions of the n-th optical image block are respectively Bny1, Bn.
If y2, Bnx1, and Bnx2, the upper, lower, left, and right boundary positions of the m-th optical image block are respectively Bmy1, Bmy2, and Bmy2.
When Bmx1 and Bmx2 are set, the vertical coordinates of the two optical image blocks are determined to be substantially equal when the conditional expressions of Bny1 ≦ Bmy2 and Bny2 ≧ Bmy1 are satisfied. Then, the light image blocks without the substantially equal vertical coordinates are eliminated. After that, the horizontal block intervals Δmn of the optical image blocks having substantially the same values are examined, and only those having a value within a predetermined range (Δ1 ≦ Δmn ≦ Δ2) are adopted (where Δ1 and Δ2 are constants).

For example, when the three vertical coordinates of the optical image blocks B1, B2, B3 are substantially equal, the interval Δ12 between B1 and B2, the interval Δ23 between B2 and B3, the interval Δ between B3 and B1.
31 is determined, and it is checked whether it is within a predetermined value. As a result, if Δ12 and Δ23 are within a predetermined value, the combination of [B1, B2] [B2, B3] is adopted as the Purkinje image candidate pair. To do. At this point, if there is one Purkinje image candidate pair (if the number of optical image blocks is two), this is adopted as the Purkinje image. If the Purkinje image cannot be determined at this point, the following processing is performed.

In the third processing, the center coordinates of the Purkinje image pair are compared with the coordinates of the center of the pupil, which are separately obtained, and the one closest to the coordinates of the center of the pupil, that is, the center coordinates of the Purkinje image pair and the pupil center. The image with the smallest distance of coordinates is selected as the Purkinje image. Note that, for convenience of description, the three steps of Purkinje image selection are collectively described, but in the actual processing, the following Step 11 is performed.
This third processing is performed after the pupil circle calculation described in 0.

The coordinates of the Purkinje image pairs P1 and P2 and the Purkinje image Pc determined as described above are P1x = (Bnx1 + Bnx2) / 2 P1y = (Bny1 + Bny2) / 2 P2x = (Bmx1 + Bmx2) / 2 P2y = (Bmy1 + Bmy2 + Bmy1) / 2 Pcx = (P1x + P2x) / 2 Pcy = (P1y + P2y) / 2 However, P1x, P1y, P2x, P
2y, Pcx, and Pcy are horizontal and vertical coordinates of the Purkinje image pairs P1 and P2 and the Purkinje image Pc. It is also assumed that the nth and mth optical image blocks are selected as Purkinje images.

Next, in step 110, the pupil circle is calculated as follows.

In the above-mentioned pupil edge extraction processing, the MPU
Reference numeral 1 stores in the memory 2 the coordinates of the extracted pupil edge, the minimum luminance MIN (L) of the line L, and the minimum luminance MIN0 of the entire image.

Therefore, first, the MPU 1 compares the minimum brightness (MIN (L)) of the area where the pupil edge is extracted with the minimum brightness (MIN0 + constant) of the entire image, and the condition of MIN (L) ≤MIN0 + constant is satisfied. If not, then the pupil edges present in the area are rejected as unsuitable.

Then, the pupil edges around the Purkinje images (candidates) obtained in the first to the second step 109 of Purkinje image selection are eliminated. This is in an area centered on the horizontal and vertical coordinates (Pnx, Pny) of the Purkinje image candidate (optical image block), for example, (Pnx-constant,
Pny-constant), (Pnx-constant, Pny + constant),
(Pnx + constant, Pny−constant), (Pnx + constant, P
It is carried out by excluding those in the quadrangle surrounded by four points (ny + constant). Since there are two Purkinje images, this processing is performed for each Purkinje image.

Further, the average value m and the standard deviation σ of the coordinates of the pupil edge candidates selected so far are obtained, and [average value m-
a * standard deviation σ-average value m + a * standard deviation σ (however, a
Is a constant)].

This calculation may be performed only in the horizontal direction or in both the horizontal and vertical directions. Also,
You may perform this process twice. That is, [average value m
The average value m ′ and the standard deviation σ ′ of those falling within the range of −a * standard deviation σ to average value m + a * standard deviation σ] are determined again, and the range [average value m′-a] defined by these two quantities is determined.
* Standard deviation σ'-average value m '+ a * standard deviation σ'] is used only.

Next, the pupil center and the pupil radius are obtained using the pupil edges selected as described above. As this method, the least square method may be used.

After that, the process proceeds to step 111, and the rotation angle of the eyeball and the viewpoint position on the focusing plate of the camera are calculated using the Purkinje image and the position of the center of the pupil. And
In step 112, the AF point and the like are determined based on this viewpoint position, and the camera is controlled.

(Second Embodiment) FIG. 6 is a flow chart showing the operation of the main part of a camera equipped with a line-of-sight detection apparatus according to the second embodiment of the present invention. The circuit configuration is the same as that of the first embodiment shown in FIG. 1, and the main routine of the camera is the first embodiment shown in FIG. The same circuit number and step number shall be given.

Similar to the first embodiment, the switch SW which is interlocked with the first stroke operation of the release button (not shown) of the camera.
When a line-of-sight detection request is made by turning on 1 etc., MP
U1 enters the visual axis detection routine from step 102.

In step 102, initialization of variables used in the calculation is performed, and in the next step 103, accumulation time is set (step 103). This is done in consideration of the presence / absence of glasses and the intensity of external light, and at the same time, IR
The IRED pair is also selected to be turned on from within the ED group 5. Then, the process proceeds to the storage control operation after step 104.

First, in step 104, the CCD
The driver circuit 3 gives an instruction for performing the charge clearing operation of No. 4
Do against. Upon receiving this instruction, the driver circuit 3 performs a clear operation to erase the charges remaining in the memory zone of the CCD 4, the charge transfer line, and the like. Then step 1
05, the I selected in step 103 above
At the same time as transmitting the IRED selection signal to the driver circuit 3 to turn on the RED pair, the accumulation signal is set to the high level to start accumulation by the CCD 4, and when the above set accumulation time has elapsed, the accumulation signal is set to the low level and accumulation is performed. To finish. The IRED pair selected in synchronization with this accumulation is turned on.

Next, the MPU 1 proceeds to the operation after step 106 for performing the processing of extracting the optical image block (Purkinje image candidate) and the pupil edge candidate.

First, in step 106, the image signal for one line of the CCD 4 is sequentially read through the driver circuit 3, A / D converted, and the value is stored in the memory 2. Then, in the next step 107, the optical image block (Purkinje image candidate) and the pupil edge candidate are extracted using this data. This processing is performed for the number of lines of the CCD 4.

The "light image block extraction" performed in step 107 will be described below with reference to the flow chart of FIG. 6 and FIGS.

First, the MPU 1 reads the data for one line in step 301, and the next step 30
At 2 and 303, it is determined whether or not the rising and falling conditions of the optical image block are satisfied. That is, it is determined whether or not the brightness has a certain value or more and the slopes at both ends are steep to some extent.

Now, let the brightness of the i-th pixel on a line be d
When [i] is set, the following condition (1) d [i] ≧ Const1 AND d [i] −d [i−a] ≧ Const2 (2) d [j] ≧ Const1 AND d [j] −d [J−a] ≧ Const2 However, if Const1, Const2, and a are constants and j is a variable of i + 1 or more, this is recognized as an optical image block (Purkinje image candidate), and in step 304, i is a rising point. The coordinate, j is stored as the falling point coordinate.

When this extraction operation is sequentially performed from the left side (smaller pixel number), the rising point coordinate i adopts the point which first satisfies the above condition (1), but the falling point coordinate j. The last point that satisfies the condition (2) above is adopted. In addition, as a matter of course, it is sufficiently possible that a plurality of optical image blocks exist in one line, and therefore this extraction calculation process is performed for all the pixels considered to be effective in that line.

When it is determined in step 305 that the optical image block has been extracted before that, in the next step 306, it is determined whether or not the rising point and the falling point just stored are the same as the optical image block. Whether or not it is determined.
That is, if the condition of [the right boundary of the extracted light image block ≦ the average of the right and left ends of the current light image block [(i + j) / 2] ≦ the left boundary of the extracted light image block] is satisfied, the same light image is obtained. It is regarded as a block, and the process proceeds to step 307. At this step,
The values of the rising edge of the optical image block (the right boundary of the optical image block), the falling edge of the optical image block (the left boundary of the optical image block), and the lower boundary of the optical image block (the vertical optical image block width) Update. The coordinates of the upper, lower, left and right boundaries of the n-th extracted block are Bny1, Bny2, Bnx1, and
When Bnx2 is set, Bnx1> i, Bnx1 = i Bnx2 <j, Bnx2 = j Bny2 = Bny2 + 1, and the value of each boundary is updated. Furthermore, for image signals within this range, the sum SI [n] of luminance (signal intensity) and the sum SIx [n], SIy of the products of luminance and their horizontal and vertical coordinates.
[N] is obtained according to the following equation, and its value is updated and stored.

SI [n] = SI [n] + Σ (d [x]) SIx [n] = SI [n] + Σ (d [x] * x] SIy [n] = SI [n] + Σ (d [ x] * Number of lines being processed L) If the condition is not satisfied, it is determined that a new optical image block is generated, and the process proceeds to step 308. In this step, the coordinates of the upper, lower, left, and right boundaries are set as follows. Memorize like.

Bnx1 = i Bnx2 = j Bny1 = (the number of read lines) Bny2 = (the number of read lines) Further, the brightness (signal intensity) of the image signal within this range
SI [n], and the sum SIx [n], SIy [n] of the products of the luminance and its horizontal and vertical coordinates are obtained according to the following equation, and the values are updated and stored.

SI [n] = Σ (d [x]) SIx [n] = Σ (d [x] * x) SIy [n] = Σ (d [x] * the number of lines L being processed) The range of x is such that the rising point i ≦ x ≦ the falling point j.

At this time, information on the upper and lower lines may be added as data.

In this case, when it is first determined that a new optical image block is generated, the coordinates of the upper, lower, left, and right boundaries are stored as described above, and the sum SI [] of the luminances (signal intensities) of the image signals within this range is stored. [n] and the sum SIx [n] and SIy [n] of the products of the luminance and their horizontal and vertical coordinates are calculated according to the following equation and stored.

SI [n] = Σ (d [x]) + Σ (e [x]) SIx [n] = Σ (d [x] * x) + Σ (e [x] * x] SIy [n] = Σ (d [x] * the number of lines being processed L) + Σ (e [x] * the number of lines being processed L-1) where e [x] is the line L- immediately before the line being processed
It is the luminance of the i-th pixel of 1. Further, the range of x in this case is a range (i ≦ x ≦ j) between the rising point i and the falling point j of the line L being processed.

Further, in the case where the optical image block (Purkinje image candidate) is detected in the previous line, even if the optical image block cannot be detected, it is within the range of the detected optical image block. Regarding the image signal, the sum SI [n] of luminance (signal intensity) and the sum SIx [n], SIy [n] of products of luminance and their horizontal and vertical coordinates are calculated according to the following equation and stored.

SI [n] = SI [n] + Σ (d [x]) SIx [n] = SI [n] + Σ (d [x] * x] SIy [n] = SI [n] + Σ (d [ x] * number of lines being processed L) When there are a plurality of optical image blocks being detected, the sum SI [n] of the luminance and the sum SIx [n of the products of the luminance and their horizontal and vertical coordinates are obtained for each of them. ], SIy [n] values are updated and stored.

Further, as a countermeasure against the case where the optical image block is branched or the two optical image blocks merge on the way, it may be considered to perform the processing of copying and absorbing the optical image block.

When the optical image block is branched, the optical image block is duplicated.

For example, in the case of branching as shown in FIG. 4A (see optical image block), two optical image blocks shown in FIG. Duplicate the overlapping part of. In the line L2, two light image blocks are extracted, and both of these two light image blocks are the light image blocks that have been extracted by that time (the light image blocks of the overlapping portion of, in FIG. 4B). Are considered the same. Therefore, in order to generate the two optical image blocks of FIG. 4B, the processing is continued by adding the overlapping portion to both.

That is, the processing is performed as if the optical image block had existed from the beginning, and in line L2, the rising point of the optical image block (the right boundary of the optical image block) and the falling point of the optical image block ( The values of the left boundary of the light image block) and the lower boundary of the light image block (light image block width in the vertical direction) are updated as follows. The coordinates of the upper, lower, left, and right boundaries of the overlapping portion are Bny1, Bny2, and
When Bnx1 and Bnx2 are set, Bnx1> i, Bnx1 = i Bnx2 <j, Bnx2 = j Bny2 = Bny2 + 1, and the boundary values are updated.

Further, with respect to image signals within this range, the sum SI [n] of luminance (signal intensity) and the luminance and its horizontal coordinate,
Sums SIx [n] and SIy [n] of products of vertical coordinates are obtained according to the following equation, and the values are updated and stored.

SI [n] = SI [A] + Σ (d [x]) SIx [n] = SI [A] + Σ (d [x] * x] SIy [n] = SI [A] + Σ (d [ x] * number of lines being processed L) The range of x here is a rising point i ≦ x ≦ falling point j. Further, SI [A], SIx [A], SIy [A] are overlapping portions. And the sum of the products of the horizontal and vertical coordinates.

Similarly, after line L2, the sum SI [n] of the value and brightness of each boundary and the sum SIx [n], SIy [n] of the product of brightness and its horizontal and vertical coordinates are updated. Go.

When two optical image blocks merge,
Absorbs the light image block.

For example, in the case of merging as shown in FIG. 5, the optical image block extracted in the line L2 is the line L.
The two optical image blocks extracted up to 1 are integrated into one optical image block by absorbing them. That is, in the line L2, the rising point of the optical image block (right side boundary of the optical image block), the falling point of the optical image block (left side boundary of the optical image block), and the lower side boundary (vertical direction) of the optical image block. The value of the optical image block width of is updated as follows.

The coordinates of the upper, lower, left, and right boundaries of the optical image block are Bay1, Bay2, Bax1, Bax2, and the coordinates of the upper, lower, left, and right boundaries of the optical image block are Bby1, Bb, respectively.
When y2, Bbx1 and Bbx2 are set, the coordinates Bny1, Bny2 and Bnx of the upper, lower, left and right boundaries of the new optical image block are set.
1, Bnx2 is: if Bax1> i, then Bnx1 = i if Bax1 ≦ i, then if Bnx1 = Bax1 Bbx2 <j, then if Bnx2 = j Bbx2 ≧ j, then if Bnx2 = Bbx2 Byy = By2 <Bby1, if Bnx2 = By2b2by2 By1 <Bby1; = Bby2 + 1 and the value of each boundary is updated.

Further, with respect to the image signal within this range, the sum SI [n] of the luminance (signal intensity) and the luminance and its horizontal coordinate,
Sums SIx [n] and SIy [n] of products of vertical coordinates are obtained according to the following equation, and the values are updated and stored.

SI [n] = SI [A] + SI [B] + Σ (d [x]) SIx [n] = SI [A] + SI [B] + Σ (d [x] * x] SIy [n] = SI [A] + SI [B] + Σ (d [x] * the number of lines L being processed) The range of x here is the rising point i ≦ x ≦ falling point j. Further, SI [A], SIx [A], SIy [A],
SI [B], SIx [B], SIy [B] are the sum of the brightness of the optical image blocks A and B and the sum of the products of the brightness and their horizontal and vertical coordinates.

After the line L2, the same process as the normal optical image block extraction process is performed on the absorbed optical image block, and the sum SI of the boundary value of the optical image block and the brightness SI is obtained.
[N] and the sum SI of the luminance and its horizontal and vertical coordinates SI
x [n] and SIy [n] are updated.

Either one of the duplication and absorption treatments may be performed, or both may be performed.

When it is determined in step 309 that the operation for one line is completed, the process proceeds to step 310, and pupil edge extraction processing is performed.

Returning to FIG. 2 again, when it is determined in step 108 that this process has been completed for all lines, the process proceeds to step 109 and the Purkinje image selection process is started.

In the first processing here, the horizontal and vertical block widths of the respective optical image blocks are examined, and only the widths of the constants Const3 or less are adopted. Objects having a larger block width than this are optical images other than the Purkinje image, such as ghosts of eyeglasses and reflections from eyelids illuminated by external light, and are therefore excluded.

In the second processing, the vertical coordinate of the optical image block adopted in the first processing is examined. If a certain light image block and a certain light image block overlap or are in contact with each other in the vertical direction, it is determined that the two light image blocks have substantially the same vertical coordinates. That is, the upper, lower, left, and right boundary positions of the n-th optical image block are respectively Bny1, Bn.
If y2, Bnx1, and Bnx2, the upper, lower, left, and right boundary positions of the m-th optical image block are respectively Bmy1, Bmy2, and Bmy2.
When Bmx1 and Bmx2 are set, when the conditional expressions of Bny1 ≦ Bmy2 and Bny2 ≧ Bmy1 are satisfied, it is determined that the vertical coordinates of the two optical image blocks are substantially equal. Then, the light image blocks without the substantially equal vertical coordinates are eliminated. After that, for optical image blocks that are substantially equal, the horizontal block interval Δm
n is checked and the value is within a predetermined range (Δ1 ≦ Δmn ≦ Δ2)
Only the ones used here (where Δ1 and Δ2 are constants). At this point, if there is one Purkinje image candidate pair (if the number of optical image blocks is two), this is adopted as the Purkinje image. If the Purkinje image cannot be determined at this point, the following processing is performed.

In the third processing, the center coordinates of the Purkinje image pair are compared with the coordinates of the center of the pupil, which are obtained separately, and the one closest to the coordinates of the center of the pupil, that is, the center coordinates of the pair of Purkinje images and the pupil center. The image with the smallest distance of coordinates is selected as the Purkinje image. Note that, for convenience of description, the three steps of Purkinje image selection are collectively described, but in the actual processing, the following Step 11 is performed.
This third processing is performed after the pupil circle calculation described in 0.

The coordinates of the Purkinje image pairs P1 and P2 and the Purkinje image Pc determined as described above are: P1x = (Bnx1 + Bnx2) / 2 P1y = (Bny1 + Bny2) / 2 P2x = (Bmx1 + Bmx2) / 2 P2y = (Bmy1 + Bmy1 + Bmy1 + Bmy1 + Bmy1 + Bmy1 + Bmy1 + Bmy1 / 2 Pcx = (P1x + P2x) / 2 Pcy = (P1y + P2y) / 2 However, P1x, P1y, P2x, P
2y, Pcx, and Pcy are horizontal and vertical coordinates of the Purkinje image pairs P1 and P2 and the Purkinje image Pc. It is also assumed that the nth and mth optical image blocks are selected as Purkinje images.

Next, the routine proceeds to step 110, where the pupil circle is calculated as follows.

In the above-mentioned pupil edge extraction processing, the MPU
Reference numeral 1 stores in the memory 2 the coordinates of the extracted pupil edge, the minimum luminance MIN (L) of the line L, and the minimum luminance MIN0 of the entire image.

Therefore, first, the MPU 1 compares the minimum brightness (MIN (L)) of the area where the pupil edge is extracted with the minimum brightness (MIN0 + constant) of the entire image, and MIN (L).
If the condition of ≦ MIN0 + constant is not satisfied, the pupil edge existing in the area is excluded as an inappropriate one.

Then, the pupil edges around the Purkinje images (candidates) obtained in the first to the second step 109 of Purkinje image selection are eliminated. This is in an area centered on the horizontal and vertical coordinates (Pnx, Pny) of the Purkinje image candidate (optical image block), for example, (Pnx-constant,
Pny-constant), (Pnx-constant, Pny + constant),
(Pnx + constant, Pny−constant), (Pnx + constant, P
It is carried out by excluding those in the quadrangle surrounded by four points (ny + constant). Since there are two Purkinje images, this processing is performed for each Purkinje image.

Further, the average value m and the standard deviation σ of the coordinates of the pupil edge candidates selected so far are obtained, and the [average value m-
a * standard deviation σ-average value m + a * standard deviation σ (however, a
Is a constant)].

This calculation may be performed only in the horizontal direction or may be performed in both the horizontal and vertical directions. Also,
You may perform this process twice. That is, [average value m
The average value m ′ and the standard deviation σ ′ of those falling within the range of −a * standard deviation σ to average value m + a * standard deviation σ] are determined again, and the range [average value m′-a] defined by these two quantities is determined.
* Standard deviation σ'-average value m '+ a * standard deviation σ'] is used only.

Next, the pupil center and the pupil radius are obtained by using the pupil edges selected as described above. As this method, the least square method may be used.

After that, the routine proceeds to step 111, where the rotation angle of the eyeball and the viewpoint position on the focus plate of the camera are calculated using the Purkinje image and the position of the pupil center. And
In step 112, the AF point and the like are determined based on this viewpoint position, and the camera is controlled.

According to the above-mentioned first and second embodiments, a light image block having a brightness of a certain value or more and having a rising or falling inclination thereof is a certain value or more is extracted, and its light image is extracted. Compared with the previously extracted optical image block, if something that can be judged to be the same exists in the previously extracted ones, the upper, lower, left, and right boundary positions of that optical image block are updated. It is registered as a new optical image block, and the boundary positions of up, down, left, and right (two-dimensional direction) are stored.
In this way, a plurality of optical image blocks (Purkinje image candidates) are obtained, and from the extracted optical image blocks (Purkinje image candidates), the block width in the horizontal and vertical directions (two-dimensional direction) and the optical image block center The Purkinje image is selected in consideration of the number of those having substantially the same vertical direction coordinates, the distance from other optical image blocks, and the positional relationship with the pupil. By doing this, the image similar to the Purkinje image such as tears, ghosts in glasses, etc. and the Purkinje image can be properly distinguished, and the influence of the extremely bright light image due to external light can be eliminated, and the Purkinje image can be accurately obtained. It is possible to find the position of the image.

Optical image block (Purkinje image candidate)
Since the center of gravity is calculated from the sum of the luminance and the sum of the products of the luminance and the horizontal and vertical coordinates, the center position of the Purkinje image can be calculated with a resolution of a pixel unit or more.

(Third Embodiment) FIGS. 7 and 8 are flowcharts showing the operation of the main part of the camera equipped with the visual axis detection device according to the third embodiment of the present invention, which is different from the other embodiments. The point is that a line sensor is used as the light receiving means and a single IERD is turned on for processing. Therefore, the circuit configuration is different from that of the first embodiment shown in FIG. 1 in that the CCD is a line sensor and the IRED group has one IRD.
The only difference is the ED, and the others are the same. Further, since the main routine of the camera is the first embodiment shown in FIG. 2, the same step numbers will be given in the description of the parts related to this operation.

Similar to the first embodiment, the switch SW interlocked with the first stroke operation of the release button (not shown) of the camera.
When a line-of-sight detection request is made by turning on 1 etc., MP
U1 enters the visual axis detection routine from step 102.

In step 102, the initialization of the variables used in the calculation is performed, and in the next step 103, the accumulation time is set. Then, the process proceeds to the storage control operation after step 104.

First, in step 104, the driver circuit 3 is instructed to perform the charge clearing operation of the line sensor. Upon receiving this instruction, the driver circuit 3 performs a clear operation to erase the charges remaining in the memory zone of the line sensor, the charge transfer line, and the like. Then, in step 105, I
At the same time as transmitting the RED drive signal to the driver circuit 3,
The accumulation signal is set to the high level to start accumulation by the line sensor, and when the accumulation time set above has elapsed, the accumulation signal is set to the low level and the accumulation is completed.

Next, the MPU 1 proceeds to the operations after step 106 for performing the process of extracting the optical image block (Purkinje image candidate) and the pupil edge candidate.

First, in step 106, the image signals on the line sensor are sequentially read through the driver circuit 3, A / D converted, and the values are stored in the memory 2. Then, in the next step 107, the optical image block (Purkinje image candidate) and the pupil edge candidate are extracted using this data.

Here, the "extraction of optical image block (Purkinje image candidate)" performed in step 107 will be described.

The MPU 1 reads data for one line. Then, an optical image block having a brightness equal to or higher than a certain fixed value is extracted. Next, it is determined whether or not the rising and falling conditions of the optical image block are satisfied. That is, it is determined whether or not the brightness has a certain value or more and the slopes at both ends are steep to some extent. Note that only the optical image block width, for example, the number of pixels having a luminance of a certain value or more and a certain value or more may be regarded as valid.

Now, let the brightness of the i-th pixel on a line be d
When [i] is set, the following condition (1) d [i] ≧ Const1 AND d [i] −d [i−a] ≧ Const2 (2) d [j] ≧ Const1 AND d [j] −d [J−a] ≧ Const2 However, if Const1, Const2, and a are constants, and j is a variable of i + 1 or more, this is recognized as an optical image block (Purkinje image candidate), and in step 204, i is a rising point. The coordinate, j is stored as the falling point coordinate. That is, the left and right boundary positions Bnx1 and Bnx2 of the optical image block are calculated as Bnx1 = i Bnx2 = j and stored. Further, the sum SI [n] of the brightness (signal intensity) and the sum SIx [n] of the products of the brightness and the horizontal coordinates of the image signals within this range are obtained and stored according to the following equation.

SI [n] = Σ (d [x]) SIx [n] = Σ (d [x] * x) Here, the range of x is the rising point i ≦ x ≦ falling point j.

Next, the "pupil edge extraction" performed in step 107 will be described with reference to the flowchart of FIG.

First, in step 401, the MPU 1 reads the data on the line sensor, and in the next step 402, the minimum value MINL and its position (coordinates) are obtained from this data. Then, in step 403, the sensor is divided in the horizontal direction, and the minimum value in each area is obtained.

Then, after step 404, the extraction of the pupil edge in the horizontal direction is started from the coordinates of the minimum value MINL.

First, in step 405, the counter is decremented to detect the left pupil edge. Then, it is checked whether or not the output signal monotonically decreases over n pixels. That is, it is checked whether the following equations d [j] <d [j-1] <d [j-2] <... <d [j- (n-1)] <d [j-n] are satisfied. . At this time, if the monotonous decrease continues, the count is continued during that time and the slope length D is obtained. Furthermore, it is checked whether or not the difference between the pixels in all the slopes and the adjacent pixels is a certain value Cde or more. For example, d [j-n] -d [j], d [j-n]
Check -d [j- (n-1)] .... By doing so, it is possible to detect a slope (a slope of the pupil edge) which rises from the pupil portion where the signal intensity is substantially equal to the minimum value, with an inclination of a certain value or more.

Next, in step 406, a process of removing the influence of the luminance change due to the eyelashes is performed.

The change in luminance due to the eyelashes is as shown in FIG. 6 as described above. Therefore, by checking whether or not there is a down edge B paired with an up edge A immediately after that, it can be determined whether or not the detected edge is eyelashes.

If the above three conditions are satisfied, the vicinity of the pixel (j-2D) (for example, j-2D ± several pixels) is examined,
If there is a value that is approximately equal to the minimum value (MIN), it is determined that this luminance change is due to the eyelashes.

Those which satisfy the above three conditions are regarded as pupil edges, and in step 407, the pixel number (j + j-D) / 2 is stored in the memory 2 as left pupil edge information.

The MPU 1 performs this process until the end of the effective range of the line sensor is reached (step 408). After that, the MPU 1 proceeds to the routine after step 409 for detecting the pupil edge on the opposite side (right side).

Similar processing is performed in the routine for detecting the pupil edge on the opposite side (right side) (steps 409 to 513).

That is, (1) It is monotonically increasing over n pixels (the slope length D is obtained).

(2) Regarding the pixels in all slopes, the difference from the adjacent pixel is equal to or more than the constant value Cde.

(3) There is no down edge immediately after the up edge (not a change in luminance due to eyelashes). If the three conditions are satisfied, the position is regarded as the pupil edge, the line number L and the pixel number (j + j + D) are stored in the memory 2 as the right pupil edge information, and the process is continued until the end of the effective range is reached. To do.

As described above, this processing is performed for all the pixels within the effective range on the line sensor.

Returning to FIG. 2 again, in step 108, the Purkinje image selection processing is performed in the same manner as in the first embodiment.

That is, in the first processing, the block width in the horizontal direction of each optical image block is examined, and only those widths which are equal to or less than the constant CH are adopted. Objects having a larger block width than this are optical images other than the Purkinje image, such as ghosts of eyeglasses and reflection of external light from the eyelids, and are excluded.
If there is only one Purkinje image candidate at this point, this is adopted as the Purkinje image. If the Purkinje image cannot be determined at this point, the following processing is performed.

In the second processing, the average value of the luminance values of the Purkinje image is first obtained, and then the values are compared, and the largest one is selected as the Purkinje image.

For example, if the blocks B1, B2, and B3 have been selected in the processing so far, the optical image blocks B1 and B
2, average values Iave1, Iave2, I of the brightness values of B3
ave3 is calculated by the following equation.

Iave1 = SI [1] / (B1x2-B1x1 + 1) Iave2 = SI [2] / (B2x2-B2x1 + 1) Iave3 = SI [3] / (B3x2-B3x1 + 1) Then, Iave1, Iave2, and Iave3 are compared, The one with the highest value is the Purkinje image.

The coordinates of the Purkinje image Pc determined as described above are calculated as Pcx = SIx [n] / SI [n]. However, Pcx is the Purkinje image Pc
Is the horizontal coordinate of. Further, it is assumed that the nth optical image block is selected as the Purkinje image (step 10).
9).

Next, the routine proceeds to step 110, where "pupil circle calculation" is performed as follows.

This "pupil circle calculation" is performed in step 5 of FIG.
From 01 to 517 is performed as follows.

In the above-mentioned pupil edge extraction processing, the MPU
Reference numeral 1 stores the coordinates of the extracted pupil edge, the lowest luminance MINn at the edge (slope) and the lowest luminance MIN0 of the entire image in the memory 2.

Therefore, first, the MPU 1 compares the minimum luminance (MIN (e)) of the area where the pupil edge is extracted with the minimum luminance (MIN0 + constant C) of the entire image, and MIN (e) ≤MIN0 + constant C If the condition is not satisfied, the pupil edge existing in the area is excluded as an inappropriate one.

Then, the pupil edge around the Purkinje image obtained in step 109 is eliminated. This is performed by excluding those in the area centered on the horizontal position Pnx of the Purkinje image, for example, those in the range surrounded by (Pnx-constant) and (Pnx + constant).

Further, the average value m and the standard deviation σ of the coordinates of the pupil edge candidates selected so far are obtained, and [average value m-
a * standard deviation σ-average value m + a * standard deviation σ (however, a
Is a constant)].

Next, the pupil center is obtained by using the pupil edge selected as described above. This can be determined by the average value of the adopted pupil edges.

After that, the routine proceeds to step 111, where the rotation angle of the eyeball and the viewpoint position on the focus plate of the camera are calculated using the Purkinje image and the position of the pupil center. And
In step 112, the AF point and the like are determined based on this viewpoint position, and the camera is controlled.

According to the third embodiment described above, a light image block having a brightness equal to or higher than a certain value and having a rising or falling slope is equal to or higher than a certain value is extracted, and the light image is extracted before that. If there is one that can be judged to be the same in the previously extracted ones as compared with the optical image block, the horizontal (horizontal) direction boundary position of that optical image block is updated, and if it does not exist, It is registered as a new optical image block and the horizontal boundary position is stored. In this way, a plurality of optical image blocks (Purkinje image candidates) are obtained, and from the extracted optical image blocks (Purkinje image candidates),
The Purkinje image is selected in consideration of the block width in the horizontal direction. By doing this, the image similar to the Purkinje image such as tears, ghosts in glasses, etc. and the Purkinje image can be properly distinguished, and the influence of the extremely bright light image due to external light can be eliminated, and the Purkinje image can be accurately obtained. It is possible to find the position of the image.

Optical image block (Purkinje image candidate)
Since the center of gravity of is obtained from the sum of the luminance and the sum of the products of the luminance and the horizontal coordinates, the center position of the Purkinje image can be calculated with a resolution of a pixel unit or more.

[0141]

As described above, according to the present invention,
A Purkinje image extracting means is provided in the Purkinje image extracting means, which detects a light image having a brightness equal to or higher than a certain value from a signal from the light receiving means and stores the light image as a Purkinje image candidate, and a Purkinje image extracting means. In the inside, a Purkinje image candidate storage means for detecting an optical image having a brightness equal to or higher than a certain value from the signal from the two-dimensional light receiving means and storing the optical image as a Purkinje image candidate is provided. The image is selected and stored as a Purkinje image candidate.

Further, according to the present invention, the rising and falling slopes of the signals of the Purkinje image candidates stored in the Purkinje image candidate storing means are detected in the Purkinje image extracting means, and based on this result, A Purkinje image selecting means for selecting a Purkinje image is provided, and the Purkinje image is selected depending on whether the rising or falling slope of each signal of the Purkinje image candidates is steep.

Further, according to the present invention, the positional relationship between the Purkinje image candidates stored in the Purkinje image candidate storage means and the pupil is discriminated in the Purkinje image extraction means, and the Purkinje image is selected based on this result. Purkinje image selection means is provided, the positional relationship between the Purkinje image candidate and the pupil (center) is determined, and the closer one is selected as the Purkinje image.

Further, according to the present invention, the number of pixels in the two-dimensional direction of each signal of the Purkinje image candidates stored in the Purkinje image candidate storage unit is detected in the Purkinje image extraction unit, and based on this result, A Purkinje image selecting means for selecting a Purkinje image is provided, and the number of pixels in the two-dimensional direction of each signal of the Purkinje image candidates is detected. I try to select Purkinje images based on less than a few things.

Further, according to the present invention, in the Purkinje image extracting means, the light image centers of the Purkinje image candidates stored in the Purkinje image candidate storing means are arranged at substantially equal intervals in the arrangement direction of the light projecting means. A Purkinje image selecting means for selecting a Purkinje image based on the Purkinje image selection means is provided, and the Purkinje images are selected based on the distances between the light image centers of the respective Purkinje image candidates that are substantially equal to the arrangement direction of the light projecting means within a predetermined interval. I have to.

Further, according to the present invention, in the Purkinje image extracting means, the number of the light image centers of the Purkinje image candidates stored in the Purkinje image candidate storing means is substantially equal to the arrangement direction of the light projecting means. Purkinje image selecting means for selecting a Purkinje image based on the Purkinje image selection means is provided, and the Purkinje image is selected based on the number of the light image centers of the respective Purkinje image candidates substantially equal to the direction of arrangement of the light projecting means. I am trying to choose.

Therefore, it is possible to eliminate the influence of an optical image of extreme brightness due to external light or the like and to accurately extract a Purkinje image.

[Brief description of drawings]

FIG. 1 is a block diagram showing a schematic configuration of a camera provided with a first embodiment device of the present invention.

2 is a flowchart showing a main operation of the camera of FIG.

3 is a flowchart showing an operation performed in step 107 of FIG.

FIG. 4 is a diagram showing an example of an optical image block on which a copying process is performed in the first embodiment of the present invention.

FIG. 5 is a diagram showing an example of an optical image block on which absorption processing is performed in the first embodiment of the present invention.

FIG. 6 is a flowchart showing an optical image block extracting operation of a camera provided with a second embodiment device of the present invention.

FIG. 7 is a flow chart showing a pupil edge extraction operation of a camera equipped with a third embodiment device of the present invention.

FIG. 8 is a flowchart showing a pupil circle calculation operation of a camera equipped with a third embodiment device of the present invention.

FIG. 9 is a circuit diagram showing a main configuration of a conventional line-of-sight detection device.

[Explanation of symbols]

 1 MPU 2 memory 4 CCD 5 IRED group

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location 7316-2K G03B 3/00 A

Claims (10)

[Claims] 1. A light projecting means for irradiating an eyeball of an observer looking into a finder system, a light receiving means for receiving a light flux reflected by the eyeball by the light projecting means, and a pupil edge extracted from a signal from the light receiving means. A pupil edge extracting means for selecting a plurality of Purkinje image candidates from the signal from the light receiving means, and a Purkinje image extracting means for extracting a Purkinje image from among these, the Purkinje image extracting means and the pupil edge extracting means, respectively. In the line-of-sight detection device having a calculation means for calculating the rotation angle of the optical axis of the eyeball based on the information and detecting the line-of-sight direction of the observer, in the Purkinje image extraction means, the signal from the light-receiving means A line-of-sight detection device comprising a Purkinje image candidate storage means for detecting a light image having a brightness of a certain value or more and storing the light image as a Purkinje image candidate. Place 2. A light projecting means for irradiating an eyeball of an observer looking into the finder system, a two-dimensional light receiving means for receiving a light flux reflected by the eyeball by the light projecting means, and a signal from the two-dimensional light receiving means. A plurality of Purkinje image candidates are selected, Purkinje image extracting means for extracting a Purkinje image from them, a pupil edge extracting means for extracting a pupil edge from a signal from the two-dimensional light receiving means, the Purkinje image extracting means and the In a line-of-sight detection device comprising a calculation unit for calculating the rotation angle of the optical axis of the eyeball based on information from each of the pupil edge extraction unit and detecting the line-of-sight direction of the observer, in the Purkinje image extraction unit, Purkinje image candidate storage means for detecting a light image having a brightness of a certain value or more from the signal from the two-dimensional light receiving means and storing the light image as a Purkinje image candidate is provided. And a line-of-sight detection device. 3. The Purkinje image candidate storage means detects an optical image having a brightness of a certain value or more from the signal from the two-dimensional light receiving means, compares the optical image with a previously detected optical image, and determines that they are the same. If what is considered exists before that, update each boundary position in the two-dimensional direction on the light-receiving surface of the optical image that is the Purkinje image candidate, and if nothing that is considered the same exists before that, The line-of-sight detection device according to claim 2, which is means for registering each boundary position in the two-dimensional direction as a new Purkinje image candidate. 4. The Purkinje image candidate storage means detects an optical image having a brightness equal to or higher than a certain value from a signal from the two-dimensional light receiving means, and the optical image as a Purkinje image candidate is detected as an optical image previously detected. In comparison, if there is something that is regarded as the same before that, each boundary position in the two-dimensional direction on the light receiving surface of the light image, the sum of luminance, the sum of the product of luminance and its horizontal coordinate, the luminance and If the sum of products of vertical coordinates is updated as the Purkinje image candidate position, and if there is no object that is considered to be the same before that, then as a new Purkinje image candidate position, each boundary position in the two-dimensional direction, sum of brightness, and brightness 3. The line-of-sight detection apparatus according to claim 2, further comprising means for registering a sum of products of the horizontal coordinates and a sum of products of the vertical coordinates. 5. The Purkinje image selecting means detects the rising and falling slopes of the signals of the respective Purkinje image candidates stored in the Purkinje image candidate storing means, and selects the Purkinje image based on the result. 5. The line-of-sight detection device according to claim 1, 3 or 4, further comprising image selection means. 6. A Purkinje image selecting means for discriminating a positional relationship between a Purkinje image candidate stored in the Purkinje image candidate storing means and a pupil in the Purkinje image extracting means and selecting a Purkinje image based on the result. The line-of-sight detection device according to claim 1, 3 or 4, characterized in that it is provided. 7. The Purkinje image extracting means detects the number of pixels in the two-dimensional direction of each signal of the Purkinje image candidates stored in the Purkinje image candidate storing means, and selects the Purkinje image based on the result. The line-of-sight detection apparatus according to claim 3 or 4, further comprising image selection means. 8. The line-of-sight detection device according to claim 3, wherein a plurality of light projecting means for irradiating the eyeball are arranged at a predetermined interval. 9. A Purkinje image is selected in the Purkinje image extracting means on the basis of the intervals between the light image centers of the Purkinje image candidates stored in the Purkinje image candidate storing means, which are substantially equal to the arrangement direction of the light projecting means. 9. The line-of-sight detection apparatus according to claim 8, further comprising a Purkinje image selecting unit for performing the above. 10. A Purkinje image is selected in the Purkinje image extraction means based on the number of light image centers of the Purkinje image candidates stored in the Purkinje image candidate storage means that are substantially equal to the arrangement direction of the light projecting means. 9. The line-of-sight detection apparatus according to claim 8, further comprising a Purkinje image selecting unit for performing the above.