JP7532094B2 - Observation device and imaging device - Google Patents

Description

The present invention relates to an observation device equipped with an imaging system that captures an image of an observer's eyeball.

Some imaging devices such as digital cameras have a gaze detection function that detects the direction of the eyeball of an observer looking into the eyepiece optical system of an optical finder or electronic viewfinder as an observation device, that is, the gaze direction, and selects an area within the imaging screen for performing autofocus, autoexposure, etc., in accordance with the gaze direction. Such a gaze detection function is realized by providing an imaging system that images the observer's eyeball, separate from the eyepiece optical system.
Patent Literature 1 discloses an imaging device that arranges a prism in an eyepiece optical system to branch an optical path and guides light from the branched optical path to an imaging system. Patent Literature 2 discloses an imaging device having an imaging system that images an eyeball from outside the eyepiece optical system in a direction tilted with respect to the optical axis of the eyepiece optical system.

Japanese Patent Application Publication No. 05-188430 Japanese Patent Application Publication No. 05-313057

However, placing an optical component such as a prism that branches the optical path within the eyepiece optical system increases the size of the observation device and the imaging device. In addition, placing an optical component that is not necessary for the eyepiece optical system within the eyepiece optical system reduces the design freedom of the eyepiece optical system.

Furthermore, when the eyeball is imaged from a direction tilted relative to the optical axis of the eyepiece optical system, the image magnification fluctuates greatly due to the variation in the distance from the eyepiece optical system to the eyeball. For example, it becomes difficult to maintain the same gaze detection accuracy when the observer is wearing glasses as when the observer is wearing naked eyes.

The present invention provides a small observation device that can capture good images of the eyeball even if the position of the eyeball changes, and an imaging device equipped with the same.

An observation device according to one aspect of the present invention has an image display element that displays an image, an eyepiece optical system that directs light from the image display element to the eyeball of an observer, and a plurality of imaging systems each including an imaging lens that forms an optical image of the eyeball and an imaging sensor that captures the optical image. The optical axis of each of the imaging lenses of the plurality of imaging systems intersects with the optical axis of the eyepiece optical system at different positions on the optical axis and at different inclination angles. The observation device is characterized in that the angles between the optical axis of the imaging lens in each of the plurality of imaging systems and the normal to the imaging surface of the imaging sensor are different from each other.

An observation device according to another aspect of the present invention has an image display element that displays an image, an eyepiece optical system that guides light from the image display element to the observer's eyeball, and a number of imaging systems that each image the eyeball. The imaging directions of the multiple imaging systems are directions that form different inclination angles with respect to the optical axis of the eyepiece optical system at different positions on the optical axis. Each of the multiple imaging systems is configured so that the inclination of the focal plane of the imaging system on the imaging sensor side with respect to the optical axis of the eyepiece optical system is smaller than the inclination of a plane perpendicular to the imaging direction with respect to the optical axis of the eyepiece optical system.

The present invention allows for a compact observation device that can capture good images of the eyeball even if the eyeball position changes.

FIG. 1 is a diagram showing the configuration of an observation device according to a first embodiment of the present invention. FIG. 2 is a diagram showing the configuration of an imaging system in the first embodiment. FIG. 4 is a diagram showing the configuration of another imaging system in the first embodiment. FIG. 11 is a diagram showing the configuration of an observation device according to a second embodiment of the present invention. FIG. 11 is a diagram showing the configuration of an imaging system in a second embodiment. FIG. 11 is a diagram showing the configuration of another imaging system in the second embodiment. FIG. 13 is a diagram showing the configuration of yet another imaging system in the second embodiment. FIG. 11 is a diagram showing the configuration of an observation device according to a third embodiment of the present invention. FIG. 2 is a diagram showing an imaging device equipped with the observation device according to the first to third embodiments.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. Fig. 1 shows the configuration of observation device 1A, which is embodiment 1 of the present invention. Observation device 1A has an image display element 3, an eyepiece optical system 2, and multiple imaging systems 4, 5. Fig. 4 shows the configuration of observation device 1B, which is embodiment 2 of the present invention. Observation device 1B has an image display element 3, an eyepiece optical system 2, and multiple imaging systems 6, 7, 8.

Figures 2 and 3 respectively show the configurations of imaging systems 4 and 5 provided in observation device 1A. Figures 5, 6 and 7 respectively show the configurations of imaging systems 6, 7 and 8 provided in observation device 1B. Each imaging system (4-8) is composed of, from the object side to the image side, an aperture (11, 14, 17, 20, 23), imaging lenses (12, 15, 18, 21, 24) and an imaging sensor (13, 16, 19, 22, 25).

Tables 1 to 5 show examples of values for imaging systems 4 to 8, respectively. Among the overall specifications in each example, "focal length" refers to the focal length of the imaging lens (mm), "imaging surface diagonal length" refers to the diagonal length (mm) of the rectangular imaging surface of the imaging sensor, and "angle of view" refers to the angle of view (°) of the imaging lens.

In the surface data, if i is the surface number counted from the object side, r is the radius of curvature (mm) of the i-th surface, z and y are the coordinates of the i-th surface in the z and y directions shown in Figures 1 and 4, respectively. θ is the inclination angle (°) of the optical axis of the imaging lens and the normal to the imaging surface of the imaging sensor with respect to the optical axis of the eyepiece optical system, and nd and νd are the refractive index of the i-th lens with respect to the d-line (587.6 nm) and the Abbe number based on the d-line, respectively. The Abbe number νd is expressed as νd = (Nd-1)/(NF-NC), where Nd, NF, and NC are the refractive indices of the Fraunhofer lines at the d-line (587.6 nm), F-line (486.1 nm), and C-line (656.3 nm). The "effective diameter" is the diameter of the area of the i-th surface through which light that reaches the imaging surface passes.

Additionally, lens surfaces with an * next to the surface number have an aspheric shape. The aspheric shape is expressed by the following formula, where the position on the optical axis of the imaging lens from the vertex of the lens surface is x, the height in a direction perpendicular to the optical axis is h, the traveling direction of light is positive, r is the paraxial radius of curvature, K is the conic constant, and A4, A6, and A8 are aspheric coefficients. "E±M" in the conic constant and aspheric coefficients means ×10 ^±M .

In the observation device 1A of Example 1 shown in FIG. 1, the image display element 3 is composed of a liquid crystal element, an organic EL element, or the like, and displays an observation image. The eyepiece optical system 2 guides light from the image display element 3 to the observer's eyeball 10. The observation device 1A is mounted on an imaging device such as a digital still camera or a video camera, and functions as an electronic viewfinder (EVF) that displays an image obtained by imaging a subject with the imaging device as an observation image.

The observation device 1A also has imaging systems 4 and 5. As shown in Figures 2 and 3, the imaging systems 4 and 5 are each composed of an aperture 11, 14, an imaging lens 12, 15 that forms an image of light that has passed through the aperture of the aperture 11, 14, and an imaging sensor 13, 16 as a photoelectric conversion element that captures (photoelectrically converts) the optical image formed by the imaging lens 12, 15, and capture an image of the eyeball 10 looking into the eyepiece optical system 2. The imaging data (image data) obtained by capturing an image of the eyeball 10 by the imaging systems 4 and 5 is used to detect the direction of the eyeball 10, i.e., the line of sight of the observer.

The imaging direction of the imaging system 4 (the direction in which the optical axis 12a of the imaging lens 12 extends) faces a position on the optical axis 2a of the eyepiece optical system 2 that is far from the eyepiece optical system 2, and the imaging direction of the imaging system 5 (the direction in which the optical axis 15a of the imaging lens 15 extends) faces a position close to the eyepiece optical system 2. In other words, the imaging systems 4 and 5 are provided to capture images of the eyeball 10 that are at positions that are different distances from the eyepiece optical system 2.

Specifically, the optical axis 12a of the imaging lens 12 of the imaging system 4 and the optical axis 15a of the imaging lens 15 of the imaging system 5 intersect with the optical axis 2a of the eyepiece optical system 2 at different positions on the optical axis 2a (positions at different distances from the eyepiece optical system 2) at different inclination angles θ4 and θ5.

In this way, the imaging systems 4 and 5 are configured to image the eyeball 10 from an imaging direction inclined with respect to the optical axis 2a of the eyepiece optical system 2, which makes it possible to prevent unnecessary increases in the optical path length caused by providing optical components such as prisms and mirrors in the eyepiece optical system as in the conventional art. In particular, with an EVF such as that in this embodiment, a large magnification is required for the eyepiece optical system, which shortens the focal length of the eyepiece optical system, so an increase in the optical path length of the eyepiece optical system is not desirable.

Furthermore, when the eyeball is imaged from an oblique direction rather than from the front, the image height at which the eyeball is imaged on the imaging sensor changes depending on the distance from the eyepiece optical system to the eyeball. Furthermore, the distance from the imaging system to the eyeball also changes depending on the distance from the eyepiece optical system to the eyeball, so the focus position of the imaging system changes and the imaging magnification for the eyeball also changes. These effects can be ignored when the distance from the eyepiece optical system to the eyeball is almost fixed, such as in glasses-type terminals and head-mounted displays, but when the distance from the eyepiece optical system to the eyeball can change significantly, such as in the EVF of an imaging device, gaze detection becomes difficult depending on the distance to the eyeball.

For this reason, in this embodiment, imaging systems 4 and 5 are provided according to the distance from the eyepiece optical system 2 to the eyeball 10, and by mitigating the changes in image height, focal position, and imaging magnification described above, the eyeball 10 can be imaged well even if the distance to the eyeball 10 changes, enabling highly accurate gaze detection. Specifically, the positions at which the optical axes 12a and 15a of the imaging lenses 12 and 15 of the imaging systems 4 and 5 intersect with the optical axis 2a of the eyepiece optical system 2 are made different from each other, thereby changing the distance at which the optical image of the eyeball 10 is formed at the center of the imaging sensors 13 and 16.

In addition, by varying the angle of view of the imaging lens (hereinafter also referred to as the angle of view of the imaging system) and focal length according to the position of the eyeball 10 corresponding to each imaging system, changes in the focus position and imaging magnification due to changes in the distance to the eyeball 10 are suppressed. Specifically, the angle of view of imaging system 4, which images the eyeball 10 at a far distance from the eyepiece optical system 2, is narrower than that of imaging system 5, which images the eyeball 10 at a close distance from the eyepiece optical system 2.

In addition, in each imaging system, the tilt angle that the optical axis of the imaging lens makes with respect to the optical axis 2a of the eyepiece optical system 2 (hereinafter referred to as the lens optical axis angle) and the tilt angle that the normal (13a, 16a) of the imaging surface of the imaging sensor makes with respect to the optical axis 2a of the eyepiece optical system 2 (hereinafter referred to as the sensor normal angle) are different from each other. The difference between the sensor normal angle and the lens optical axis angle (= sensor normal angle - lens optical axis angle) is called the sensor tilt angle Δθ. Furthermore, in this embodiment, the sensor tilt angle Δθ4 of imaging system 4 and the sensor tilt angle Δθ5 of imaging system 5 are also different from each other.

When the eyeball 10 is imaged from an imaging direction tilted with respect to the optical axis 2a of the eyepiece optical system 2, if the lens optical axis angle and the sensor normal angle are the same (i.e., the sensor tilt angle is 0), the focal plane, which is the plane on which the imaging system is focused, is a plane that is perpendicular to the optical axis of the imaging lens and is greatly tilted from the plane perpendicular to the optical axis 2a of the eyepiece optical system 2. However, in order to image not only the cornea of the eyeball 10 but also the sclera, pupil, and even eyelids so that they are in focus, it is desirable for the focal plane of the imaging system to be a plane that has as small an inclination as possible with respect to the plane perpendicular to the optical axis 2a of the eyepiece optical system 2 (the plane directly facing the eyeball 10). For this reason, in this embodiment, the imaging sensor in each imaging system is tilted with respect to the imaging lens to appropriately set the sensor tilt angle, so that the inclination of the focal plane of the imaging system with respect to the plane perpendicular to the optical axis 2a of the eyepiece optical system 2 is smaller than the plane perpendicular to the optical axis (imaging direction) of the imaging lens.

As a specific example, in the imaging system 4 shown in FIG. 2, the sensor tilt angle Δθ4 is set so that the inclination of the focal plane 12c on the observer side of the imaging system 4 with respect to the plane 2b perpendicular to the optical axis 2a of the eyepiece optical system 2 is smaller than the inclination of the plane 12b perpendicular to the optical axis 12a of the imaging lens 12. Note that at this time, the inclination of the focal plane on the imaging sensor side of the imaging system 4 with respect to the optical axis 2a of the eyepiece optical system 2 is smaller than the inclination of the plane 12b perpendicular to the optical axis 12a of the imaging lens 12 with respect to the optical axis of the eyepiece optical system.

The following describes the conditions that are desirably satisfied by multiple imaging systems in this embodiment and other embodiments described below.

When the widest angle of view among the angles of view of a plurality of imaging systems is ω1 and the narrowest angle of view is ω2, it is desirable to satisfy the following conditional expression (1).
-0.30≦tan(ω2)/tan(ω1)≦0.80 (1)
Conditional formula (1) shows a condition regarding the angle of view for providing a plurality of imaging systems that can sufficiently respond to changes in the distance between the eyepiece optical system and the eyeball. If tan(ω2)/tan(ω1) is below the lower limit of conditional formula (1), the imaging performance of one of the imaging systems will be reduced, causing distortion and the like, and the imaging magnification for the eyeball will be reduced, which is undesirable. If tan(ω2)/tan(ω1) is above the upper limit of conditional formula (1), the angles of view of the plurality of imaging systems will be too close to each other, making it impossible to sufficiently reduce the effect of changes in the distance between the eyeball, which is undesirable.

It is more preferable that the numerical range of conditional expression (1) is as follows:
-0.20≦tan(ω2)/tan(ω1)≦0.70 (1a)
It is more preferable that the numerical range of conditional expression (1) is as follows:
-0.10≦tan(ω2)/tan(ω1)≦0.60 (1b)
Furthermore, when the largest lens optical axis angle among the absolute values of the lens optical axis angles of a plurality of imaging systems is θ1 and the smallest lens optical axis angle is θ2, it is desirable to satisfy the following conditional expression (2).
1.1≦tan(θ1)/tan(θ2)≦8.0 (2)
Conditional formula (2) shows a condition related to the lens optical axis angle for providing a plurality of imaging systems that can sufficiently respond to changes in the distance between the eyepiece optical system and the eyeball. If tan(θ1)/tan(θ2) is below the lower limit of conditional formula (2), it is not preferable because it is not possible to reduce the influence of changes in the distance of the eyeball. If tan(θ1)/tan(θ2) is above the upper limit of conditional formula (2), it is not preferable because the optical path of the imaging system may interfere with the optical path of the eyepiece optical system.

It is more preferable that the numerical range of conditional expression (2) is as follows:
1.5≦tan(θ1)/tan(θ2)≦6.0 (2a)
It is more preferable that the numerical range of conditional expression (2) is as follows:
1.7≦tan(θ1)/tan(θ2)≦5.0 (2b)
In addition, when the absolute value of the lens optical axis angle of each imaging system is θa and the absolute value of the sensor normal angle is θb, it is desirable to satisfy the following conditional expression (3).
0.70≦tan(θa)/tan(θb)<1.00 (3)
Conditional formula (3) shows a condition for bringing the focal plane of the imaging system closer to a plane perpendicular to the optical axis of the eyepiece optical system. If tan(θa)/tan(θb) is below the lower limit of conditional formula (3), the imaging system becomes large, which is not preferable. If tan(θa)/tan(θb) is above the upper limit of conditional formula (3), the total length of the imaging system becomes long in order to bring the inclination of the focal plane on the observer side of the imaging system closer to a plane perpendicular to the optical axis of the eyepiece optical system, which is not preferable.

It is more preferable that the numerical range of conditional expression (3) is as follows:
0.75≦tan(θa)/tan(θb)<1.00 (3a)
It is more preferable that the numerical range of conditional expression (3) is as follows:
0.80≦tan(θa)/tan(θb)<1.00 (3b)
Furthermore, when the absolute value of the lens optical axis angle of each imaging system is θn, it is desirable to satisfy the following conditional expression (4).
10°≦θn≦70° (4)
Conditional expression (4) shows a condition for ensuring the optical performance of the imaging system while suppressing the influence on the eyepiece optical system. If θn is below the lower limit of conditional expression (4), the position of the imaging system becomes too close to the eyepiece optical system, which may affect the optical performance of the eyepiece optical system, which is not preferable. If θn is above the upper limit of conditional expression (4), the inclination of the imaging system with respect to the eyepiece optical system becomes too large, which makes it difficult to ensure the optical performance of the imaging system, which is also not preferable.

It is more preferable that the numerical range of conditional expression (4) is as follows:
10°≦θn≦65° (4a)
It is more preferable that the numerical range of conditional expression (4) is as follows:
10°≦θn≦60° (4b)
Furthermore, when the focal length of the imaging lens of each imaging system is f, the diagonal length of the imaging surface of the image sensor is H, and the maximum value of H/f of the multiple imaging systems is H1/f1 and the minimum value is H2/f2, it is desirable to satisfy the following conditional expression (5):
1.1≦(H1/f1)/(H2/f2)≦2.5 (5)
Conditional formula (5) shows a condition for improving the gaze detection accuracy by the multiple imaging systems even if the distance between the eyepiece optical system and the eyeball varies. If (H1/f1)/(H2/f2) is below the lower limit of conditional formula (5), the characteristics of the multiple imaging systems are too close, and the change in the gaze detection accuracy due to the variation in the distance of the eyeball increases, which is not preferable. If (H1/f1)/(H2/f2) is above the upper limit of conditional formula (5), the difference in the characteristics of the multiple imaging systems is too large, and there is a risk of the number of imaging systems increasing, which is not preferable.

It is more preferable that the numerical range of conditional expression (5) is as follows:
1.1≦(H1/f1)/(H2/f2)≦2.1 (5a)
It is more preferable that the numerical range of conditional expression (5) is as follows:
1.1≦(H1/f1)/(H2/f2)≦1.9 (5b)
Furthermore, when the curvature of the imaging lens on the eyeball side of each imaging system is R1 and the curvature of the imaging lens on the imaging sensor side is R2, it is desirable that each imaging system includes at least one imaging lens that satisfies the following conditional expression (6):
-4.00≦(R1+R2)/(R1-R2)<1.00 (6)
Conditional formula (6) indicates a condition that is desirably satisfied by the imaging lens in order to achieve both optical performance for line of sight detection and compactness in each imaging system. If (R1+R2)/(R1-R2) is below the lower limit of conditional formula (6), the imaging lens will become large, which is undesirable. If (R1+R2)/(R1-R2) is above the upper limit of conditional formula (6), the image surface curvature, coma aberration, etc. of the imaging lens will increase, which is undesirable, and the optical performance of the imaging lens will deteriorate.

It is more preferable that the numerical range of conditional expression (6) is set as follows:
-3.50≦(R1+R2)/(R1-R2)<1.00 (6a)
It is more preferable that the numerical range of conditional expression (5) is as follows:

-3.30≦(R1+R2)/(R1-R2)<1.00 (6b)
In the numerical example of the first embodiment, tan(θa)/tan(θb) in the conditional expression (2), H/f in the conditional expression (5), and (R1+R2)/(R1-R2) in the conditional expression (6) are shown. 1 and 2. In addition, tan(ω2)/tan(ω1) in conditional expression (1), tan(θ1)/tan(θ2) in conditional expression (2), and (H1/f1 )/(H2/f2) are summarized in Table 6.

Figure 4 shows the configuration of observation device 1B, which is Example 2 of the present invention. The image display element 3 and eyepiece optical system 2 in observation device 1B are the same as the image display element 3 and eyepiece optical system 2 in Example 1. In this example, by increasing the number of imaging systems 6 to 8 to three compared to Example 1, which has two imaging systems 4 and 5, it is possible to obtain higher line of sight detection accuracy in response to changes in the distance from the eyepiece optical system 2 to the eyeball 10.

As shown in Figures 5, 6 and 7, the imaging systems 6, 7 and 8 are each composed of an aperture 17, 20 and 23, imaging lenses 18, 21 and 24 that form an image using light that has passed through the apertures of the apertures 17, 20 and 23, and imaging sensors 19, 22 and 25 as photoelectric conversion elements that capture the optical image formed by the imaging lenses 18, 21 and 24, and capture an image of the eyeball 10 looking into the eyepiece optical system 2. As in Example 1, the imaging data obtained by capturing an image of the eyeball 10 using the imaging systems 6 to 8 is used to detect the direction of the eyeball 10, i.e., the line of sight of the observer.

The imaging direction of imaging system 6 faces a position far away from the eyepiece optical system 2, and the imaging direction of imaging system 7 faces a position close to the eyepiece optical system 2. The imaging direction of imaging system 8 faces a position at an intermediate distance from the eyepiece optical system 2. In other words, imaging systems 6 to 8 are provided to capture images of the eyeball 10 at positions that are different distances from the eyepiece optical system 2.

Specifically, the optical axis 18a of the imaging lens 18 of the imaging system 6, the optical axis 21a of the imaging lens 21 of the imaging system 7, and the optical axis 24a of the imaging lens 24 of the imaging system 8 intersect with the optical axis 2a of the eyepiece optical system 2 at different positions (positions at different distances from the eyepiece optical system 2) and at different inclination angles θ6, θ7, and θ8.

In this embodiment, the angles of view of the imaging systems 6 to 8 are also different from one another. Specifically, the angles of view of the imaging system 6, which images the eyeball 10 at a close distance from the eyepiece optical system 2, are narrower in this order than the imaging system 8, which images the eyeball 10 at an intermediate distance, and the imaging system 7, which images the eyeball 10 at a far distance.

Furthermore, in this embodiment, as shown in Figures 5 to 7, the normals 19a, 22a, and 25a of the imaging surfaces of the imaging sensors 19, 22, and 25 in the imaging systems 6 to 8 are tilted relative to the optical axes 18a, 21a, and 24a of the imaging lenses 18, 21, and 24, respectively, to appropriately set the sensor tilt angles Δθ6, Δθ7, and Δθ8, so that the inclination of the focal surface on the observer side of each imaging system relative to the plane perpendicular to the optical axis 2a of the eyepiece optical system 2 is made smaller than the plane perpendicular to the optical axis of the imaging lens.

In the numerical example of Example 2, tan(θa)/tan(θb) in conditional formula (2), H/f in conditional formula (5), and (R1+R2)/(R1-R2) in conditional formula (6) are shown in Tables 3 to 5. In addition, tan(ω2)/tan(ω1) in conditional formula (1), tan(θ1)/tan(θ2) in conditional formula (2), and (H1/f1)/(H2/f2) in conditional formula (5) are shown in Table 6.
Although three imaging systems are provided in this embodiment, the number of imaging systems may be four or more.

In the above-described first and second embodiments, the normal of the imaging surface of the imaging sensor is tilted with respect to the optical axis of the imaging lens in each imaging system, so that the inclination of the focal plane of the imaging system on the imaging sensor side with respect to the optical axis of the eyepiece optical system is smaller than the inclination of the surface perpendicular to the optical axis of the imaging lens (imaging direction) with respect to the optical axis of the eyepiece optical system. However, as long as the inclination of the focal plane of the imaging system on the imaging sensor side with respect to the optical axis of the eyepiece optical system is smaller than the inclination of the surface perpendicular to the imaging direction with respect to the optical axis of the eyepiece optical system. For example, the imaging lens constituting the imaging system may be composed of multiple lenses. In addition, in order to reduce the size of the observation device, a configuration in which the optical path is bent by disposing a mirror in the imaging system, etc., may be adopted. In this case, it is sufficient that the conditions of this embodiment are satisfied when the mirror is unfolded.

Figure 8 shows the configuration of an observation device 101 according to a third embodiment of the present invention. The observation device 101 has an image display element 103, an eyepiece optical system 102, and imaging systems 104 and 105 that capture an image of an observer's eyeball 109. The image display element 103 and the eyepiece optical system 102 are the same as the image display element 3 and the eyepiece optical system 2 in the first embodiment.

The observation device 101 also has an illumination system 106 that irradiates the eyeball 109 with infrared light when the eyeball 109 is imaged by the imaging system 104, and an illumination system 107 that irradiates the eyeball 109 with infrared light when the eyeball 109 is imaged by the imaging system 104. The illumination systems 106 and 107 irradiate the eyeball 109 at different distances from the eyepiece optical system 102 with infrared light from directions tilted at different angles with respect to the optical axis of the eyepiece optical system 102. Infrared light is used so that the irradiation is not visible to the observer.

The observation device 101 is connected to a processing circuit (processing means) 108. The processing circuit 108 calculates the gaze direction of the observer using the imaging data obtained by the imaging systems 104 and 105. In this case, the processing circuit 108 switches between the imaging systems 104 and 105 that obtain imaging data (main imaging data) mainly used for gaze detection depending on the distance from the eyepiece optical system 102 to the eyeball 109. This is because the imaging system that provides the highest gaze detection accuracy differs depending on the distance to the eyeball 109. The distance to the eyeball 109 may be calculated using the parallax of the imaging data obtained from the imaging systems 104 and 105. The gaze direction can be calculated from the coordinates of the reflected image of infrared light irradiated from the illumination systems 106 and 107 to the eyeball 109, or from imaging data of a wide area of the face centered on the eyeball 109.

The processing circuit 108 may also correct the gaze direction obtained from the main imaging data using imaging data (sub-imaging data) obtained from an imaging system other than the imaging system that obtains the main imaging data. For example, sub-imaging data from another imaging system that has a wider angle of view than the imaging system that obtains the main imaging data may include images of areas of the face other than the eyeballs 109. By using this sub-imaging data to correct the gaze direction calculated using the main imaging data, the gaze accuracy of the gaze direction can be improved compared to when only the main imaging data is used.

According to the above-described first to third embodiments, it is possible to realize an observation device that is compact, yet capable of capturing good images of the eyeball even when the eyeball position changes, and that can maintain good gaze detection accuracy.

In the above embodiments 1 to 3, the case where multiple imaging systems are arranged above and below the eyepiece optical system has been described. This is because the effective area of a typical eyepiece optical system is narrower in the vertical direction than in the horizontal direction, so arranging multiple imaging systems above and below the eyepiece optical system makes it easier to prevent the observation device from becoming larger. However, the multiple imaging systems may be arranged anywhere on a circumference centered on the optical axis of the eyepiece optical system.

In addition, in the first to third embodiments, the optical system of the imaging system is described as being composed of an imaging lens and an aperture, but a cover glass may be placed in front of the aperture, or an optical member such as a prism may be placed either in front of or behind the imaging lens.

Figure 9 shows an imaging device 200 such as a digital still camera or video camera equipped with any one of the observation devices 1A, 1B, and 101 of Examples 1 to 3.

The imaging device 200 captures an image of a subject formed by an imaging lens 201 using an imaging sensor 202 such as a CCD sensor or CMOS sensor. The imaging signal output from the imaging sensor 202 is input to an arithmetic processing circuit (processing means) 203. The arithmetic processing circuit 203 performs various image processes on the imaging signal to generate captured image data. The captured image data is output to an observation device as an EFV and displayed on an image display element (3, 103) in the observation device. The user can view the image displayed on the image display element by looking into the eyepiece optical system (2, 102) in the observation device with his/her eyeball (10, 109).

At this time, the arithmetic processing circuit 203 uses the imaging data obtained from the imaging system (4-8, 104, 105) of the observation device to calculate (acquire) the line of sight of the user (observer) like the processing circuit 108 described above. Then, from this line of sight direction, it calculates the user's gaze position on the imaging screen. Furthermore, the arithmetic processing circuit 203 selects an area including the gaze position from within the imaging screen, and performs processes such as automatic exposure and autofocus using the imaging image data of the selected area.

By using such a small observation device with high gaze detection accuracy, it is possible to realize an imaging device that is small yet capable of performing processes such as auto exposure and auto focus well.

The above-described embodiments are merely representative examples, and various modifications and variations are possible when implementing the present invention.

1A, 1B, 101 Observation device 2, 102 Eyepiece optical system 3, 103 Image display element 4, 5, 6, 7, 8, 104, 105 Imaging system 12, 15, 18, 21, 24 Imaging lens 13, 16, 19, 22, 25 Imaging sensor

Claims (13)

an image display element for displaying an image;
an eyepiece optical system that guides light from the image display element to an observer's eye;
a plurality of imaging systems each including an imaging lens for forming an optical image of the eyeball and an imaging sensor for capturing the optical image;
an optical axis of each of the imaging lenses of the plurality of imaging systems intersects with an optical axis of the eyepiece optical system at different positions on the optical axis and at different inclination angles;
an imaging system in which the optical axis of the imaging lens in each of the plurality of imaging systems and the normal to the imaging surface of the imaging sensor form angles different from one another; The observation device according to claim 1, characterized in that the angle between the optical axis of the imaging lens and the normal to the imaging surface is set so that the inclination of the focal plane on the imaging sensor side of the imaging system with respect to the optical axis of the eyepiece optical system is smaller than the inclination of a plane perpendicular to the optical axis of the imaging lens with respect to the optical axis of the eyepiece optical system. the plurality of imaging systems have different angles of view;
Among the angles of view of each of the plurality of imaging systems, the widest angle of view is ω1 and the narrowest angle of view is ω2.
-0.30≦tan(ω2)/tan(ω1)≦0.80
3. The observation apparatus according to claim 1, wherein the following conditions are satisfied: Among the tilt angles of each of the plurality of imaging systems, the tilt angle with the largest absolute value is θ1, and the tilt angle with the smallest absolute value is θ2.
1.1≦tan(θ1)/tan(θ2)≦8.0
4. The observation apparatus according to claim 1, wherein the following conditions are satisfied: Let θa be the absolute value of the tilt angle in each of the plurality of imaging systems, and θb be the absolute value of the angle between the normal to the imaging surface and the optical axis of the eyepiece optical system.
0.70≦tan(θa)/tan(θb)<1.00
5. The observation apparatus according to claim 1, wherein the following condition is satisfied: When the absolute value of the tilt angle in each of the plurality of imaging systems is θn,
10°≦θn≦70°
6. The observation apparatus according to claim 1, wherein the following conditions are satisfied: Let f be the focal length of the imaging lens in each of the multiple imaging systems, H be the diagonal length of the imaging surface of the imaging sensor, and let H1/f1 be the maximum value of H/f for each of the multiple imaging systems and H2/f be the minimum value.
1.1≦(H1/f1)/(H2/f2)≦2.5
7. The observation apparatus according to claim 1, wherein the following conditions are satisfied: In each of the plurality of imaging systems, the curvature of the imaging lens on the eyeball side is R1, and the curvature of the imaging lens on the imaging sensor side is R2.
-4.00≦(R1+R2)/(R1-R2)<1.00
8. The observation apparatus according to claim 1, further comprising at least one imaging system which satisfies the following condition: an image display element for displaying an image;
an eyepiece optical system that guides light from the image display element to an observer's eye;
a plurality of imaging systems each capturing an image of the eyeball;
imaging directions of the plurality of imaging systems are directions that form different inclination angles with respect to an optical axis of the eyepiece optical system at different positions on the optical axis,
An observation device characterized in that each of the multiple imaging systems is configured so that the inclination of the focal plane of the imaging system on the imaging sensor side with respect to the optical axis of the eyepiece optical system is smaller than the inclination of a plane perpendicular to the imaging direction with respect to the optical axis of the eyepiece optical system. The observation device according to any one of claims 1 to 9, characterized in that it has a processing means for acquiring the orientations of the eyeballs at different distances from the eyepiece optical system using main imaging data obtained from one of the multiple imaging systems. The observation device according to claim 10, characterized in that the processing means switches the imaging system for obtaining the main imaging data depending on the distance from the eyepiece optical system to the eyeball. The processing means includes:
12. The observation device according to claim 10, wherein the direction of the eyeball acquired using the main imaging data is corrected using sub-imaging data from an imaging system different from the imaging system that obtains the main imaging data. The observation device according to any one of claims 10 to 12,
An imaging device comprising: an imaging unit that selects an area within an imaging screen in accordance with the direction of the eyeball obtained by the processing means.