JP7559605B2 - Image display device and method - Google Patents

Description

The present invention relates to an image display device and method.

Conventionally, there is known an image display device that uses an optical element to relay an exit pupil formed by an optical unit to place an eyepoint in the air, and allows a user to view an image when placing his/her eye (i.e., pupil) within the eyepoint by irradiating light that forms an image into the optical unit. Here, a retroreflective element can be used as the optical element (see Patent Document 1). Such a display device has attracted attention because it provides a particularly high level of security, since only a single user who places his/her eye within the eyepoint can view the image.
Patent Document 1: JP 2009-288696 A

In a first aspect of the present invention, there is provided an image display device comprising: a display unit having an optical unit having an input side focus at a position where an image is generated and forming an exit pupil on the output side from which light forming the image is output; and an optical element that reflects or transmits the light output from the optical unit to form an eyepoint at a conjugate position of the exit pupil; a detection unit that detects the position of a user's eye; and a guidance unit that guides the user's eye relative to the position of the eyepoint based on the position of the user's eye and the position of the eyepoint detected by the detection unit , wherein the guidance unit has a tactile generation device that generates a tactile sensation at at least one location near the position of the eyepoint .

In a second aspect of the present invention, there is provided an image display method comprising: a step of detecting the position of a user's eye; a step of guiding the user's eye relative to the position of the eyepoint based on the position of the user's eye detected in the detection step; a step of guiding the user's eye relative to the position of the eyepoint based on the position of the eyepoint formed in the air by an optical unit having an input side focus at the position where an image is generated and forming an exit pupil on the output side from which light forming the image is output; and an optical element that reflects or transmits the light output from the optical unit to form an eyepoint at a conjugate position of the exit pupil; and a step of displaying an image, wherein the guiding step further generates a tactile sensation at at least one location near the position of the eyepoint .

Note that the above summary of the invention does not list all of the features of the present invention. Also, subcombinations of these features may also be inventions.

1 shows a schematic configuration of an image display device according to a first embodiment in a side view. 1 is a front view showing a schematic configuration of an image display device according to a first embodiment. 1 is a top view showing a schematic configuration of an image display device according to a first embodiment. 2 shows the configuration of a control system of the image display device according to the first embodiment. 3 shows a flow of an image display method according to the first embodiment. 13 is a schematic side view showing the configuration of an image display device according to a second embodiment. 13 is a front view showing a schematic configuration of an image display device according to a second embodiment. 13 is a top view illustrating a schematic configuration of an image display device according to a second embodiment. 13 shows the configuration of a control system of an image display device according to a second embodiment. 13 shows an example of control of the haptic generation position and intensity by the haptic generation device (when the user is away from the image display device). 13 shows an example of control of the haptic generation position and intensity by the haptic generation device (when the user is approaching the image display device). 13 shows an example of control of the haptic generation position and intensity by the haptic generation device (when the user's eyes are at the eyepoint). 13 shows another example of control of the position and intensity of haptics generated by the haptic generating device (when the user is facing left). 13 shows another example of control of the position and intensity of haptics generated by the haptic generating device (when the user is facing to the right). 10 shows a flow of an image display method according to a second embodiment. 13 is a schematic side view showing the configuration of an image display device according to a third embodiment. 13 is a front view showing a schematic configuration of an image display device according to a third embodiment. 13 is a top view illustrating a schematic configuration of an image display device according to a third embodiment. 13 shows the configuration of a control system of an image display device according to a third embodiment. 1 is a side view showing a schematic configuration of an image display device capable of rotating an optical element. 13 shows a flow of an image display method according to a third embodiment.

The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

1A, 1B, and 1C are schematic side, front, and top views, respectively, showing the configuration of an image display device 100 according to a first embodiment. The image display device 100 is a device that displays an image Im so that the user 99 can see it without physically contacting any of the components, and includes a display unit 10, a detection unit 20, and a guidance unit 30. The image Im may be a still image or a video, and may be a monochrome image or a color image. Furthermore, with respect to the user 99 viewing the image Im, the left-right direction (the user's eye width direction) is the X-axis direction, the front-back direction is the Y-axis direction, and the up-down direction is the Z-axis direction.

The display unit 10 is a unit that forms an eyepoint Exp in the air, and displays an image Im whose entire angle of view is visible within the field of view when the user 99 places both eyes within the position of the eyepoint Exp (also called an aerial eyepiece), and has a case 19, a display device 11, an optical unit 12 (12L, 12R), a half mirror 13, and an optical element 14. The display unit 10 is supported on the floor or wall via a frame (not shown).

The case 19 is, for example, a hollow box having triangular sides, and supports the display device 11 and the optical unit 12 (12L, 12R) on its upper surface, and holds the half mirror 13 and the optical element 14 inside.

The display device 11 is a device that displays an image Im and has left-eye and right-eye display screens 11L and 11R. The left-eye and right-eye display screens 11L and 11R may be, for example, liquid crystal displays, and are supported by the housings of the optical units 12L and 12R described later and arranged on the left and right sides of the upper surface of the case 19. The display device 11 receives an image signal from the outside via a communication device (not shown) or reads an image signal stored in a storage device (not shown), and displays the left-eye and right-eye images ImL and ImR on the left-eye and right-eye display screens 11L and 11R, respectively. In this specification, the symbols L and R are added to the symbols indicating the left-eye and right-eye members, but these symbols are omitted when there is no need to distinguish between the left eye and the right eye. Instead of arranging the display screens separately for the left eye and the right eye, the display screen may be arranged as a common display screen for the left eye and the right eye.

The optical unit 12 is a unit that functions as an afocal objective lens that refracts light that forms the image Im, particularly parallel light beams, and emits them as parallel light beams, and has optical units 12L and 12R for the left and right eyes. The optical units 12L and 12R for the left and right eyes each include one or more optical elements such as lens elements, are held in a housing, and are arranged on the left and right sides of the top surface of the case 19. The optical units 12L and 12R have a focus on the input side (i.e., the +Z side) at the position where the images ImL and ImR are generated, i.e., on the display screens 11L and 11R, and form an exit pupil (not shown) on the output side (i.e., the -Z side) where the light that forms the images ImL and ImR is output.

The half mirror 13 and the optical element 14 are elements that direct the light output from the optical unit 12 in the +Y direction toward the user 99, and are held within the case 19.

The half mirror 13 is a semi-transparent mirror element, and is supported on an inclined surface of the case 19 facing the +Y and -Z directions. As a result, the half mirror 13 is disposed between the optical unit 12 and the optical element 14 in the optical path of the light output from the image Im, and reflects the light output from the optical unit 12 in the -Z direction toward the optical element 14 in the -Y direction, and transmits the light output from the optical element 14 in the +Y direction.

The optical element 14 is a retroreflective element in which, for example, a corner reflector having two plane mirrors at an angle of 90 degrees is arrayed on a plane, and is supported on the -Y surface of the case 19. As a result, the optical element 14 retroreflects in the +Y direction the light output from the optical unit 12 and reflected in the -Y direction by the half mirror 13, thereby relaying (i.e., reshaping) the exit pupils (not shown) of the optical units 12L and 12R to their conjugate positions, and forming eyepoints ExpL and ExpR at a position of, for example, 200 mm on the +Y side of the case 19. Here, the eyepoint Exp has, for example, a diameter (width in the X-axis and Z-axis directions) of the interpupillary width (approximately 50 mm) of the user 99, a depth (width in the Y-axis direction) of about 100 mm, or an angle of view of about 50 degrees.

In addition, instead of the half mirror 13 at the position where the half mirror 13 is disposed, a retrotransmissive element in which a laminated portion in which multiple reflective surfaces are stacked so that the multiple reflective surfaces intersect with each other may be used as the optical element 14. In such a case, the display device 11 and the optical unit 12 are disposed on the -Y side of the optical element 14, and light forming the image Im is output from the optical unit 12 in the +Y direction toward the optical element 14. As a result, the optical element 14 transmits the light output from the optical unit 12 retrotransmittently in the +Y direction, relaying (i.e., re-forming) the exit pupils (not shown) of the optical units 12L and 12R to their conjugate positions, and forming eyepoints ExpL and ExpR on the +Y side of the case 19.

In the display unit 10 configured as described above, when the left eye display screen 11L displays the left eye image ImL, the light forming the left eye image ImL is sent along the optical axes 10L, 10L' through the optical unit 12L, the half mirror 13, and the optical element 14, and is focused at the eye point ExpL. When the right eye display screen 11R displays the right eye image ImR, the light forming the right eye image ImR is sent along the optical axes 10R, 10R' through the optical unit 12R, the half mirror 13, and the optical element 14, and is focused at the position of the eye point ExpR. In this way, the display unit 10 forms eyepoints ExpL and ExpR for the left and right eyes of the user 99, respectively, so that when the user 99 places the left and right eyes (i.e., pupils, or simply eyes) 99L and 99R within the positions of the eyepoints ExpL and ExpR and places the eyepiece in midair, the user can view the entire angle of view of the left and right eye images ImL and ImR with the left and right eyes, respectively. Here, by displaying parallax images as the left and right eye images ImL and ImR, the user 99 can also view a stereoscopic image.

The detection unit 20 is a unit that detects the position and orientation of the eyes 99L, 99R of the user 99. As an example, the detection unit 20 has two stereo cameras 21 arranged at a distance from each other on the left and right in front of the upper surface of the case 19, and uses these to capture the head, particularly the face, of the user 99. The stereo camera 21 may be installed inside the case 19 and configured to capture the face of the user 99 through the half mirror 13. The images obtained by the two stereo cameras 21, particularly the parallax images, detect not only the left-right position (X position), up-down position (Z position), and front-back position (Y position) of the eyes 99L, 99R of the user 99, but also their orientations. These detection results are transferred to the guidance unit 30. When detecting the position and orientation of the eyes 99L, 99R of the user 99, as an example, the position and orientation of the pupils of the left eye and right eye of the user are detected.

In addition, instead of the stereo camera 21, the detection unit 20 may include a range finder that uses ultrasonic waves or infrared lasers to measure the distance to the user's head by a time-of-flight (ToF) method, thereby detecting the position and orientation of the user's 99 eyes 99L, 99R.

The guidance unit 30 is a unit that guides the eyes 99L, 99R of the user 99 relatively to the positions of the eyepoints ExpL, ExpR based on the positions of the eyes 99L, 99R and the eyepoint position of the user 99 detected by the detection unit 20, and in this embodiment has a display monitor 31 such as a liquid crystal display and a speaker 32 such as a directional speaker or a bone conduction speaker. The display monitor 31 is installed in a position where the user 99 can see the display, in this example, on the left side of the case 19. The speaker 32 is installed in a position where the user 99 can hear the sound, in this example, on the lower left side of the case 19.

Based on the detection result by the detection unit 20, the guidance unit 30 uses the display monitor 31 and the speaker 32 to visually and/or audibly guide the user 99 to move the position of the eyes 99L, 99R. For example, a video guide is displayed on the display monitor 31 and/or an audio guide is played by the speaker 32 to move the eyes 99L, 99R to the right (left) when the eyes 99L, 99R are to the left (right) of the eye points ExpL, ExpR, to lower (raise) the head when the eyes are above (below), to step back (step forward) when the eyes are forward (rear), and to look to the right (left) when the eyes are looking to the left (right).

Figure 2 shows the configuration of the control system 50 of the image display device 100. The control system 50 includes the display unit 10, the detection unit 20, and the guidance unit 30 described above. These are connected to each other so that they can send and receive signals.

Figure 3 shows flow S100 of the image display method according to the first embodiment. Note that flow S100 starts when the detection unit 20 detects the approach of a user 99 to the image display device 100.

In step S102, the detection unit 20 detects the positions of the eyes 99L, 99R of the user 99 relative to the positions of the eyepoints ExpL, ExpR. The detection unit 20 detects the X position, Y position, Z position, and orientation of the eyes 99L, 99R of the user 99 using the stereo camera 21, and transfers the detection results to the guidance unit 30.

In step S104, the detection unit 20 determines whether the positions of the eyes 99L, 99R of the user 99 are away from the positions of the eyepoints ExpL, ExpR based on the detection results obtained in step S102. At this time, for example, the detection unit 20 detects a relative positional deviation between the positions of the eyes 99L, 99R of the user 99 and the positions of the eyepoints ExpL, ExpR (for example, the center positions of the eyepoints ExpL, ExpR). If it is determined that the eyes 99L, 99R are away from the positions of the eyepoints ExpL, ExpR, i.e., that the positions of the eyes 99L, 99R and the positions of the eyepoints ExpL, ExpR are relatively misaligned, the process proceeds to step S106, and if it is determined that the eyes 99L, 99R are located within the positions of the eyepoints ExpL, ExpR, respectively, the process proceeds to step S108.

In step S106, the guidance unit 30 generates a video guide and/or an audio guide based on the detection result obtained in step S102 and the positions of the eyepoints ExpL and ExpR, and guides the eyes 99L and 99R of the user 99 relative to the positions of the eyepoints ExpL and ExpR. The guidance unit 30 displays a video guide on the display monitor 31 and/or plays an audio guide through the speaker 32 to, for example, move to the right (left) when the eyes 99L and 99R are located to the left (right) of the positions of the eyepoints ExpL and ExpR, lower (raise) the head when the eyes are located above (below), step back (step forward) when the eyes are located forward (rear), and turn right (left) when facing left (right).

After step S106 is executed, steps S102 to S106 are repeated until step S104 determines that the eyes 99L and 99R are located within the eyepoints ExpL and ExpR, respectively, that is, the user 99 has placed his or her eyes in the air.

In step S108, the display unit 10 displays images ImL and ImR as the user 99's eyes 99L and 99R enter the positions of eyepoints ExpL and ExpR, respectively. This allows the user 99 to view the entire angle of view of the left and right eye images ImL and ImR with his left and right eyes, respectively. Here, if the display unit 10 displays parallax images as the left and right eye images ImL and ImR, the user 99 can view a stereoscopic image. Note that if the eyepoints ExpL and ExpR are sufficiently large with respect to the user 99's eyes 99L and 99R, the display unit 10 may display images ImL and ImR after detecting in step S108 that one of the user's eyes 99L and 99R has entered the position of eyepoints ExpL and ExpR.

Note that flow S100 ends when the detection unit 20 detects that the user 99 has moved away from the image display device 100.

The image display device 100 of the first embodiment comprises a display unit 10 having an optical unit 12 having an input side focus at the position where an image Im is generated and forming an exit pupil on the output side from which light forming the image is output, and an optical element 14 which reflects or transmits light output from the optical unit to form an eyepoint Exp at a conjugate position of the exit pupil, a detection unit 20 which detects the position of the eyes 99L, 99R of a user 99, and a guidance unit 30 which generates a video guide and/or an audio guide based on the detection result of the detection unit and the position of the eyepoint, and guides the user's eyes relative to the position of the eyepoint. Since the image display device 100 does not have an eyepiece that is in physical contact, it is expected that some users will have difficulty placing their eyes on the display. However, the detection unit 20 detects the position of the user's eyes relative to the eyepoint position, and the guidance unit generates a video guide and/or audio guide based on the detection result to guide the user's eyes relatively to the eyepoint position, allowing the user to easily align their eyes within the eyepoint position formed in the air by the display unit (i.e., place the eye in the air) by following the guidance. In addition, the use of a retroreflective element or a retrotransparent element as the optical element 14 expands the eyepoint, increasing the degree of freedom in position setting when guiding the user's eyes relatively to the eyepoint position, making it easier to align the eyes within the eyepoint position.

4A, 4B, and 4C are schematic side, front, and top views of the configuration of the image display device 110 according to the second embodiment. The image display device 110, like the image display device 100 described above, is a device that displays an image Im so that the user 99 can see it without physically contacting the components, and includes a display unit 10, a detection unit 20, and a guidance unit 30d. The image Im may be a still image or a video, and may be a monochrome image or a color image. Also, with respect to the user 99 viewing the image Im, the left-right direction (user's eye width direction) is the X-axis direction, the front-back direction is the Y-axis direction, and the up-down direction is the Z-axis direction. The display unit 10 and the detection unit 20 are configured in the same manner as those in the image display device 100 described above.

Figure 5 shows the configuration of the control system 50 of the image display device 110. The control system 50 includes the display unit 10, the detection unit 20, and the guidance unit 30d described above. These are connected to each other so that they can send and receive signals.

The guidance unit 30d is a unit that guides the user's 99 eyes 99L, 99R relatively to the positions of the eyepoints ExpL, ExpR based on the detection result of the positions of the user's 99 eyes 99L, 99R by the detection unit 20, and has a haptic generation device 33 (33L, 33R) in this embodiment. The haptic generation device 33 guides the user's eyes 99L, 99R relatively to the positions of the eyepoints ExpL, ExpR by pressure including wind pressure or vibration including ultrasonic vibration. In particular, the haptic generation device 33 in this embodiment is a display device in which multiple ultrasonic transducers are arranged in an array, and thereby generates an intensity distribution (i.e., a sparse/dense distribution) of ultrasonic waves in the air to give a haptic sensation to at least one point on the surface of the human body located near the position of the eyepoint Exp. The haptic generation device 33 includes two haptic generation devices 33L and 33R that are arranged on the left and right sides of the top surface of the case 19, respectively, facing the positions of the eye points ExpL and ExpR.

As shown in Figures 4A to 4C, the haptic generation devices 33L and 33R generate haptics 98 at a position on the forehead of the user 99 that is slightly above the center of the eyepoints ExpL and ExpR, i.e., at a position where the eyes 99L and 99R are aligned within the positions of the eyepoints ExpL and ExpR, respectively. This allows the user 99 to know the positions of the eyepoints ExpL and ExpR through the haptics 98 felt on the forehead and align the eyes 99L and 99R to them.

The haptic generation devices 33L, 33R may control the position at which the haptic 98 is generated and its intensity according to the position of the user's 99's eyes 99L, 99R relative to the positions of the eyepoints ExpL, ExpR, particularly the position in the Y-axis direction. Figures 6A to 6C show an example of controlling the generation position and intensity of the haptic 98. If the detection unit 20 does not detect the user 99 or detects it sufficiently far away, the haptic generation devices 33L, 33R do not generate the haptic 98.

As shown in FIG. 6A, when the detection unit 20 detects the approach of the user 99 to the eye point Exp, the guidance unit 30d generates a haptic sensation 98 using the haptic sensation generating devices 33L and 33R. Here, since the user 99 is still away from the image display device 100, the haptic sensation generating devices 33L and 33R generate a weak haptic sensation on the forehead of the user 99.

6B, when the user 99 approaches the image display device 100, the haptic generation devices 33L and 33R move the haptic 98 toward the eye point Exp in accordance with the movement of the user 99 based on the detection result of the position of the user 99 by the detection unit 20, and increase the strength of the haptic 98. As a result, the user 99 receives a haptic 98 on the forehead that becomes stronger as the eyes 99L and 99R approach the positions of the eye points ExpL and ExpR.

As shown in FIG. 6C, when the user 99 reaches the image display device 100 and the eyes 99L, 99R enter the positions of the eyepoints ExpL, ExpR, respectively, the haptic generating devices 33L, 33R detect the position of the user 99 using the detection unit 20 and further increase the intensity of the haptic 98 (or increase it to the maximum intensity). In this way, as the eyes 99L, 99R of the user 99 approach the positions of the eyepoints ExpL, ExpR, the position at which the haptic 98 is generated is moved in the Y-axis direction to match the position of the user 99's forehead and the intensity of the haptic 98 is increased, so that the user 99 can know the positions of the eyepoints ExpL, ExpR through the haptic 98 and can easily place the eyes 99L, 99R at the positions of the eyepoints ExpL, ExpR.

The induction unit 30d may change the intensity of the haptic 98 according to the attributes of the user 99 using the haptic generating devices 33L and 33R. The user attributes include any one of age, sex, and weight. The detection unit 20 detects the user attributes, and according to the detection result, the induction unit 30d reduces the intensity of the haptic 98 for women and children, for example, and increases the intensity for a large user.

In addition, instead of generating one haptic 98 at the forehead of the user 99, the haptic generating devices 33L, 33R may generate two haptics 98L, 98R at the left and right temples or cheeks of the user 99 as described below, and move the positions at which they are generated in the Y-axis direction and control the intensity according to the positions of the eyes 99L, 99R of the user 99 relative to the positions of the eyepoints ExpL, ExpR.

Based on the orientation of the user's 99's eyes 99L, 99R and the position of eyepoint Exp (ExpL, ExpR) detected by the detection unit 20, the haptic generation devices 33L, 33R may generate haptics 98L, 98R to the left and/or right of the position of eyepoint Exp (ExpL, ExpR), i.e., at the positions of the left and right temples of the user 99 with the eyes 99L, 99R aligned within the positions of the eyepoints ExpL, ExpR, respectively, and may further control the intensity of each of the haptics 98L, 98R. Figures 7A and 7B show another example of controlling the generation position and intensity of the haptics 98.

As shown in FIG. 7A, when the detection unit 20 detects that the eyes 99L, 99R of the user 99 are facing leftward (white arrow) relative to the positions of the eyepoints ExpL, ExpR, the guidance unit 30d generates a weak (or strong) haptic 98L and a strong (or weak) haptic 98R using the haptic generation devices 33L, 33R. This causes the user 99 to consciously rotate his/her head to the right (black arrow). As a result, the user 99 can turn his/her head to the right through the haptics 98L, 98R, thereby placing the eyes 99L, 99R within the positions of the eyepoints ExpL, ExpR, respectively.

As shown in FIG. 7B, when the detection unit 20 detects that the eyes 99L, 99R of the user 99 are facing to the right (white arrow) relative to the eyepoints ExpL, ExpR, the guidance unit 30d generates a strong (or weak) haptic 98L and a weak (or strong) haptic 98R using the haptic generation devices 33L, 33R. This causes the user 99 to be conscious of rotating his/her head to the left (black arrow). As a result, the user 99 can turn his/her head to the left through the haptics 98L, 98R, thereby placing the eyes 99L, 99R within the positions of the eyepoints ExpL, ExpR, respectively.

In this way, the haptic generating devices 33L, 33R generate haptics 98L, 98R to the left and right of the positions of the eyepoints ExpL, ExpR, respectively, to provide haptics to the left and right temples of the user 99, and by controlling the strength of these haptics 98L, 98R based on the detection results of the orientation of the user 99's eyes 99L, 99R, the user 99 can determine the orientation of the eyes 99L, 99R through the strength of the haptics felt at the left and right temples, and place them within the positions of the eyepoints ExpL, ExpR.

The rotation of the user's 99 head by controlling the generation position and strength of the haptics 98L and 98R described with reference to Figures 7A and 7B may be performed following the guidance of the user's 99 to the eye point Exp by controlling the generation position and strength of the haptics 98 described with reference to Figures 6A to 6C. For example, the position at which the haptics 98 is generated and its strength may be controlled according to the position of the user 99 in the X-axis direction. In this case, if the user's 99 eyes 99L and 99R are shifted in the -X direction relative to the positions of the eye points ExpL and ExpR, the haptic generation devices 33L and 33R generate the haptics 98L strongly (or weakly) and the haptics 98R weakly (or strongly). The user 99 can move in the +X direction through the haptics 98L and 98R to bring the eyes 99L and 99R into the positions of the eye points ExpL and ExpR, respectively. The haptic generation devices 33L and 33R may be movable or tiltable in the X-axis, Y-axis, and Z-axis directions by a driving device (not shown). This allows the user 99 to reliably feel the haptic sensation of wind pressure, pressure, etc. by driving the haptic generation devices 33L and 33R to generate wind pressure, pressure, etc. in the direction of the user 99 when the user 99 is away from the image display device 100.

When the detection unit 20 detects that the eyes 99L, 99R of the user 99 have entered the positions of the eyepoints ExpL, ExpR, for example, that the eyes 99L, 99R have remained within the positions of the eyepoints ExpL, ExpR for longer than a predetermined time, the guidance unit 30d stops the tactile generation devices 33L, 33R. Following this, the display unit 10 may display the image Im.

Figure 8 shows flow S200 of the image display method according to the second embodiment. Note that flow S200 starts when the detection unit 20 detects the approach of a user 99 to the image display device 110.

In step S202, the detection unit 20 detects the position (X position, Y position, and Z position) and orientation of the user's 99 eyes 99L and 99R relative to the position of the eye point Exp (ExpL, ExpR). The detection result is transferred to the guidance unit 30d.

In step S204, the detection unit 20 determines whether the positions of the eyes 99L, 99R of the user 99 are away from the positions of the eyepoints ExpL, ExpR based on the detection results obtained in step S202. If it is determined that the eyes 99L, 99R are away from the positions of the eyepoints ExpL, ExpR, the process proceeds to step S206, and if it is determined that the eyes 99L, 99R are located within the positions of the eyepoints ExpL, ExpR, respectively, the process proceeds to step S208.

In step S206, the guidance unit 30d generates a tactile guide using the tactile generating devices 33L and 33R based on the detection results obtained in step S202 and the positions of the eyepoints ExpL and ExpR, and guides the eyes 99L and 99R of the user 99 relatively to the positions of the eyepoints ExpL and ExpR.

After step S206 is executed, steps S202 to S206 are repeated until step S204 determines that the eyes 99L and 99R are located within the eyepoints ExpL and ExpR, respectively, that is, the user 99 has placed his or her eyes in the air.

As a result, when the detection unit 20 detects that the eyes 99L, 99R of the user 99 are approaching the position of the eye point Exp, the guidance unit 30d generates a haptic 98 by the haptic generation devices 33L, 33R as shown in FIG. 6A. When the detection unit 20 detects that the user 99 is approaching the image display device 100, the guidance unit 30d moves the haptic 98 toward the position of the eye point Exp in accordance with the movement of the user 99 based on the detection result of the position of the user 99 by the detection unit 20 and increases its intensity as shown in FIG. 6B. When the detection unit 20 detects that the user 99 has reached the image display device 100 and that the eyes 99L, 99R have reached the positions of the eye points ExpL, ExpR, the guidance unit 30d further increases the intensity of the haptic 98 (or increases it to the maximum intensity) by the haptic generation devices 33L, 33R by detecting the position of the user 99 by the detection unit 20.

In this way, the tactile generating devices 33L, 33R move the position at which the tactile 98 is generated to match the position of the user 99's forehead as the user's 99 eyes 99L, 99R approach the positions of the eyepoints ExpL, ExpR, and increase the intensity of the tactile 98, allowing the user 99 to know the position of the eyepoint Exp through the tactile 98 and to move the eyes 99L, 99R closer to the positions of the eyepoints ExpL, ExpR.

Furthermore, when the detection unit 20 detects that the eyes 99L, 99R of the user 99 have reached the positions of eyepoints ExpL, ExpR but are facing left or right relative to them, the guidance unit 30d generates a weak or strong haptic 98L and a strong or weak haptic 98R using the haptic generation devices 33L, 33R as shown in Figures 7A and 7B. This causes the user 99 to be conscious of turning his head to the right or left. As a result, the user 99 can turn his head to the right or left through the haptics 98L, 98R, thereby placing the eyes 99L, 99R within the positions of the eyepoints ExpL, ExpR, respectively.

In this way, the haptic generating devices 33L, 33R generate haptics 98L, 98R to the left and right of the positions of the eyepoints ExpL, ExpR, respectively, to provide haptics to the left and right temples of the user 99, and by controlling the strength of these haptics 98L, 98R based on the detection results of the orientation of the user 99's eyes 99L, 99R, the user 99 can determine the orientation of the eyes 99L, 99R through the strength of the haptics felt at the left and right temples, and place them within the positions of the eyepoints ExpL, ExpR.

In step S208, when the detection unit 20 detects that the eyes 99L, 99R of the user 99 have entered the positions of eyepoints ExpL, ExpR, respectively, the guidance unit 30d stops the tactile generation devices 33L, 33R and ends the tactile guide. When the detection unit 20 detects that the eyes 99L, 99R of the user 99 have entered the positions of eyepoints ExpL, ExpR, respectively, the tactile generation devices 33L, 33R may intermittently change the wind pressure or pressure. The user 99 can recognize that the eyes 99L, 99R have entered the positions of eyepoints ExpL, ExpR by sensing the intermittent changes in wind pressure or pressure as a touch.

In step S210, the display unit 10 displays the image Im (ImL, ImR). This allows the user 99 to view the entire angle of view of the left and right eye images ImL, ImR with the left and right eyes, respectively. Here, if the display unit 10 displays parallax images as the left and right eye images ImL, ImR, the user 99 can view a stereoscopic image.

Note that flow S200 ends when the detection unit 20 detects that the user 99 has moved away from the image display device 110.

The image display device 110 according to the second embodiment includes a display unit 10 having an optical unit 12 that has a focus on the input side at the position where an image Im is generated and forms an exit pupil on the output side where light forming the image is output, an optical element 14 that reflects or transmits the light output from the optical unit to form an eye point Exp at a conjugate position of the exit pupil, a detection unit 20 that detects the position of the user's eyes 99L, 99R, and a guidance unit 30d that generates a tactile guide based on the detection result of the detection unit and the position of the eye point and guides the user's eyes relatively to the position of the eye point. Since the image display device 100 does not have an eyepiece unit that makes physical contact, it is expected that some users will have difficulty placing their eyes on the display unit. However, the detection unit 20 detects the position of the user's eyes relative to the position of the eye point, and the guidance unit generates a tactile guide based on the detection result to guide the user's eyes relatively to the position of the eye point, so that the user can easily align their eyes within the position of the eye point formed in the air by the display unit according to the guidance (i.e., placing their eyes in the air).

9A, 9B, and 9C are schematic side, front, and top views of the configuration of the image display device 120 according to the third embodiment. The image display device 120, like the image display devices 100 and 110 described above, is a device that displays an image Im so that the user 99 can see it without physically contacting the components, and includes a display unit 10, a detection unit 20, and a guidance unit 30dd. The image Im may be a still image or a video, and may be a monochrome image or a color image. In addition, the left-right direction (user's eye width direction) is the X-axis direction, the front-back direction is the Y-axis direction, and the up-down direction is the Z-axis direction, based on the user 99 viewing the image Im. The display unit 10 and the detection unit 20 are configured in the same manner as those in the image display devices 100 and 110 described above. However, the case 19 of the display unit 10 is fixed onto the wall surface W, and the optical unit 12 (12L, 12R) is supported on the case 19 facing downwards through an opening (not shown) provided on the top surface of the case 19 by the driving devices 34, 35, and 36.

Figure 10 shows the configuration of the control system 50 of the image display device 120. The control system 50 includes the display unit 10, the detection unit 20, and the guidance unit 30dd described above. These are connected to each other so that they can send and receive signals.

The guidance unit 30dd is a unit that guides the eyes 99L, 99R of the user 99 relatively to the positions of the eye points ExpL, ExpR based on the detection results of the positions of the eyes 99L, 99R of the user 99 by the detection unit 20, and in this embodiment includes driving devices 34, 35, and 36.

The drive device 34 drives the optical unit 12 in the XY directions, and in this embodiment in particular drives the left-eye and right-eye optical units 12L, 12R as a unit, and has an X drive device 34X and a Y drive device 34Y. For example, a linear motor may be used as the X drive device 34X and the X drive device 34X. The X drive device 34X holds the ±Y ends of each of the display devices 11L, 11R to which the optical units 12L, 12R are fixed, and drives them in the X-axis direction. The Y drive device 34Y holds the ±X ends of the X drive device 34X and drives it in the Y-axis direction.

The driving device 35 is a device that drives the driving device 34 in the Z-axis direction. For example, a linear motor may be used as the driving device 35. The driving device 35 is fixed onto the upper surface of the case 19, and holds the -Y end of the driving device 34 to drive it in the Z-axis direction.

The guidance unit 30dd controls the driving devices 34, 35 based on the detection results by the detection unit 20 to drive the optical units 12L, 12R in a direction parallel to and/or perpendicular to the optical axes 10L, 10R (parallel to the Z axis). When the optical units 12L, 12R are driven in the Z axis direction, the positions of the eye points ExpL, ExpR move in the Y axis direction along the optical axes 10L', 10R' (parallel to the Y axis) (see the arrow (solid line) shown in Figure 9A). When the optical units 12L, 12R are driven in the Y axis direction, the positions of the eye points ExpL, ExpR move in the Z axis direction (see the arrow (dotted line) shown in Figure 9A). When the optical units 12L, 12R are driven in the X axis direction, the positions of the eye points ExpL, ExpR move in the X axis direction (see the arrows shown in Figures 9B and 9C).

As shown in FIG. 11, the guidance unit 30dd may further include a drive device 36 that tilts the half mirror 13 relative to the optical axes 10L', 10R' (parallel to the Y-axis). For example, a rotary motor may be used as the drive device 36. The guidance unit 30dd moves the positions of the eye points ExpL, ExpR up and down by tilting the half mirror 13 up and down using the drive device 36 based on the detection results by the detection unit 20 (see the arrows shown in FIG. 11). For example, when a short user 99 such as a child or infant is detected, the half mirror 13 is rotated downward to lower the positions of the eye points ExpL, ExpR. Furthermore, the guidance unit 30dd may move the positions of the eye points ExpL, ExpR left and right by swinging the half mirror 13 left and right. Based on the detection results from the detection unit 20, the guidance unit 30dd can drive the optical unit 12 (12L, 12R) in the X-axis, Y-axis, and Z-axis directions and/or tilt the half mirror 13 to move the positions of the eye points ExpL and ExpR forward/backward, up/down, and left/right to match the positions of the eyes 99L and 99R of the user 99.

Figure 12 shows a flow S300 of the image display method according to the third embodiment. Note that flow S300 starts when the detection unit 20 detects the approach of a user 99 to the image display device 120.

In step S302, the detection unit 20 detects the position (X position, Y position, and Z position) and orientation of the user's 99 eyes 99L and 99R relative to the position of the eye point Exp (ExpL, ExpR). The detection result is transferred to the guidance unit 30dd.

In step S304, the detection unit 20 determines whether the positions of the eyes 99L, 99R of the user 99 are away from the positions of the eyepoints ExpL, ExpR based on the detection results obtained in step S302. If it is determined that the eyes 99L, 99R are away from the positions of the eyepoints ExpL, ExpR, the process proceeds to step S306, and if it is determined that the eyes 99L, 99R are located within the positions of the eyepoints ExpL, ExpR, respectively, the process proceeds to step S308.

In step S306, the guidance unit 30dd controls the driving devices 34, 35, and 36 based on the detection result obtained in step S302 and the positions of the eye points ExpL and ExpR to drive the optical unit 12 (12L, 12R) and tilt the optical element 14, thereby guiding the eyes 99L, 99R of the user 99 relatively to the positions of the eye points ExpL, ExpR.

After step S306 is executed, steps S302 to S306 are repeated until step S304 determines that the eyes 99L and 99R are located within the eyepoints ExpL and ExpR, respectively, that is, the user 99 has placed his or her eyes in the air.

As a result, the guiding unit 30dd moves the positions of the eyepoints ExpL and ExpR in the Y-axis direction along the optical axes 10L' and 10R' (parallel to the Y-axis) by driving the optical units 12L and 12R in the Z-axis direction (see the arrow (solid line) in FIG. 9A), moves the positions of the eyepoints ExpL and ExpR in the Z-axis direction by driving the optical units 12L and 12R in the Y-axis direction (see the arrow (dotted line) in FIG. 9A), and moves the positions of the eyepoints ExpL and ExpR in the X-axis direction by driving the optical units 12L and 12R in the X-axis direction (see the arrows in FIG. 9B and FIG. 9C). Furthermore, the guiding unit 30dd moves the positions of the eyepoints ExpL and ExpR up and down by tilting the optical element 14 up and down with respect to the optical axes 10L' and 10R' (parallel to the Y-axis) (see the arrows in FIG. 11).

In this way, the driving devices 34, 35, 36 can drive the optical unit 12 (12L, 12R) in the X-axis, Y-axis, and Z-axis directions and/or tilt the optical element 14 to move the positions of the eye points ExpL, ExpR forward/backward, up/down, and left/right to match the positions of the eyes 99L, 99R of the user 99.

In step S308, when the detection unit 20 detects that the eyes 99L, 99R of the user 99 are within the positions of the eye points ExpL, ExpR, respectively, the guidance unit 30dd stops the driving devices 34, 35, 36 to position the optical unit 12 (12L, 12R) and the optical element 14.

In step S310, the display unit 10 displays the image Im (ImL, ImR). This allows the user 99 to view the entire angle of view of the left and right eye images ImL, ImR with the left and right eyes, respectively. Here, if the display unit 10 displays parallax images as the left and right eye images ImL, ImR, the user 99 can view a stereoscopic image.

Note that flow S300 ends when the detection unit 20 detects that the user 99 has moved away from the image display device 120.

The image display device 120 of the third embodiment comprises a display unit 10 having an optical unit 12 having an input side focus at the position where an image Im is generated and forming an exit pupil on the output side from which light forming the image is output, and an optical element 14 which reflects or transmits light output from the optical unit to form an eyepoint Exp at a conjugate position of the exit pupil, a detection unit 20 which detects the position of the eyes 99L, 99R of a user 99 relative to the position of the eyepoint, and a guidance unit 30dd which controls drive devices 34, 35, 36 based on the detection result of the detection unit to drive the optical unit and/or tilt the optical element to guide the user's eyes relative to the position of the eyepoint. Since the image display device 100 does not have an eyepiece that is in physical contact, it is expected that some users will have difficulty placing their eyes on it. However, the detection unit 20 detects the position of the user's eyes, and the guidance unit drives the optical unit and/or tilts the optical element based on the detection result and the eyepoint position to guide the user's eyes relative to the eyepoint position, allowing the user to easily align their eyes within the eyepoint position formed in the air by the display unit (i.e., placing the eye on the screen in the air).

The image display devices 100, 110, 120 configured as described above may be configured to receive information input by a user using the detection unit 20. The detection unit 20 may use the stereo camera 21 to detect facial movements of the user 99, such as nodding or shaking of the head, finger gestures, such as finger shape and movement, eye size, line of sight, changes in facial expression, and the number of blinks, thereby accepting input of confidential information, such as a PIN number. The detection unit 20 may also detect the face and irises to authenticate the user.

Also, the image display device 100 excluding the detection unit 20 shown in FIG. 1A to FIG. 1C may be arranged inverted with respect to the XY plane, or the image display device 100 including the detection unit 20 may be arranged inverted with respect to the XY plane. Also, instead of guiding the user's eye relatively to the eyepoint position based on the position of the user's eye and the position of the eyepoint, the user's eye may be relatively guided within the pupil area using the position of the user's eye and the pupil area formed by relaying (i.e., re-forming) the exit pupils (not shown) of the optical units 12L and 12R to their conjugate positions (the position of the eyepoint). For example, if the pupil area is sufficiently large, as long as the user's left eye and right eye are each within the respective left and right pupil areas formed at the position of the eyepoint, the user can view the image Im (ImL, ImR) displayed by the display unit 10 without vignetting even if the left eye and right eye are not located at the center of the left and right pupils formed at the position of the eyepoint.

The present invention has been described above using an embodiment, but the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiment. It is clear from the claims that forms with such modifications or improvements can also be included in the technical scope of the present invention.

The order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and it should be noted that the processes may be performed in any order, unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is explained using "first," "next," etc. for convenience, it does not mean that it is necessary to perform the processes in this order.

10...display unit, 10L, 10R...optical axis, 11...display device, 11L...display screen for left eye, 11R...display screen for right eye, 12...optical unit, 12L...optical unit for left eye, 12R...optical unit for right eye, 13...half mirror, 14...optical element, 19...case, 20...detection unit, 21...stereo camera, 30, 30d, 30dd...guidance unit, 31...display monitor, 32...speaker, 33...tactile Generation device, 33, 33L, 33R...tactile generation device, 34, 35, 36...drive device, 34X...X drive device, 34Y...Y drive device, 50...control system, 98...tactile sense, 98L, 98R...tactile sense, 99...user, 99L, 99R...eyes, 100, 110, 120...image display device, Exp, ExpL, ExpR...eye point, Im...image, ImL...image for left eye, ImR...image for right eye, W...wall surface.

Claims (17)

a display unit having an optical unit having an input side focal point at a position where an image is generated and forming an exit pupil at an output side from which light forming the image is output, and an optical element that reflects or transmits the light output from the optical unit to form an eyepoint at a conjugate position of the exit pupil;
A detection unit that detects the position of a user's eyes;
A guidance unit that guides the user's eyes relatively to the position of the eye point based on the position of the user's eyes and the position of the eye point detected by the detection unit;
Equipped with
The guidance unit includes a tactile sense generating device that generates a tactile sense at least at one location near the position of the eyepoint . The image display device according to claim 1 , wherein the haptic sense generating device guides the user's eye relative to the position of the eye point by pressure or vibration. The image display device according to claim 1 , wherein the haptic sense generating device generates a haptic sense above the position of the eyepoint. The image display device according to claim 3 , wherein the haptic sense generating device controls a position or intensity at which a haptic sense is generated depending on a position of the user's eye relative to the position of the eyepoint. The image display device according to claim 4 , wherein the haptic sense generating device changes an intensity of the haptic sense in accordance with an attribute of the user, the attribute of the user including any one of age, sex, and weight. The detection unit further detects an orientation of the user's eyes,
The image display device according to claim 1 , wherein the haptic sense generating device generates a haptic sense to the left or right of the eye point position based on the user's eye direction and eye point position detected by the detection unit. 6. The image display device according to claim 1, wherein the detection unit further detects the direction of the user's eyes, and the tactile generation device generates tactile sensations to the left and right of the eye point position, and controls the intensity of each tactile sensation based on the direction of the user's eyes and the eye point position detected by the detection unit. The image display device according to claim 1 , wherein the guidance unit generates a tactile sensation by the tactile sensation generating device when the detection unit detects that the user's eye is approaching the position of the eyepoint . The image display device according to claim 1 , wherein the guidance unit stops the haptic sense generating device when the detection unit detects that the user's eye has entered within the position of the eyepoint . 10. The image display device according to claim 1, wherein the guidance unit includes a driving device that drives the optical unit, and controls the driving device based on a detection result by the detection unit to drive the optical unit in a direction parallel to and / or a direction intersecting the optical axis. The detection unit further detects an orientation of the user's eyes,
The image display device according to claim 10 , wherein the guidance section controls the drive device based on a detection result by the detection section to tilt the optical element with respect to an optical axis. The image display device according to claim 10 , wherein the driving device drives the optical units corresponding to the left and right eyes of a user as a unit. The image display device according to claim 1 , wherein the guiding unit guides the user's eye relative to the position of the eye point by visual and/or audio. the optical element is a retroreflective element that reflects light output from the optical unit,
The image display device according to claim 1 , wherein the display unit further comprises a half mirror between the optical element and the optical unit in an optical path of the light. 14. The image display device according to claim 1, wherein the optical element is a retrotransmissive element in which a laminated portion, in which a plurality of reflective surfaces are laminated, is laminated such that the plurality of reflective surfaces intersect with each other. The image display device according to claim 1 , wherein the display unit forms two eyepoints for a left eye and a right eye of a user. detecting a position of a user's eyes;
a step of guiding the user's eye relatively to the position of the eyepoint based on the position of the user's eye detected in the detection step, and the position of an eyepoint formed in the air by an optical unit having an input side focus at a position where an image is generated and forming an exit pupil on an output side from which light forming the image is output, and an optical element that reflects or transmits the light output from the optical unit to form an eyepoint at a conjugate position of the exit pupil;
displaying the image;
Equipped with
The image display method , in the guiding step, further comprises generating a haptic sensation at at least one location in the vicinity of the eyepoint position .

Images