JPH0819004A - Head mount display device - Google Patents

Abstract

PURPOSE:To allow the user to enjoy an image under an environment close to that of a conventional television receiver and to view a scenery of an external field as required. CONSTITUTION:A scenery image picked up by video cameras 24R, 24L, and an image desired to be watched such as a movie picture or a television program entered externally are synthesized by a synthesis switch 32 and displayed on liquid crystal display devices 14R, 14L. When the viewer directs its face as required into a direction of the display device, the motion is sensed by a sensor and a read control circuit 48, a rotary angle detection circuit 50 and an image rotation circuit 52 provide a motion in the reverse direction to a watched image. Thus, an environment close to that of a stationary television receiver is realized for the display device.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a head mounted display which a viewer wears on his or her head.

[0002]

2. Description of the Related Art Image display units are provided close to both eyes of a viewer, and an image display device mounted on a head for use, a so-called head mounted display (H
Examples of MD) include those disclosed in JP-A-4-61495 and JP-A-5-183943. What is disclosed in these is a pair of image display units that have a spectacles-like appearance and are configured to be worn on the head so as to be worn over the ears, and that face the left and right eyes. There is. As disclosed in Nikkei Electronics, 1993, 14, No. 571, there is also a headband that is worn on the head.

A liquid crystal element, an EL element, or the like is used as the image display section, which is provided in front of both eyes. By displaying the left-eye image and the right-eye image respectively on the left and right display portions, it is possible to enjoy a stereoscopic image. There is also a transmissive HMD in which a lens for reflecting an image is placed inside a spectacle-shaped screen so that the image displayed on the upper part of the screen is reflected and projected on the screen for viewing.
This type of image can be seen through the screen at the same time as the outside, together with the image on the screen.

[0004]

However, the above background art has the following disadvantages. (1) The non-transparent type is convenient for enjoying only the image, but since the outside world is blocked, the HMD must be removed once to see the outside world, which is not necessarily convenient. I can't say. On the other hand, the transmissive type does not necessarily cause the above-described inconvenience because the external scenery and the viewed image appear to overlap each other. However, even if the head moves, the scenery outside moves, but the viewing image occupies a certain position in the field of view. For this reason, when it is necessary to see both the scenery required for the outside world and the viewing image, it is necessary to move and fix the head so that both of them can be seen.

(2) The position of the display image in the visual field of both eyes is fixed regardless of whether it is a non-transmissive type or a transmissive type. Therefore, even if the head is moved, the viewing image is always located at a fixed position within the visual field. On the other hand, when watching television in the room, turning the head and looking in a different direction causes the image on the television to disappear from view.
As is clear from a comparison between this and the HMD, the two environments differ greatly in the environment in which the image is viewed. That is, since it is impossible to avoid viewing images with an HMD, it is considered that the degree of oppression and fatigue when viewing for a long time increases.

The present invention focuses on the above points,
It is an object of the present invention to provide a user-friendly head mounted display capable of satisfactorily seeing a desired landscape of the outside world as needed. Another object is to provide a head-mounted display that allows users to enjoy images in an environment similar to that of a conventional television and has less fatigue even when viewed for a long time.

[0007]

In order to achieve the above-mentioned object, the present invention detects the movement of the main body of the device by a sensor, and according to the detection result, the relative display position of the viewing image with respect to the background image. Since a reverse movement is applied to the display, a display environment similar to that of a stationary television is realized even though it is a head mount type. According to a main aspect of the invention, a microphone is provided for obtaining external sound. Then, the sound mixing means mixes the external sound and the sound of the viewing image in accordance with the movement of the horizontal sensor so that the sound image position of the viewing image coincides with the image position.
The above and other objects, features and advantages of the present invention will be apparent from the following detailed description and the accompanying drawings.

[0008]

Description of the Preferred Embodiments Although there may be many embodiments of the head mounted display of the present invention, a suitable number of embodiments will now be shown and described in detail.

<First Embodiment> FIG. 1 shows a plan view of mechanical components of the first embodiment. As shown in the figure,
The whole body is goggle shape, and the body 1 on the front side of the head
Image display units 12R and 12L are provided at 0 so as to be positioned close to both eyes when worn. The image display units 12R and 12R are respectively configured by liquid crystal displays 14R and 14L and backlights 16R and 16L. In addition, the image display unit 12
Eyepiece lenses 18R and 18L are provided on the naked eye sides of R and 12L, respectively. Wearing arms 20R and 20L corresponding to temples of spectacles are extended on both sides of the main body 10, respectively. Speakers 22R and 22L are provided inside the ends of the wearing arms 20R and 20L, respectively.

Further, in the first embodiment, the left and right end portions of the front surface of the main body 10 have a video camera 24R facing forward (to the left in the drawing).
24L are provided, and vertical sensors 26R and 26L are provided inside the left and right ends, respectively. A horizontal sensor 28 is provided at the center of the main body 10.

Of these, the video cameras 24R, 24
L is for imaging the visual field ranges of the right eye and the left eye, respectively. The vertical sensors 26R and 26L are sensors for detecting the relative position of the visual field in the vertical direction, and the horizontal sensor 28 is a sensor for detecting the relative position of the visual field in the horizontal direction. In addition, the vertical sensors 26R, 2
The rotation of the visual field can be detected from the 6L detection output.

Next, the electrical components of the first embodiment are as shown in FIG. It should be noted that the circuit block shown in the figure has only one of the left and right circuits.
One similar circuit is provided. The basic operation, that is, the operation in the case where the above-described sensors 26R, 26L, 28 are not taken into consideration, will be described. R, G, B image signals of a landscape imaged by the video cameras 24R, 24L are generated by the synchronization signal generation circuit 30. It is output based on the synchronization signal supplied from and is supplied to the synthesizing switch 32.

On the other hand, an external input image signal such as a movie is color demodulated by the color demodulation circuit 34, and then the A / D converter 3
At 6, it is converted into a digital signal. Then, it is stored in the memory 38 in synchronization with the input image signal. At this time, by adjusting the writing control circuit 40, the pixels of the image stored in the memory 38 are thinned out or interpolated, so that the image can be enlarged or reduced or the aspect ratio can be changed according to the viewer's desire. Is done.

The image signal stored in the memory 38 is read out at the same timing as the synchronizing signal of the video camera 24, converted into an analog signal by the D / A converter 42, and then supplied to the synthesizing switch 32. Synthesis switch 3
In 2, the landscape image supplied from the video camera 24 side and the viewing image such as a movie are combined.

This is shown in FIG. The video cameras 24R and 24L respectively capture landscape images PBR and PBL as shown in FIG.
The synthesizing switch 32 performs a synthesizing process in which the viewing image is prioritized over the background image, that is, a process of outputting a region having a viewing image on one screen and outputting a background image of a region having no viewing image. Done. Therefore, as shown in FIG. 6B, the viewing images PMR and PML are combined with the landscape images PBR and PBL, respectively. This is a picture-in-picture on a television or VTR.
It can be compared to the function of a picture. The combined image signals for the left and right eyes are displayed on the liquid crystal displays 14R and 14L.
Is supplied to each.

On the other hand, liquid crystal displays 14R and 14L
The display scanning is performed by the scanning timing signal supplied from the horizontal / vertical timing circuit 44. The horizontal / vertical timing circuit 44 includes a synchronizing signal generating circuit 3
Since the operation is performed based on the synchronization signal supplied from 0, the scanning of the liquid crystal displays 14R and 14L is also performed in synchronization with the image signal output of the video cameras 24R and 24L. That is, the circuit operations from the image signal output from the video cameras 24R and 24L and the image signal reading from the memory 38 to the display on the liquid crystal displays 14R and 14L are all performed in synchronization.

In the liquid crystal displays 14R and 14L, the light from the backlights 16R and 16L driven by the backlight driving device 46 is received, and a composite image for the left and right eyes shown in FIG. 3B is displayed. . These composite images are observed by both eyes of the viewer through the eyepiece lenses 18R and 18L.

By the way, in the first embodiment, the detection control side of the above-mentioned sensors 26R, 26L and 28 is the read control circuit 48.
The read control circuit 48 controls the data read from the memory 38. Further, the detection output sides of the vertical sensors 26R and 26L are connected to the rotation angle detection circuit 50, and the output side of the rotation angle detection circuit 50 is connected to the image rotation circuit 52. The image rotation circuit 52 is for rotating and outputting the image supplied from the memory 38.

When these operations are described, that is, when the HMD shown in FIG. 1 moves in the horizontal direction, this is detected by the horizontal sensor 28. The read control circuit 48 controls the signal read timing from the memory 38 in response to this horizontal movement. As a result, the video camera 24
The display timing of the viewing image with respect to the background image sent from R and 24L is displaced, and as a result, the relative display position is displaced. The read control circuit 48 controls the data read from the memory 38 so that the moving direction of the viewing image is opposite to the moving direction detected by the horizontal sensor 28. This state is shown in FIG. 4A, and if the detection movement direction of the horizontal sensor 28 is the arrow FA, the liquid crystal displays 14R, 14 will be described.
The viewing images PMR and PML on L are indicated by opposite arrows FB.
Will move in the direction.

Next, if the HMD is moved in the vertical direction, this is detected by the vertical sensors 26R and 26L. The read control circuit 48 responds to this vertical movement in the memory 3
The timing of signal reading from 8 is controlled. Also in this case, the read control circuit 48 controls the data read from the memory 38 so that the moving direction of the viewing image is opposite to the moving direction detected by the vertical sensors 26R and 26L. This is shown in FIG. 4 (B),
The direction of detection and movement of the vertical sensors 26R and 26L8 is arrow F.
If it is C or FD, the liquid crystal display 14R,
The viewing images PMR and PML on 14L move in the opposite arrow FE directions by their average movement amounts.

Further, at this time, the vertical sensors 26R, 26
From the detection result of L, the rotation angle detection circuit 50 shown in FIG.
The rotation angle θ of the HMD is detected. For example, in FIG.
As shown in (C), if the detection direction of the vertical sensor 26R is the arrow FF and the movement amount is Δa, and the detection direction of the vertical sensor 26L is the arrow FG is the movement amount Δb, the distance between the vertical sensors is S, and θ = The rotation angle θ is obtained by calculating tan ⁻¹ ((Δa + Δb) / S). The image rotation circuit 52 rotates the image corresponding to −θ which is the reverse of the rotation angle θ. As a result, the liquid crystal display 14
In R and 14L, as shown in FIG.
MR and PML are rotated and displayed.

As described above, when viewed from the viewer, the direction in which the viewed image is displayed is fixed, and when the viewer looks in another direction, the viewed image deviates from the field of view and the background image becomes visible. . The background image at this time is the video camera 2
It is a photographed image of the outside world by 4R and 24L, and is an image in the direction in which the face of the viewer faces. Therefore, the viewer is the HMD
Even during use, you can see the image in that direction by moving your face in that direction if necessary. In other words, it is as if you are watching a stationary television.

For example, it is assumed that the viewer gradually moves his face to the left in the display state shown in FIG. 3B, as indicated by arrow FH in FIG. 3C. Then, the viewing images PMR, PM on the liquid crystal displays 14R, 14L
L moves in the opposite direction indicated by arrow FI in FIG. The same applies to vertical movement and rotation.

An initial setting switch 54 is connected to the read control circuit 48 and the rotation angle detection circuit 50.
This makes it possible to initialize the relative relationship between the desired posture of the viewer when the HMD is mounted on the head and the positions of the viewed images PMR and PML in the visual field.

Next, the overall operation of the first embodiment will be described. The viewer wears the wearing arms 20R and 20L of the HMD shown in FIG. In this state, the liquid crystal displays 14R and 14L have a video camera 24
Landscape images PBR and PBL captured by R and 24L
Then, images PMR and PML desired to be viewed such as a movie or a television program input from the outside are displayed in an overlapping manner (see FIG. 3B).

When the viewer changes the direction of the face in this state, the change is detected by the sensors 26R, 26L and 28, and the display position of the viewed image changes as shown in FIG. Therefore, the viewer takes a desired posture, such as sitting on a sofa seat, and faces an appropriate direction so that the viewed image is at a desired position within the display field of view. Then, in this state, the initial setting switch 54 is operated. Then, the read control circuit 48 and the rotation angle detection circuit 50 perform an initial setting operation ignoring the sensor input, and the display position of the viewing image on the liquid crystal displays 14R and 14L is fixed even if the field of view is changed.

In this state, the viewer releases the initial setting switch 54. Then, the read control circuit 48 and the rotation angle detection circuit 50 perform the above-described read control and rotation angle detection operations with reference to the sensor output at the time of cancellation. That is, the display position of the viewing image on the liquid crystal displays 14R and 14L changes as shown in FIG. 4 based on the sensor output.

When the viewer needs to turn his face in that direction when necessary, the background image in that direction is displayed on the liquid crystal displays 14R and 14L and the viewed image moves, so You can easily know the state of the outside world.

Next, the voice will be described. In Figure 6,
An audio signal processing circuit is shown. In the figure, the output side of a microphone (hereinafter simply referred to as "microphone") 70 for obtaining external sound is connected to the input sides of sound mixing circuits 72R and 72L configured by variable resistors. The microphone 70 is placed at an appropriate position, for example, in FIG.
It is provided above the horizontal sensor 28 in the HMD shown in FIG.

The R sound and monaural sound are input to the sound mixing circuit 72R. Further, L sound and monaural sound are input to the sound mixing circuit 72L. The sliding terminals of the audio mixing circuits 72R and 72L are adapted to move in accordance with the detection result of the horizontal sensor 28. The output sides of the audio mixing circuits 72R and 72L are amplifiers 74R and 74L.
Are connected to the speakers 22R and 22L, respectively. Such an audio signal processing circuit is housed in an appropriate position of the HMD.

Next, the operation of voice processing will be described.
First, as shown in FIG. 7A, it is assumed that the viewer faces the front direction when viewing the viewed image. At this time, the horizontal sensor 28 is in a state of not detecting horizontal movement. In the audio mixing circuits 72R and 72L, since the sliding terminal is located at the center, R audio and L audio are output respectively. Therefore, the R and L audio signals are reproduced and output from the speakers 22R and 22L, respectively. That is, the audio output corresponds to the viewing image.

Next, as shown in FIG. 7B, it is assumed that the viewer turns to the left to see the outside world. Then, the horizontal sensor 28 detects the movement, and the sound mixing circuit 72
In R and 72L, the sliding terminal moves in the “left” direction.
Therefore, in the audio mixing circuit 72R, monaural audio R + L
Is output, and the external sound of the microphone 70 is output in the audio mixing circuit 72L. That is, the right ear that virtually faces the viewing image PM can hear the monaural sound, and the opposite left ear can hear the external sound.

Further, as shown in FIG. 7C, it is assumed that the viewer turns to the right to see the outside world. Then, the horizontal sensor 28 detects the movement, and the sound mixing circuit 72
In R and 72L, the sliding terminal moves in the “right” direction.
Therefore, the sound mixing circuit 72R outputs the external sound of the microphone 70, and the sound mixing circuit 72L outputs the monaural sound R + L. That is, a monaural sound can be heard by the left ear that virtually faces the viewing image PM, and an external sound can be heard by the opposite right ear. In this way, the voice and the external sound are mixed in correspondence with the detection movement amount of the horizontal sensor 28, and the position of the viewing image PM and the position of the sound source correspond to each other.

As described above, according to the first embodiment, the desired scenery of the outside world can be viewed satisfactorily, and it is very convenient. In addition, images can be enjoyed in an environment similar to that of a conventional stationary television, and even if viewed for a long time, there is little fatigue. Furthermore, the size of the viewing image can be changed, which is convenient for observing the external landscape. Further, since the position of the viewing image and the position of the sound source are made to coincide with each other, the discomfort of wearing the HMD is reduced, and the information of the outside world can be obtained audibly, so that the operating environment is a stationary television. It will be closer to John.

<Embodiment 2> Next, Embodiment 2 will be described with reference to FIG. The above-described Example 1 is an example of a non-transmissive HMD, but this Example 2 is a transmissive HMD.
It is an example of MD. In the case of this transmissive type, since the viewer can see the outside world through the goggles, a video camera for capturing an image of the outside world is not necessary. For this reason,
The components related to the inside video camera have been removed,
Only the components that control the display position of the viewing image supplied from the outside are sufficient. In addition, since the second embodiment is of a simplified type, the components of the display position control based on the rotational movement are omitted.
Therefore, only one vertical sensor is required.

In FIG. 5, the viewing image signal input from the outside is subjected to color demodulation by the color demodulation circuit 34 and then supplied to the liquid crystal displays 14R and 14L as it is. This viewing image signal is also input to the sync signal separation circuit 60, where the horizontal and vertical sync signals HD and VD are separated. The horizontal synchronizing signal HD is supplied to the horizontal timing circuit 62.
Is input to the liquid crystal displays 14R and 14L. Further, the vertical synchronizing signal VD is supplied to the vertical timing circuit 6
4 and the vertical scanning timing signal obtained therefrom is similarly supplied to the liquid crystal displays 14R and 14L. On the liquid crystal displays 14R and 14L, the image input from the color demodulation circuit 34 is displayed based on these timing signals.

In this case, the detection output of the horizontal sensor 28 is supplied to the horizontal timing circuit 62, whereby the horizontal timing is adjusted so that the viewing image moves in the direction opposite to the moving direction of the face. . On the other hand, the detection output of the vertical sensor 26R (or 26L) is supplied to the vertical timing circuit 64, whereby the vertical timing is adjusted so that the viewing image moves in the direction opposite to the moving direction of the face.

As described above, in the second embodiment, the display position of the viewing image changes in the horizontal and vertical opposite directions corresponding to the face direction. Therefore, the outside world can be seen from the goggles, and the same effect as that of the first embodiment can be obtained. In addition, Example 2
Since it is a simple type, the size of the viewing image does not change.

<Other Embodiments> The present invention can be variously modified based on the above disclosure, and includes, for example, the following. (1) In the first embodiment, the video cameras are provided for the left and right eyes, but of course, only one video camera may be provided. (2) In the second embodiment, the initial setting operation performed in the first embodiment may be performed by the horizontal and vertical timing circuits 62 and 64. Since the second embodiment is a transmissive type, the viewing image is displayed so as to be superimposed on the background image. Therefore, it is convenient to perform the initial setting so that the viewing image is located in a monotone place, for example, a part such as a white wall in the outside scenery.

(3) Although the liquid crystal display is directly viewed by both eyes in the above embodiment, it may be viewed indirectly by using a mirror or the like. In addition, various design changes can be made to achieve the same effect, such as detecting the movement of the image captured by the video camera instead of the horizontal and vertical sensors and moving the viewing image in the opposite direction. Is.

[0041]

As described above, the present invention has the following effects. (1) Since the movement of the HMD is detected and the movement opposite to that is added to the viewing image, an environment similar to that of a stationary television can be realized, and the desired external scenery can be created by simply moving the face. Easy to see and easy to use. (2) Images can be enjoyed in an environment similar to that of a conventional stationary television, and there is no feeling of pressure even when viewed for a long time and there is little fatigue.

[Brief description of drawings]

FIG. 1 is a plan view showing a main part of a mechanical structure according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a main part of an electrical configuration of the first embodiment.

FIG. 3 is a diagram showing the operation of the first embodiment.

FIG. 4 is a diagram showing the operation of the first embodiment.

FIG. 5 is a block diagram showing a main part of an electrical configuration of a second embodiment.

FIG. 6 is a block diagram showing a voice processing system according to the first embodiment.

FIG. 7 is a diagram showing an operation of the voice processing system according to the first embodiment.

[Explanation of symbols]

10 ... HMD main body 12R, 12L ... Image display part (image display means, image projection means) 14R, 14L ... Liquid crystal display 16R, 16L ... Backlight 18R, 18L ... Eyepiece 20R, 20L ... Wearing arm 22R, 22L ... Speaker 24R , L ... Video camera 26R, 26L ... Vertical sensor (sensor means) 28 ... Horizontal sensor (sensor means) 30 ... Synchronous signal generating circuit 32 ... Synthesizing switch 34 ... Color demodulation circuit 36 ... A / D converter 38 ... Memory (position Control means) 40 ... Writing control circuit 42 ... D / A converter 44 ... Horizontal / vertical timing circuit 46 ... Backlight drive device 48 ... Readout control circuit (position control means) 50 ... Rotation angle detection circuit (position control means) 52 ... Image rotation circuit (position control means) 54 ... Initial setting switch 60 ... Sync separation circuit 62 ... Horizontal switch Vertical circuit (position control means) 70 ... Microphones 72R, 72L ... Audio mixing circuits (audio mixing means) 74R, 74L ... Amplifiers PBR, PBL ... Background images PM, PMR, PML ... Appreciation image

Claims (2)

[Claims] 1. An HMD main body mounted on a head; an image display means for displaying an image at a position close to both eyes in the main body; an image pickup means for picking up a visual field of both eyes; Image combining means for supplying the image display means with an externally supplied image signal superimposed on the background image signal; sensor means for detecting the movement of the display device body; corresponding to the detection result of the sensor means And a position control means for giving a reverse movement to the composition position of the viewed image by the image composition means. 2. An HMD main body equipped with a screen capable of seeing the outside world and mounted on the head; an image projection means for projecting a viewing image on the screen in the main body; and a movement of the display device main body. A head mount display comprising: a sensor means; a position control means for giving an opposite movement to a projection position of a viewing image on the screen by the image projection means in response to a detection result of the sensor means.