JP2007334149A - Head mount display apparatus for hearing-impaired persons - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a head mount display apparatus for a hearing-handicapped person with which many and diversified surrounding sounds, such as human voices and environmental sounds, can be visually recognized. <P>SOLUTION: The head mount display apparatus for the hearing-handicapped person includes a spectacle frame 10 and spectacle lenses 11, a plurality of microphones M1 to M4 attached in prescribed positions of the spectacle frame 10, a recognition means which recognizes the surrounding sounds based on the microphone signals S1 to S4 output from the plurality of the microphones M1 to M4 and outputs the character information corresponding to the recognition results, and a display means 14 which displays the character information in a part of the visual field through the spectacle lenses 11. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a head-mounted display device for a hearing impaired person, and more particularly to a head-mounted display device for a hearing impaired person that can recognize ambient sounds and visually display information useful for the hearing impaired person.

  2. Description of the Related Art Conventionally, a head-mounted display device is known in which a head-mounted display is configured as an eyeglass type and a character or graphic image can be displayed in a part of the field of view (see, for example, Patent Document 1). Such a head-mounted display device can be configured to be small and light, and the user can obtain a natural wearing feeling.

JP-A-11-346336

  The conventional head-mounted display device can be used comfortably in accordance with displayed information regardless of whether it is indoors or outdoors. However, when the user is a hearing-impaired person, it is difficult to use comfortably in a situation where attention is paid to the surrounding situation such as outdoors.

  In general, a hearing-impaired person who has difficulty in recognizing sound cannot obtain information provided by sound generated indoors or outdoors, for example. For example, in order to know a visitor indoors, visual means instead of sounds such as knocks and buzzers are necessary. Also, sufficient attention is always required for dangerous situations such as the approach of a car outdoors. As described above, there is a need for a means for allowing a hearing impaired person to recognize information related to sounds generated in the surroundings and obtaining such information without excessive burden.

  Therefore, the present invention has been made to solve these problems, and displays information useful for recognizing ambient sounds and human speech when worn by a hearing impaired person. An object of the present invention is to provide a head-mounted display device for a hearing impaired person that can be configured to be lightweight, provide a good wearing feeling, and can support the daily life of a hearing impaired person.

  In order to solve the above problems, a head mounted display device for a hearing impaired person according to the invention of claim 1 is provided at a predetermined position of the spectacle frame (10), the spectacle lens (11), and the spectacle frame (10). Ambient sounds are recognized based on one or more attached microphones (M1 to M4) and microphone signals (S1 to S4) output from the one or more microphones (M1 to M4), and corresponding to the recognition result. A recognition means (12) for outputting character information and a display means (14) for displaying the character information in a part of the field of view through the spectacle lens (11).

  According to the head mounted display device for the hearing impaired person configured as described above, the ambient sound input from the microphone in a state worn by the user is recognized by the recognition unit, and the character information corresponding to the recognition result is near the spectacle lens. Are displayed by the display means. Therefore, when used by a hearing impaired person, a voice that is spoken by a nearby person, an environmental sound, and the like can be visually recognized as character information by a structure that is small, light, and has a good wearing feeling.

  In the present invention, the recognition unit may include a voice recognition unit that recognizes a voice uttered by a human and outputs a word corresponding to a recognition result as the character information.

  In the present invention, the recognition means may include environmental sound recognition means for recognizing environmental sounds under various circumstances and outputting onomatopoeia corresponding to recognition results as the character information.

  The present invention further comprises means for determining the levels of the plurality of microphones based on the level of the microphone signal, and specifying the direction of the sound source based on the determination result, wherein the display means displays the direction of the specified sound source. You may comprise so that it may display with the said character information.

  In the present invention, the display means may be configured to visually display the determined level for each of the plurality of microphones.

  According to the present invention, the head mounted display device integrated with the glasses is configured to display the character information corresponding to the recognition result of the ambient sound input from the microphone. It is possible to visually recognize ambient sounds including human voices and environmental sounds as character information. Especially when worn by hearing-impaired people, it can provide multifaceted useful information such as grasping human voices and perceiving dangers in a variety of indoor and outdoor situations. be able to.

Hereinafter, a head mounted display device for a hearing impaired person to which the present invention is applied will be described with reference to the drawings.
FIG. 1 is a diagram illustrating a schematic configuration of a head mounted display device for a hearing impaired person according to the present embodiment. As shown in FIG. 1, the head mounted display device for a hearing impaired person of this embodiment includes a spectacle frame 10, a spectacle lens 11, a control unit 12, a support member 13, a display unit 14, and four microphones. M1, M2, M3, and M4 are provided.

  In the above configuration, the spectacle frame 10 and the spectacle lens 11 have the appearance of general spectacles, but are formed in a special structure in order to integrate the above-described members. The right temple 10a of the spectacle frame 10 is attached with a control unit 12 incorporating various circuits used for control or the like or a battery. A display unit 14 is attached to one end of a support member 13 integrated with the control unit 12. The support member 13 is formed in a curved shape, and is adjusted so that the display unit 14 is positioned near the outer surface of one spectacle lens 11. The support member 13 has a built-in wiring for passing an image signal or the like sent from the control unit 12 to the display unit 14. As the display unit 14, for example, a small liquid crystal panel can be used.

  Microphones M1 and M2 are attached to the right temple 10a of the spectacle frame 10 at two locations on the front side and the rear side. Similarly, microphones M3 and M4 are attached to the left temple 10b of the spectacle frame 10 in a symmetrical arrangement as described above. Note that wirings (not shown) for electrically connecting the four microphones M1 to M4 and the control unit 12 are built in the right temple 10a and the left temple 10b.

  In the state where the spectacle frame 10 is mounted on the user's head, the image of the display unit 14 controlled by the control unit 12 is displayed on a part of the field of view seen by the user through the spectacle lens 11. Thereby, the user can see useful information on the image of the display unit 14 without being disturbed by the user.

  Next, FIG. 2 is a block diagram showing a circuit configuration of the head mounted display device for the hearing impaired person of the present embodiment. In FIG. 2, the control unit 21 included in the control unit 12, the power source 22, the recognition processing unit 23, the image processing unit 24, the memory 25, the liquid crystal driving unit 31 and the liquid crystal panel 32 included in the display unit 14, and four pieces. Microphones M1 to M4 are shown. The operation of the entire head mounted display device for the hearing impaired person of the present embodiment is controlled by the control unit 21. The control unit 21 is, for example, a CPU that executes a control program stored in the memory 25, and sends a control signal to each unit at a predetermined timing.

  As the power source 22, for example, a small battery that can be repeatedly charged can be used. In this case, a charging terminal for a battery is provided at a desired location on the spectacle frame 10 or the control unit 12, and charging can be performed using a charging stand or connector for mounting the head mounted display device for the hearing impaired when not worn. It is desirable to have a structure. The power source 22 may be configured such that a user carries a battery provided separately from the control unit 12 and is connected to the control unit 12 with a cable.

  The recognition processing unit 23 receives the microphone signals S1, S2, S3, and S4 from the four microphones M1 to M4. The recognition processing unit 23 converts the four microphone signals S1 to S4 into digital signals, performs level detection processing, and executes speech recognition processing or environmental sound automatic recognition processing. Details of each process will be described later.

  Each of the microphones M1 to M4 has directivity. Then, as shown in FIG. 3, in a state where the user has attached the head mounted display device for the hearing impaired person, the arrangement is such that the directivity of each other faces different directions. That is, as indicated by arrows in FIG. 3, the directivity directions of the microphone M1 are arranged in the direction D1, the microphone M2 in the direction D2, the microphone M3 in the direction D3, and the microphone M4 in the direction D4. With reference to the user's head, the direction D1 corresponds to the right front, the direction D2 corresponds to the right rear, the direction D3 corresponds to the left front, and the direction D4 corresponds to the left rear.

  In the recognition processing unit 23, the levels of the microphones M1 to M4 are determined stepwise based on the microphone signals S1 to S4. The control unit 21 can roughly specify the direction of the sound source based on the determined size comparison of each level. Of the four microphone signals S1 to S4, the maximum microphones M1 to M4 may be specified as the direction of the sound source, but in order to specify the angle in a finer direction, interpolation is performed using the four microphone signals S1 to S4. Calculations and the like may be performed.

  The voice recognition process in the recognition processing unit 23 is a process of analyzing a voice uttered by a human and extracting a predetermined word as character information. For this reason, various voice patterns are recorded in the memory 25, and a voice pattern having a matching feature is selected as a recognition result by comparing with the microphone signals S1 to S4. The recognition processing unit 23 can employ both speech recognition processing for a specific speaker that specifies a speaker and speech recognition processing for an unspecified speaker that does not characterize the speaker.

  The environmental sound automatic recognition processing in the recognition processing unit 23 is a process of analyzing environmental sounds under various conditions such as a living sound such as a knocking of a door, a driving sound of a car outdoors or a sound of a siren, and taking out as character information. It is. Therefore, various environmental sound feature patterns are recorded in the memory 25, and the closest feature pattern is selected by comparing the microphone signals S1 to S4 with each feature pattern.

  In the environmental sound automatic recognition processing, an onomatopoeia generally expressing the environmental sound can be used as the character information corresponding to the feature pattern. For example, it can be expressed by an onomatopoeia such as “one one” for a dog cry and “tonton” for a door knock. By using such onomatopoeia, it is possible to recognize at least an approximate environmental sound while avoiding misrecognition due to specifying a recognized environmental sound object.

  The recognition result of the recognition processing unit 23 is sent to the image processing unit 24 via the control unit 21. In the image processing unit 24, an image signal including character information corresponding to the recognition result of the recognition processing unit 23 is generated. The image signal generated by the image processing unit 24 is sent to the display unit 14 through the wiring in the support member 13, and predetermined character information is displayed on the liquid crystal panel 32 driven by the liquid crystal driving unit 31.

  Next, a specific display example on the display unit 14 will be described. FIG. 4 shows a first display example of the display unit 14 as viewed from the user. The first display example is an example in the case of a voice uttered by a human. As shown in FIG. 4 (a), a part on the display screen 14a of the user's field of view is disposed, the character information of "Hello" therein are displayed. On the left side of the character information, “left front” is displayed as the voice arrival direction. In the first display example, when a specific speaker such as a family member or a non-specific speaker speaks, a large level is detected mainly by the microphone M3 in FIG. Corresponds to the case where the information is recognized.

  On the other hand, FIG. 4B is an example in which FIG. 4A is slightly modified. In the case of FIG. 4B, the recognition result is the same as that in FIG. 4A, but a different expression is adopted for the input states of the microphones M1 to M4. That is, level meters corresponding to the levels of the microphones M1 to M4 are arranged over four lines, and the character information of “Hi” is displayed in the line of the microphone M3 indicating the maximum level. As a visual effect by such display, in addition to being able to recognize the direction of voice arrival, the user can also grasp the levels of the microphones M1 to M4 in all directions. Moreover, based on the change of the level meter of each microphone M1-M4, the movement state of a sound source can be grasped | ascertained.

  Next, FIG. 5 shows a second display example of the display unit 14 as viewed from the user. The second display example is an example in the case of traveling sound of an automobile or the like outdoors. As shown in FIG. 5A, the character information of the onomatopoeia “boo” is displayed in the display screen 14a similar to FIG. On the left side of the character information, “right rear” is displayed as the voice arrival direction. In the second display example, when the vehicle travels behind the user outdoors, a large level is detected mainly by the microphone M2 in FIG. 1 and the character information is recognized by the recognition result in the recognition processing unit 23. Corresponds to the case. In the second embodiment, for example, even when the object that emits sound is not an automobile, there is a possibility that construction sound having the same characteristics may be recognized.

  On the other hand, FIG. 5B is an example in which FIG. 5A is slightly modified. In FIG.5 (b), the expression similar to FIG.4 (b) is employ | adopted about the input state of microphone M1-M4. That is, level meters corresponding to the levels of the microphones M1 to M4 are arranged over four lines, and the character information of the “boo” is displayed in the line of the microphone M2 indicating the maximum level. Thereby, in addition to obtaining the same visual effect as in FIG. 4B, it is possible to easily detect the danger of a hearing impaired person particularly outdoors.

  Embodiment described above is not limited to said structure, There exist various modifications. In the above embodiment, the head mounted display device for hearing impaired persons provided with the four microphones M1 to M4 has been described, but the number of microphones to be installed can be freely changed. It is desirable to increase the number of microphones in order to accurately determine the direction of arrival of the sound source. However, when indoor use is the center, the specification of the direction of arrival of the sound source is omitted and only one microphone is provided. You can also.

  In the above embodiment, the configuration in which the glasses and the head mounted display device are integrated has been described. However, the glasses and the head mounted display may be configured to be detachable as separate units. For example, when the number of microphones is one, a head mounted display device for a hearing impaired person that can be attached to and detached from general glasses can be realized by adopting a structure in which a microphone is attached to the control unit 12.

  In the above embodiment, a vibrator member that vibrates when a predetermined signal is supplied to a predetermined position of the spectacle frame 10 may be attached. Thereby, like a mobile phone, when a specific ambient sound is recognized, a user can be alerted by sensing vibration of the vibrator member.

It is a figure which shows schematic structure of the head mounted display apparatus for hearing impaired persons of this embodiment. It is a block diagram which shows the circuit structure of the head mounted display apparatus for hearing impaired persons of this embodiment. It is a figure explaining the directivity of microphone M1-M4. FIG. 10 is a diagram showing a first display example of the display unit 14. FIG. 10 is a diagram showing a second display example of the display unit 14.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Eyeglass frame 11 ... Eyeglass lens 12 ... Control unit 13 ... Supporting member 14 ... Display unit 14a ... Display screen 21 ... Control part 22 ... Power supply 23 ... Recognition processing part 24 ... Image processing part 25 ... Memory 31 ... Liquid crystal drive part 32 ... Liquid crystal panels M1, M2, M3, M4 ... Microphones S1, S2, S3, S4 ... Microphone signals

Claims (5)

A spectacle frame and a spectacle lens;
One or a plurality of microphones attached to a predetermined position of the spectacle frame;
Recognizing means for recognizing ambient sounds based on microphone signals output from the one or more microphones and outputting character information corresponding to the recognition results;
Display means for displaying the character information in a part of the field of view through the spectacle lens;
A head mounted display device for the hearing impaired.   The head mounted display for a hearing impaired person according to claim 1, wherein the recognition means includes voice recognition means for recognizing a voice uttered by a human and outputting a word corresponding to a recognition result as the character information. apparatus.   The auditory perception according to claim 1 or 2, wherein the recognition means includes environmental sound recognition means for recognizing environmental sounds under various situations and outputting pseudo-sound words corresponding to recognition results as the character information. Head mounted display device for the handicapped. Further comprising means for determining the level of the plurality of microphones based on the level of the microphone signal, and specifying the direction of the sound source based on the determination result;
The head mounted display device for a hearing impaired person according to claim 1, wherein the display unit displays the direction of the specified sound source together with the character information.   5. The head mounted display device for a hearing impaired person according to claim 4, wherein the display means visually displays the determined level for each of the plurality of microphones.

Images