CN115268621A - VR head-mounted device and VR virtual reality equipment - Google Patents

Abstract

The invention relates to the technical field of VR (virtual reality), in particular to a VR head-mounted device and VR virtual reality equipment.A mobile power supply, a host and a display unit are arranged in VR glasses, and the host, the display unit, a listening and speaking module and at least three lip language identification modules are respectively connected with the mobile power supply; the at least three lip language recognition modules and the listening and speaking modules are respectively arranged on the VR glasses, and the at least three lip language recognition modules, the listening and speaking modules and the display unit are respectively connected with the host computer in the VR glasses; when wearing VR head-mounted device, at least three lip language identification module all sends to the host computer after reading wearer's lip language information, generates the information of putting outward after the host computer calculates the lip language information, and host computer control listening and speaking module broadcast information of putting outward, when VR glasses environment had the audio frequency, listening and speaking module identification audio frequency and give the wearer with the audio frequency transmission. A VR headset is provided that enables a deaf-mute to perform an immersive experience.

Description

VR head-mounted device and VR virtual reality equipment

Technical Field

The invention relates to the technical field of VR (virtual reality), in particular to a VR head-mounted device and VR virtual reality equipment.

Background

The Virtual Reality technology is a computer simulation system capable of creating and experiencing a Virtual world, a Virtual environment is generated by a computer, the Virtual environment is a system simulation of multi-source information fusion, interactive three-dimensional dynamic views and entity behaviors, and a user is immersed in the environment. With the rapid development of the VR technology, the VR technology tends to mature and obtains diversified applications, and has related applications in the fields of education, medical treatment, movies, military, construction, and the like, and with the help of the VR virtual reality device interests, more and more companies apply the VR technology to the system design and gradually integrate into the daily life of the public, while the current VR technology integrated hearing technology only relates to the external technologies such as speakers, but for the deaf, such VR virtual reality devices cannot enable the deaf to experience the characteristics of immersion, interactivity, multi-perceptibility, and the like. The deaf-mute can not hear the sound emitted by the VR virtual reality equipment like a normal person, and can not normally communicate when performing voice interaction.

In summary, the technical problem actually solved by the present invention is how to provide a VR headset for deaf-mute to experience in an immersive manner.

Disclosure of Invention

In order to overcome the technical defects, the invention aims to provide a VR (virtual reality) head-mounted device and VR virtual reality equipment, and provides the VR head-mounted device which can enable a deaf-mute to perform immersive experience.

The invention discloses a VR head-mounted device, which comprises VR glasses, a listening and speaking module and at least three lip language recognition modules; the mobile power supply, the host and the display unit are arranged in the VR glasses, and the host, the display unit, the listening and speaking module and the at least three lip language identification modules are respectively connected with the mobile power supply; the at least three lip language recognition modules and the listening and speaking modules are respectively arranged on the VR glasses, and the at least three lip language recognition modules, the listening and speaking modules and the reality unit are respectively connected with the host computer in the VR glasses; when wearing VR head-mounted device, at least three lip language identification module all faces wearer's lip to send to the host computer after reading wearer's lip language information, the host computer generates the information of putting outward after calculating lip language information, and host computer control listening and speaking module broadcast information of putting outward, when VR glasses environment had the audio frequency, listening and speaking module discernment audio frequency and give the wearer with the audio frequency transmission.

Preferably, the listening and speaking module comprises a loudspeaker and a bone conduction device; wherein, the speaker plays the information of putting outward, and bone conduction device discerns and transmits the audio frequency.

Preferably, the at least three lip language recognition modules comprise a first lip language recognition module, a second lip language recognition module and a third lip language recognition module; wherein, first lip language identification module, second lip language identification module, third lip language identification module are in the same face of VR glasses.

Preferably, when the wearer wears the VR headset, the first lip language recognition module, the second lip language recognition module and the third lip language recognition module are respectively arranged on one surface of the VR glasses facing the lips of the wearer.

Preferably, the first lip language recognition module and the third lip language recognition module are respectively arranged on one side of the VR glasses facing the ears of the wearer, and the second lip language recognition module is arranged between the first lip language recognition module and the second lip language recognition module.

Preferably, the first lip language recognition module, the second lip language recognition module and the third lip language recognition module are respectively arranged on one side of the VR glasses, which is far away from the wearer.

Preferably, the listening and speaking modules are on either side of the VR glasses facing the wearer's ears when the wearer is wearing the VR headset.

Preferably, the VR headset comprises at least two listening and speaking modules, and the at least two listening and speaking modules are respectively on a side of the VR glasses facing the ear of the wearer.

In view of the above, another object of the present invention is to provide a VR virtual reality device, which includes at least one VR headset as described above.

Preferably, the VR virtual reality device is capable of communicating with the VR headset, and the VR headset is capable of reading audio in a virtual environment formed by the VR virtual reality device, and the bone conduction device delivers the audio to the wearer.

After the technical scheme is adopted, compared with the prior art, the VR headset device has the advantages that the VR headset device can enable a deaf-mute to perform immersive experience, under the technologies of bone conduction and lip language recognition, the deaf-mute can directly participate in education through the VR headset device, the application range of the VR headset device is enlarged, the deaf-mute can be added into VR virtual reality equipment and can also experience VR virtual display equipment, when the deaf-mute communicates with a normal person, the voice can be directly expressed through the lip language recognition and the loudspeaker, and obviously, the loudspeaker can customize the type of the voice.

Drawings

Fig. 1 is a schematic diagram of a VR headset and VR virtual reality equipment of the present invention;

fig. 2 is a schematic diagram of issuing at least three lip language recognition modules of a VR headset and VR virtual reality equipment according to the present invention.

Reference numerals are as follows:

1 is VR glasses, 2 is listening and speaking module, 3 is first lip language identification module, 4 is second lip language identification module, 5 is the third lip language identification module.

Detailed Description

The advantages of the invention are further illustrated in the following description of specific embodiments in conjunction with the accompanying drawings.

Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, like numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the exemplary embodiments below are not intended to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used in this disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and/or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

It is to be understood that although the terms first, second, third, etc. may be used herein to describe various information, such information should not be limited to these terms. These terms are only used to distinguish one type of information from another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present disclosure. The word "if," as used herein, may be interpreted as "at … …" or "at … …" or "in response to a determination," depending on the context.

In the description of the present invention, it is to be understood that the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used merely for convenience of description and for simplicity of description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed in a particular orientation, and be operated, and thus, are not to be construed as limiting the present invention.

In the description of the present invention, unless otherwise specified and limited, it is to be noted that the terms "mounted," "connected," and "connected" are to be interpreted broadly, and may be, for example, a mechanical connection or an electrical connection, a communication between two elements, a direct connection, or an indirect connection via an intermediate medium, and specific meanings of the terms may be understood by those skilled in the art according to specific situations.

In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in themselves. Thus, "module" and "component" may be used in a mixture.

Referring to fig. 1 and fig. 2, the present embodiment provides a VR headset, including VR glasses 1, a listening and speaking module 2, and at least three lip language recognition modules; wherein, a mobile power supply, a host and a display unit are arranged in the VR glasses 1; the mobile power supply supplies power to all components in the VR headset, the mobile power supply can be a portable mobile electric power storage power supply, the host machine is a core element of the whole VR headset, functions of the host machine comprise communication, calculation, control and the like, the display unit is used for displaying reality, in the listening and speaking module 2 and the at least three lip language recognition modules, the listening and speaking module 2 replaces the deaf-mute to listen and speak, and the at least three lip language recognition modules respectively shoot the lips of the deaf-mute in real time. Specifically, at least three lip language identification modules and a listening and speaking module 2 are respectively arranged on VR glasses 1, and at least three lip language identification modules and a listening and speaking module 2 are respectively connected with a host in VR glasses 1, so that the host can communicate and/or control with at least three lip language identification modules and a listening and speaking module 2, specifically, the at least three lip language identification modules respectively shoot the lips of deaf people in real time, and the at least three lip language identification modules respectively send shot lip language information to the host, when the host receives the lip language information shot by the at least three lip language identification modules, lip language analysis and identification are carried out through a lip language identification algorithm in the host, the host generates external information through the lip language identification information after calculation, and the host sends the external information to a display unit for display, and simultaneously the host plays the external information through the listening and speaking module 2. When wearing VR head mounted device, at least three lip language identification module all towards the lip of wearer, guarantees that at least three lip language identification module can accurate shooting wearer's lip. And the host computer, the display element, mobile power source is connected respectively to listening and speaking module and at least three lip language identification module, mobile power source is the host computer, the display element, listening and speaking module 2 and at least three lip language identification module power supply, and the host computer is connected the display element respectively, listening and speaking module 2 and at least three lip language identification module carry out communication and/or control connection, host computer control listening and speaking module 2 plays the information of putting outward, when VR glasses 1 located the environment have the audio frequency, listening and speaking module 2 discernment audio frequency and give the person of wearing with the audio frequency transfer.

It should be noted that, since the host, the display unit and the mobile power supply are in the VR glasses 1, the host, the display unit and the mobile power supply are not labeled, but according to the description of the embodiment, those skilled in the art can understand the positions of the host, the display unit and the mobile power supply and the functions thereof.

It should be noted that, in this embodiment, the mobile power supply that supplies power to the host, the display unit, the listening and speaking module 2, and the at least three lip recognition modules is not the only implementation, but may be directly connected by a cable or another implementation that supplies power to the host, the real unit, the listening and speaking module 2, and the at least three lip recognition modules.

It should be noted that, the listening and speaking module 2 includes a speaker and a bone conduction device; the speaker plays the external information formed by calculating the lip language information shot by the at least three lip language recognition modules by the host, the bone conduction device transmits the audio of the environment where the VR glasses 1 are located to the wearer, and it should be noted that the bone conduction device described above transmits the audio of the environment where the VR glasses 1 are located to the wearer, where the environment where the VR glasses 1 are located in the audio can be the audio in a real environment or a virtual environment, the audio in the real environment obviously generates decibel sounds, and the audio in the virtual environment is a virtual audio that can appear in a certain scene, such as sounds of fighting, collision, character and character roles.

It should be noted that, the at least three lip language recognition modules include a first lip language recognition module 3, a second lip language recognition module 4, and a third lip language recognition module 5, and the first lip language recognition module 3, the second lip language recognition module 4, and the third lip language recognition module 5 are all on the same surface of the VR glasses 1, it should be noted that, in this embodiment, only three lip language recognition modules are used for description, and it is not limited to only three lip language recognition modules on the VR headset, and under the condition that allows or necessary, the lip language recognition modules can be arbitrarily added, such as a fifth lip language recognition module, a sixth lip language recognition module, and the like.

It should be noted that at least three lip language recognition modules are all high definition cameras.

It should be noted that, when the wearer wears the VR headset, the first lip language recognition module 3, the second lip language recognition module 4, and the third lip language recognition module 5 are respectively located on one surface of the VR glasses 1 facing the lips of the wearer, and the first lip language recognition module 3 and the third lip language recognition module 5 are respectively located on one side of the VR glasses 1 facing the ears of the wearer, as shown in fig. 2, that is, the first lip language recognition module 3 and the third lip language recognition module 5 are respectively located on the left and right sides of one surface of the VR glasses 1 facing the lips of the wearer, and the second lip language recognition module 4 is located between the first lip language recognition module 3 and the second lip language recognition module 5 and faces the lips of the wearer.

The first lip language recognition module 3, the second lip language recognition module 4, and the third lip language recognition module 5 are respectively disposed on the VR glasses 1 on the side away from the wearer.

When the wearer wears the VR headset, the listening and speaking module 2 is located on any side of the VR glasses facing the ears of the wearer, that is, the listening and speaking module 2 is approximately located at the ears of the wearer. Moreover, the VR headset at least includes two listening and speaking modules 2, and the two listening and speaking modules 2 are respectively arranged on one side of the VR glasses facing to the ears of the wearer, and the listening and speaking modules 2 are not shown in the drawings and arranged on both sides of the VR glasses, but it can be understood by those skilled in the art from the above description.

It is another object of this embodiment to provide a VR virtual reality device that includes at least one VR headset as provided in the above embodiments, and the VR virtual reality device is capable of communicating with the VR headset, and the VR headset is capable of reading audio in a virtual environment formed by the VR virtual reality device, and the bone conduction device delivers this portion of audio to the wearer.

It should be noted that the VR headset can be used alone or in combination with the VR virtual reality device, and when the VR headset is used alone, the VT headset communicates with a real person, and when the VR headset is used in the VR virtual reality device, the VR headset communicates with a virtual environment person, a character, an NPC, and the like.

A bone conduction device, a speaker, and a lip recognition module of a VR headset, as shown in fig. 1 and 2. Bone conduction devices (not shown in the drawing) are arranged on the left side and the right side of the VR glasses 1; the lip language recognition modules are distributed on the lower side of the VR glasses 1, and 3 high-definition cameras are arranged in the scheme and are respectively distributed on the middle position of the leftmost edge and the rightmost edge of the lower side of the VR glasses 1 and the middle position of the foremost edge of the lower side of the VR glasses 1.

The functions of each module are as follows: and voice information in the VR world is transmitted to the bone conduction devices through the bone conduction technology by the bone conduction devices on the left side and the right side, and then the deaf-mute can receive the voice transmission information through the bone conduction devices. The high-definition camera of the VR glasses 1, which is right opposite to the lips, is a main camera, the lips are shot in real time, so that the lips act when a user speaks when the user uses the glasses, lip language analysis and recognition are carried out by combining a lip language recognition algorithm, information which the deaf-mute wants to express is displayed on a display unit of VR virtual reality equipment, and the lip language information can be simultaneously displayed through a loudspeaker. The high definition cameras of the left side and the right side of the lower side of the VR glasses 1 are distributed to aim at the lips from the left side and the right side, the action of collecting the lips from the left side and the right side is collected in real time, collected lip language information and information collected by the main camera are integrated, and then lip language algorithm analysis and recognition are carried out. And the external loudspeaker is used for externally playing voice information of the user.

It should be noted that the embodiments of the present invention have been described in terms of preferred embodiments, and not by way of limitation, and that those skilled in the art can make modifications and variations of the embodiments described above without departing from the spirit of the invention.

Claims (10)

1. A VR headset comprises VR glasses, a listening and speaking module and at least three lip language recognition modules; wherein,

a mobile power supply, a host and a display unit are arranged in the VR glasses, and the host, the display unit, the listening and speaking module and the at least three lip language identification modules are respectively connected with the mobile power supply;

the at least three lip language identification modules and the listening and speaking module are respectively arranged on the VR glasses, and the at least three lip language identification modules, the listening and speaking module and the display unit are respectively connected with the host in the VR glasses;

when the VR headset is worn, the at least three lip language identification modules face the lips of a wearer, lip language information of the wearer is read and then sent to the host, the host generates play-out information after calculating the lip language information, the host controls the listening and speaking module to play the play-out information, and when audio exists in the environment where the VR glasses are located, the listening and speaking module identifies the audio and transmits the audio to the wearer.

2. The VR headset of claim 1, wherein the listening and speaking module comprises a speaker, a bone conduction device; wherein,

the loudspeaker plays the outgoing information, and the bone conduction device identifies and transmits the audio.

3. The VR headset of claim 1, wherein the at least three lip recognition modules include a first lip recognition module, a second lip recognition module, a third lip recognition module; wherein,

the first lip language recognition module, the second lip language recognition module and the third lip language recognition module are arranged on the same surface of the VR glasses.

4. The VR headset of claim 3 wherein the first, second, and third lip recognition modules are on respective sides of the VR glasses facing lips of the wearer when the wearer wears the VR headset.

5. The VR headset of claim 4 wherein the first and third lip recognition modules are on respective sides of the VR glasses facing the wearer's ears and the second lip recognition module is between the first and second lip recognition modules.

6. The VR headset of claim 5, wherein the first, second, and third lip recognition modules are on respective sides of the VR glasses that are distal from the wearer.

7. The VR headset of claim 1, wherein the dictation module is on either side of the VR glasses that face the wearer's ears when the wearer wears the VR headset.

8. The VR headset of claim 7, wherein the VR headset includes at least two of the listening and speaking modules, and the at least two listening and speaking modules are on respective sides of the VR glasses that face towards the wearer's ears.

9. A VR virtual reality apparatus comprising at least one VR headset of any one of claims 1 to 8.

10. The VR virtual reality device of claim 9, wherein the VR virtual reality device is capable of communicating with the VR headset, and the VR headset is capable of reading audio in a virtual environment formed by the VR virtual reality device, and the bone conduction device communicates the audio to the wearer.

Images