CN104065951B - Video capture method, video broadcasting method and intelligent glasses - Google Patents

Abstract

The video shooting method, video playing method and smart glasses provided by the embodiments of the present invention can obtain video data of different viewpoints and audio data of different channels through two camera devices and two recording devices respectively, so the present invention realizes 3D video Filmed in real time. Since the installation position of the camera device corresponds to the eyes, and the installation position of the recording device corresponds to the ears, the video shot by the invention can make the viewer have an immersive feeling and be more real. The invention can automatically focus and shoot the surrounding scenery, so that the captured picture can form a 3D video. The present invention can send video and audio data in real time, so that users can enter remote virtual scenes at any time, obtain current user's viewpoint position information or video attention point information, and truly realize 3D video real-time interaction.

Description

Video shooting method, video playing method, and smart glasses

The present invention claims the priority of the Chinese patent application with the application number 201320876456.5 and the title of "a kind of smart glasses" submitted to the China Patent Office on December 26, 2013, the entire content of which is incorporated in the present invention by reference.

technical field

The invention relates to the technical field of video interaction, in particular to a video shooting method, a video playing method and smart glasses.

Background technique

With the rapid development of technology, wearable devices have become the development trend of the next generation of electronic products. And with the continuous popularization of the network, the continuous improvement of network speed, and the continuous improvement of the computing speed of the processor, the demand for real-time transmission of 3D video and stereo sound will gradually become a reality.

Although 3D technology exists currently, the current 3D technology focuses on playing 3D video. That is, the user can watch 3D images such as 3D movies, programs or games by wearing 3D glasses. Although many 3D TVs are now equipped with cameras, 3D TVs can be used for video chat with the help of related software applications. However, these cameras (even the two cameras that simulate glasses) do not focus on the surrounding scenery when shooting video, which is actually equivalent to the camera directly shooting the scenery, so the scenery seen by the user's eyes is the same, not 3D The image cannot form a 3D effect. Therefore, the user cannot experience the real 3D video viewing effect, let alone the real-time interaction of the 3D video between users.

Contents of the invention

Aiming at the above problems, the present invention proposes a video shooting method, a video playing method and smart glasses to realize 3D video interaction and give users a more realistic video experience.

The smart glasses for realizing 3D video real-time interaction proposed by the present invention are realized by adding a camera device, an autofocus device, a recording device, a display device, a listening device, a processing device and a network communication device to the glasses frame.

Specifically, the present invention provides a kind of smart glasses. The smart glasses are respectively equipped with a camera device for capturing video and a display device for both eyes to watch the video on the glasses frame close to the human eyes; The recording device for collecting audio frequency and the listening device for listening to audio frequency; the autofocus device is connected with the camera device to control the camera device to focus and capture video; the processing device is connected with the camera device, display device, recording device and listening device, and has a compression unit and The decompression unit, the compression unit is used to compress the video data received from the camera device and the audio data received from the recording device, and the decompression unit is used to decompress the video and audio data received from the network communication device for subsequent use The display device displays video and uses the listening device to listen to audio; the network communication device is connected to the processing device for sending out the video and audio data compressed by the compression unit, or receiving compressed video and audio data from the outside and sending it to the decompression unit.

The present invention also provides a video shooting method, which is applied to smart glasses. The glasses frames of the smart glasses are provided with a first camera, a second camera, a first recording device and a second recording device. The first The installation position of the camera corresponds to the left eye, the installation position of the second camera corresponds to the right eye, the installation position of the first recording device corresponds to the left ear, and the installation position of the second recording device Corresponding to the right ear, the method includes:

Control the first camera device and the second camera device to focus, obtain the first video data captured by the first camera device, and the second video data captured by the second camera device, and obtain the Obtaining the first audio data collected by the first recording device and obtaining the second audio data collected by the second recording device;

performing compression processing on the first video data, the second video data, the first audio data, and the second audio data to obtain compressed video data and compressed audio data;

Send the compressed video data and compressed audio data to other devices.

Optionally, the glasses frames of the smart glasses are further provided with a first display device, a second display device, a first listening device and a second listening device, and the installation position of the first display device corresponds to the left eye, The installation position of the second display device corresponds to the right eye, the installation position of the first listening device corresponds to the left ear, and the installation position of the second listening device corresponds to the right ear, and the method further includes :

Receive compressed video data and compressed audio data sent by other devices;

Decompressing compressed video data and compressed audio data sent by other devices to obtain third video data, fourth video data, third audio data, and fourth audio data;

Send the third video data to the first display device for display, send the fourth video data to the second display device for display, and send the third audio data to the first listening device for playback , sending the fourth audio data to the second listening device for playing.

Optionally, the controlling the first camera device and the second camera device to focus includes: controlling the first camera device and the second camera device to focus on the same point.

The present invention also provides a video playing method, which is applied to smart glasses. The glasses frames of the smart glasses are provided with a first display device, a second display device, a first listening device, and a second listening device. The first The installation position of the display device corresponds to the left eye, the installation position of the second display device corresponds to the right eye, the installation position of the first listening device corresponds to the left ear, and the installation position of the second listening device Corresponding to the right ear, the method includes:

Receive compressed video data and compressed audio data sent by other devices;

Decompressing compressed video data and compressed audio data sent by other devices to obtain first video data, second video data, first audio data, and second audio data;

Send the first video data to the first display device for display, send the second video data to the second display device for display, and send the first audio data to the first listening device for playback , sending the second audio data to the second listening device for playing.

The video shooting method, video playing method and smart glasses provided by the embodiments of the present invention can obtain video data of different viewpoints and audio data of different channels through two camera devices and two recording devices respectively, so the present invention realizes 3D video Filmed in real time. Since the installation position of the camera device corresponds to the eyes, and the installation position of the recording device corresponds to the ears, the video shot by the invention can make the viewer have an immersive feeling and be more real. The invention can automatically focus and shoot the surrounding scenery, so that the captured picture can form a 3D video. Moreover, the present invention can send video and audio data in real time, so that the user can enter the remote virtual scene at any time, obtain the current user's viewpoint position information or video attention point information, and truly realize 3D video real-time interaction. In addition, the video picture taken by the present invention changes with the change of the user's viewpoint position information, so that the video picture corresponding to the deflection angle direction of the remote scene can be obtained in real time according to the deflection change of the user's head perspective, so as to achieve intelligent control. level, significantly improving the user's video interaction experience.

Description of drawings

Specific embodiments of the smart glasses of the present invention will be described below with reference to the accompanying drawings, wherein:

FIG. 1 is a schematic diagram of the basic structure of smart glasses according to a preferred embodiment of the present invention;

FIG. 2 is a schematic diagram of intelligent 3D video interaction realized by smart glasses according to a preferred embodiment of the present invention;

3 is a schematic diagram of 3D shooting of smart glasses according to a preferred embodiment of the present invention;

FIG. 4 is a flow chart of a video shooting method provided by an embodiment of the present invention;

FIG. 5 is a flow chart of another video shooting method provided by an embodiment of the present invention;

FIG. 6 is a flowchart of a video playing method provided by an embodiment of the present invention.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Fig. 1 is the basic structural diagram of the smart glasses of the preferred embodiment of the present invention, as shown in Fig. 1, the smart glasses of the present invention are respectively provided with camera devices 11 and 12 for capturing video on the glasses frames near the human eyes, and for both eyes Display devices 21 and 22 for viewing video respectively, and recording devices 13 and 14 for collecting audio frequency, and listening devices 23 and 24 for listening to audio frequency are respectively arranged on the spectacle frame near the human ear.

In addition, the smart glasses in the preferred embodiment of the present invention are also provided with an auto-focus module, a processing device and a network communication device. The functions of these devices are described in detail below with reference to FIG. 2 , which is a schematic diagram of intelligent 3D video interaction realized by smart glasses according to a preferred embodiment of the present invention. As shown in FIG. 2 , the smart glasses 1 are the collection end of the 3D video, and the smart glasses 2 are the viewing end of the 3D video. The processing device of the smart glasses 1 compresses the video data captured by the receiving cameras 11 and 12 and the audio data collected by the receiving and recording devices 13 and 14, for example, the compression unit of the processing device of the smart glasses 1 can compress the above video data and audio data. The data is compressed. The smart glasses 1 send the compressed data obtained from the processing device to the smart glasses 2 by means of the network communication device. The network communication device of the smart glasses 2 receives the compressed data sent by the smart glasses 1 . The processing device of the smart glasses 2 decompresses the compressed data. For example, the decompression unit of the processing device of the smart glasses 2 decompresses the compressed data, and provides the decompressed video to the display devices 21 and 22 of the smart glasses 2 for synchronization. Play, and the audio frequency is provided to listening devices 23 and 24 for synchronous listening. The network communication device can use wireless communication methods such as wifi to access the Internet, so as to perform data transmission by means of the Internet.

Different from the dual cameras of the glasses in the prior art that independently shoot the scenes in the field of view, the autofocus device of the smart glasses of the present invention enables the camera device to focus and shoot at the same point, and its working principle is shown in Figure 3. It is shown in the 3D shooting schematic diagram of the preferred embodiment. As shown in FIG. 3 , the camera devices 11 and 12 are symmetrically installed on the spectacle frame close to the human eyes, and the shooting angles and focal lengths of the camera devices 11 and 12 remain the same. Therefore, the shooting point is always at the same position in the direction vertically forward from the center point of the two camera devices, so that the camera devices 11 and 12 can shoot the same camera point. The smart glasses of the present invention can use a control device (such as a chip) to realize that the shooting angles and focal lengths of the camera devices 11 and 12 are the same, and converge on the same shooting point. Specifically, the control device controls the camera devices 11 and 12 at the same time, so that the shooting angles of the camera devices are the same, so that the photographed point is in a direction vertically forward from the center point of the two camera devices. Then, the control device controls the camera devices 11 and 12 to adjust the focus at the same time until the captured picture is clear, so that the camera devices 11 and 12 can simultaneously focus and shoot the same point. Alternatively, either camera 11 or 12 may be set as the main camera, the main camera shoots the shooting point, and changes the shooting angle and focal length until the picture is clear. Then, the control device controls another camera device to take pictures with the same focal length and shooting angle parameters as the main camera device. Thus, it is also possible to realize that the photographed points of the photographing devices 11 and 12 are always in the direction vertically forward from the center point of the two photographing devices, so as to achieve the effect that the two photographing devices photograph the same photographing point. Of course, it is also possible to use a mechanical adjustment method to allow the camera devices 11 and 12 to shoot the same shooting point at the same time, such as setting buttons or knobs on the smart glasses of the present invention, and changing the camera devices 11 and 12 synchronously and symmetrically by adjusting the buttons or knobs. The shooting angle and focal length make the camera 12 and the camera 11 change the shooting angle and focal length at the same time, and the changed shooting angle and focal length are the same, so that the camera 11 and 12 can simultaneously focus and shoot the same shooting point.

In order to make the technical solution of the present invention easier to understand, a specific application scenario of the smart glasses of the present invention will be described below. For example, A and B are friends, A travels, and wants to share what he sees and hears with B in real time while traveling. In the existing way, A can only take photos and share them with B through the mobile phone camera, or take a video and share them with B. Obviously, such a sharing method is not real-time and interactive, and the photos and videos viewed by B have no 3D effect. Using the smart glasses of the present invention, A and B can perform real-time 3D video interaction only by wearing the smart glasses. Specifically, A puts on the smart glasses of the present invention and sets it to a shooting mode, and B puts on the smart glasses of the present invention and sets it to a viewing mode. While A is watching, the camera devices 11 and 12 of A's smart glasses capture the video of the scene he sees, and the recording devices 13 and 14 collect the audio frequency of the sound he hears, and compress it through the compression unit of the processing device , and then transmitted to B's smart glasses by the network communication device in real time. The network communication device of B's smart glasses receives the video and audio data transmitted by A's smart glasses, decompresses them through the decompression unit of the processing device, presents 3D video effects on the display devices 21 and 22, and displays them on the listening devices 23 and 24 Stereo sound is played on the Internet, so that B can share in real time what A sees and sounds that A hears. There is an immersive feeling, which greatly improves the user experience.

Not only that, but the present invention can add the function of voice real-time communication, that is, add a voice real-time communication device on the smart glasses to realize the function of voice communication. As a result, users can also perform real-time voice communication while interacting with 3D videos in real time, thereby enhancing the interactive experience between users. In addition, a storage device can also be added to the smart glasses, so that the user can store video and audio data, so that the smart glasses can be used to watch later. Moreover, the smart glasses of the present invention can not only realize one-to-one 3D video real-time interactive experience, but also realize one-to-many 3D video real-time interactive experience, and the network communication device of one smart glasses terminal can simultaneously send multiple smart glasses terminals Video and audio data to realize real-time 3D video interaction with multiple users. In addition, the application of the present invention is not limited to glasses, and can also be applied to helmets or other wearable devices.

The embodiment of the present invention also provides a video shooting method, which is applied to the smart glasses 1 shown in FIG. 2 or the smart glasses shown in FIG. 1, the smart glasses 1 shown in FIG. A first camera device 12, a second camera device 11, a first recording device 14 and a second recording device 13 are provided, the installation position of the first camera device 12 corresponds to the left eye, and the second camera device 11 The installation position corresponds to the right eye, the installation position of the first recording device 14 corresponds to the left ear, and the installation position of the second recording device 13 corresponds to the right ear, as shown in Figure 4, the method can include :

S001. Control the first camera device 12 and the second camera device 11 to focus, and obtain the first video data captured by the first camera device 12 and the second video data captured by the second camera device 11. For video data, the first audio data collected by the first recording device 14 is obtained, and the second audio data collected by the second recording device 13 is obtained;

Wherein, the manner of controlling the first camera device 12 and the second camera device 11 to focus may be: controlling the first camera device 12 and the second camera device 11 to focus on the same point.

S002. Perform compression processing on the first video data, the second video data, the first audio data, and the second audio data to obtain compressed video data and compressed audio data;

S003. Send the compressed video data and compressed audio data to other devices.

A video shooting method provided by an embodiment of the present invention can respectively acquire video data of different viewpoints and audio data of different channels through two camera devices and two recording devices, so the present invention realizes real-time shooting of 3D video. Since the installation position of the camera device corresponds to the eyes, and the installation position of the recording device corresponds to the ears, the video shot by the invention can make the viewer have an immersive feeling and be more real. The invention can automatically focus and shoot the surrounding scenery, so that the captured picture can form a 3D video. Moreover, the present invention can send video and audio data in real time, so that the user can enter the remote virtual scene at any time, obtain the current user's viewpoint position information or video attention point information, and truly realize 3D video real-time interaction. In addition, the video picture taken by the present invention changes with the change of the user's viewpoint position information, so that the video picture corresponding to the deflection angle direction of the remote scene can be obtained in real time according to the deflection change of the user's head perspective, so as to achieve intelligent control. level, significantly improving the user's video interaction experience.

On the basis of the embodiment shown in FIG. 4, the embodiment of the present invention also provides another video shooting method. As shown in FIG. 1, a first display device 22, a first display device 22, The second display device 21, the first listening device 24 and the second listening device 23, the installation position of the first display device 22 corresponds to the left eye, and the installation position of the second display device 21 corresponds to the right eye, The installation position of the first listening device 24 corresponds to the left ear, and the installation position of the second listening device 23 corresponds to the right ear, as shown in Figure 5, the method may also include:

S004. Receive compressed video data and compressed audio data sent by other devices;

S005. Decompress the compressed video data and compressed audio data sent by other devices to obtain third video data, fourth video data, third audio data, and fourth audio data;

S006. Send the third video data to the first display device 22 for display, send the fourth video data to the second display device 21 for display, and send the third audio data to the first listening device 24 for playing, and the fourth audio data is sent to the second listening device 23 for playing.

There may be various execution sequences of S004 to S006 and S001 to S003, which are not limited in the present invention.

It can be understood that the smart glasses in this embodiment are equipped with an imaging device, a recording device, a display device and a listening device at the same time, so not only video shooting but also video playback can be performed. Since video data captured by different camera devices are sent to corresponding display devices for display, and audio data collected by different recording devices are sent to corresponding listening devices for playback, the present invention can also realize 3D video playback and stereo playback. Users can watch 3D video and listen to stereo sound through the smart glasses.

The present invention also provides a video playing method, which is applied to the smart glasses shown in FIG. 1 or the smart glasses 2 shown in FIG. 2 . The smart glasses shown in FIG. 1 and the smart glasses 2 shown in FIG. There are a first display device 22, a second display device 21, a first listening device 24 and a second listening device 23, the installation position of the first display device 22 corresponds to the left eye, and the installation position of the second display device 21 The position corresponds to the right eye, the installation position of the first listening device 24 corresponds to the left ear, and the installation position of the second listening device 23 corresponds to the right ear, as shown in Figure 6, the method may include:

S100. Receive compressed video data and compressed audio data sent by other devices;

S200. Decompress compressed video data and compressed audio data sent by other devices to obtain first video data, second video data, first audio data, and second audio data;

S300. Send the first video data to the first display device 22 for display, send the second video data to the second display device 21 for display, and send the first audio data to the first listening device 24 for playback, and send the second audio data to the second listening device 23 for playback.

In the video playback method provided by this embodiment, video data captured by different camera devices can be sent to corresponding display devices for display, and audio data collected by different recording devices can be sent to corresponding listening devices for playback. Therefore, the present invention can Realize 3D video playback and stereo playback. Users can watch 3D video and listen to stereo sound through smart glasses, so as to get a more realistic video interactive experience.

It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is a relationship between these entities or operations. There is no such actual relationship or order between them. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

Each embodiment in this specification is described in a related manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, refer to part of the description of the method embodiment.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present invention are included in the protection scope of the present invention.

Claims (7)

1. A kind of smart glasses, it is characterized in that, be used for taking and playing 3D video, two camera devices (11, 12) for capturing video and two video recording devices (11, 12) for two eyes to watch video respectively are arranged on the glasses frame close to human eye position respectively. The display devices (21, 22) are respectively provided with two recording devices (13, 14) for collecting audio frequency and two listening devices (23, 24) for listening to the audio frequency on the glasses frame near the position of the human ear, and the smart glasses are also An autofocus device, a processing device, and a network communication device are provided; wherein, The autofocus device is connected with the two camera devices (11, 12), and is used to control the two camera devices (11, 12) to focus and capture video, and the autofocus device of the smart glasses is controlled by the control device. The above two camera devices (11, 12) focus on the same point at the same time, so that the shooting angles and focal lengths of the camera devices (11, 12) are the same, wherein, the same shooting angles and focal lengths of the camera devices (11, 12) include: control The device controls the camera 11 and 12 simultaneously, so that the shooting angle of the camera is the same, and the control device controls the camera 11 and 12 to adjust the focal length simultaneously until the picture taken is clear; or, the camera 11 or 12 is set to either The main camera device, the main camera device shoots the shooting point, changes the shooting angle and focal length until the picture is clear, and then the control device controls another camera device to have the same focal length and shooting angle parameters as the main camera device to shoot; The processing device and the two camera devices (11, 12), the two display devices (21, 22), the two recording devices (13, 14) and the two listening devices (23, 24) connection, for compressing the video data from the two camera devices (11, 12) and the audio data from the two recording devices (13, 14), and compressing the compressed video from the network communication device performing decompression processing with the audio data, so that the two display devices (21, 22) display the video and the two listening devices (23, 24) listen to the audio; The network communication device is connected with the processing device, and is used for sending the compressed video and audio data from the processing device, or receiving the compressed video and audio data and sending them to the processing device. 2 . The smart glasses according to claim 1 , wherein the smart glasses are further provided with a voice real-time communication device, and the voice real-time communication device is connected to the processing device. 3. The smart glasses according to claim 1, wherein the smart glasses are further provided with a storage device for storing video and audio data. 4. The smart glasses according to claim 1, characterized in that, the control device of the smart glasses controls the two camera devices (11, 12) to adjust simultaneously to achieve the same shooting angle and the same focal length. 5. The smart glasses according to claim 1, wherein the control device of the smart glasses controls the shooting angle and focal length of the first camera (11) until the scene is clear, and then controls the second camera (12) Shoot with the same shooting angle and focal length as the first camera device (11). 6. A video shooting method, characterized in that it is applied to smart glasses, the smart glasses are used to shoot 3D videos, and the glasses frame of the smart glasses is provided with a first camera, a second camera, a first recording device and a second recording device, the installation position of the first camera is corresponding to the left eye, the installation position of the second camera is corresponding to the right eye, and the installation position of the first recording device is corresponding to the left ear Correspondingly, the installation position of the second recording device corresponds to the right ear, and the method includes: Controlling the first camera device and the second camera device to focus on the same point to achieve the same shooting angle and focal length of the camera devices, wherein, realizing the same shooting angle and focal length of the camera devices (11, 12) includes: the control device Camera 11 and 12 are controlled simultaneously, so that the shooting angle of camera is the same, and control device controls camera 11 and 12 to adjust focal length simultaneously, until the picture of shooting is clear; device, the main camera device shoots the shooting point, changes the shooting angle and focal length until the picture is clear, and then the control device controls another camera device to shoot with the same focal length and shooting angle parameters as the main camera device; obtain the The first video data captured by the first camera device, and the second video data captured by the second camera device, the first audio data collected by the first recording device, and the second video data captured by the second recording device The collected second audio data; performing compression processing on the first video data, the second video data, the first audio data, and the second audio data to obtain compressed video data and compressed audio data; Send the compressed video data and compressed audio data to other devices. 7. The method according to claim 6, wherein a first display device, a second display device, a first listening device and a second listening device are also provided on the spectacle frame of the smart glasses, and the first The installation position of the display device corresponds to the left eye, the installation position of the second display device corresponds to the right eye, the installation position of the first listening device corresponds to the left ear, and the installation position of the second listening device Corresponding to the right ear, the method further includes: Receive compressed video data and compressed audio data sent by other devices; Decompressing compressed video data and compressed audio data sent by other devices to obtain third video data, fourth video data, third audio data, and fourth audio data; Send the third video data to the first display device for display, send the fourth video data to the second display device for display, and send the third audio data to the first listening device for playback , sending the fourth audio data to the second listening device for playing.