CN107040794A - Video broadcasting method, server, virtual reality device and panoramic virtual reality play system - Google Patents

Abstract

The application provides a kind of video broadcasting method, server, virtual reality device and panoramic virtual reality play system, and methods described includes：Compression original video code stream obtains preview video code stream, sends preview video code stream to virtual reality device；The playing request of virtual reality device is received, wherein, playing request is used for the region-of-interest for determining user's selection；Original video code stream is handled according to playing request, video code flow is played in generation, wherein, the image sharpness of the region-of-interest in broadcasting video code flow is higher than the image sharpness outside region-of-interest；Send and play video code flow to virtual reality device.The application can be by sending the less previewing video files of data volume to user, selection attentinal contents are browsed to user, and the region-of-interest selected according to user reduces the picture quality of the image-region in addition to region-of-interest, when user perspective switches, real-time update viewing area is high-definition image, ensure the picture quality of user's region-of-interest, while experience to user's panoramic high-definition, the network bandwidth is saved to greatest extent.

Description

Video playback method, server, virtual reality device, and panoramic virtual reality playback system

technical field

The present application relates to the video field, in particular to a video playing method, a server, a virtual reality device and a panoramic virtual reality playing system.

Background technique

In the prior art, virtual reality technology is a computer simulation system that can create and experience a virtual world. It uses a computer to generate a simulated environment, which is a multi-source information fusion, interactive three-dimensional dynamic scene and entity behavior. A system simulation immerses the user in the environment.

With the development of virtual reality technology, video playback through virtual reality equipment is becoming more and more popular. However, because the scene captured by the video capture device is relatively large, the amount of data transmitted to the video stream of the virtual reality device is also relatively large, which requires relatively high network bandwidth, and the virtual reality device switches images according to the viewing angle of the user. Screen, if the data volume is relatively large, when switching the image screen, the loading speed of the image will be very slow, which will affect the user experience.

Especially when applied in the field of panoramic images, the video acquisition device collects the panoramic video, and then uploads the panoramic video to the server. When the virtual reality device requests the server to send the panoramic video, if the image quality of the whole panoramic image is high, the network Bandwidth will have particularly high requirements. When the user changes the angle of view, because the resolution of the collected video is relatively high, the amount of transmitted data is relatively large, and the loading of the image is very slow, which affects the user experience. In addition, the user may only pay attention to part of the content in the panoramic image. All images in the image are transmitted to the VR device in the original image quality, which will result in a waste of bandwidth resources.

Contents of the invention

This application provides a video playback method, server, virtual reality device and panoramic virtual reality playback system, which can solve the problem that the video stream of the virtual reality device has a large amount of data and the network bandwidth is limited when the video is played through the virtual reality device. The requirements will be relatively high, and because of the large amount of transmitted data, the image loading will be very slow when the user switches the viewing angle, which will affect the user experience and even cause a waste of bandwidth resources.

According to the first aspect of the present application, the present application provides a video playing method, which is applied to a server. The video playing method includes: receiving the original video code stream; receiving a play request from a virtual reality device, wherein the play request is used to determine the user selection area of interest; according to the playback request, the original video code stream is processed to generate a video code stream for playback, wherein the image definition of the area of interest in the video code stream is higher than that of the image outside the area of interest; send the video code stream for playback Stream to a virtual reality device.

Preferably, in the step of processing the original video code stream according to the playback request, it includes: extracting the original image frame in the original video code stream, and extracting the image corresponding to the area of interest from the original image frame according to the playback request; The tag of the image of the same tag is obtained to obtain the original image frame of the same tag; the original image frame of the same tag is compressed according to the set image quality.

Preferably, before the step of receiving the play request of the virtual reality device, it includes: extracting the original image frame in the original video code stream, dividing the original image frame into a plurality of sub-image blocks according to the set rules; compressing the sub-image blocks to have At least two set resolutions; in the step of processing the original video code stream according to the playback request, including: extracting sub-image blocks with the first set resolution corresponding to the attention area; extracting the original image frame except the attention area Sub-image blocks of the second resolution corresponding to the remaining areas outside the sub-image blocks of the first resolution and having the same label as the sub-image blocks of the first resolution, wherein the first resolution is higher than the second resolution.

Preferably, in the step of processing the original video code stream according to the playback request, it also includes: splicing the image corresponding to the region of interest and the original image frame with the same mark as the image corresponding to the region of interest into a playback image frame, and playing the image frame Compose the playback video code stream.

Preferably, in the step of processing the original video code stream according to the playback request, it also includes: splicing the sub-image blocks corresponding to the attention area and the sub-image blocks corresponding to the remaining areas except the attention area into playback image frames, and playing Image frames form the playback video code stream.

Preferably, after the step of generating the playing video code stream, the video playing method further includes: buffering and playing the video code stream; before the step of processing the original video code stream according to the playing request, the video playing method further includes: judging whether there is a playing Request the cache of the corresponding playback video code stream, if yes, send the playback video code stream to the virtual reality device, otherwise, continue to process the original video code stream according to the playback request.

According to the second aspect of the present application, the present application provides a video playing method, which is applied to a virtual reality device. The video playing method includes: generating a play request according to the area of interest selected by the user, and sending the play request to the server; receiving and processing the server The returned playback video code stream; play the playback video code stream.

Preferably, in the step of receiving and processing the playback video code stream returned by the server, it includes: splicing the image corresponding to the region of interest and the original image frame with the same mark as the image corresponding to the region of interest into a playback image frame, and the playback image frame consists of Play video code stream.

Preferably, in the step of receiving and processing the playback video code stream returned by the server, it also includes: splicing the sub-image blocks corresponding to the attention area and the sub-image blocks corresponding to other areas except the attention area into playback image frames, and playing Image frames form the playback video code stream.

According to the third aspect of the present application, the present application provides a server, the server includes: an original video code stream receiving module configured to receive the original video code stream; a play request receiving module configured to receive a play request from a virtual reality device, Wherein, the play request is used to determine the attention area selected by the user; the processing module is configured to process the original video code stream according to the play request, and generate a play video code stream, wherein, the picture definition of the attention area in the play video code stream The picture definition is higher than that outside the attention area; the sending module is configured to send the playback video stream to the virtual reality device.

Preferably, the processing module includes: a first extraction unit configured to extract the original image frame in the original video code stream, and extract an image corresponding to the area of interest from the original image frame according to the playback request; the acquisition unit is configured to extract the image corresponding to the area of interest according to the area of interest The corresponding tag of the image acquires the original image frame with the same tag; the first compression unit is configured to compress the original image frame with the same tag according to the set image quality.

Preferably, the server further includes a division module, which includes: a division unit configured to extract the original image frame in the original video code stream, decode and divide the original image frame into a plurality of sub-image blocks according to set rules; the second compression unit , configured to compress the sub-image block so that it has at least two set resolutions; the processing module includes: a second extraction unit configured to extract the sub-image block corresponding to the region of interest with the first set resolution; the third extraction A unit that extracts a sub-image block with a second resolution corresponding to the rest of the original image frame except for the area of interest and having the same label as the sub-image block of the first resolution, wherein the first resolution is higher than the sub-image block of the first resolution Second resolution.

Preferably, the processing module further includes: a first splicing unit configured to splice the image corresponding to the region of interest and the original image frame having the same mark as the image corresponding to the region of interest into a playback image frame, and the playback image frame forms a playback video stream .

Preferably, the processing module further includes: a second splicing unit, configured to splice the sub-image blocks corresponding to the region of interest and the sub-image blocks corresponding to the remaining regions except the region of interest into playback image frames, and the playback image frames form a playback video stream.

Preferably, the server further includes: a caching module, configured to cache the video code stream for playing after the video code stream is generated; a judging module, configured to judge whether the original video code stream is processed according to the playback request There is a cache of the playing video code stream corresponding to the playing request, if so, send the playing video code stream to the virtual reality device, otherwise, continue to process the original video code stream according to the playing request.

According to the fourth aspect of the present application, the present application provides a virtual reality device. The virtual reality device includes: a play request generation module configured to generate a play request according to the area of interest selected by the user, and send the play request to the server; video receiving processing The module is configured to receive and process the playing video stream returned by the server; the playing module is configured to play the playing video stream.

Preferably, the video receiving processing module further includes: a first splicing module configured to splice the image corresponding to the region of interest and the original image frame with the same mark as the image corresponding to the region of interest into a playback image frame, and the playback image frame forms a playback video stream.

Preferably, the video receiving processing module further includes: a second splicing module, configured to splice the sub-image blocks corresponding to the attention area and the sub-image blocks corresponding to other areas except the attention area into playback image frames, and the playback image frames consist of Play video code stream.

According to the fifth aspect of the present application, the present application provides a panoramic virtual reality playback system, including: a panoramic camera device for shooting panoramic images, the server as described above, and the virtual reality device as described above, wherein the panoramic camera device Take a video, and send the captured original video stream to the server, and the server receives the original video stream.

The beneficial effect of the present application is that: by generating the playback video code stream according to the attention area selected by the user, the image quality of other areas in the playback video code stream except the attention area can be reduced, which can greatly reduce the impact on the network during the transmission of the video code stream. Bandwidth requirements, and when the user switches the image screen of the area of interest, the loading image screen of the area of interest can be quickly focused on, which improves user experience and does not cause waste of bandwidth resources.

Description of drawings

FIG. 1 is an application scene diagram of a video playback method in the first embodiment of the present application;

Fig. 2 is the flowchart of the video playing method in the second embodiment of the present application;

Fig. 3 is the flowchart of the video playing method in the third embodiment of the present application;

FIG. 4 is a schematic diagram of dividing the original image frame into sub-image blocks in the video playback method in the fourth embodiment of the present application;

FIG. 5 is a flowchart of a video playback method in the fourth embodiment of the present application;

FIG. 6 is a flowchart of a video playback method in the fifth embodiment of the present application;

Fig. 7 is a schematic diagram of a video playback device in the sixth embodiment of the present application;

Fig. 8 is a schematic diagram of a video playback device in the seventh embodiment of the present application;

FIG. 9 is a schematic diagram of a video playback device in an eighth embodiment of the present application;

FIG. 10 is a schematic diagram of a video playback device in the ninth embodiment of the present application.

detailed description

The present application will be described in further detail below through specific embodiments in conjunction with the accompanying drawings.

First some terminology is described. VR, the full name is Virtual Reality, Chinese is virtual reality.

The idea of this application is to reduce the image quality of other areas in the playback video stream except for the area of interest by generating the playback video stream according to the area of interest selected by the user, thus greatly reducing the impact on the network bandwidth during video stream transmission. requirements, and when the user switches the image screen of the area of interest, the loading image screen of the area of interest can be quickly focused on, which improves user experience and does not cause waste of bandwidth resources.

Embodiment one:

As shown in FIG. 1 , FIG. 1 is an application scenario of a video playing method according to an embodiment of the present disclosure, in which the application scenario includes: a panoramic camera device 100 , a virtual reality device 101 and a server 102 . Among them, the panoramic camera device 100 and the virtual reality device 101 are connected in communication with the server 102, and the video captured by the panoramic camera device 100 is transmitted to the server 102, and the server 102 processes it and sends it to the virtual reality device 101 for display. In practical applications, the panoramic camera device 100 may be a panoramic camera device composed of one panoramic camera, or a panoramic camera device composed of two or more cameras. The virtual reality device may include: a modeling device, a three-dimensional visual display device, a sound device, an interactive device, a head-mounted virtual reality display device or virtual reality glasses, and the like. In addition, the virtual reality device 101 can also communicate with a terminal device to receive video or images transmitted by the terminal device. In practical applications, the terminal device can be any intelligent terminal with Internet access capabilities, such as smart phones, tablet computers wait.

Embodiment two:

As shown in FIG. 2, FIG. 2 is a flowchart of a video playback method according to an example of the present disclosure, and the method may include the following steps:

In step S201, an original video code stream is received.

In this embodiment, the original video code stream from the video acquisition device is received. The video capture device is a panoramic camera device composed of one panoramic camera, or a panoramic camera device composed of two or more cameras.

In step S202, a play request from the virtual reality device is received, wherein the play request is used to determine the attention area selected by the user.

The user focus area may be a user visible area, or may be one or more focus areas selected by the user in the video screen. For example, the user wears a virtual reality device or VR glasses, and the user's attention area is the user's visible area. For another example, when applied to a panoramic image, the user may select multiple regions in the panoramic image as regions of interest.

In step S203, the original video code stream is processed according to the play request to generate a play video code stream, wherein the picture definition of the video content corresponding to the attention area is higher than that outside the attention area.

Because the image of the corresponding video content in the area of interest is what the user cares about, as long as the image definition of this area is high enough, it can be guaranteed that the content that the user cares about can be presented to the user, and the image screen outside the area of interest does not need to be distinguished The smaller the resolution, the more the amount of data that needs to be transmitted can be reduced. In this way, the network bandwidth occupied by the video code stream is much smaller than the network bandwidth occupied by the video code stream captured by the video capture device in the prior art when it is directly transmitted to the terminal. bandwidth, greatly reducing the burden on the server.

In step S204, send the playing video code stream to the virtual reality device.

In this embodiment, the transmitted video code stream may include: images of the region of interest with higher resolution and images of other regions other than the region of interest are spliced together, or non-spliced images with higher resolution The code stream of the images of the attention area of the highest rate and the images of other areas outside the attention area is spliced, and the latter needs to be spliced on the terminal device, which is not limited in this embodiment.

In another feasible embodiment, after step S203, further steps are included:

Cache and play the video code stream. That is, as long as a playback request is received, the image corresponding to the region of interest in the playback request is acquired and cached on the storage device of the server.

Before S203, also include steps:

Judging whether there is a cache of the playing video code stream corresponding to the playing request, if yes, sending the playing video code stream to the virtual reality device, otherwise, continuing to process the original video code stream according to the playing request.

As long as the server receives a playback request from the virtual reality device, it will cache the corresponding video stream, so that when any virtual reality device connected to the server sends the same playback request to the server, that is, as long as the user's attention area corresponding to the playback request is on the server When the processed image cache is stored on the server, the server does not need to process it, and can directly send this type of cache to the terminal device. No matter how many terminal devices access the server, the same type of request only needs to perform data processing once, which greatly reduces the computing burden of the server, reduces the network bandwidth, and can support unlimited data access.

It can be seen from the above embodiment that the video playback method of this embodiment can reduce the image quality of the video file, save the network bandwidth to the greatest extent, and at the same time, ensure the clarity of the user's attention area, because the amount of data to be processed is reduced, This enables users to switch screens in real time, avoiding the problem of too long loading time of image screens due to excessive data volume, and improving user experience.

Embodiment three:

Please refer to FIG. 3. FIG. 3 is a flow chart of a video playback method according to another example of the present disclosure. The method may include the following steps:

In step S301, an original video code stream is received.

In step S302, a play request from the virtual reality device is received, wherein the play request is used to determine the attention area selected by the user.

The user-focused area may be a user-visible area, or one or more user-selected areas in a video screen. For example, the user wears a virtual reality device or VR glasses, and the user's attention area is the user's visible area. For another example, when applied to a panoramic image, the user may select multiple areas in the panoramic image as the attention area.

In step S303, the original video code stream is processed according to the play request to generate a play video code stream, wherein the picture definition of the attention area is higher than that of the picture outside the attention area.

In this embodiment, step S303 may include:

Step S303A: Extract the original image frame in the original video code stream, and extract the image corresponding to the attention area from the original image frame according to the playback request.

Step S303B: According to the label of the image corresponding to the attention area, obtain the original image frame with the same label. In this embodiment, the mark is a timestamp. That is, the original image frame with the same time stamp is acquired according to the time stamp of the image corresponding to the region of interest. In other embodiments, it can also be implemented by setting labels for images. There is no limitation on the marking method here, as long as the original image frame from which the image corresponding to the region of interest comes from is found.

Step S303C: Compress the original image frames with the same mark according to the set image quality.

In step S304, send the playing video code stream to the virtual reality device.

The sent playback video code stream may include: an image of the region of interest with a higher resolution and images of other regions outside the region of interest are spliced together, or an image of the region of interest with a higher resolution that has not been spliced And the code stream of images in other areas outside the area of interest, the latter needs to be spliced on the terminal device. In this embodiment, there is no limitation to the two ways of splicing the images first and then sending them to the device terminal, or sending the images to the device terminal and then splicing the images by the device terminal.

In another feasible embodiment, after step S303, further steps are included:

Cache and play the video code stream. That is, as long as a playback request is received, the image corresponding to the visible area is acquired.

Before S303, also include steps:

Judging whether there is a cache of the playing video code stream corresponding to the playing request, if yes, sending the playing video code stream to the virtual reality device, otherwise, continuing to process the original video code stream according to the playing request.

As long as the server receives a playback request from the virtual reality device, it will cache the corresponding video stream, so that when any virtual reality device connected to the server sends the same playback request to the server, that is, as long as the user's attention area corresponding to the playback request is on the server When the processed image cache is stored on the server, the server does not need to process it, and can directly send this type of cache to the terminal device. No matter how many terminal devices access the server, the same type of request only needs to perform data processing once, which greatly reduces the computing burden of the server, reduces the network bandwidth, and can support unlimited data access.

It can be seen from the above embodiment that the video playback method of this embodiment can reduce the image quality of the video file, save the network bandwidth to the greatest extent, and at the same time, ensure the clarity of the user's attention area, because the amount of data to be processed is reduced, This enables users to switch screens in real time, avoiding the problem of too long loading time of image screens due to excessive data volume, and improving user experience.

Embodiment four:

Please refer to FIG. 4 and FIG. 5. FIG. 5 is a flowchart of a video playback method according to another example of the present disclosure. The method may include the following steps:

In step S501, the original image frame in the original video code stream is extracted, and the original image frame is divided into a plurality of sub-image blocks according to set rules.

As shown in FIG. 4, FIG. 4 is a tiling map of the original image frame, which divides the original image frame into M*N sub-image blocks. The position of the sub-image block is determined by the coordinates of its four corners on the original image frame. For example, taking the point in the lower left corner of the original image frame as the origin, the position of the sub-image block in the lower left corner in the image frame passes the coordinates (0, 0), (1, 0), (0, 1) of its four corners, (1, 1) to represent. It can also be expressed in other ways, which is not limited here.

In step S502: Compressing the sub-image blocks to have at least two set resolutions.

For example, according to the network bandwidth of the terminal device, the sub-image blocks may be compressed into standard-definition resolution, high-definition resolution, and ultra-definition resolution, that is, 360P, 720P, and 1080P.

In step S503, a play request from the virtual reality device is received, wherein the play request is used to determine the attention area selected by the user.

The user focus area may be a user visible area, or may be one or more focus areas selected by the user in the video screen. For example, the user wears a virtual reality device or virtual reality glasses, and the user's attention area is the user's visible area. For another example, when applied to a panoramic image, the user may select multiple regions in the panoramic image as regions of interest.

Please continue to refer to FIG. 5 , the size of the user's attention area is the size of J*K sub-tiles. If the moving range of the user's attention area relative to the last attention area is less than the set distance, it is considered that the attention area has no change. For example, when the user interacts through VR glasses, the user's visible area is the attention area. When the user's eyeballs move, the visible area also changes. If the change between the current visible area and the previous visible area does not exceed the sub-image block When the size is half of the size, it is considered that the visible area has not changed.

In step S504, the original video code stream is processed according to the play request to generate a play video code stream, wherein the picture definition of the attention area is higher than that of the picture outside the attention area.

In this embodiment, step S504 may include:

Step S504A: Extracting sub-image blocks corresponding to the attention area and having the first set resolution.

Step S504B: Extract sub-image blocks of the second resolution corresponding to the remaining areas in the original image frame except the area of interest and having the same label as the sub-image blocks of the first resolution, wherein the first resolution is higher than second resolution.

In other embodiments, step S504 may also include step 504C: splicing the sub-image blocks corresponding to the attention area and the sub-image blocks corresponding to other areas except the attention area into playback image frames, and the playback image frames form the playback video stream .

In step S505, send the playing video code stream to the virtual reality device.

In this embodiment, the transmitted video code stream may include: images of the region of interest with higher resolution and images of other regions other than the region of interest are spliced together, or non-spliced images with higher resolution The code stream of the image of the area of interest of the highest rate and the images of the rest of the area outside the area of interest. In the latter case, splicing needs to be performed on the virtual reality device side, which is not limited in this embodiment.

In another feasible embodiment, after step S503, further steps are included:

Cache and play the video code stream. That is, as long as a playback request is received, the image corresponding to the attention area is acquired.

Based on the above steps, before S503, the steps include:

Judging whether there is a cache of the playing video code stream corresponding to the playing request, if yes, sending the playing video code stream to the virtual reality device, otherwise, continuing to process the original video code stream according to the playing request.

As long as the server receives a playback request from the virtual reality device, it will cache the corresponding video stream, so that when any virtual reality device connected to the server sends the same playback request to the server, the server does not need to splicing, and directly sends this type of cache to the virtual reality device. No matter how many terminal devices access the server, the same type of request only needs to perform data processing once, which greatly reduces the computing burden of the server and can support unlimited data access.

It can be seen from the above-mentioned embodiment that the video playing method of this embodiment can divide the image of the live content into sub-image blocks, and then encode them into multiple resolutions. The combination can easily adjust the appropriate image quality and size, save network bandwidth, and ensure that the image of the user's attention area maintains the original quality, and because the data volume is reduced, the image can be loaded quickly when the user switches the attention area. Improved user experience.

Embodiment five:

As shown in FIG. 6, FIG. 6 is a flowchart of a video playback method according to another example of the present disclosure, which is applied to a virtual reality device. The method may include the following steps:

In step S601: generate a play request according to the attention area selected by the user, and send the play request to the server.

In this embodiment, the user's selection of the attention area may be realized by detecting the attention area of the user's eyeballs, or by the user operating the controller, or by detecting the user's gesture action.

Specifically, the user selects the attention area by detecting the user's eye focus area through the user's wearing virtual reality device, or by detecting the user's gesture through the user's wearing virtual reality device, or by the user operating the controller.

Please continue to refer to FIG. 5 , the size of the user's attention area is the size of J*K sub-tiles. It should be noted here that if the moving range of the user's attention area relative to the last attention area is less than a set distance, it is considered that the attention area has not changed. For example, when the user interacts through VR glasses, the user's visible area is the attention area. When the user's eyeballs move, the visible area also changes. If the change between the current visible area and the previous visible area does not exceed the sub-image block When the size is a quarter of the time, it is considered that the visible area has not changed. The setting distance is set according to the actual usage situation of the user, and there is no limitation here.

In step S602: receiving and processing the playback video code stream returned by the server.

In this embodiment, the received playback video code stream may include: images of the region of interest with higher resolution and images of other regions other than the region of interest are stitched together, or images with higher resolution that have not been spliced The code stream of the image of the area of interest and the images of the rest of the area outside the area of interest. If it is the latter, the sub-image blocks corresponding to the attention area need to be spliced with the sub-image blocks corresponding to other areas outside the attention area on the virtual reality device side.

For the fourth embodiment, the received playback video code stream includes sub-image blocks with the first resolution corresponding to the area of interest and sub-image blocks with the second resolution in the remaining areas, and also includes a protocol for splicing the two .

In step S603: playing the playing video code stream.

In this embodiment, in playing the video code stream, only the image corresponding to the attention area that the user's eyes focus on maintains the original definition, and the images corresponding to the other areas have lower definition.

It should be noted that the ROI here is not limited to one, and there may be multiple ROIs, that is, there are multiple ROIs with relatively high definition in the live broadcast image. In addition to live video, video on demand can also use the above method.

It should be noted that, although operations of the methods of the present invention are depicted in the figures in a particular order, this does not require or imply that operations be performed in that particular order, or that all illustrated operations must be performed to achieve desirable results. Conversely, the steps depicted in the flowcharts may be performed in an altered order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step for execution, and/or one step may be decomposed into multiple steps for execution.

Embodiment six:

Corresponding to the embodiment of the video playing method applied to the server shown in FIG. 2 , the present disclosure also provides an embodiment of the server.

As shown in FIG. 7 , FIG. 7 shows a schematic diagram of a server according to an example of the present disclosure, and the server may include:

The original video code stream receiving module 701 is configured to receive the original video code stream;

The play request receiving module 702 is configured to receive a play request from the virtual reality device, wherein the play request is used to determine the attention area selected by the user;

The processing module 703 is configured to process the original video code stream according to the playback request, and generate a playback video code stream, wherein the picture definition of the attention area in the playback video code stream is higher than the picture definition outside the attention area;

The sending module 704 is configured to send the playing video code stream to the virtual reality device.

In another embodiment, the video playback device also includes:

The caching module is configured to cache and play the video code stream after the video code stream is generated;

The judging module is configured to, before the step of processing the original video code stream according to the play request, judge whether there is a cache of the play video code stream corresponding to the play request, and if so, send the play video code stream to the virtual reality device, otherwise , continue to process the original video code stream according to the playback request.

Embodiment seven:

Corresponding to the embodiment of the video playing method applied to the server shown in FIG. 3 , the present disclosure also provides an embodiment of the server.

As shown in FIG. 8, FIG. 8 shows a schematic diagram of a server according to another exemplary embodiment of the present disclosure, and the server may include:

The original video code stream receiving module 801 is configured to receive the original video code stream.

The play request receiving module 802 is configured to receive a play request from the virtual reality device, wherein the play request is used to determine the attention area selected by the user.

The processing module 803 is configured to process the original video code stream according to the play request to generate a play video code stream, wherein the picture definition of the attention area is higher than that of the picture outside the attention area.

The processing module 803 may include:

The first extracting unit 8031 is configured to extract the original image frame in the original video code stream, and extract the image corresponding to the attention area from the original image frame according to the playback request.

The acquiring unit 8032 is configured to acquire the original image frame with the same marker according to the marker of the image corresponding to the attention area.

The first compression unit 8033 is configured to compress the original image frames with the same mark according to the set image quality.

The sending module 804 is configured to send the playing video code stream to the virtual reality device.

The sent playback video code stream may include: an image of the region of interest with a higher resolution and images of other regions outside the region of interest are spliced together, or an image of the region of interest with a higher resolution that has not been spliced And the code stream of images in other areas outside the area of interest, the latter needs to be spliced on the terminal device. In this embodiment, there is no limitation to the two ways of splicing the images first and then sending them to the device terminal, or sending the images to the device terminal and then splicing the images by the device terminal.

In another feasible embodiment, the video playback device also includes:

The cache module is configured to cache and play video code streams. That is, as long as a playback request is received, the image corresponding to the visible area and the corresponding image outside the visible area are acquired.

The judging module is configured to, before the step of processing the original video code stream according to the play request, judge whether there is a cache of the play video code stream corresponding to the play request, and if so, send the play video code stream to the virtual reality device, otherwise , continue to process the original video code stream according to the playback request.

Embodiment eight:

Corresponding to the embodiment of the video playing method applied to the server shown in FIG. 5 , the present disclosure also provides an embodiment of the server.

As shown in FIG. 9, FIG. 9 shows a schematic diagram of a server according to another exemplary embodiment of the present disclosure, and the server may include:

The division module 901 is configured to extract the original image frame in the original video code stream, and divide the original image frame into a plurality of sub-image blocks according to set rules.

The second compression module 902 is configured to compress the sub-image blocks to have at least two set resolutions.

The play request receiving module is configured to receive a play request from the virtual reality device, wherein the play request is used to determine the attention area selected by the user.

The processing module 903 is configured to process the original video code stream according to the play request, and generate a play video code stream, wherein the picture definition of the attention area is higher than that of the picture outside the attention area.

The processing module 903 may include:

The first extraction unit 9031 is configured to extract the original image frame in the original video code stream, and extract the image corresponding to the region of interest from the original image frame according to the playback request;

The acquiring unit 9032 is configured to acquire an original image frame with the same marker according to the marker of the image corresponding to the region of interest;

The first compression unit 9033 is configured to compress the original image frames with the same mark according to the set image quality.

In other embodiments, the processing module 903 may further include: a second splicing unit configured to splice the sub-image blocks corresponding to the attention area and the sub-image blocks corresponding to other areas except the attention area into a playback image frame, and play Image frames form the playback video code stream.

The sending module 904 is configured to send the playing video code stream to the virtual reality device.

In this embodiment, the transmitted video code stream may include: images of the region of interest with higher resolution and images of other regions other than the region of interest are spliced together, or non-spliced images with higher resolution The code stream of the image of the area of interest of the highest rate and the images of the rest of the area outside the area of interest. In the latter case, splicing needs to be performed on the virtual reality device side, which is not limited in this embodiment.

In this embodiment, the transmitted video code stream may include: images of the region of interest with higher resolution and images of other regions other than the region of interest are spliced together, or non-spliced images with higher resolution The code stream of the image of the area of interest of the highest rate and the images of the rest of the area outside the area of interest. In the latter case, splicing needs to be performed on the virtual reality device side, which is not limited in this embodiment. In another embodiment, the video playback device also includes:

The caching module is configured to cache and play the video code stream after the video code stream is generated;

The judging module is configured to, before the step of processing the original video code stream according to the play request, judge whether there is a cache of the play video code stream corresponding to the play request, and if so, send the play video code stream to the virtual reality device, otherwise , continue to process the original video code stream according to the playback request.

Embodiment nine:

Corresponding to the embodiment of the video playing method applied to the virtual reality device shown in FIG. 6 , the present disclosure also provides an embodiment of the virtual reality device.

As shown in FIG. 10 , FIG. 10 shows a schematic diagram of a virtual reality device according to another exemplary embodiment of the present disclosure, and the virtual reality device may include:

The play request generating module 1001 is configured to generate a play request according to the attention area selected by the user, and send the play request to the server.

The video receiving and processing module 1002 is configured to receive and process the playback video stream returned by the server.

The playing module 1003 is configured to play the playing video code stream.

It should be noted that the ROI here is not limited to one, and there may be multiple ROIs, that is, there are multiple ROIs with relatively high definition in the live broadcast image. In addition to live video, video on demand can also use the above method.

Embodiment ten:

The present application provides a panoramic virtual reality playback system, including: a panoramic camera device for shooting panoramic images, the above-mentioned server, and a virtual reality device. Among them, the panoramic shooting device shoots video and sends the captured original video code stream to the server, the server receives the original video code stream, the virtual reality device sends a playback request, and the server sends the processed video code stream to On the virtual reality device side, the virtual reality device plays the processed video stream.

The beneficial effects of the present application are: by generating a preview video code stream for the user to select an area of interest, generating a playback video code stream according to the area of interest selected by the user, reducing the image quality of other areas in the playback video code stream except for the area of interest, so that It can greatly reduce network bandwidth, improve user experience, and will not cause resource waste.

Those skilled in the art can understand that all or part of the steps of the various methods in the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, Random access memory, magnetic disk or optical disk, etc.

The above content is a further detailed description of the present application in conjunction with specific implementation modes, and it cannot be considered that the specific implementation of the present application is limited to these descriptions. For those of ordinary skill in the technical field to which the present application belongs, some simple deduction or replacement can also be made without departing from the inventive concept of the present application.

Claims (19)

1. a kind of video broadcasting method, it is characterised in that it is applied to server, the video broadcasting method includes：

Receive original video code stream；

The playing request of the virtual reality device is received, wherein, the playing request is used for the concern area for determining user's selection Domain；

The original video code stream is handled according to the playing request, video code flow is played in generation, wherein, it is described to play The image sharpness of region-of-interest in video code flow is higher than the image sharpness outside region-of-interest；

The broadcasting video code flow is sent to the virtual reality device.

2. video broadcasting method as claimed in claim 1, it is characterised in that original regarded to described according to playing request described In the step of frequency code stream is handled, including：

The original image frame in the original video code stream is extracted, is extracted according to the playing request from the original image frame The corresponding image of the region-of-interest；

According to the mark of the corresponding image of the region-of-interest, the original image frame of same tag is obtained；

The original image frame of same tag is compressed according to setting picture quality.

3. video broadcasting method as claimed in claim 1, it is characterised in that in broadcasting for the reception virtual reality device Before the step of putting request, including：

The original image frame in original video code stream is extracted, by the original image frame according to setting regular partition as multiple subgraphs As block；

Compress the subimage block and make it have at least two setting resolution ratio；

It is described the original video code stream is handled according to playing request the step of in, including：

Extract the corresponding subimage block with the first setting resolution ratio of the region-of-interest；

Extract remaining region in the original image frame in addition to the region-of-interest corresponding and differentiated with described first The subimage block with second resolution of the same tag of the subimage block of rate, wherein, the first resolution is higher than described Second resolution.

4. video broadcasting method as claimed in claim 2, it is characterised in that original regarded to described according to playing request described In the step of frequency code stream is handled, in addition to：By the corresponding image of the region-of-interest and with the region-of-interest pair The original image frame for the image same tag answered is spliced into broadcasting picture frame, and the broadcasting picture frame constitutes the broadcasting video codes Stream.

5. video broadcasting method as claimed in claim 3, it is characterised in that original regarded to described according to playing request described In the step of frequency code stream is handled, in addition to：By the corresponding subimage block of the region-of-interest and except the region-of-interest it The outer corresponding subimage block in remaining region is spliced into broadcasting picture frame, and the broadcasting picture frame constitutes the broadcasting video codes Stream.

6. the video broadcasting method as described in claim 1,4 or 5 any one, it is characterised in that regarded in the generation broadcasting After the step of frequency code stream, the video broadcasting method also includes：Cache the broadcasting video code flow；

It is described the original video code stream is handled according to playing request the step of before, the video broadcasting method is also Including：

The corresponding caching for playing video code flow of the playing request is judged whether, is regarded if it is, sending the broadcasting Frequency code flow to the virtual reality device, otherwise, and continuation is handled the original video code stream according to the playing request.

7. a kind of video broadcasting method, it is characterised in that it is applied to virtual reality device, the video broadcasting method includes：

The region-of-interest selected according to user generates the playing request, and sends the playing request to the server；

Receive and handle the broadcasting video code flow that the server is returned；

Play the broadcasting video code flow.

8. video broadcasting method as claimed in claim 7, it is characterised in that it is described receive and handle the server return The broadcasting video code flow the step of in, including：

By the corresponding image of the region-of-interest and the original image with image same tag corresponding with the region-of-interest Frame is spliced into broadcasting picture frame, and the broadcasting picture frame constitutes the broadcasting video code flow.

9. video broadcasting method as claimed in claim 7, it is characterised in that it is described receive and handle the server return The broadcasting video code flow the step of in, in addition to：

By the corresponding subimage block of region-of-interest subimage block corresponding with remaining region in addition to the region-of-interest Broadcasting picture frame is spliced into, the broadcasting picture frame constitutes the broadcasting video code flow.

10. a kind of server, it is characterised in that the server includes：

Original video code stream receiving module, is configured as receiving original video code stream；

Playing request receiving module, is configured as receiving the playing request of the virtual reality device, wherein, the playing request Region-of-interest for determining user's selection；

Processing module, is configured as handling the original video code stream according to the playing request, and video is played in generation Code stream, wherein, the picture that the image sharpness for playing the region-of-interest in video code flow is higher than outside region-of-interest is clear Degree；

Sending module, is configured as sending the broadcasting video code flow to the virtual reality device.

11. server as claimed in claim 10, it is characterised in that the processing module includes：

First extraction unit, is configured as extracting the original image frame in the original video code stream, according to the playing request The corresponding image of the region-of-interest is extracted from the original image frame；

Acquiring unit, is configured as the mark according to the corresponding image of the region-of-interest, obtains the original image of same tag Frame；

First compression unit, is configured as the original image frame of same tag being compressed according to setting picture quality.

12. server as claimed in claim 10, it is characterised in that the server also includes division module, and it includes：

Division unit, is configured as extracting the original image frame in original video code stream, decodes and press the original image frame According to setting regular partition as multiple subimage blocks；

Second compression unit, is configured to compress the subimage block and makes it have at least two setting resolution ratio；

The processing module includes：

Second extraction unit, is configured as extracting the corresponding subimage block with the first setting resolution ratio of the region-of-interest；

3rd extraction unit, extract remaining region in the original image frame in addition to the region-of-interest it is corresponding and with The subimage block with second resolution of the same tag of the subimage block of the first resolution, wherein, described first point Resolution is higher than the second resolution.

13. server as claimed in claim 11, it is characterised in that the processing module also includes：First concatenation unit, quilt It is configured to the corresponding image of the region-of-interest and the original graph with image same tag corresponding with the region-of-interest As frame is spliced into broadcasting picture frame, the broadcasting picture frame composition broadcasting video code flow.

14. server as claimed in claim 12, it is characterised in that the processing module also includes：Second concatenation unit, quilt It is configured to the corresponding subimage block of region-of-interest subgraph corresponding with remaining region in addition to the region-of-interest Block is spliced into broadcasting picture frame, and the broadcasting picture frame constitutes the broadcasting video code flow.

15. the server as described in claim 10,12 or 13 any one, it is characterised in that the server also includes：

Cache module, is configured as after the broadcasting video code flow is generated, caches the broadcasting video code flow；

Judge module, was configured as before the step of being handled according to playing request the original video code stream, judged With the presence or absence of the corresponding caching for playing video code flow of the playing request, if it is, sending the broadcasting video code flow extremely The virtual reality device, otherwise, continuation are handled the original video code stream according to the playing request.

16. a kind of virtual reality device, it is characterised in that the virtual reality device includes：

Playing request generation module, the region-of-interest for being configured as being selected according to user generates the playing request, and to described Server sends the playing request；

Video reception processing module, is configured to receive and process the broadcasting video code flow that the server is returned；

Playing module, is configured as playing the broadcasting video code flow.

17. virtual reality device as claimed in claim 16, it is characterised in that the video reception processing module also includes： First concatenation module, is configured as by the corresponding image of the region-of-interest and with image phase corresponding with the region-of-interest The original image frame of isolabeling is spliced into broadcasting picture frame, and the broadcasting picture frame constitutes the broadcasting video code flow.

18. virtual reality device as claimed in claim 16, it is characterised in that the video reception processing module also includes： Second concatenation module, is configured as the corresponding subimage block of the region-of-interest and remaining area in addition to the region-of-interest The corresponding subimage block in domain is spliced into broadcasting picture frame, and the broadcasting picture frame constitutes the broadcasting video code flow.

19. a kind of panoramic virtual reality play system, it is characterised in that including：Panoramic shooting for photographing panorama picture is set Server standby, as described in claim 14 to 21 any one and virtual as described in claim 22 to 26 any one Real world devices, wherein, the panorama shooting device shoots video, and the original video code stream photographed is sent to service Device.