CN116760965B

CN116760965B - Panoramic video encoding method, device, computer equipment and storage medium

Info

Publication number: CN116760965B
Application number: CN202311017490.1A
Authority: CN
Inventors: 赵志立
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2023-08-14
Filing date: 2023-08-14
Publication date: 2023-12-22
Anticipated expiration: 2043-08-14
Also published as: CN116760965A

Abstract

The application relates to a panoramic video coding method, a device, computer equipment and a storage medium, and can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic, auxiliary driving and the like. The method comprises the following steps: projecting the panoramic video frame to be coded to a cylindrical surface through equidistant columnar projection to obtain a spherical matrix; determining a viewing angle point of an expected viewing angle of a panoramic video frame to be encoded at a spherical matrix; determining the spherical distance from a pixel point to a viewing angle point in the spherical matrix; and carrying out video coding processing on the panoramic video frame to be coded based on the spherical distance, and inversely correlating the coding image quality of the pixel point corresponding to the spherical distance with the spherical distance to obtain panoramic video coding data of the panoramic video frame to be coded. According to the panoramic video coding method and device, the image quality of the coded video is dynamically adjusted through the expected view angle of the panoramic video, so that the code rate of the video coding process is effectively utilized, and the compression effect in the panoramic video coding process is improved.

Description

Panoramic video encoding method, device, computer equipment and storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a panoramic video encoding method, apparatus, computer device, and storage medium.

Background

With the development of computer technology and photographic technology, panoramic video technology appears, and panoramic video is a video that uses a three-dimensional camera to shoot 360 degrees in all directions, and when watching video, users can randomly adjust the video to watch up, down, left and right. The production and manufacturing process of the panoramic video comprises video acquisition, video stitching, projection transformation and video coding. The playing link comprises video decoding, projective transformation and rendering.

However, in the current coding link of the panoramic video production process, the compression coding is generally performed by using a common video processing method, however, the compression effect of the compression mode is poor.

Disclosure of Invention

In view of the foregoing, it is desirable to provide a panoramic video encoding method, apparatus, computer device, and computer-readable storage medium that can improve the compression effect in the panoramic video production process.

In a first aspect, the present application provides a panoramic video encoding method. The method comprises the following steps:

projecting the panoramic video frame to be coded to a cylindrical surface through equidistant columnar projection to obtain a spherical matrix;

determining a viewing angle point of an expected viewing angle of the panoramic video frame to be encoded at the spherical matrix;

Determining the spherical distance from the pixel point in the spherical matrix to the view angle point;

and carrying out video coding processing on the panoramic video frame to be coded based on the spherical distance, and inversely correlating the coding image quality of the pixel point corresponding to the spherical distance with the spherical distance to obtain panoramic video coding data of the panoramic video frame to be coded.

In a second aspect, the present application further provides a panoramic video encoding apparatus. The device comprises:

the video projection module is used for projecting the panoramic video frame to be coded to a cylindrical surface through equidistant columnar projection to obtain a spherical matrix;

the viewing angle identification module is used for determining that the expected viewing angle of the panoramic video frame to be encoded is at the angle point of the spherical matrix;

the spherical distance identification module is used for determining the spherical distance from the pixel point to the view angle point in the spherical matrix;

and the video coding module is used for carrying out video coding processing on the panoramic video frame to be coded based on the spherical distance, and enabling the coding image quality of the pixel point corresponding to the spherical distance to be inversely related to the spherical distance so as to obtain panoramic video coding data of the panoramic video frame to be coded.

In a third aspect, the present application also provides a computer device. The computer device comprises a memory storing a computer program and a processor which when executing the computer program performs the steps of:

In a fourth aspect, the present application also provides a computer-readable storage medium. A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:

projecting the panoramic video frame to be encoded to a spherical matrix through equidistant columnar projection;

determining a viewing angle point of an expected viewing angle of a panoramic video frame to be encoded at a spherical matrix;

determining the spherical distance from a pixel point to a viewing angle point in the spherical matrix;

According to the panoramic video coding method, the device, the computer equipment and the storage medium, firstly, the panoramic video frame to be coded is projected to the spherical matrix through equidistant columnar projection to obtain the panoramic video frame in a matrix form, then, the view angle point of the expected view angle of the panoramic video frame to be coded in the spherical matrix is determined, namely, the position corresponding to the expected view angle of a user side in the panoramic video is determined in the spherical matrix, and therefore, the spherical distance from the pixel point to the view angle point in the spherical matrix is determined; and the panoramic video frame to be encoded is subjected to video encoding processing based on the spherical distance, the encoding image quality of the pixel points corresponding to the spherical distance is inversely related to the spherical distance through video encoding, panoramic video encoding data of the panoramic video frame to be encoded is obtained, the encoding processing is performed on the panoramic video through the spherical distance, the image quality of the pixel points which are far away from the view point in the panoramic video is poorer, the higher video compression ratio can be obtained under the condition that the same image quality video is provided for the viewing angle, and the higher image quality can be provided for the viewing angle under the condition that the video compression ratio is the same, so that the compression effect of the video is improved. According to the method and the device, the spherical distances from different areas of the panoramic video frame to the expected viewing angle are determined through the expected viewing angle of the panoramic video, so that the coding image quality of the panoramic video frame is dynamically adjusted, the image quality of different areas in the panoramic video frame is controlled, the point image quality far away from the viewing angle point is poorer, the point image quality near the viewing angle point is better, the code rate is effectively utilized, the higher video compression ratio can be obtained under the condition that the same image quality is provided for the viewing angle, and the higher image quality can be improved for the viewing angle under the condition that the video compression ratio is the same, so that the compression effect of the video is improved.

Drawings

FIG. 1 is an application environment diagram of a panoramic video encoding method in one embodiment;

FIG. 2 is a flow chart of a panoramic video encoding method in one embodiment;

FIG. 3 is a schematic illustration of a mid-range cylindrical projection in one embodiment;

FIG. 4 is a schematic diagram of a desired viewing perspective of a spherical matrix in one embodiment;

FIG. 5 is a feedback link diagram of a panoramic video for real-time bi-directional interaction in one embodiment;

FIG. 6 is a flow diagram of a fabrication process in one embodiment;

FIG. 7 is a flow chart illustrating a process from creation to playback in one embodiment;

FIG. 8 is a flowchart of a panoramic video encoding method according to another embodiment;

FIG. 9 is a block diagram of a panoramic video encoding apparatus in one embodiment;

fig. 10 is an internal structural view of a computer device in one embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

The panoramic video coding method provided by the embodiment of the application can be applied to an application environment shown in fig. 1. Wherein the terminal 102 communicates with the server 104 via a network. The data storage system may store data that the server 104 needs to process. The data storage system may be integrated on the server 104 or may be located on the cloud or other servers. After the user at the terminal 102 side shoots the panoramic video, the shot panoramic video can be encoded through the terminal, and the encoded panoramic video related data is submitted to the server 104, so that processing operations such as distribution and transmission of the panoramic video can be performed through the server 104. Firstly, the terminal 102 can shoot panoramic video through a panoramic video shooting device, then for the shot panoramic video, the terminal 102 also needs to finish video coding, firstly, the terminal 102 can project a panoramic video frame to be coded to a spherical matrix through equidistant columnar projection; determining a viewing angle point of an expected viewing angle of a panoramic video frame to be encoded at a spherical matrix; determining the spherical distance from a pixel point to a viewing angle point in the spherical matrix; and carrying out video coding processing on the panoramic video frame to be coded based on the spherical distance, and inversely correlating the coding image quality of the pixel point corresponding to the spherical distance with the spherical distance to obtain panoramic video coding data of the panoramic video frame to be coded. The encoded video is then transmitted to the server 104. The terminal 102 may be, but not limited to, various desktop computers, notebook computers, smart phones, tablet computers, internet of things devices, and portable wearable devices, where the internet of things devices may be smart speakers, smart televisions, smart air conditioners, smart vehicle devices, and the like. The portable wearable device may be a smart watch, smart bracelet, headset, or the like. The server 104 may be implemented as a stand-alone server or as a server cluster of multiple servers.

In one embodiment, as shown in fig. 2, a panoramic video encoding method is provided, and the method is applied to the terminal 102 in fig. 1 for illustration, and includes the following steps:

step 201, projecting a panoramic video frame to be coded to a cylindrical surface through equidistant columnar projection to obtain a spherical matrix.

The equidistant columnar projection is a projection mode, and is also the most commonly used projection conversion mode in panoramic video production links. As shown in fig. 3, the acquired panoramic video may be stitched into a sphere in three-dimensional space, the frame of the panoramic video is a spherical inner surface, and the equidistant columnar projections may map the spherical inner surface to a rectangle, which is the same as the relationship between the globe and the world map plan. Each point on the sphere can be represented by a longitude and a latitude, the longitude corresponding to the lateral direction of the rectangle and the latitude corresponding to the longitudinal direction of the rectangle. The rectangle is 0 in the middle, 180 degrees on the leftmost side, 180 degrees on the rightmost side, 90 degrees on the uppermost side and 90 degrees on the lowermost side. The cylindrical surface then refers to the side surface of a cylinder tangential to the spherical picture of the panoramic video frame to be encoded. For the obtained spherical matrix, a coordinate system is established in the center of the spherical matrix, the left and right sides of the matrix correspond to coordinates (0, -pi) and (0, pi), and the upper and lower sides of the matrix correspond to coordinates (0, pi/2) and (0, -pi/2). In the playing link, a rectangular image can be obtained after decoding, and then the rectangular image is projected to the spherical surface through equidistant columnar projection inverse transformation, and according to a preset value and user selection, one area on the spherical surface is selected as a user watching area for rendering and playing. The user's viewing angle is at the center of the sphere, and at any time the user sees a small area on the sphere.

Specifically, after the user collects the panoramic video through the photographing device of the panoramic video, in order to perform encoding of the panoramic video on the terminal 102, the terminal 102 may be used to complete the encoding process of the collected panoramic video through the panoramic video encoding method of the present application. In the encoding stage of panoramic video, compression encoding can be generally performed by using a processing method of a general video. However, the difference between the normal video and the panoramic video includes that the viewer can see the whole picture of the normal video at any moment, but the panoramic video cannot see the whole picture, and only one area can be seen, and the area only occupies a small part of the whole image (the viewing angle is limited and is far smaller than 360 °). The common video coding compression method is adopted, and the whole picture is processed identically, so that the code rate of the video is increased or the image quality of the video is reduced, and the compression effect is not improved. Therefore, the compression code rate of the video is considered to be adjusted based on the viewing angle of the user, in order to determine the distance between the picture locked by the viewing angle of the user and each pixel point in the panoramic video picture, the panoramic video frame to be encoded can be projected to the cylindrical surface through equidistant columnar projection to obtain a spherical matrix, and then the spherical distance is determined based on the spherical matrix, so that the image quality control processing of the panoramic video encoding process is performed.

Step 203, determining a viewing angle point at which a desired viewing angle of a panoramic video frame to be encoded is at a spherical matrix.

The desired viewing angle refers to a main viewing angle of a panoramic video when a user views the panoramic video, and for the panoramic video, the user cannot see the whole picture when viewing the panoramic video, but only sees a region, and the region only occupies a small part of the whole image. The viewing angle at which the user views this image is the desired viewing angle of the encoded panoramic video frame. For pre-generated panoramic video, the desired viewing angle may be, in particular, the optimal viewing angle. For panoramic video, although a user can choose to watch any angle, there is often an optimal viewing angle, for example, panoramic video of stage performance, stage direction is the optimal viewing angle, and side and rear pictures can provide immersion. The optimal viewing angle can be changed along with the playing time of the panoramic video, and the optimal viewing angle of the panoramic video frame is known only at each time. In addition, for generating the panoramic video in real time, the desired viewing angle can be adjusted according to feedback from the user side. The view point refers to a point in the center of the user's field of view, and the range of the picture watched by the user can be estimated through the view point.

Specifically, after obtaining the spherical matrix, a desired viewing angle at which the user views the panoramic video may be further determined in the spherical matrix, and a corresponding view point may be determined. And then performing coding optimization processing based on the view point. Wherein for a desired viewing angle, the data may be preset according to the type of panoramic video or determined according to viewing feedback of the user in real time. After the desired viewing angle is determined, a corresponding view point of the desired viewing angle in the spherical matrix may be determined. For example, as shown in fig. 4, let the desired viewing angle of the nth frame video be α. The user is located at the center of the sphere, α is a point on the sphere, and is located at the center of the user's field of view. As mentioned above, according to the application scenario, the desired viewing angle may be a preset optimal viewing angle of the user, or may be a real-time feedback prediction calculation of the current viewing angle. The view angle α is described by two parameters, longitude λ and latitude Φ. From the longitude λ and the dimension Φ, the corresponding position in the rectangular image can be found, then (λ1·Φ1) in fig. 4 is the desired viewing angle.

Step 205, determining a spherical distance from a pixel point to a viewing angle point in the spherical matrix.

For the pixel points in the spherical matrix, the spherical matrix corresponds to a complete panoramic video frame to be encoded, the pixel points on the spherical matrix can be restored to the pixel points of the panoramic video frame to be encoded, and in order to determine the distance from the pixel points in the spherical matrix to the view points, the spherical distance from the pixel points in the spherical matrix to the view points can be determined. The spherical distance is the length of the shortest connecting line between two points on the spherical surface, namely the length of a minor arc of a large circle passing through the two points between the two points.

Specifically, since the straight line distance between two points in the spherical rectangular image cannot represent the true distance on the spherical surface, for example, the leftmost side and the rightmost side in the spherical rectangular image are far away in the rectangular image, and are the same longitude (east longitude 180 ° and west longitude 180 °) on the spherical surface. Therefore, the scheme of the application needs to calculate the spherical distance between the two points from the pixel point to the view point in the spherical matrix, so as to realize panoramic video coding. In particular embodiments, the calculation of the spherical distance may be accomplished by various methods of calculating the spherical distance.

Step 207, performing video encoding processing on the panoramic video frame to be encoded based on the spherical distance, and inversely correlating the encoding image quality of the pixel point corresponding to the spherical distance with the spherical distance to obtain panoramic video encoding data of the panoramic video frame to be encoded.

Video coding refers to a manner of converting a file in an original video format into a file in another video format by a compression technique. The encoded image quality is an image quality parameter of the video picture after the video encoding process. The quality of the encoded picture can be controlled specifically by picture quality parameters. After the picture quality parameters are obtained, the relevant parameters of video coding can be determined by the picture quality parameters, in a specific embodiment, for example, in a video coding mode of h.264, entropy coding parameters in the coding process can be determined by the picture quality parameters, quantized data is coded by using huffman coding, context adaptive coding and other technologies, and compressed bit stream data is output as final panoramic video coding data.

Specifically, after the spherical distance is determined, the image quality of the pixel points in the spherical matrix can be determined based on the spherical distance. The spherical distance can be regarded as a parameter value for adjusting the image quality, the coding image quality of each pixel point in the panoramic video frame to be coded can be adjusted through the spherical distance, and for the specific adjusted value, the image quality obtained by adjustment is inversely related to the spherical distance, and the farther the spherical distance from the pixel point to the visual angle point is, the worse the image quality is, and the larger the compression ratio is; conversely, the better the image quality, the smaller the compression ratio. In a specific embodiment, the quantization parameter may be controlled by the spherical distance, and then the encoded image quality of the pixel point corresponding to the spherical distance may be controlled by the quantization parameter. For the control process of the coding image quality, an initial quantization parameter can be determined by a quantization parameter algorithm, and then an adjustment coefficient of the quantization parameter is determined by the spherical distance to correspondingly adjust the quantization parameter, so as to obtain a corresponding image quality quantization parameter.

According to the panoramic video coding method, firstly, the panoramic video frame to be coded is projected to the spherical matrix through equidistant columnar projection to obtain the panoramic video frame in a matrix form, then, the view angle point of the expected view angle of the panoramic video frame to be coded in the spherical matrix is determined, namely, the position corresponding to the expected view angle of the user side in the panoramic video is determined in the spherical matrix, and therefore, the spherical distance from the pixel point in the spherical matrix to the view angle point is determined; and the panoramic video frame to be encoded is subjected to video encoding processing based on the spherical distance, the encoding image quality of the pixel points corresponding to the spherical distance is inversely related to the spherical distance through video encoding, panoramic video encoding data of the panoramic video frame to be encoded is obtained, the encoding processing is performed on the panoramic video through the spherical distance, the image quality of the pixel points which are far away from the view point in the panoramic video is poorer, the higher video compression ratio can be obtained under the condition that the same image quality video is provided for the viewing angle, and the higher image quality can be provided for the viewing angle under the condition that the video compression ratio is the same, so that the compression effect of the video is improved. According to the method and the device, the spherical distances from different areas of the panoramic video frame to the expected viewing angle are determined through the expected viewing angle of the panoramic video, so that the coding image quality of the panoramic video frame is dynamically adjusted, the image quality of different areas in the panoramic video frame is controlled, the point image quality far away from the viewing angle point is poorer, the point image quality near the viewing angle point is better, the code rate is effectively utilized, the higher video compression ratio can be obtained under the condition that the same image quality is provided for the viewing angle, and the higher image quality can be improved for the viewing angle under the condition that the video compression ratio is the same, so that the compression effect of the video is improved.

In one embodiment, step 201 includes: acquiring a spherical image corresponding to a panoramic video frame to be encoded; determining projection of the longitude lines and latitude lines of the spherical image on the corresponding cylindrical surface of the spherical image; and expanding the cylindrical surface along the bus to obtain a spherical matrix.

The spherical image corresponding to the panoramic video frame to be encoded refers to the picture content formed by all the pictures of one frame of panoramic video through splicing, and a user can view the picture from the center of the panoramic video to the periphery, so that the user can see the video picture at any angle around, and the picture is the spherical inner surface. And the longitude and latitude of the spherical image, each point on the sphere is represented by longitude and latitude, and after the spherical image is projected onto the corresponding cylindrical surface, the longitude corresponds to the transverse direction of the rectangle, and the latitude corresponds to the longitudinal direction of the rectangle.

Specifically, the scheme of the application can realize the expansion of the spherical surface of the panoramic video frame to be encoded in an equidistant columnar projection mode, so that the distance between the pixel points in the panoramic video frame to be encoded is calculated. When equidistant columnar projection is carried out, a spherical image corresponding to a panoramic video frame to be encoded is required to be acquired firstly, and a projection cylinder corresponding to the spherical image is determined at the same time, wherein the cylinder can be a cylinder tangent to the spherical image. After the cylinder is determined, the spherical image can be projected onto the side surface of the cylinder, and the projection of the longitude line and latitude line of the spherical image on the corresponding cylindrical surface of the spherical image is determined; and finally, expanding the cylindrical surface along the bus to obtain the spherical matrix. In this embodiment, after obtaining the spherical image of the panoramic video, the spherical image is projected onto the cylindrical surface to determine the projection of the spherical image corresponding to the cylindrical surface, so as to obtain the spherical matrix, thereby effectively ensuring the validity of the spherical projection process, accurately calculating the spherical distance, and improving the coding accuracy of the coding process.

In one embodiment, the method further comprises: acquiring user visual angle feedback information of a panoramic video frame to be encoded; determining a user viewing area based on the user viewing angle feedback information; the center of the user viewing area is taken as the desired viewing angle of the panoramic video frame to be encoded.

The user visual angle feedback information of the panoramic video frame to be encoded refers to a viewing visual angle of the generated panoramic video watched by a user in real time, and the user visual angle feedback information can be realized by constructing a real-time feedback channel in advance. Through the real-time feedback path, the region watched by the user can be obtained in the process of watching the panoramic video by the user, and then the expected viewing angle can be identified.

Specifically, for a panoramic video that is produced and played simultaneously, a user viewing angle feedback mode may be adopted to determine an expected viewing angle of a panoramic video frame to be encoded in real time, specifically, as shown in fig. 5, the panoramic video content may be produced by encoding in real time, then the produced panoramic video content is sent to a terminal where a user is located for playing, the user may select a playing viewing angle of the panoramic video on the terminal, then the terminal for playing may first determine a user viewing area selected by a current viewing angle of the user, so as to obtain user viewing angle feedback information, and the user viewing angle feedback information is sent to one end of the panoramic video content to be encoded through a real-time feedback channel, so that a subsequent encoding process may determine the user viewing area by using the user viewing angle feedback information, and then identify a viewing angle of the panoramic video to be encoded by the user based on the user viewing area, that is, i.e., a center of the user viewing area is the expected viewing angle of the panoramic video frame to be encoded. In this embodiment, the expected viewing angle in the panoramic video encoding process is dynamically adjusted based on the feedback of the user viewing the panoramic video, so that the panoramic video encoding data obtained by encoding can fit the viewing requirement of the user, and the effectiveness of the panoramic video encoding process is ensured.

In one embodiment, the method further comprises: acquiring a viewing angle time axis of a panoramic video to be encoded; determining a playing time node corresponding to a panoramic video frame to be encoded; and searching the expected viewing angle corresponding to the playing time node in the viewing angle time axis.

The viewing angle time axis refers to time axis data preset based on data such as the type or content of the panoramic video, and is used for adjusting an expected viewing angle in the playing process of the panoramic video. Each panoramic video frame to be encoded corresponds to a node in the viewing perspective timeline.

Specifically, in addition to determining the desired viewing angle by means of angle feedback, the solution of the present application also supports determining the desired viewing angle at each moment by means of presetting the viewing angle time axis. For example panoramic video of stage performances, stage direction is generally the best viewing angle, and side and rear pictures can provide immersion. Therefore, in the process of the dance, the corresponding viewing angle time axis can be synchronously designed directly according to the optimal display angle of the performance. And then in the process of shooting the panoramic video, determining a playing time node corresponding to the panoramic video frame to be coded according to the shooting time point corresponding to the panoramic video frame to be coded. And searching a corresponding expected viewing angle in a viewing angle time axis based on the playing time node, so as to carry out subsequent encoding processing.

In one embodiment, step 205 comprises: determining macro blocks to which pixel points in the spherical matrix belong; and determining the spherical distance from the center point of the macro block to the view angle point, and taking the spherical distance from the center point of the macro block to the view angle point as the spherical distance from the pixel point in the spherical matrix to the view angle point.

Among them, macroblock is a basic concept in video coding technology. Different compression strategies are implemented for different locations by dividing the picture into blocks of different sizes.

Specifically, in order to reduce the calculation complexity in the video coding process, the spherical matrix can be converted into the macro block to calculate the spherical distance, so that the video coding is realized, in the specific coding process, the spherical matrix can be divided into different macro blocks, and then when the distance is calculated, the distance between calculation points can be converted into the distance of the calculation macro block, namely, the spherical distance from the center point of the calculation macro block to the view angle point is calculated. In one embodiment, the panoramic video frame to be encoded is image-encoded by using a YUV encoding mode, and at this time, for the division of macro blocks, one macro block is composed of one luminance pixel block and two additional chrominance pixel blocks. The luminance block is a 16x16 size pixel block, while the size of two chrominance image pixel blocks depends on the sampling format of their image, such as: for a YUV420 sampled image, the chroma block is an 8x8 size block of pixels. In this embodiment, by determining the macro block of the pixel point in the image and then performing the calculation of the spherical distance based on the position of the macro block, the complexity of the calculation process of the spherical distance can be effectively reduced, and the calculation efficiency of the video coding process can be improved.

In one embodiment, determining the spherical distance from the center point of the macroblock to the view point comprises: determining longitude and latitude information of a center point of a macro block and longitude and latitude information of a view point; and determining the spherical distance from the pixel point to the view angle point in the spherical matrix through a semi-normal vector formula based on the longitude and latitude information of the center point of the macro block and the longitude and latitude information of the view angle point.

Specifically, in the scheme of the application, the spherical distance from the pixel point in the spherical matrix to the view angle point can be determined through a semi-normal vector formula, and when the spherical distance is calculated, longitude and latitude information of the point on the spherical matrix corresponds to information of the point in a coordinate axis, so that the spherical distance from the pixel point in the spherical matrix to the view angle point can be determined through the semi-normal vector formula. For the view point (λ1, Φ1) and the center point (λ2, Φ2) of the macroblock, the corresponding sphere distance d calculation formula satisfies:

where r is the sphere radius, the value of sphere radius is preferably 1 because only the relative distance is required to be considered when calculating the spherical distance, and no absolute value is considered. In other embodiments, other formulas for calculating the spherical distance between two points can be used to calculate the spherical distance from the center point of the macroblock to the view point. In the embodiment, the spherical distance from the pixel point to the view angle point in the spherical matrix is determined through a semi-normal vector formula, so that the length of the spherical distance between two points can be accurately identified, the calculation of the picture quality parameters in the panoramic video coding process is accurately realized, and the efficiency and the accuracy of panoramic video coding are ensured.

In one embodiment, step 207 comprises: determining a picture quality parameter of a pixel point in the spherical matrix based on the spherical distance, wherein the picture quality parameter is positively correlated with the spherical distance and negatively correlated with the coding picture quality of the pixel point; and carrying out video coding processing on the panoramic video frame to be coded based on the picture quality parameters to obtain panoramic video coding data of the panoramic video frame to be coded.

The quantization parameter QP (quantization parameter) among the picture quality parameters is an important parameter in the encoding process, and the setting of this parameter determines the encoding quality of the picture. Quantization is a process of mapping continuous values of a signal into a plurality of discrete amplitude values, and the mapping of the signal values into one-to-many values is realized. The coding quality of each position in the panoramic video frame to be coded can be controlled by controlling the picture quality parameters of the pixel points in the spherical matrix. The picture quality parameter is positively correlated with the spherical distance and negatively correlated with the encoded picture quality of the pixel. Namely, the farther the spherical distance from the pixel point to the view point is, the larger the QP value is, the larger the quantization error is, the worse the image quality is, and the larger the compression ratio is; conversely, the smaller the QP value, the smaller the quantization error, the better the image quality, and the smaller the compression ratio.

Specifically, after the spherical distance is determined, the image quality parameters of the pixel points in the spherical matrix can be determined based on the spherical distance. The spherical distance can be regarded as a value for adjusting the quantization parameter, the coding image quality of each pixel point in the panoramic video frame to be coded can be adjusted through the spherical distance, and the adjusted image quality parameter is positively correlated with the spherical distance and negatively correlated with the coding image quality of the pixel point. The farther the spherical distance from the pixel point to the view point is, the larger the QP value is, the larger the quantization error is, the worse the image quality is, and the larger the compression ratio is; conversely, the smaller the QP value, the smaller the quantization error, the better the image quality, and the smaller the compression ratio. For the control process of the coding image quality, an initial quantization parameter can be determined by a quantization parameter algorithm, and then an adjustment coefficient of the quantization parameter is determined by the spherical distance to correspondingly adjust the quantization parameter, so as to obtain a corresponding image quality quantization parameter. And finally, carrying out video coding processing on the panoramic video frame to be coded based on the picture quality parameters to obtain panoramic video coding data of the panoramic video frame to be coded. In this embodiment, the video encoding process is performed on the panoramic video frame to be encoded by adjusting the picture quality parameter mode, so that accuracy of picture quality adjustment in the panoramic video encoding process can be effectively ensured, and encoding effect is ensured.

In one embodiment, determining the picture quality parameters for the pixel points in the spherical matrix based on the spherical distance comprises: determining initial quantization parameters of a panoramic video frame to be encoded, and acquiring image quality control parameters of the panoramic video frame to be encoded; comparing the image quality control parameters with the spherical distance to determine the image quality coefficient of the pixel points in the spherical matrix; and obtaining the picture quality parameters of the pixel points in the spherical matrix based on the product of the picture quality coefficients and the initial quantization parameters.

The initial quantization parameter refers to a quantization parameter value obtained by encoding the panoramic video frame to be encoded according to the application, for example, a quantization parameter value determined by intra-frame encoding or inter-frame encoding. The image quality control parameter is a parameter value set according to the image quality requirement in the encoding process and is used for calculating the image quality coefficient so as to control the change value of the quantization parameter in the encoding process.

Specifically, in the scheme of the application, the compression rate of the video coding process is controlled mainly by adjusting the quantization parameter, and the specific adjustment mode of the quantization parameter is mainly based on the initial quantization parameter determined by the video coding process. Thus, the initial quantization parameter may be determined on the basis of the panoramic video frame to be encoded first. And then determining the image quality coefficient which is specifically used for adjusting the image quality of the pixel points through the image quality control parameter and the spherical distance, and finally obtaining the image quality parameter of the pixel points in the spherical matrix based on the product of the image quality coefficient and the initial quantization parameter. In one embodiment, the picture quality parameter QP satisfies the following formula:

Wherein,for the initial quantization parameter, d is the sphere distance, ranging from 0 to pi (semicircle length). a is an image quality control parameter designed to control the strength of QP adjustment. For example, let a=pi, the QP minimum is/>Maximum value is 2->. In particular embodiments, the spherical distance may be calculated using the distance of the macroblock from the view point. In this embodiment, by determining an initial quantization parameter of a panoramic video frame to be encoded and then calculating a picture quality parameter of a pixel point in a spherical matrix based on a spherical distance and a picture quality control parameter, the picture quality parameter corresponding to each point in a picture can be effectively identified, thereby realizing accurate encoding processing operation.

In one embodiment, determining initial quantization parameters for a panoramic video frame to be encoded comprises: performing frame pretreatment on panoramic video frames to be coded to obtain pretreated video frames; carrying out intra-frame coding treatment on the preprocessed video frames to obtain intra-frame quantization parameters, and carrying out inter-frame coding treatment on the preprocessed video frames to obtain the intra-frame quantization parameters; an initial quantization parameter is determined based on a sum of the intra quantization parameter and the frame quantization parameter.

The intra-frame prediction refers to encoding a frame of video, and first performing intra-frame prediction on a current block. Intra prediction predicts the value of the current block using neighboring blocks that have already been encoded. Inter prediction refers to inter prediction of the current block. Inter prediction predicts the value of a current block using a block of a previous frame or a subsequent frame that has been encoded.

Specifically, the calculation process of the initial quantization parameter specifically includes two processes of intra-frame coding and inter-frame coding, and the preprocessing operation is needed before the calculation of the quantization parameter of the panoramic video frame to be coded. Items of preprocessing operations include color space conversion, clipping, scaling, filtering, etc., primarily to enhance the encoding effect. After the preprocessing is completed, the intra-frame coding processing and the inter-frame coding processing can be directly performed to determine corresponding initial quantization parameters. For the process, the pre-processing video frame can be directly subjected to intra-frame coding processing to obtain an intra-frame quantization parameter, and the pre-processing video frame is subjected to inter-frame coding processing to obtain the intra-frame quantization parameter; the initial quantization parameter is then determined based on the sum of the intra quantization parameter and the frame quantization parameter. In this embodiment, the encoding process of the panoramic video frame to be encoded is optimized through preprocessing, and then the quantization parameter is determined through the encoding process of the intra-frame encoding process and the inter-frame encoding process, so that the accuracy and the efficiency of the quantization parameter calculation can be effectively ensured.

In one embodiment, performing an intra-frame encoding process on the preprocessed video frame to obtain the intra-frame quantization parameter includes: converting pixel data of the preprocessed video frame into macro block data; performing motion prediction processing on macro block data of the preprocessed video frame to obtain macro block motion information; and obtaining an intra-frame quantization parameter based on the macroblock motion information.

Specifically, for the process of intra-frame encoding processing, pixel data of a preprocessed video frame can be converted into macro block data and then encoded, specifically, the pixel data is converted into macro block data in a 16x16 format, and then motion prediction and compensation are performed on each macro block to determine corresponding motion information of the macro block; finally, the motion information is quantized to reduce the data quantity, so as to obtain the intra-frame quantization parameter. In a specific embodiment, the calculation process of the frame quantization parameter can directly obtain the previous video frame and the next video frame of the preprocessed video frame; performing differential coding on the previous video frame and the next video frame to obtain brightness information, and performing color space conversion processing on the previous video frame and the next video frame to obtain color information; and finally, obtaining the frame quantization parameter based on the brightness information and the color information. And in the inter-frame coding process, information is acquired for compression mainly by comparing the front frame with the rear frame, so that the frame quantization parameter is obtained. Therefore, for the YUV color-coded image, the previous video frame and the next video frame can be differentially coded to obtain brightness information, and the previous video frame and the next video frame are subjected to color space conversion processing to obtain color information. And then giving the comparison result of the two to determine the corresponding frame quantization parameter, so as to obtain the final initial quantization parameter. In this embodiment, the motion information of the video frame is preprocessed to identify the intra-frame quantization parameter, and the frame quantization parameter is identified by comparing the front frame and the rear frame of the preprocessed video frame, so as to obtain the final initial quantization parameter, and the accuracy of calculating the initial quantization parameter can be effectively improved, so that the coding effect of the panoramic video is ensured.

The application scene is used for applying the panoramic video coding method. Specifically, the application of the panoramic video coding method in the application scene is as follows:

after a user shoots a panoramic video through a terminal, in order to code and transmit the panoramic video more effectively, the panoramic video coding method can be used for realizing coding processing of panoramic video data shot at the terminal. The manufacturing process of panoramic video coding in the application can be shown with reference to fig. 6, and includes processes of acquisition, splicing, projection transformation, coding and the like, wherein the acquisition refers to a process of shooting panoramic video through shooting equipment of panoramic video such as a panoramic camera, the splicing refers to a process of splicing content shot by the panoramic camera into a complete spherical panoramic video frame, and the projection transformation refers to a process of projecting the panoramic video frame to be coded onto a cylindrical surface through equidistant columnar projection to obtain a spherical matrix. And the coding refers to the process of panoramic video coding processing based on the spherical matrix.

Firstly, after a spherical image corresponding to a panoramic video frame to be encoded is acquired through stitching, projection of the longitude line and the latitude line of the spherical image on a cylindrical surface corresponding to the spherical image can be determined; and expanding the cylindrical surface along the generatrix to obtain a spherical matrix. Firstly, determining a viewing angle point of an expected viewing angle of a panoramic video frame to be encoded in a spherical matrix, and obtaining the expected viewing angle through viewing feedback, namely obtaining user viewing angle feedback information of the panoramic video frame to be encoded; determining a user viewing area based on the user viewing angle feedback information; the center of the user viewing area is taken as the desired viewing angle of the panoramic video frame to be encoded. The time axis of the viewing angle of the panoramic video to be encoded can be obtained through the set time axis; determining a playing time node corresponding to a panoramic video frame to be encoded; and searching the expected viewing angle corresponding to the playing time node in the viewing angle time axis. Then, the spherical distance from the pixel point in the spherical matrix to the view angle point can be determined, the pixel point in the spherical matrix is converted into a macro block for calculation, and the macro block to which the pixel point in the spherical matrix belongs is determined; and determining the spherical distance from the center point of the macro block to the view angle point, and taking the spherical distance from the center point of the macro block to the view angle point as the spherical distance from the pixel point in the spherical matrix to the view angle point. For the calculation method, the longitude and latitude information of the center point of the macro block and the longitude and latitude information of the view point can be determined; and determining the spherical distance from the pixel point to the view angle point in the spherical matrix through a semi-normal vector formula based on the longitude and latitude information of the center point of the macro block and the longitude and latitude information of the view angle point. Finally, the initial quantization parameter of the panoramic video frame to be encoded can be determined, and the image quality control parameter of the panoramic video frame to be encoded is obtained; comparing the image quality control parameters with the spherical distance to determine the image quality coefficient of the pixel points in the spherical matrix; and obtaining the picture quality parameters of the pixel points in the spherical matrix based on the product of the picture quality coefficients and the initial quantization parameters. And carrying out video coding processing on the panoramic video frame to be coded based on the picture quality parameters to obtain panoramic video coding data of the panoramic video frame to be coded. For the initial quantization parameter of the panoramic video frame to be encoded, the pre-processing video frame is required to be subjected to intra-frame encoding processing to obtain an intra-frame quantization parameter, and the pre-processing video frame is subjected to inter-frame encoding processing to obtain an inter-frame quantization parameter; an initial quantization parameter is determined based on a sum of the intra quantization parameter and the frame quantization parameter. The motion prediction processing can be carried out on macro block data of the preprocessed video frame for the intra-frame quantization parameter to obtain macro block motion information; and obtaining an intra-frame quantization parameter based on the macroblock motion information. For the frame quantization parameter, a previous video frame and a next video frame of the preprocessed video frame are obtained; performing differential coding on the previous video frame and the next video frame to obtain brightness information, and performing color space conversion processing on the previous video frame and the next video frame to obtain color information; based on the luminance information and the color information, a frame quantization parameter is obtained. After obtaining the panoramic video encoded data of the panoramic video frame to be encoded, the encoded panoramic video encoded data may be transmitted, and then the process from panoramic video production to playing may be referred to as fig. 7.

In one embodiment, the full flow of the panoramic video encoding method of the present application may be shown with reference to fig. 8, including:

step 801, acquiring a spherical image corresponding to a panoramic video frame to be encoded. Step 803, determining projection of the longitude line and the latitude line of the spherical image on the corresponding cylindrical surface of the spherical image. And step 805, expanding the cylindrical surface along the bus to obtain a spherical matrix. Step 807, obtaining user view feedback information of the panoramic video frame to be encoded. Step 809 determines a user viewing area based on the user perspective feedback information. Step 811, taking the center of the user viewing area as the desired viewing angle of the panoramic video frame to be encoded. Step 813, determining a viewing angle point at which a desired viewing angle of the panoramic video frame to be encoded is at the spherical matrix. Step 815, determining the macro block to which the pixel point in the spherical matrix belongs. In step 817, the spherical distance from the center point of the macro block to the viewing angle point is determined, and the spherical distance from the center point of the macro block to the viewing angle point is used as the spherical distance from the pixel point in the spherical matrix to the viewing angle point. And step 819, determining initial quantization parameters of the panoramic video frame to be encoded, and obtaining image quality control parameters of the panoramic video frame to be encoded. In step 821, the image quality control parameters and the spherical distance are compared to determine the image quality coefficient of the pixel point in the spherical matrix. Step 823, obtaining the picture quality parameters of the pixel points in the spherical matrix based on the product of the picture quality coefficient and the initial quantization parameters. Step 825, performing video encoding processing on the panoramic video frame to be encoded based on the picture quality parameter to obtain panoramic video encoding data of the panoramic video frame to be encoded.

It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

Based on the same inventive concept, the embodiment of the application also provides a panoramic video encoding device for realizing the panoramic video encoding method. The implementation of the solution provided by the apparatus is similar to the implementation described in the above method, so the specific limitation in one or more embodiments of the panoramic video encoding apparatus provided below may be referred to the limitation of the panoramic video encoding method hereinabove, and will not be described herein.

In one embodiment, as shown in fig. 9, there is provided a panorama video encoding apparatus, comprising:

the video projection module 902 is configured to project the panoramic video frame to be encoded onto a cylindrical surface through equidistant columnar projection, so as to obtain a spherical matrix.

A viewing angle identification module 904 for determining a viewing angle point at which a desired viewing angle of a panoramic video frame to be encoded is at the spherical matrix.

The spherical distance identifying module 906 is configured to determine a spherical distance from a pixel point to a viewing angle point in the spherical matrix.

The video encoding module 908 is configured to perform video encoding processing on the panoramic video frame to be encoded based on the spherical distance, and inversely correlate the encoding image quality of the pixel point corresponding to the spherical distance with the spherical distance, so as to obtain panoramic video encoding data of the panoramic video frame to be encoded.

In one embodiment, video projection module 902 is specifically configured to: acquiring a spherical image corresponding to a panoramic video frame to be encoded; determining projection of the longitude lines and latitude lines of the spherical image on the corresponding cylindrical surface of the spherical image; and expanding the cylindrical surface along the bus to obtain a spherical matrix.

In one embodiment, the device further comprises a feedback viewing angle adjustment module for: acquiring user visual angle feedback information of a panoramic video frame to be encoded; determining a user viewing area based on the user viewing angle feedback information; the center of the user viewing area is taken as the desired viewing angle of the panoramic video frame to be encoded.

In one embodiment, the device further comprises a time axis visual angle adjustment module for: acquiring a viewing angle time axis of a panoramic video to be encoded; determining a playing time node corresponding to a panoramic video frame to be encoded; and searching the expected viewing angle corresponding to the playing time node in the viewing angle time axis.

In one embodiment, the spherical distance identification module 906 is specifically configured to: determining macro blocks to which pixel points in the spherical matrix belong; and determining the spherical distance from the center point of the macro block to the view angle point, and taking the spherical distance from the center point of the macro block to the view angle point as the spherical distance from the pixel point in the spherical matrix to the view angle point.

In one embodiment, the spherical distance identification module 906 is further configured to: determining longitude and latitude information of a center point of a macro block and longitude and latitude information of a view point; and determining the spherical distance from the pixel point to the view angle point in the spherical matrix through a semi-normal vector formula based on the longitude and latitude information of the center point of the macro block and the longitude and latitude information of the view angle point.

In one embodiment, the video encoding module 908 is configured to: determining a picture quality parameter of a pixel point in the spherical matrix based on the spherical distance, wherein the picture quality parameter is positively correlated with the spherical distance and negatively correlated with the coding picture quality of the pixel point; and carrying out video coding processing on the panoramic video frame to be coded based on the picture quality parameters to obtain panoramic video coding data of the panoramic video frame to be coded.

In one embodiment, the video encoding module 908 is specifically configured to: determining initial quantization parameters of a panoramic video frame to be encoded, and acquiring image quality control parameters of the panoramic video frame to be encoded; comparing the image quality control parameters with the spherical distance to determine the image quality coefficient of the pixel points in the spherical matrix; and obtaining the picture quality parameters of the pixel points in the spherical matrix based on the product of the picture quality coefficients and the initial quantization parameters.

In one embodiment, the video encoding module 908 is further to: performing frame pretreatment on panoramic video frames to be coded to obtain pretreated video frames; carrying out intra-frame coding treatment on the preprocessed video frames to obtain intra-frame quantization parameters, and carrying out inter-frame coding treatment on the preprocessed video frames to obtain the intra-frame quantization parameters; an initial quantization parameter is determined based on a sum of the intra quantization parameter and the frame quantization parameter.

In one embodiment, the video encoding module 908 is further to: converting pixel data of the preprocessed video frame into macro block data; performing motion prediction processing on macro block data of the preprocessed video frame to obtain macro block motion information; and obtaining an intra-frame quantization parameter based on the macroblock motion information.

In one embodiment, the video encoding module 908 is further to: acquiring a previous video frame and a next video frame of the preprocessed video frame; performing differential coding on the previous video frame and the next video frame to obtain brightness information, and performing color space conversion processing on the previous video frame and the next video frame to obtain color information; based on the luminance information and the color information, a frame quantization parameter is obtained.

The respective modules in the panoramic video encoding apparatus described above may be implemented in whole or in part by software, hardware, and a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

In one embodiment, a computer device is provided, which may be a terminal, and an internal structure diagram thereof may be as shown in fig. 10. The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input means. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for carrying out wired or wireless communication with an external terminal, and the wireless mode can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a panoramic video encoding method. The display unit of the computer equipment is used for forming a visual picture, and can be a display screen, a projection device or a virtual reality imaging device, wherein the display screen can be a liquid crystal display screen or an electronic ink display screen, the input device of the computer equipment can be a touch layer covered on the display screen, can also be a key, a track ball or a touch pad arranged on a shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the structure shown in fig. 10 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.

In an embodiment, there is also provided a computer device comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.

In one embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps of the method embodiments described above.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to comply with the related laws and regulations and standards of the related countries and regions.

Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as Static Random access memory (Static Random access memory AccessMemory, SRAM) or dynamic Random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The above examples only represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A panoramic video encoding method, the method comprising:

projecting a panoramic video frame to be encoded onto a cylindrical surface through equidistant columnar projection to obtain a spherical matrix, wherein the spherical matrix is a rectangular image obtained by expanding the projected cylindrical surface along a generatrix;

determining a viewing angle point of an expected viewing angle of the panoramic video frame to be encoded in the spherical matrix, wherein the viewing angle point is a pixel point in the center of a user field of view when a user views the panoramic video, and the viewing angle point is used for estimating the range of a picture watched by the user;

2. The method of claim 1, wherein projecting the panoramic video frame to be encoded onto a cylindrical surface by equidistant columnar projection to obtain a spherical matrix comprises:

acquiring a spherical image corresponding to a panoramic video frame to be encoded;

determining projection of the longitude line and the latitude line of the spherical image on a cylindrical surface corresponding to the spherical image;

and expanding the cylindrical surface along a bus to obtain a spherical matrix.

3. The method according to claim 1, wherein the method further comprises:

acquiring user visual angle feedback information of the panoramic video frame to be encoded;

determining a user viewing area based on the user viewing angle feedback information;

taking the center of the user viewing area as a desired viewing angle of the panoramic video frame to be encoded.

4. The method according to claim 1, wherein the method further comprises:

Acquiring a viewing angle time axis of an expected viewing angle in the playing process of the panoramic video to be encoded;

determining a playing time node corresponding to the panoramic video frame to be encoded;

and searching the expected viewing angle corresponding to the playing time node in the viewing angle time axis.

5. The method of claim 1, wherein the determining the spherical distance of the pixel point in the spherical matrix to the view point comprises:

determining macro blocks to which pixel points in the spherical matrix belong;

and determining the spherical distance from the center point of the macro block to the view point, and taking the spherical distance from the center point of the macro block to the view point as the spherical distance from the pixel point in the spherical matrix to the view point.

6. The method of claim 5, wherein the determining the spherical distance of the center point of the macroblock to the view point comprises:

determining longitude and latitude information of a center point of the macro block and longitude and latitude information of the view point;

and determining the spherical distance from the pixel point in the spherical matrix to the view point through a semi-normal vector formula based on the longitude and latitude information of the center point of the macro block and the longitude and latitude information of the view point.

7. The method according to any one of claims 1 to 6, wherein the performing video encoding processing on the panoramic video frame to be encoded based on the spherical distance, and inversely correlating the encoded image quality of the pixel point corresponding to the spherical distance with the spherical distance, to obtain panoramic video encoding data of the panoramic video frame to be encoded includes:

determining a picture quality parameter of a pixel point in the spherical matrix based on the spherical distance, wherein the picture quality parameter is positively correlated with the spherical distance and negatively correlated with the coding picture quality of the pixel point;

and carrying out video coding processing on the panoramic video frame to be coded based on the picture quality parameters to obtain panoramic video coding data of the panoramic video frame to be coded.

8. The method of claim 7, wherein the determining the picture quality parameters for the pixel points in the spherical matrix based on the spherical distance comprises:

determining initial quantization parameters of the panoramic video frame to be encoded, and acquiring image quality control parameters of the panoramic video frame to be encoded;

comparing the image quality control parameter with the spherical distance to determine the image quality coefficient of the pixel points in the spherical matrix;

And obtaining the picture quality parameters of the pixel points in the spherical matrix based on the product of the picture quality coefficients and the initial quantization parameters.

9. The method of claim 8, wherein the determining initial quantization parameters for the panoramic video frame to be encoded comprises:

performing frame preprocessing on the panoramic video frame to be encoded to obtain a preprocessed video frame;

performing intra-frame coding processing on the preprocessed video frame to obtain an intra-frame quantization parameter, and performing inter-frame coding processing on the preprocessed video frame to obtain the intra-frame quantization parameter;

an initial quantization parameter is determined based on a sum of the intra quantization parameter and the frame quantization parameter.

10. The method of claim 9, wherein the intra-coding the pre-processed video frame to obtain the intra-quantization parameter comprises:

converting pixel data of the preprocessed video frame into macroblock data;

performing motion prediction processing on the macro block data of the preprocessed video frame to obtain macro block motion information;

and obtaining an intra-frame quantization parameter based on the macro block motion information.

11. The method of claim 9, wherein the inter-encoding the pre-processed video frame to obtain the frame quantization parameter comprises:

Acquiring a previous video frame and a next video frame of the preprocessed video frame;

performing differential encoding on the previous video frame and the next video frame to obtain brightness information, and performing color space conversion processing on the previous video frame and the next video frame to obtain color information;

and obtaining the frame quantization parameter based on the brightness information and the color information.

12. A panoramic video encoding apparatus, said apparatus comprising:

the video projection module is used for projecting the panoramic video frame to be coded onto the cylindrical surface through equidistant columnar projection to obtain a spherical matrix, wherein the spherical matrix is a rectangular image obtained by expanding the projected cylindrical surface along a generatrix;

the viewing angle identification module is used for determining a viewing angle point of an expected viewing angle of the panoramic video frame to be encoded at the spherical matrix, wherein the viewing angle point is a pixel point of the center of a user view when the user views the panoramic video, and the viewing angle point is used for estimating the range of a picture watched by the user;

the spherical distance identification module is used for determining the spherical distance from the pixel point in the spherical matrix to the view angle point;

13. The apparatus of claim 12, wherein the video projection module is specifically configured to: acquiring a spherical image corresponding to a panoramic video frame to be encoded; determining projection of the longitude line and the latitude line of the spherical image on a cylindrical surface corresponding to the spherical image; and expanding the cylindrical surface along a bus to obtain a spherical matrix.

14. The apparatus of claim 12, further comprising a feedback viewing angle adjustment module to: acquiring user visual angle feedback information of the panoramic video frame to be encoded; determining a user viewing area based on the user viewing angle feedback information; taking the center of the user viewing area as a desired viewing angle of the panoramic video frame to be encoded.

15. The apparatus of claim 12, further comprising a time axis viewing angle adjustment module to: acquiring a viewing angle time axis of an expected viewing angle in the playing process of the panoramic video to be encoded; determining a playing time node corresponding to the panoramic video frame to be encoded; and searching the expected viewing angle corresponding to the playing time node in the viewing angle time axis.

16. The apparatus of claim 12, wherein the spherical distance identification module is specifically configured to: determining macro blocks to which pixel points in the spherical matrix belong; and determining the spherical distance from the center point of the macro block to the view point, and taking the spherical distance from the center point of the macro block to the view point as the spherical distance from the pixel point in the spherical matrix to the view point.

17. The apparatus of claim 16, wherein the spherical distance identification module is further configured to: determining longitude and latitude information of a center point of the macro block and longitude and latitude information of the view point; and determining the spherical distance from the pixel point in the spherical matrix to the view point through a semi-normal vector formula based on the longitude and latitude information of the center point of the macro block and the longitude and latitude information of the view point.

18. The apparatus of any one of claims 12 to 17, wherein the video encoding module is configured to: determining a picture quality parameter of a pixel point in the spherical matrix based on the spherical distance, wherein the picture quality parameter is positively correlated with the spherical distance and negatively correlated with the coding picture quality of the pixel point; and carrying out video coding processing on the panoramic video frame to be coded based on the picture quality parameters to obtain panoramic video coding data of the panoramic video frame to be coded.

19. The apparatus of claim 18, wherein the video encoding module is specifically configured to: determining initial quantization parameters of the panoramic video frame to be encoded, and acquiring image quality control parameters of the panoramic video frame to be encoded; comparing the image quality control parameter with the spherical distance to determine the image quality coefficient of the pixel points in the spherical matrix; and obtaining the picture quality parameters of the pixel points in the spherical matrix based on the product of the picture quality coefficients and the initial quantization parameters.

20. The apparatus of claim 19, wherein the video encoding module is further configured to: performing frame preprocessing on the panoramic video frame to be encoded to obtain a preprocessed video frame; performing intra-frame coding processing on the preprocessed video frame to obtain an intra-frame quantization parameter, and performing inter-frame coding processing on the preprocessed video frame to obtain the intra-frame quantization parameter; an initial quantization parameter is determined based on a sum of the intra quantization parameter and the frame quantization parameter.

21. The apparatus of claim 20, wherein the video encoding module is further configured to: converting pixel data of the preprocessed video frame into macroblock data; performing motion prediction processing on the macro block data of the preprocessed video frame to obtain macro block motion information; and obtaining an intra-frame quantization parameter based on the macro block motion information.

22. The apparatus of claim 20, wherein the video encoding module is further configured to: acquiring a previous video frame and a next video frame of the preprocessed video frame; performing differential encoding on the previous video frame and the next video frame to obtain brightness information, and performing color space conversion processing on the previous video frame and the next video frame to obtain color information; and obtaining the frame quantization parameter based on the brightness information and the color information.

23. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any one of claims 1 to 11 when the computer program is executed.

24. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 11.