WO2020220941A1 - Point cloud encoding method, point cloud decoding method, apparatuses, and storage medium - Google Patents

Abstract

The present application relates to the field of data processing, and provides a point cloud encoding method, a point cloud decoding method, apparatuses, and a storage medium. The method comprises: firstly, obtaining auxiliary information of a patch to be encoded and a first identifier of said patch. Because the number of first-type patches in a current point cloud group may be the same in each point cloud frame, if the first identifier is a first value, and said patch is a first patch in a current point cloud frame, the auxiliary information and the first identifier of said patch may be encoded into a code stream. If the first identifier is the first value, and said patch is not the first patch in the current point cloud frame, the auxiliary information of said patch may be encoded into the code stream, and the first identifier of said patch is not required to be encoded into the code stream, thereby simplifying the format of being encoded into the code stream, reducing bit overheads of the code stream, and improving the encoding efficiency.

Description

Point cloud coding method, point cloud decoding method, device and storage medium

This application claims the priority of the Chinese patent application filed on April 30, 2019 with the application number 201910364032.2 and the invention title "point cloud encoding method, point cloud decoding method, device and storage medium", the entire content of which is incorporated by reference In this application.

Technical field

This application relates to the field of data processing technology, and in particular to a point cloud encoding method, point cloud decoding method, device and storage medium.

Background technique

With the continuous development of 3D sensing technology, the collection of point clouds has become more and more convenient, and the quality of the collected point clouds has become higher and higher, and the scale has also become larger. Therefore, how to effectively encode and decode the point cloud has become an urgent problem to be solved.

Summary of the invention

This application provides a point cloud encoding method, point cloud decoding method, device and storage medium, which can solve the problem of how to effectively encode and decode point clouds in related technologies. The technical solution is as follows:

In a first aspect, a point cloud encoding method is provided, the method includes: obtaining auxiliary information of a patch to be encoded (point cloud block) and a first identifier of the patch to be encoded; when the first identifier is a first value, Encode the auxiliary information of the patch and the first identifier into the code stream, and the patch to be encoded is the first patch in the current point cloud frame; or, when the first identifier is the first value, encode the auxiliary information of the patch to be encoded into the code Stream, the patch to be coded is the non-first patch in the current point cloud frame; the first identifier is the first value to indicate that the patch to be coded is the first type of patch, and the auxiliary information of the patch to be coded includes three-dimensional information.

It should be noted that the patch to be encoded may be any patch included in the current point cloud frame to be encoded. The first identifier may indicate the type of patch to be encoded. The first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.

Where the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch. The first type of patch can be a global matching patch in the current point cloud group. The auxiliary information of the patch to be encoded includes three-dimensional information. Including two-dimensional information, rotation transformation information and normal coordinate axis information.

When the first type of patch exists in the current point cloud group, since the number of the first type of patch in the current point cloud group can be the same in each point cloud frame, and when the number of the first type of patch is greater than 1, The first-type patches in each point cloud frame in the current point cloud group can be arranged in the same order, so when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame, Indicates that the patch to be encoded is the first patch of the first type in the current point cloud frame. At this time, the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, the patch to be encoded is When the current point cloud frame is not the first patch, it indicates that the patch to be coded is the non-first type I patch in the current point cloud frame. At this time, the auxiliary information of the patch to be coded can be encoded into the code stream instead of the waiting The first identifier of the encoded patch is encoded into the code stream, so that the format of the encoded code stream can be simplified.

Because the first-type patch has matching patches in all point cloud frames in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinates between the first-type patches in the current point cloud group The axis information is the same, but the three-dimensional information may be different. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be coded may include three-dimensional information, but not two-dimensional information, rotation transformation information and normal direction. Coordinate axis information.

In a possible situation, the patches in each point cloud frame can be arranged in the order of the first type of patch, the second type of patch and the third type of patch, and then the first type of patch and the second type of patch are also used when encoding. The type patch and the third type patch are coded sequentially. So if there is a first-type patch in the current point cloud group, the first patch in each point cloud frame is the first-type patch. The number of the first-type patches in each point cloud frame in the current point cloud group can be the same, and when the number of the first-type patches is greater than 1, the number of the first type of patches in each point cloud frame in the current point cloud group One type of patches can be arranged in the same order, that is, there is a matching relationship between the first type of patches at the same position in each point cloud frame.

However, in order to easily distinguish which point cloud frames can form a point cloud group, as shown in Figure 10, for the first point cloud frame in the current point cloud group, the first identification of all patches included in the first point cloud frame can be Set to the third value, the patches in other point cloud frames are still arranged in the order of the first type of patch, the second type of patch, and the third type of patch. Therefore, when the first identifier of the patch to be encoded is the first value, it can be determined that the current point cloud frame is the non-first point cloud frame in the current point cloud group. Moreover, the current point cloud frame may be the Kth point cloud frame in the current point cloud group, and K is an integer greater than or equal to 3. That is, the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be coded in the manner of S902.

Among them, because the point cloud frames in the current point cloud group can be numbered according to two numbering methods. The first numbering method: point cloud frame 0, point cloud frame 1, point cloud frame 2...; the second numbering method: point cloud frame 1, point cloud frame 2, point cloud frame 3... Therefore, the above Kth point cloud frame can refer to the point cloud frame 2 in the first numbering method and any point cloud frame after the point cloud frame 2, or it can refer to the point cloud frame 3 and the point cloud frame in the second numbering method. Any point cloud frame after point cloud frame 3. Regardless of the numbering method of the point cloud frames in the current point cloud group, the Kth point cloud frame can refer to the third point cloud frame in the actual current point cloud group and the point cloud frames after the third.

Based on the above description, the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be coded in the above manner, and for the second point cloud frame in the current point cloud group Point cloud frames can be encoded in the following two ways:

The first possible implementation manner, if the current point cloud frame is the second point cloud frame in the current point cloud group, then, when the first identifier of the patch to be encoded is the first value, and the patch to be encoded is the current point cloud For the first patch in the frame, the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded can be incorporated into the code stream. When the first identifier of the patch to be coded is the first value, and the patch to be coded is not the first patch in the current point cloud frame, the auxiliary information of the patch to be coded can be encoded into the code stream instead of the patch to be coded The first identifier is incorporated into the code stream.

Wherein, in the process of setting the first identifiers of all patches included in the first point cloud frame in the current point cloud group to the third value, the number of first-type patches in the current point cloud group can be obtained.

Moreover, the operation of encoding the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded into the code stream may be: editing the first identifier of the patch to be encoded Enter the code stream, and then encode the number of first-type patches in the current point cloud group into the code stream, and finally, encode the auxiliary information of the patch to be encoded into the code stream. That is, the number of the first-type patch in the current point cloud group is compiled into the code stream after the first identifier of the patch to be encoded.

It should be noted that the fixed-length encoding method can be used to encode the number of patches of the first type into the code stream, and the variable-length encoding method can also be used to encode the number of patches of the first type into the code stream. Not limited.

In the second possible implementation manner, if the current point cloud frame is the second point cloud frame in the current point cloud group, then when the first identifier of the patch to be coded is the first value, the first value of the patch to be coded can be set An identification and auxiliary information are compiled into the code stream. That is, the first identification and auxiliary information of all the first-type patches in the second point cloud frame need to be encoded into the code stream, so that the number of the first-type patches in the current point cloud group can be indicated without The number of the first type patch in the current point cloud group is compiled into the code stream.

Regardless of the foregoing implementation manners for encoding the first-type patch in the second point cloud frame, the number of the first-type patch in the current point cloud group can be determined through the second point cloud frame. In this way, for the third point cloud frame in the current point cloud group and the point cloud frames after the third point cloud frame, only the first patch, that is, the first identifier of the first first type patch can be coded It is not necessary to encode the first identifier of the non-first first-type patch into the code stream.

In summary, when the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group frame. In this way, there is no need to program the length of the point cloud frame of the current point cloud group in the code stream. In addition, for the third point cloud frame in the current point cloud group and the point cloud frames after the third point cloud frame, only the first identifier of the first first type patch can be compiled into the code stream, instead of the non-first point cloud frame. The first identifier of the first type patch is encoded into the code stream, which improves the coding efficiency and reduces the bit overhead of the code stream.

It is worth noting that, in a possible implementation, the first patch in the current point cloud frame can correspond to an identification information, which can also be called an index, and this identification information can uniquely indicate the first patch in the current point cloud frame patch, and the identification information of the first patch can usually be incorporated into the code stream. For other patches in the current point cloud frame, a counter can be used to determine which patch is the current point cloud frame. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.

It should be noted that the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).

As an example, when the first identifier is the first value, the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded. Specifically, the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded. The 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited. The fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream.

Wherein, using differential encoding to encode the three-dimensional information of the patch to be encoded into the code stream refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be encoded into the code stream.

Wherein, when the first identifier is the second value or the third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream.

Wherein, when the first identifier is the second value, it indicates that the patch to be encoded is a second type of patch. The second type of patch can be a local matching patch in the current point cloud group. The auxiliary information of the patch to be encoded includes two-dimensional information and three-dimensional information. Information and identification information of the reference patch of the patch to be encoded. When the first identifier is the third value, it indicates that the patch to be coded is a third type of patch. The third type of patch can be a non-matching patch in the current point cloud group. The auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, Rotation transformation information and normal coordinate axis information.

Because the second type of patch has a matching relationship in the reference frame of the current point cloud frame, and the rotation transformation information and normal coordinate axis information between the second type of patch and its reference patch are the same, and the two-dimensional information and the three-dimensional information It may be different. The third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value The auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information. When the first identifier is the second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

It should be noted that the two-dimensional information of the patch to be encoded may include the offset (2d_shift_u) of the patch to be encoded in the current point cloud frame occupancy map along the u axis, and the patch to be encoded in the current point cloud frame occupancy map along the v axis The direction shift (2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v). In addition, the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex. The syntax element of the rotation transformation information of the patch to be encoded may be rotation. Furthermore, the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded. The normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.

As an example, when the first identifier is the second value, the two-dimensional information and three-dimensional information of the patch to be encoded can be compiled into the code stream according to the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded. The differential encoding method encodes the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream, and a more accurate predictive differential method can also be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream. The specific encoding method is not used. limited. The identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method. When the first identifier is the third value, the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal axis information and the first identifier of the patch to be encoded into the code stream, or variable The long encoding method encodes the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and first identifier of the patch to be encoded into the code stream, which is not limited in the embodiment of the present application.

Among them, using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be encoded, and The difference between the three-dimensional information of the coded patch and the three-dimensional information of the reference patch of the patch to be coded is encoded into the code stream.

Based on the above description, for the current point cloud frame, the first identifier of the first first-type patch in the current point cloud frame, the first identifier of each second-type patch, and the first identifier of each third-type patch in the current point cloud frame The logo needs to be programmed into the code stream. The auxiliary information of each patch included in the current point cloud frame also needs to be compiled into the code stream. There may be the following three possible implementation methods for the encoding method of the first identification and auxiliary information.

In the first possible implementation manner, the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches. The auxiliary information of each patch included in the current point cloud frame can be individually packaged and compiled into a code stream. For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream. Then, the auxiliary information of each patch included in the current point cloud frame is packaged one by one and compiled into the code stream.

In the second possible implementation mode, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately And compiled into the code stream, the auxiliary information of each patch included in the current point cloud frame can also be packaged separately and compiled into the code stream. For example, you can first encode the first identifier of the first patch in the current point cloud frame into the code stream, and then encode the auxiliary information of the first patch into the code stream. Then, the auxiliary information of other first-type patches in the current point cloud frame is compiled into the code stream one by one. Finally, for the remaining patches, that is, the second type of patch and the third type of patch, the coding can be performed in a way that the first identifier is first compiled, and then the auxiliary information is compiled.

In a third possible implementation manner, the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches. For the auxiliary information of each patch included in the current point cloud frame, the same type of information in the auxiliary information of each patch can be packaged together and compiled into the code stream. Different types of information in the auxiliary information of each patch can be individually packaged and compiled into the code stream.

For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream. Suppose that the auxiliary information of each patch includes rotation transformation information and normal coordinate axis information, while some auxiliary information of patches includes two-dimensional information or three-dimensional information, and some auxiliary information of patches does not include two-dimensional information or For three-dimensional information, the rotation transformation information and normal coordinate axis information in the auxiliary information of each patch are packaged together and compiled into the code stream. The two-dimensional information and three-dimensional information in the auxiliary information of each patch are individually packaged and compiled into the code stream.

Among them, the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively. The reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.

In order to further reduce the bit overhead of the code stream, the patches in each point cloud frame in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch. For the three types of patches, if the patch to be coded is the first third type of patch, then the fixed-length encoding method may be used for encoding, or the variable-length encoding method may be used, which is not limited in the embodiment of the present application. If the patch to be coded is not the first type 3 patch, at this time, according to the 3D information of the first type 3 patch, the 2D information and 3D information of the patch to be coded can be encoded into the code stream using the differential encoding method, and other The information is also encoded using the fixed-length encoding method described above.

The different values of the first identifier indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, and for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch , Only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.

In a second aspect, a point cloud decoding method is provided. The method includes: when the point cloud block patch to be decoded is the first patch in the current point cloud frame, parsing the first identifier of the patch to be decoded from the code stream; When the first identifier is the first value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the three-dimensional information of the patch to be decoded; when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of patches of the first type in the reference point cloud frame of the current point cloud frame, parse the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; Decode the auxiliary information of the patch and reconstruct the patch to be decoded.

Since the first identifier of the first patch in the current point cloud frame is encoded in the code stream, when the point cloud block to be decoded is the first patch in the current point cloud frame, the to be decoded can be parsed from the code stream The first ID of the patch.

In a possible implementation manner, it can be determined whether the code stream includes the identification information of the patch to be decoded, and if the identification information of the patch to be decoded is included, it can be determined that the patch to be decoded is the first patch in the current point cloud frame. If the identification information of the patch to be decoded is not included, it can be determined that the patch to be decoded is the non-first patch in the current point cloud frame. At this time, the number of the patch to be decoded in the current point cloud frame can be determined according to the counter. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.

It should be noted that when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point cloud frames in the current point cloud group. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.

As an example, the number of patches of the first type can be parsed from the code stream by a method that matches the encoding end. For example, fixed-length decoding or variable-length decoding can be used to parse the number of patches of the first type from the code stream.

It can be known from the method provided in the above first aspect that when the first identifier is the first value, it can indicate that the current point cloud frame is the non-first point cloud frame in the current point cloud group. At this time, if the current point cloud frame is the second point cloud frame in the current point cloud group, the patch to be decoded can be decoded through the following two possible implementation methods.

The first possible implementation is the first identification of the first patch in the second point cloud frame in the current point cloud group, and the number of the first type of patch included in each point cloud frame in the current point cloud group Code stream together. At this point, if the current point cloud frame is the second point cloud frame in the current point cloud group, and the patch to be decoded is the first patch in the current point cloud frame, the first patch to be decoded can be parsed from the code stream. Identification, when the first identification is the first value, the number of the first-type patches included in each point cloud frame in the current point cloud group is analyzed from the code stream. Then parse the auxiliary information of the patch to be decoded from the code stream.

In the second possible implementation manner, the first identifiers of the first type patch included in the second point cloud frame in the current point cloud group are all encoded into the code stream. At this time, if the current point cloud frame is in the current point cloud group If the patch to be decoded is the first patch in the second point cloud frame, the first identifier of the patch to be decoded can be parsed from the code stream. When the first identifier is the first value, the patch to be decoded is parsed from the code stream. Supplementary information.

It can be seen from the method provided by the above first aspect that during the encoding process of the first-type patch, the first identifier of the non-first first-type patch is not encoded into the code stream, so when the patch to be decoded is the nth in the current point cloud frame When n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first first-type patch in the current point cloud frame. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.

Since the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information, when the patch to be decoded is the first type of patch The auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded. At this time, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.

As an example, a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream. For other auxiliary information of the patch to be decoded, for example, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded, the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, and then according to the The identification information of the reference frame determines the reference frame of the current point cloud frame, and then obtains the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame, and refers to the patch The two-dimensional information, rotation transformation information and normal coordinate axis information of the patch are used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded. Among them, the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch. For ease of description, the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame. In this way, the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame The auxiliary information of the patch is determined. In this way, the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc. In addition, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can be obtained from the decoded point cloud, and the obtained two-dimensional information, The rotation transformation information and the normal coordinate axis information are used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded.

In other words, when the first identifier is the first value, after the three-dimensional information of the patch to be decoded is parsed from the code stream, the auxiliary information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the to-be-decoded patch to assist The information includes two-dimensional information, rotation transformation information, and normal coordinate axis information. In this way, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the decoded patch.

When the patch in the point cloud is coded in the order of the first type of patch, the second type of patch and the third type of patch, the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded A patch with the same position in the current point cloud frame.

Based on the above description, if the patch to be decoded is the non-first patch in the current point cloud frame, the counter can be used to determine the number of patches in the current point cloud frame to be decoded. In this way, when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, the patch to be decoded can be determined It is the non-first first-type patch in the current point cloud frame, so that the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.

Among them, the number of first-type patches included in each point cloud frame in the current point cloud group can be determined in two ways. The first possible implementation is that after decoding the first identifier of the first patch in the second point cloud frame in the current point cloud group, each point in the current point cloud group can be directly parsed from the code stream The number of first-type patches included in the cloud frame. In the second possible implementation manner, after decoding the first identifiers of all patches of the second point cloud frame in the current point cloud group, the number of patches with the first identifier as the first value can be determined, so as to determine The number of first-type patches included in the second point cloud frame, that is, the number of first-type patches included in each point cloud frame in the current point cloud group. That is, the number of first-type patches included in each point cloud frame in the current point cloud group can be determined through the second point cloud frame.

In a possible situation, the second point cloud frame in the current point cloud group can be used as the reference point cloud frame of the third point cloud frame, and the third point cloud frame can be used as the reference point cloud frame of the fourth point cloud frame Point cloud frame, and so on. In this way, the third point cloud frame and the point cloud frames after the third can determine whether n in the nth patch in the current point cloud frame is greater than 1 and based on the number of the first type of patch in the reference point cloud frame The number of the first-type patch in the reference point cloud frame of the current point cloud frame is less than or equal to, so as to determine whether the patch to be decoded is the first-type patch.

In addition, since the first identifier of the first patch in the current point cloud frame, and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately and compiled into the code stream, Can be packaged together and compiled into code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for patches that take the second value or the third value other than the first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of the first patch to be decoded in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be parsed in sequence. For the method of packing together and compiling into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , Wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis. Similarly, when the same type of information in the auxiliary information of each patch included in the current point cloud frame is also packaged and compiled into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, you can The code stream parses the same type of information in the auxiliary information of each patch included in the current point cloud frame. That is, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.

As an example, a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream. For example, a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.

Among them, when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of first-type patches in the reference point cloud frame of the current point cloud frame, the to-be-decoded patch is parsed from the code stream The first ID of the patch.

Since in the encoding process of the second type patch and the third type patch, the first identifier is encoded into the code stream, so when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current When the point cloud frame refers to the number of the first-type patch in the point cloud frame, it indicates that the patch to be decoded is a second-type patch or a third-type patch. At this time, the first identifier of the patch to be decoded can be parsed from the code stream.

Wherein, when the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, the three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.

It should be noted that when the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.

When the first identifier is the second value, the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis. For the rotation transformation information and normal coordinate axis information of the patch to be decoded, the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained. Axis information, and use the rotation transformation information and normal coordinate axis information of the reference patch as the rotation transformation information and normal coordinate axis information of the patch to be decoded. For the two-dimensional information and three-dimensional information of the patch to be decoded, as an example, a method that matches the encoding end can be used for decoding. For example, the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. That is, for the second type of patch, two-dimensional information and three-dimensional information can be parsed from the code stream, while the rotation transformation information and the normal coordinate axis information are derived or obtained, thus avoiding the need to include in the code stream The rotation transformation information and the normal coordinate axis information also reduce the bit overhead of the code stream and improve the decoding efficiency at the same time.

In other words, when the first identifier is the second value, after the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded are parsed from the code stream, it can also be based on the patch to be decoded. Refer to the identification information of the patch to obtain the rotation transformation information and the normal coordinate axis information of the patch to be decoded. In this way, reconstructing the patch to be decoded according to the auxiliary information of the patch to be decoded includes: reconstructing the patch to be decoded according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded.

Wherein, when the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

It should be noted that when the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group. At this point, the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.

As an example, a method that matches the encoding end can be used for decoding. For example, fixed-length decoding can be used to parse the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded from the code stream. The variable-length decoding method can also be used to parse the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded from the code stream, which is not limited in the embodiment of the present application. However, if the auxiliary information of the first third-type patch is coded according to fixed-length coding or variable-length coding, the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by differential coding, and other auxiliary information is coded Encode with fixed-length encoding. Then, for the non-first third-type patch, according to the two-dimensional information and three-dimensional information of the first third-type patch, the two-dimensional information and three-dimensional information of the patch to be decoded are parsed from the code stream by differential decoding, and fixed The long decoding method parses the rotation transformation information and normal coordinate axis information of the patch to be decoded from the code stream. That is, for the third type of patch, the non-first third type of patch two-dimensional information and three-dimensional information can be parsed from the code stream, and the rotation transformation information and the normal coordinate axis information are derived or obtained, so as to avoid In order to encode the rotation transformation information and the normal coordinate axis information in the code stream, the bit overhead of the code stream is also reduced, and the decoding efficiency is improved.

Among them, the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively. The application embodiment does not limit this.

Based on the above description, the first type patch, the second type patch, and the third type patch in each point cloud frame in the current point cloud group can be arranged in sequence. At this time, if the patch to be decoded is the first in the current point cloud frame A patch, then, after parsing the first identifier of the patch to be decoded from the code stream, it can be determined whether the current point cloud frame is the first point cloud in the current point cloud group according to the value of the first identifier. That is, when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group. In this way, there is no need to program the length of a point cloud group in the code stream.

It should be noted that after obtaining the auxiliary information of the patch to be decoded, that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded, the auxiliary information can be reproduced according to but not limited to these auxiliary information. To construct the patch to be decoded, this application does not limit the specific information needed to reconstruct the patch to be decoded.

In summary, that is, for the first type of patch and the second type of patch, only part of the auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained, which can simplify the format of the code stream. , Reduce the bit overhead of the code stream, thereby improving the efficiency of decoding.

In a third aspect, a point cloud coding method is provided. The method includes: obtaining auxiliary information of a point cloud block patch to be coded and a first identifier of the patch to be coded; when the first identifier is a first value, the patch to be coded The auxiliary information, the first identifier, and the second identifier of the patch to be encoded are encoded into the code stream, and the patch to be encoded is the first patch in the current point cloud frame; or, when the first identifier is the first value, the patch to be encoded The auxiliary information and the second identifier of the patch to be encoded are encoded into the code stream, the patch to be encoded is the non-first patch in the current point cloud frame; the first identifier is the first value to indicate that the patch to be encoded is the first type of patch ; Wherein the second identifier is the fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; or the second identifier is the fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream and is to be encoded The auxiliary information of the patch includes three-dimensional information.

The first type of patch has matching patches in all point cloud frames in the current point cloud group, and the two-dimensional information, rotation transformation information and normal coordinate axis between the first type of patches in the current point cloud group The information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch can also be the same. In this case, when the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include the second identifier.

It should be noted that the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. The syntax element of the second identification may be override_3d_shift_data_flag. When the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information. When the second identifier is the fifth value, it can indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include three-dimensional information. The specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used. The syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.

Wherein, when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.

Wherein, when the first identifier is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

Since the number of the first-type patches in each point cloud frame in the current point cloud group can be the same, when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame, Indicates that the patch to be encoded is the first patch of the first type in the current point cloud frame. At this time, the auxiliary information, the first identifier and the second identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, When the patch to be coded is the non-first patch in the current point cloud frame, it indicates that the patch to be coded is the non-first type 1 patch in the current point cloud frame, and the auxiliary information and the second identifier of the patch to be coded can be coded Instead of coding the first identifier of the patch to be coded into the code stream, the format of the code stream can be simplified. The second identifier is an identifier used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. Information, in this way, can further reduce the bit overhead of the code stream, thereby improving the coding efficiency.

In a fourth aspect, a point cloud decoding method is provided. The method includes: when the point cloud block patch to be decoded is the first patch in the current point cloud frame, parsing the first identifier of the patch to be decoded from the code stream; When an identifier is the first value, analyze the second identifier of the patch to be decoded from the code stream; when the second identifier is the fourth value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; When the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame, the second identifier of the patch to be decoded is parsed from the code stream; when the second identifier is the fourth value, according to the The auxiliary information of the decoded patch refers to the auxiliary information of the patch to be decoded to determine the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; The auxiliary information of the patch to be decoded is reconstructed.

It should be noted that when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the 3D information of the patch to be decoded is the same as the 3D information of the reference patch of the patch to be decoded. At this time, the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of the reference patch of the patch to be decoded. Specifically, the auxiliary information of the reference patch of the patch to be decoded may be used as the auxiliary information of the patch to be decoded. When the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information. At this time, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded. As an example, a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.

Since the second identifiers of the non-first first-type patch in the current point cloud frame are all encoded into the bitstream, when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current When the point cloud frame refers to the number of the first-type patch in the point cloud frame, that is, when the patch to be decoded is not the first first-type patch, the second identifier of the patch to be decoded can be parsed from the code stream.

Since the first identifier of the first patch in the current point cloud frame is encoded into the code stream, when the point cloud block to be decoded is the first patch in the current point cloud frame, the patch to be decoded is parsed from the code stream When the first identifier is the first value, it indicates that the patch to be decoded is the first patch of the first type. At this time, the second identifier of the patch to be decoded can be parsed from the code stream. When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first A first-type patch, at this time, the second identifier of the patch to be decoded can be parsed from the code stream. Wherein, when the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. At this time, the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.

In a fifth aspect, a point cloud coding method is provided, the method comprising: obtaining auxiliary information of a point cloud block to be coded and a first identifier of the patch to be coded; when the first identifier is a first value, the patch to be coded When the first identifier is the second value or the third value, the auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream; where the first identifier is the first value to indicate The patch to be coded is the first type of patch; the first identifier is the second value to indicate that the patch to be coded is the second type of patch, and the auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information and the reference patch of the patch to be coded Identification information; where the first identification is a third value to indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

When the first identifier is the first value, it indicates that the patch to be encoded is the first type of patch. In a possible implementation, the first-type patch has patches with matching relationships in all point cloud frames in the current point cloud group, and two-dimensional information between the first-type patches in the current point cloud group , 3D information, rotation transformation information and normal coordinate axis information are all the same. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value, the first identifier of the patch to be coded is encoded into the code stream instead of the auxiliary information of the patch to be coded. Code stream.

Since the two-dimensional information, rotation transformation information and normal coordinate axis information and three-dimensional information among the first-type patches in the current point cloud group can be the same, when the first identifier of the patch to be encoded is the first value, it indicates that the The coded patch is the first type of patch. At this time, the first identifier of the patch to be coded can be encoded into the code stream, instead of coding the auxiliary information of the patch to be coded into the code stream, so that the format of the code stream can be simplified. In addition, the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, and for different types of patches, the content of the auxiliary information encoded in the code stream may be different, and for the first type of patch and the second type of patch , Only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.

In a sixth aspect, a point cloud decoding method is provided. The method includes: parsing a first identifier of a point cloud block patch to be decoded from a code stream; when the first identifier is a first value, according to the reference patch of the patch to be decoded The auxiliary information determines the auxiliary information of the patch to be decoded. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; when the first identifier is the second value, it is parsed from the code stream The auxiliary information of the patch to be decoded. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded; when the first identifier is the third value, parse the patch to be decoded from the code stream Auxiliary information, the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information; according to the auxiliary information of the patch to be decoded, the patch to be decoded is reconstructed.

Because the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially. For the method of packing together and compiling into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , Wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.

As an example, a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream. For example, a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.

Because the first type of patch does not encode auxiliary information into the code stream during the encoding process, when the first identifier is the first value, that is, when the patch to be decoded is the first type of patch, the reference patch of the patch to be decoded can be Auxiliary information to obtain 2D information, 3D information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

As an example, the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, so as to determine the reference frame of the current point cloud frame according to the identification information of the reference frame, and then from the reference frame of the current point cloud frame Obtain the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded, and use the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch as the information to be decoded Patch's two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. Among them, the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch. For ease of description, the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame. In this way, the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame The auxiliary information of the patch is determined. In this way, the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc. In addition, for the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded, the position of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can also be obtained from the decoded point cloud. Two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information, and the obtained two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information.

When the patch in each point cloud frame in the current point cloud group is encoded in the order of the first type of patch, the second type of patch and the third type of patch, the reference patch of the patch to be decoded can be the reference of the current point cloud frame The patch whose position in the frame is the same as the position of the patch to be decoded in the current point cloud frame.

Among them, because the auxiliary information of the second type of patch that is incorporated into the code stream during the encoding process includes two-dimensional information, three-dimensional information, and identification information of the reference patch, but does not include rotation transformation information and normal coordinate axis information, so when When the decoded patch is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and information of the patch to be decoded. Normal axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.

In summary, for the first type of patch, only the first identifier needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. For the second type of patch, only part of the auxiliary information needs to be parsed from the code stream. The auxiliary information can also be derived or obtained. In this way, the format of the parsing code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.

In a seventh aspect, a point cloud encoding device is provided, the device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the point cloud block patch to be encoded and a first identifier of the patch to be encoded; auxiliary information encoding module , Used to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the first value, the patch to be encoded is the first patch in the current point cloud frame; or the auxiliary information encoding module, using When the first identifier is the first value, the auxiliary information of the patch to be encoded is encoded into the code stream, and the patch to be encoded is the non-first patch in the current point cloud frame; the first identifier is the first value to indicate the The coded patch is the first type of patch, and the auxiliary information of the patch to be coded includes three-dimensional information.

Among them, the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.

Among them, the auxiliary information encoding module is also used for:

When the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into the code stream;

The first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;

The first identifier is a third value to indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

Among them, the auxiliary information encoding module is also used for:

The first identifiers of the multiple patches included in the current point cloud frame are encoded into the code stream, the first identifiers of the multiple patches include the first identifiers of the patch to be encoded, and the first identifiers of the multiple patches are excluding the patch to be encoded The remaining first identifier besides the first identifier is the second value or the third value.

Among them, the first value, the second value and the third value are binary 10, 11 and 00 respectively,

Or, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.

The first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.

Wherein, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.

Among them, the first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the third type of patch is a non-matching patch in the current group.

In an eighth aspect, a point cloud decoding device is provided. The device includes: an auxiliary information decoding module, configured to parse the to-be-decoded point cloud block patch from the current point cloud frame when the patch to be decoded is the first patch in the current point cloud frame The first identifier of the patch; the auxiliary information decoding module is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value. The auxiliary information includes the three-dimensional information of the patch to be decoded; the auxiliary information The decoding module is also used for when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, from The auxiliary information of the patch to be decoded is parsed in the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

Among them, the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.

Among them, the auxiliary information decoding module is also used for:

When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patch in the reference point cloud frame of the current point cloud frame, parse the patch to be decoded from the code stream First logo

When the first identifier is the second value, parse the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded;

When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.

Among them, the auxiliary information decoding module is also used for:

When the patch to be decoded is the first patch in the current point cloud frame, the first identifiers of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers parsed include the patch to be decoded A first identifier, and the remaining first identifiers among the plurality of first identifiers except the first identifier of the patch to be decoded are the second value or the third value.

Among them, the first value, the second value and the third value are binary 10, 11 and 00 respectively,

Or, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.

Wherein, when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is the third value, the current point cloud frame is the first point cloud frame.

Wherein, when the first identifier is the first value, the auxiliary information decoding module is further configured to:

Obtain the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;

The reconstruction module is also used to:

According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.

Wherein, when the first identifier is the second value, the auxiliary information decoding module is also used to:

Obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;

The reconstruction module is also used to:

According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.

The reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.

In a ninth aspect, a point cloud encoding device is provided, the device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the point cloud block patch to be encoded and a first identifier of the patch to be encoded; auxiliary information encoding module , Used to encode the auxiliary information of the patch to be encoded, the first identifier and the second identifier of the patch to be encoded into the code stream when the first identifier is the first value, the patch to be encoded is the first patch in the current point cloud frame Or auxiliary information encoding module, when the first identifier is the first value, the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded into the code stream, and the patch to be encoded is the non-information in the current point cloud frame The first patch; where the first identifier is the first value to indicate that the patch to be encoded is the first type of patch; where the second identifier is the fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; or, the first The second flag is the fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the bitstream, and the auxiliary information of the patch to be encoded includes three-dimensional information.

In a tenth aspect, a point cloud decoding device is provided. The device includes: an auxiliary information decoding module for analyzing the to-be-decoded point cloud block patch from the current point cloud frame when the patch to be decoded is the first patch in the current point cloud frame The first identifier of the patch; the auxiliary information decoding module is also used to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value; the auxiliary information decoding module is also used to When the identifier is the fourth value, the auxiliary information of the patch to be decoded is determined according to the auxiliary information of the reference patch of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information Including the three-dimensional information of the patch to be decoded; the auxiliary information decoding module is also used when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the reference point cloud frame of the current point cloud frame When the number of the first type of patch, the second identifier of the patch to be decoded is parsed from the code stream; the auxiliary information decoding module is also used for when the second identifier is the fourth value, according to the reference patch of the patch to be decoded The auxiliary information determines the auxiliary information of the patch to be decoded; when the second identifier is the fifth value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; the reconstruction module is used according to The auxiliary information of the patch to be decoded is reconstructed.

In an eleventh aspect, a point cloud encoding device is provided, the device comprising: a point cloud block patch information acquisition module for acquiring auxiliary information of the point cloud block patch to be encoded and a first identifier of the patch to be encoded; auxiliary information encoding The module is used to encode the first identifier of the patch to be encoded into the code stream when the first identifier is the first value; the auxiliary information encoding module is also used to when the first identifier is the second value or the third value, The auxiliary information of the patch to be encoded and the first identifier are encoded into the code stream; the first identifier is the first value to indicate that the patch to be encoded is a first type of patch; the first identifier is the second value to indicate the patch to be encoded It is the second type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded; the first identifier is a third value to indicate that the patch to be encoded is a third-type patch, The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

In a twelfth aspect, a point cloud decoding device is provided. The device includes: an auxiliary information decoding module for parsing the first identifier of the point cloud block patch to be decoded from the code stream; the auxiliary information decoding module is also used for When the first identifier is the first value, the auxiliary information of the patch to be decoded is determined according to the auxiliary information of the reference patch of the patch to be decoded. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis. Information; the auxiliary information decoding module is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the second value. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information and the patch to be decoded Reference patch identification information; the auxiliary information decoding module is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the third value. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information Information, rotation transformation information, and normal coordinate axis information; the reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

Wherein, when the first identifier is the second value, the auxiliary information decoding module is also used to:

Obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;

The reconstruction module is also used to:

According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.

In a thirteenth aspect, a device for encoding point cloud data is provided. The device may include a memory and an encoder. The memory is used to store point cloud data. The encoder is used to implement the point cloud coding method provided by the first aspect or any possible design of the first aspect, or the encoder is used to implement the points provided by the third aspect or any possible design of the third aspect. The cloud coding method, or the encoder is used to execute the point cloud coding method provided by any possible design of the fifth aspect or the fifth aspect.

In a fourteenth aspect, a device for decoding point cloud data is provided. The device may include a memory and a decoder. The memory is used to store point cloud data in the form of a code stream. The decoder is used to implement the point cloud decoding method provided by the foregoing second aspect or any possible design of the second aspect, or the decoder is used to implement the points provided by any possible design of the foregoing fourth aspect or the fourth aspect The cloud decoding method, or the decoder is used to execute the point cloud decoding method provided by the sixth aspect or any one of the possible designs of the sixth aspect.

In a fifteenth aspect, an encoding device is provided, including: a memory and a processor; wherein the memory is used for storing program code; the processor is used for calling the program code to execute the first aspect or the third aspect or the fifth aspect. Point cloud coding method provided by the aspect.

In a sixteenth aspect, there is provided a decoding device, including: a memory and a processor; wherein the memory is used to store program code; the processor is used to call the program code to execute the second aspect or the fourth aspect or the sixth aspect. The point cloud decoding method provided by the aspect.

The present application also provides a computer-readable storage medium, including program code, which when run on a computer, causes the computer to execute the first aspect and its possible designs, or the third aspect and its possible designs, Or any point cloud coding method provided by the fifth aspect and its possible designs.

This application also provides a computer-readable storage medium, including program code, which when run on a computer, causes the computer to execute the second aspect and its possible designs, or the fourth aspect and its possible designs, or Any point cloud decoding method provided by the sixth aspect and its possible designs.

This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the first aspect or the first aspect, or execute the third aspect or the third aspect Any possible design, or execute part or all of the steps of the point cloud coding method provided by the fifth aspect or any possible design of the fifth aspect.

This application also provides a computer program product, which when the computer program product runs on a computer, causes the computer to execute any possible design of the second aspect or the second aspect, or execute the fourth aspect or the fourth aspect Any possible design, or execute part or all of the steps of the point cloud decoding method provided by the sixth aspect or any possible design of the sixth aspect.

It should be understood that the beneficial effects of any codec device, computer-readable storage medium, and computer program product provided above can be referred to the beneficial effects of the method embodiments provided in the corresponding aspects above, and details are not repeated.

The technical solution provided in this application can at least bring about the following beneficial effects:

In this embodiment of the application, first obtain the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded. Since the number of the first-type patches in each point cloud frame in the current point cloud group can be the same, when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame, Indicates that the patch to be encoded is the first patch of the first type in the current point cloud frame. At this time, the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, the patch to be encoded is When the current point cloud frame is not the first patch, it indicates that the patch to be coded is the non-first type I patch in the current point cloud frame. At this time, the auxiliary information of the patch to be coded can be encoded into the code stream instead of the waiting The first identifier of the encoded patch is encoded into the code stream, so that the format of the encoded code stream can be simplified. Further reduce the bit overhead of the code stream, thereby improving the coding efficiency.

Description of the drawings

FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application;

Fig. 2 is a schematic block diagram of an encoder that can be used in an embodiment of the present application;

Fig. 3 is a schematic diagram of a point cloud applicable to an embodiment of the present application;

FIG. 4 is a schematic diagram of a point cloud patch applicable to an embodiment of the present application;

FIG. 5 is a schematic diagram of an occupancy map of a point cloud applicable to an embodiment of the present application;

Fig. 6 is a schematic block diagram of a decoder that can be used in an embodiment of the present application;

FIG. 7 is a schematic diagram of a frame group that can be used in an embodiment of the present application;

FIG. 8 is a schematic diagram of the first current point cloud group provided by an embodiment of this application;

FIG. 9 is a flowchart of the first point cloud encoding method provided by an embodiment of the application;

FIG. 10 is a schematic diagram of a second current point cloud group provided by an embodiment of this application;

FIG. 11 is a flowchart of the first point cloud decoding method provided by an embodiment of this application;

FIG. 12 is a flowchart of a second point cloud encoding method provided by an embodiment of this application;

FIG. 13 is a flowchart of a second point cloud decoding method provided by an embodiment of this application;

FIG. 14 is a flowchart of a third point cloud encoding method provided by an embodiment of this application;

15 is a flowchart of a third point cloud decoding method provided by an embodiment of this application;

FIG. 16 is a schematic block diagram of a first encoding device provided by an embodiment of this application;

FIG. 17 is a schematic block diagram of a first decoding device provided by an embodiment of this application;

FIG. 18 is a schematic block diagram of a second encoding device provided by an embodiment of this application;

FIG. 19 is a schematic block diagram of a second decoding apparatus provided by an embodiment of this application;

FIG. 20 is a schematic block diagram of a third encoding device provided by an embodiment of this application;

FIG. 21 is a schematic block diagram of a third decoding apparatus provided by an embodiment of this application;

FIG. 22 is a schematic block diagram of an implementation manner of a decoding device used in an embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

The term "plurality" in the embodiments of the present application refers to two or more. In the description of this application, unless otherwise specified, "/" means or, for example, A/B can mean A or B; "and/or" in this document is only an association relationship describing associated objects, It means that there can be three kinds of relationships, for example, A and/or B, which can mean: A alone exists, A and B exist at the same time, and B exists alone. In addition, in order to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with substantially the same function and effect. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and order of execution, and words such as "first" and "second" do not limit the difference.

Before explaining the embodiments of the present application in detail, first introduce the implementation environment of the embodiments of the present application:

An implementation environment provided by an embodiment of the application includes a point cloud decoding system. FIG. 1 is a schematic block diagram of a point cloud decoding system provided by an embodiment of this application. Among them, the term "point cloud decoding" or "decoding" may generally refer to point cloud encoding or point cloud decoding. Referring to FIG. 1, the point cloud decoding system includes a source device 10, a destination device 20, a link 30 and a storage device 40. Among them, the source device 10 can generate coded point cloud data. Therefore, the source device 10 may also be referred to as a point cloud encoding device. The destination device 20 may decode the encoded point cloud data generated by the source device 10. Therefore, the destination device 20 may also be referred to as a point cloud decoding device. The link 30 can receive the coded point cloud data generated by the source device 10 and can transmit the coded point cloud data to the destination device 20. The storage device 40 can receive the coded point cloud data generated by the source device 10, and can store the coded point cloud data, so that the destination device 20 can directly obtain the coded point cloud from the storage device 40 data. Alternatively, the storage device 40 may correspond to a file server or another intermediate storage device that can save the encoded point cloud data generated by the source device 10, so that the destination device 20 may stream or download the stored data of the storage device 40 Coded point cloud data.

Both the source device 10 and the destination device 20 may include one or more processors and a memory coupled to the one or more processors. The memory may include random access memory (RAM), read-only memory ( read-only memory, ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, which can be used to store the desired program in the form of instructions or data structures that can be accessed by a computer Any other media of the code, etc. For example, both the source device 10 and the destination device 20 may include desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, Televisions, cameras, display devices, digital media players, video game consoles, on-board computers, or the like.

The link 30 may include one or more media or devices capable of transmitting the encoded point cloud data from the source device 10 to the destination device 20. In one possible implementation, the link 30 may include one or more communication media that enable the source device 10 to directly send the encoded point cloud data to the destination device 20 in real time. In the embodiment of the present application, the source device 10 may modulate the coded point cloud data according to a communication standard, which may be a wireless communication protocol or the like, and may send the modulated point cloud data to the destination device 20. The one or more communication media may include wireless and/or wired communication media. For example, the one or more communication media may include a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, and the packet-based network may be a local area network, a wide area network, or a global network (for example, the Internet). The one or more communication media may include routers, switches, base stations, or other devices that facilitate communication from the source device 10 to the destination device 20, and the embodiment of the present application does not specifically limit this.

In a possible implementation manner, the storage device 40 may store the received encoded point cloud data sent by the source device 10, and the destination device 20 may directly obtain the encoded point cloud data from the storage device 40 . In this way, the storage device 40 may include any of a variety of distributed or locally accessed data storage media. For example, any of the multiple distributed or locally accessed data storage media may be a hard disk drive, Blu-ray disc, digital versatile disc (DVD), compact disc read-only memory (CD-ROM), flash memory, volatile or non-volatile memory, or used to store Any other suitable digital storage media for encoding point cloud data, etc.

In a possible implementation, the storage device 40 can correspond to a file server or another intermediate storage device that can store the encoded point cloud data generated by the source device 10, and the destination device 20 can be stored via streaming or downloading. Point cloud data stored by the device 40. The file server may be any type of server capable of storing the coded point cloud data and transmitting the coded point cloud data to the destination device 20. In a possible implementation, the file server may include a network server, a file transfer protocol (FTP) server, a network attached storage (NAS) device, or a local disk drive. The destination device 20 can obtain the coded point cloud data through any standard data connection (including an Internet connection). Any standard data connection can include a wireless channel (for example, Wi-Fi connection), a wired connection (for example, digital subscriber line (DSL), cable modem, etc.), or is suitable for obtaining the encoded data stored on the file server The combination of the two point cloud data. The transmission of the encoded point cloud data from the storage device 40 may be a streaming transmission, a download transmission, or a combination of both.

The point cloud decoding system shown in FIG. 1 is only one possible implementation, and the technology of the present application can not only be applied to the source device 10 that can encode the point cloud shown in FIG. The destination device 20 for decoding point cloud data may also be applicable to other devices that can encode point clouds and decode encoded point cloud data, which is not specifically limited in the embodiment of the present application.

In the point cloud decoding system shown in FIG. 1, the source device 10 includes a data source 120, an encoder 100 and an output interface 140. In some embodiments, the output interface 140 may include a regulator/demodulator (modem) and/or a transmitter, where the transmitter may also be referred to as a transmitter. The data source 120 may include a point cloud capture device (for example, a camera, etc.), a point cloud archive containing previously captured point cloud data, a point cloud feed interface for receiving point cloud data from a point cloud content provider, and/or A computer graphics system used to generate point cloud data, or a combination of these sources of point cloud data.

The data source may send a point cloud to the encoder 100, and the encoder 100 may encode the point cloud received from the data source 120 to obtain encoded point cloud data. The encoder can send the encoded point cloud data to the output interface. In some embodiments, the source device 10 directly sends the encoded point cloud data to the destination device 20 via the output interface 140. In other embodiments, the encoded point cloud data may also be stored on the storage device 40 for the destination device 20 to obtain later and use for decoding and/or playback.

In the embodiment of FIG. 1, the destination device 20 includes an input interface 240, a decoder 200 and a display device 220. In some embodiments, the input interface 240 includes a receiver and/or a modem. The input interface 240 can receive the encoded point cloud data via the link 30 and/or from the storage device 40, and then send it to the decoder 200. The decoder 200 can decode the received encoded point cloud data to obtain the Decoded point cloud data. The decoder may send the decoded point cloud data to the display device 220. The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20. Generally, the display device 220 displays the decoded point cloud data. The display device 220 may be any one of multiple types of display devices. For example, the display device 220 may be a liquid crystal display (LCD), a plasma display, or an organic light-emitting diode (OLED). Display or other type of display device.

Although not shown in FIG. 1, in some aspects, the encoder 100 and the decoder 200 may each be integrated with an audio encoder and decoder, and may include an appropriate multiplexer-demultiplexer (multiplexer- demultiplexer, MUX-DEMUX) unit or other hardware and software, used for encoding both audio and video in a common data stream or separate data streams. In some embodiments, if applicable, the MUX-DEMUX unit may conform to the ITU H.223 multiplexer protocol, or other protocols such as the user datagram protocol (UDP).

Each of the encoder 100 and the decoder 200 can be any of the following circuits: one or more microprocessors, digital signal processing (DSP), application specific integrated circuit, ASIC ), field-programmable gate array (FPGA), discrete logic, hardware, or any combination thereof. If the application is partially implemented in software, the device may store instructions for the software in a suitable non-volatile computer-readable storage medium, and may use one or more processors to execute the instructions in hardware So as to implement the technology of this application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) can be regarded as one or more processors. Each of the encoder 100 and the decoder 200 may be included in one or more encoders or decoders, and any of the encoders or the decoders may be integrated as a combined encoding in the corresponding device Part of the decoder/decoder (codec).

This application may generally refer to the encoder 100 as "signaling" or "sending" certain information to another device such as the decoder 200. The term "signaling" or "sending" may generally refer to the transmission of syntax elements and/or other data used to decode compressed point cloud data. This transmission can occur in real time or almost in real time. Alternatively, this communication may occur after a period of time, for example, when the syntax element is stored in a computer-readable storage medium in the encoded bitstream during encoding, and the decoding device may then store the syntax element on this medium. Retrieve the syntax element at any time.

FIG. 2 is a schematic block diagram of an encoder 100 provided by an embodiment of the application. Fig. 2 illustrates the point cloud compression (PCC) coding framework of MPEG (moving pictrue expert group) as an example. 2, the encoder 100 may include a point cloud block information generation module 101, a packing module 102, a depth map generation module 103, a texture map generation module 104, a depth map filling module 105, a texture map filling module 106, image-based or video-based Encoding module 107, occupancy map encoding module 108, auxiliary information encoding module 109, multiplexing module 110, point cloud occupancy map down-sampling module 111, point cloud occupancy map filling module 112, point cloud reconstruction module 113, and point cloud filtering module 114 .

The point cloud block information generation module 101 may receive one or more point cloud groups sent by the data source 120, where each point cloud group may include one or more frames of point clouds. For ease of description, the current point cloud group will be unified in the following Take the current point cloud frame in as an example. The point cloud block information generation module 101 can determine the three-dimensional coordinates of each point included in the current point cloud frame in the three-dimensional space coordinate system, and the normal direction vector of each point in the three-dimensional space, and determine each point according to the determined The normal direction vector in the three-dimensional space and the predefined projection plane divide the current point cloud frame into multiple patches (point cloud blocks). Each patch includes one or more points in the current point cloud frame. The point cloud block information generating module 101 can also determine the occupancy map of each patch and the depth map of each patch. The point cloud block information generating module 101 can also determine the three-dimensional information of each patch and the normal coordinate axis information of each patch, which can be used as auxiliary information of each patch. In addition, the point cloud block information generating module 101 can divide the patch into three types, namely the first type patch, the second type patch, and the third type patch. The first identifier is used to indicate the type of each patch. The point cloud block information generating module 101 may send the first identification of each patch and the auxiliary information of each patch in the current point cloud frame to the auxiliary information encoding module 109 for encoding, which may also be called compression encoding. Among them, the three-dimensional information of each patch may include the offset of the patch to be encoded in the 3D space along the tangent direction, the offset of the patch to be encoded in the 3D space along the double tangent direction, and the offset of the patch to be encoded in the 3D space. Offset in the normal direction. The point cloud block information generating module 101 may also send the occupancy map of each patch and the auxiliary information of each patch to the packaging module 102. In addition, the point cloud block information generating module 101 can also send the depth map of each patch to the depth map generating module 103.

The packing module 102 may pack the received occupancy map of each patch and the auxiliary information of each patch sent by the point cloud block information generating module 101 to obtain the occupancy map of the current point cloud frame. Specifically, the packaging module 102 can arrange the occupancy map of each patch in a specific order, for example, in descending order (or ascending order) of the width/height of the occupancy map of each patch, and then arrange the occupancy map of each patch in descending order (or ascending order). In the order of the occupancy map, insert the occupancy map of each patch into the available area of the occupancy map of the current point cloud frame to obtain the occupancy map of the current point cloud frame and the packing information of the patch. The packing module 102 may send the occupancy map of the current point cloud frame and the packing information of the patch to the auxiliary information encoding module 109 for compression encoding. Among them, the packing information of the patch may include the two-dimensional information of each patch in the occupancy map of the current point cloud frame, the identification information of the reference patch of each patch, the rotation transformation information of each patch, and the reference of the current point cloud frame The identification information of the frame. Among them, the two-dimensional information of each patch in the occupancy map of the current point cloud frame may be referred to as the two-dimensional information of each patch. The two-dimensional information of each patch can include the offset of each patch along the u axis in the current point cloud frame occupancy map, the offset of each patch along the v axis in the current point cloud frame occupancy map, and each patch The current point cloud frame occupies the width of the figure, and each patch occupies the height of the figure in the current point cloud frame. In addition, the packing module 102 may also send the packing information of the patch to the depth map generating module 103 and the point cloud occupancy map down-sampling module 111.

It should be noted that, in order to more intuitively understand the point cloud, the patch of the point cloud, and the occupancy map of the point cloud involved in the technology of this application, see Figures 3, 4 and 5, and Figure 3 is an embodiment applicable to the application. Fig. 4 is a schematic diagram of the patch of the point cloud of the frame, and Fig. 5 is the occupancy diagram of each patch of the point cloud of the frame shown in Fig. 4, the frame point obtained by packing by the packing module 102 Schematic diagram of cloud occupancy map. The point cloud shown in FIG. 3 may be the current point cloud frame in the embodiment of the present application, the patch of the point cloud shown in FIG. 4 may be the patch of the current point cloud frame in the embodiment of the present application, and the point shown in FIG. 5 The cloud occupancy map may be the occupancy map of the current point cloud frame in the embodiment of the present application.

The auxiliary information encoding module 109 receives the occupancy map of the current point cloud frame and the packing information of the patch sent by the packing module 102, as well as the first identification of each patch and the first identification of each patch sent by the point cloud block information generating module 101. After the auxiliary information, the current point cloud frame and each patch included in the current point cloud frame can be encoded to obtain a code stream including the encoded auxiliary information, and then the obtained code stream including the encoded auxiliary information is sent to the complex Use module 110. Among them, the two-dimensional information of each patch and the identification information of the reference patch of each patch in the packaging information of the patch may also be referred to as auxiliary information of each patch.

The depth map generation module 103 may receive the patch packaging information sent by the packaging module 102 and the depth map of each patch sent by the point cloud block information generation module 101, and then according to the packaging information of each patch and each The depth map of the patch generates the depth map of the current point cloud frame. Then the generated depth map of the current point cloud frame is sent to the depth map filling module 105 to fill the blank pixels in the depth map of the current point cloud frame to obtain the filled depth map of the current point cloud frame. The depth map filling module 105 may send the obtained filled depth map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled depth map of the current point cloud frame, Obtain the reconstructed current point cloud depth map and the code stream including the encoded current point cloud frame depth map, and may send the obtained reconstructed current point cloud frame depth map to the point cloud reconstruction module 113 , And send the code stream including the encoded depth map of the current point cloud frame to the multiplexing module 110.

The point cloud occupancy map down-sampling module 111 may perform down-sampling processing on the received occupancy map of the current point cloud frame sent by the packing module 102 to obtain a low-resolution occupancy map of the current point cloud frame. After that, the point cloud occupancy map down-sampling module 111 may also send the occupancy map of the current low-resolution point cloud frame to the occupancy map encoding module 108 and the point cloud occupancy map filling module 112. The occupancy map encoding module 108 can encode the received occupancy map of the current low-resolution point cloud frame to obtain a code stream including the occupancy map of the encoded low-resolution current point cloud frame, and the occupancy map encoding module 108 can also The code stream including the coded low-resolution occupancy map of the current point cloud frame may be sent to the multiplexing module 110. The point cloud occupancy map filling module 112 fills the occupancy map of the current point cloud frame with the original resolution according to the received occupancy map of the low-resolution current point cloud frame to obtain the filled occupancy map of the current point cloud frame, The occupancy map of the filled current point cloud frame has the original resolution.

The point cloud reconstruction module 113 may be based on the received occupancy map of the filled current point cloud frame sent by the point cloud occupancy map filling module 112, and the reconstructed current point cloud frame sent by the image or video-based encoding module 107. The depth map, patch packing information and patch auxiliary information reconstruct the geometry of the current point cloud frame to output the reconstructed point cloud. In addition, the point cloud reconstruction module 113 can also output the reconstructed points in the reconstructed point cloud. Correspondence with patch, and the packaging position of the reconstructed point in the reconstructed point cloud. The point cloud reconstruction module 113 can send the reconstructed point cloud, the corresponding relationship between the reconstructed point in the reconstructed point cloud and the patch to the point cloud filtering module 114, and the point cloud reconstruction module 113 can also reconstruct the reconstructed point cloud. The packing positions of the points are sent to the texture generation module 104.

The point cloud filtering module 114 may filter the reconstructed point cloud after receiving the reconstructed point cloud sent by the point cloud reconstruction module 113 and the corresponding relationship between the reconstructed points in the reconstructed point cloud and the patch. Specifically, defects such as obvious noise points and gaps in the reconstructed point cloud can be removed to obtain a filtered reconstructed point cloud, which can also be referred to as a smooth reconstructed point cloud. Or it can be said that the point cloud filtering block 114 can perform smoothing processing on the reconstructed point cloud.

The texture map generating module 104 receives the smooth reconstructed point cloud sent by the point cloud filtering module 114, the packed position of the reconstructed point in the reconstructed point cloud sent by the point cloud reconstruction module 113, and is sent by the data source 120 After the current point cloud frame, the texture map of the current point cloud frame can be generated according to the smooth reconstructed point cloud, the packaging position of the reconstructed point in the reconstructed point cloud and the current point cloud frame, and the generated current point cloud The texture map of the frame is sent to the texture map filling module 106 to fill the blank pixels in the texture map of the current point cloud frame to obtain the filled texture map of the current point cloud frame. The texture map filling module 106 may send the obtained filled texture map of the current point cloud frame to the image or video-based encoding module 107 to perform image or video-based encoding on the filled texture map of the current point cloud frame, Obtain the code stream including the texture map of the reconstructed current point cloud frame. The image or video-based encoding module 107 may also send the obtained code stream including the reconstructed texture map of the current point cloud frame to the multiplexing module 110.

Through the above description, the image or video-based encoding module 107, the occupancy map encoding module 108, and the auxiliary information encoding module 109 can send the obtained code streams to the multiplexing module 110, and the multiplexing module 110 can combine the received code streams To form a combined code stream, and send the combined code stream to the output interface 140. The output interface 140 may send the combined code stream to the decoder 200.

It should be understood that the encoder 100 shown in FIG. 2 is only an embodiment provided by the present application. In a specific implementation manner, the encoder 100 may include more or less modules than those shown in FIG. 2 Module. The embodiments of the present application do not specifically limit this.

FIG. 6 is a schematic block diagram of a decoder 200 according to an embodiment of the application. Figure 6 illustrates the MPEG PCC decoding framework as an example. Referring to Figure 6, the decoder 200 may include a demultiplexing module 201, an image or video-based decoding module 202, an occupancy map decoding module 203, an auxiliary information decoding module 204, a point cloud occupancy map filling module 205, and point cloud geometric information reconstruction Structuring module 206, point cloud filtering module 207, and point cloud texture information reconstruction module 208.

The demultiplexing module 201 may receive the combined code stream sent by the output interface 140 of the encoder 100 through the input interface 204, and send the combined code stream to the corresponding decoding module. Specifically, the demultiplexing module 201 sends the code stream of the texture map of the current point cloud frame encoded and the code stream of the depth map of the encoded current point cloud frame to the image or video-based decoding module 202, and The code stream including the occupancy map of the current point cloud frame with the encoded low resolution is sent to the occupancy map decoding module 203, and the code stream including the encoded auxiliary information is sent to the auxiliary information decoding module 204.

The image or video-based decoding module 202 can decode the received bitstream including the texture map of the encoded current point cloud frame and the bitstream including the depth map of the encoded current point cloud frame to obtain the reconstructed current The texture map information of the point cloud frame and the depth map information of the reconstructed current point cloud frame, and the reconstructed texture map information of the current point cloud frame can be sent to the point cloud texture information reconstruction module 208, The constructed depth map information of the current point cloud frame is sent to the point cloud geometric information reconstruction module 206. The occupancy map decoding module 203 can decode the received bitstream including the occupancy map of the current point cloud frame with encoded low resolution to obtain the occupancy map information of the reconstructed current point cloud frame with low resolution, and The reconstructed low-resolution current point cloud frame occupancy map information is sent to the point cloud occupancy map filling module 205. The point cloud occupancy map filling module 205 can obtain the reconstructed occupancy map information of the current point cloud frame with the original resolution according to the occupancy map information of the reconstructed low-resolution current point cloud frame, and then the reconstructed has The occupancy map information of the current point cloud frame at the original resolution is sent to the geometric information reconstruction module 206 of the point cloud. For the convenience of description, the reconstructed occupancy map information of the current point cloud frame with the original resolution will be uniformly referred to as the occupancy map information of the reconstructed current point cloud frame.

The auxiliary information decoding module 204 may decode the received code stream including the encoded auxiliary information to obtain auxiliary information, and may send the auxiliary information to the geometric information reconstruction module 206 of the point cloud. The specific decoding method will be described in subsequent embodiments.

The geometric information reconstruction module 206 of the point cloud can be based on the received depth map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202, and the reconstructed information sent by the point cloud occupancy map filling module 205. The occupancy map information of the current point cloud frame and the auxiliary information sent by the auxiliary information decoding module 204 reconstruct the geometry of the current point cloud frame to obtain the reconstructed point cloud. The reconstructed point cloud is similar to the reconstructed point cloud obtained by the point cloud reconstruction module 112 in the encoder 100, and the specific reconstruction process can refer to the reconstruction process of the point cloud reconstruction module 112 in the encoder 100. I won't repeat it here. The geometric information reconstruction module 206 of the point cloud may also send the reconstructed point cloud to the point cloud filtering module 207. The point cloud filtering module 207 can filter the reconstructed point cloud according to the received reconstructed point cloud to obtain a smooth reconstructed point cloud. The specific filtering process can refer to the filtering process of the point cloud filtering module 114 in the encoder 100 , I won’t repeat it here. The point cloud filtering module 207 may send the smooth reconstructed point cloud to the texture information reconstruction module 208 of the point cloud. The texture information reconstruction module 208 of the point cloud receives the smooth reconstructed point cloud sent by the point cloud filtering module 207 and the texture map information of the reconstructed current point cloud frame sent by the image or video-based decoding module 202. Afterwards, the texture information of the reconstructed point cloud can be reconstructed to obtain the reconstructed point cloud reconstructed by the texture information.

It should be understood that the decoder 200 shown in FIG. 6 is only an example, and in a specific implementation, the decoder 200 may include more or less modules than those shown in FIG. 6. The embodiment of the present application does not limit this.

For ease of understanding, the technical terms involved in the embodiments of the present application will be described first.

In an example, the current point cloud group may be a group of frame (GOF), and a frame group may include 32 frames of point clouds, that is, the current point cloud group may include 32 frames of point clouds. Of course, the number of frames of the point cloud included in the current point cloud group may also be other values, which is not limited in the embodiment of the present application. In an example, a frame group may also be divided into multiple subgroups, each subgroup may include one or more frames of point clouds, and the current point cloud group may be any one of the multiple subgroups. For example, referring to FIG. 7, FIG. 7 is a schematic diagram of a frame group provided by an embodiment of the application. As shown in FIG. 7, a frame group can be divided into 10 subgroups, and the first subgroup includes the first, second, and third subgroups. Frame point cloud, the second subgroup includes the 4th frame point cloud, the third subgroup includes the 5th, 6, 7, and 8th frame point clouds,..., the 10th subgroup includes the 30th, 31st, and 32th frame point clouds, The current point cloud group can be any one of these 10 subgroups. FIG. 7 only exemplarily shows the point clouds included in the first subgroup, the second subgroup, the third subgroup, and the tenth subgroup, and the point clouds included in the other subgroups are not described in detail. In summary, the current point cloud group can be a frame group, or the current point cloud group can be a subgroup of a frame group. Of course, the current point cloud group can also be other forms of point cloud groups, such as custom ones. The point cloud group is not limited in this embodiment of the application.

Based on the above description, patches can be divided into three types, namely the first type of patch, the second type of patch and the third type of patch. Exemplarily, according to the matching situation of the patch in the current point cloud group, the patch can also be divided into a global matching patch (SKIP_patch), a local matching patch (local_patch), and a non-matching patch (I_INTRA, or P_INTRA). That is, the first type of patch can be a global matching patch, the second type of patch can be a local matching patch, and the third type of patch can be a non-matching patch. For ease of description, taking the patch to be coded as an example, the three types of patches are described. It should be understood that the patch to be coded is any patch in the current point cloud frame, and the current point cloud frame is any of the current point cloud groups. A frame of point cloud. If the patch to be coded is the first type of patch, the patch to be coded has a matching relationship in all the point cloud frames in the current point cloud group, this can be called a global matching patch; if the patch to be coded is the second For the patch type, the patch to be encoded has a patch with a matching relationship in the reference frame of the current point cloud frame, but only a patch with a matching relationship exists in a part of the point cloud frame in the current point cloud group. This can be called local Matching patch; if the patch to be coded is the third type of patch, there is no patch with a matching relationship in any point cloud frame in the current point cloud group for the patch to be coded. This can be called a non-matching patch.

In a possible implementation manner, referring to FIG. 8, the reference frame of the current point cloud frame may be located in the current point cloud group. Exemplarily, the reference frame of the current point cloud frame may be the previous point cloud frame of the current point cloud frame. In addition, the patches included in all point cloud frames in the current point cloud group can be arranged in the order of the first type of patch, the second type of patch, and the third type of patch. Among them, the first type of patch in the current point cloud group is The number in each point cloud frame is the same, and when the number of the first-type patches is greater than 1, the first-type patches in each point cloud frame in the current point cloud group can also be arranged in the same order , That is, there is a matching relationship between the first-type patches at the same position in each point cloud frame. For example, the w-th first-type patch in each point cloud frame has a matching relationship with each other, and w is less than or equal to the number of first-type patches.

Generally, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the first-type patch in the occupancy map of the current point cloud frame are the same, but the three-dimensional information is different. However, in some cases, the two-dimensional information of the first-type patch can also be different, or the three-dimensional information can also be the same. That is, the rotation transformation information and the normal coordinate axis information of the first type patch may be the same, and the two-dimensional information and the three-dimensional information may be the same or different. In this way, for two-dimensional information and three-dimensional information, there may be three situations. For ease of description, the following takes the patch to be coded as an example. In the first possible situation, when the patch to be encoded is the first type of patch, the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded Same as the three-dimensional information of the patch to be encoded. In the second possible situation, when the patch to be encoded is the first type of patch, the two-dimensional information of the reference patch of the patch to be encoded is the same as the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded is the same as The three-dimensional information of the patch to be encoded is different. In the third possible situation, when the patch to be encoded is the first type of patch, the two-dimensional information of the reference patch of the patch to be encoded is different from the two-dimensional information of the patch to be encoded, and the three-dimensional information of the reference patch of the patch to be encoded The information is different from the three-dimensional information of the patch to be encoded.

When the patches included in all the point cloud frames in the current point cloud group are arranged in the manner shown in FIG. 8, the reference patch of the patch to be coded may be the same position in the reference frame of the current point cloud frame where the patch to be coded is located Patch. For example, if the patch to be coded is in the third position in the current point cloud frame, the third patch in the reference frame of the current point cloud frame is the reference patch of the patch to be coded.

The rotation transformation information may be rotation angle information, coordinate axis transformation information, or mirror transformation information. Wherein, the rotation angle information includes a specific rotation angle value or rotation angle index. For example, if the rotation angle value of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle value of the patch to be coded is 20, it means that the patch to be coded is rotated by 20°; if the rotation angle of the patch to be coded is A value of 80 means that the patch to be coded is rotated by 80°. For another example, if the rotation angle index of the patch to be coded is 0, it means that the patch to be coded does not rotate; if the rotation angle index of the patch to be coded is 1, it means that the patch to be coded is rotated by 15°; if the rotation angle of the patch to be coded is If the index is 2, it means that the patch to be coded is rotated by 30°; if the rotation angle index of the patch to be coded is 3, it means that the patch to be coded is rotated by 45°. The rotation angle information can also be characterized in other forms, which is not specifically limited here. For example, it can also be described in the form of a matrix, especially when accelerated processing is performed in a GPU (graphic process unit, image processor), the use of the matrix form will increase the processing speed.

It should be noted that the rotation of the patch to be encoded can be clockwise or counterclockwise.

If the rotation transformation information is the coordinate axis exchange information, the coordinate axis exchange information can be expressed as the coordinate axis exchange flag. For example, when the coordinate axis exchange flag is false or 0, it indicates the tangent coordinate axis of the patch to be encoded and the double tangent coordinate of the patch to be encoded The axes are not swapped; for example, when the coordinate axis swap flag is ture or 1, the tangent coordinate axis of the patch to be coded and the double tangent coordinate axis of the patch to be coded are swapped. For example, the index of the coordinate axis of the patch to be encoded can be 0, 1, or 2. Assuming that the index of the X axis is 0, the index of the Y axis is 1, and the index of the Z axis is 2, then the tangent axis index of the patch to be encoded and the double tangent axis index of the patch to be encoded can be 0, 1, and 2. Any two. In general, the tangent axis of the patch can be called the U axis, and the double tangent axis of the patch can be called the V axis. Assuming that under certain circumstances, the obtained tangent axis index of the current patch and the double tangent axis index of the patch to be encoded are 0 and 2, respectively. If the axis exchange flag is false or 0, the tangent of the patch to be encoded is not exchanged The index of the coordinate axis and the index of the double tangent axis of the patch to be encoded; if the axis exchange flag is true or 1, then the index of the tangent axis of the patch to be encoded and the index of the double tangent axis of the patch to be encoded are exchanged, namely The index of the tangent axis of the patch to be coded is updated to 2, and the index of the double tangent axis of the patch to be coded is updated to 0.

Next, the first point cloud coding method provided by the embodiment of the present application will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the encoder 100 shown in FIG. 2, any of the following point cloud encoding methods may be the point cloud decoding system. The execution by the encoder 100, more specifically, may be executed by the auxiliary information encoding module 109 in the encoder 100.

FIG. 9 is a flowchart of a point cloud encoding method provided by an embodiment of the present application, and the method is applied to a point cloud decoding system. Referring to Figure 9, the method includes:

S901: Obtain auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded.

It should be noted that the patch to be encoded may be any patch included in the current point cloud frame to be encoded. The first identifier may indicate the type of patch to be encoded. The first identified syntax element may be patch_mode or patch_type, etc. The value of the first identifier is different, and the type of the patch to be encoded is different.

S902: When the first identifier is the first value, encode the auxiliary information of the patch to be encoded and the first identifier into the code stream, and the patch to be encoded is the first patch in the current point cloud frame; or, when the first identifier is the first When the value is one, the auxiliary information of the patch to be encoded is encoded into the code stream, and the patch to be encoded is the non-first patch in the current point cloud frame.

Where the first identifier is the first value, it can indicate that the patch to be encoded is the first type of patch. The first type of patch can be a global matching patch in the current point cloud group. The auxiliary information of the patch to be encoded includes three-dimensional information. Including two-dimensional information, rotation transformation information and normal coordinate axis information.

Generally, because the first-type patch has matching patches in all point cloud frames in the current point cloud group, and the two-dimensional information and rotation transformation information between the first-type patches in the current point cloud group The information is the same as the normal axis, but the three-dimensional information may be different. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value, the auxiliary information of the patch to be coded may include three-dimensional information, but not two-dimensional information, rotation transformation information and normal direction. Coordinate axis information.

In a possible situation, the patches in each point cloud frame can be arranged in the order of the first type of patch, the second type of patch and the third type of patch, and then the first type of patch and the second type of patch are also used when encoding. The type patch and the third type patch are coded sequentially. So if there is a first-type patch in the current point cloud group, the first patch in each point cloud frame is the first-type patch. The number of the first-type patches in each point cloud frame in the current point cloud group can be the same, and when the number of the first-type patches is greater than 1, the number of the first type of patches in each point cloud frame in the current point cloud group One type of patches can be arranged in the same order, that is, there is a matching relationship between the first type of patches at the same position in each point cloud frame. If the first-type patch does not exist in the current point cloud group, the first patch in each point cloud frame may be the second-type patch or the third-type patch. If the first patch in each point cloud frame is a third-type patch, it can indicate that only this point cloud frame is included in the current point cloud group, and the first identifiers of all patches in this point cloud frame are third value.

For the case where the first patch in each point cloud frame in the current point cloud group is the first type of patch or the second type of patch, in order to distinguish which point cloud frames can form a point cloud group, as shown in Figure 10, for For the first point cloud frame in the current point cloud group, the first identifier of all patches included in the first point cloud frame can be set to the third value, and the patches in other point cloud frames are still in accordance with the first type of patch and the second type. The patch and the third type of patch are arranged in the order. Therefore, when the first identifier of the patch to be encoded is the first value, it can be determined that the current point cloud frame is the non-first point cloud frame in the current point cloud group. Moreover, the current point cloud frame may be the Kth point cloud frame in the current point cloud group, and K is an integer greater than or equal to 3. That is, the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be coded in the manner of S902.

Among them, because the point cloud frames in the current point cloud group can be numbered according to two numbering methods. The first numbering method: point cloud frame 0, point cloud frame 1, point cloud frame 2...; the second numbering method: point cloud frame 1, point cloud frame 2, point cloud frame 3... Therefore, the above Kth point cloud frame can refer to the point cloud frame 2 in the first numbering method and any point cloud frame after the point cloud frame 2, or it can refer to the point cloud frame 3 and the point cloud frame in the second numbering method. Any point cloud frame after point cloud frame 3. Regardless of the numbering method of the point cloud frames in the current point cloud group, the Kth point cloud frame can refer to the third point cloud frame in the actual current point cloud group and the point cloud frames after the third.

Based on the above description, the third point cloud frame in the current point cloud group and each point cloud frame after the third point cloud frame can be encoded in the manner of S902, and for the second point cloud frame in the current point cloud group A point cloud frame can be encoded in the following two ways:

The first possible implementation manner, if the current point cloud frame is the second point cloud frame in the current point cloud group, then, when the first identifier of the patch to be encoded is the first value, and the patch to be encoded is the current point cloud For the first patch in the frame, the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded can be incorporated into the code stream. When the first identifier of the patch to be coded is the first value, and the patch to be coded is not the first patch in the current point cloud frame, the auxiliary information of the patch to be coded can be encoded into the code stream instead of the patch to be coded The first identifier is incorporated into the code stream.

Wherein, in the process of setting the first identifiers of all patches included in the first point cloud frame in the current point cloud group to the third value, the number of first-type patches in the current point cloud group can be obtained.

Moreover, the operation of encoding the first identifier of the patch to be encoded, the number of first-type patches in the current point cloud group, and the auxiliary information of the patch to be encoded into the code stream may be: editing the first identifier of the patch to be encoded Enter the code stream, and then encode the number of first-type patches in the current point cloud group into the code stream, and finally, encode the auxiliary information of the patch to be encoded into the code stream. That is, the number of the first-type patch in the current point cloud group is compiled into the code stream after the first identifier of the patch to be encoded.

It should be noted that the fixed-length encoding method can be used to encode the number of patches of the first type into the code stream, and the variable-length encoding method can also be used to encode the number of patches of the first type into the code stream. Not limited.

In the second possible implementation manner, if the current point cloud frame is the second point cloud frame in the current point cloud group, then when the first identifier of the patch to be coded is the first value, the first value of the patch to be coded can be set An identification and auxiliary information are compiled into the code stream. That is, the first identification and auxiliary information of all the first-type patches in the second point cloud frame need to be encoded into the code stream, so that the number of the first-type patches in the current point cloud group can be indicated without The number of the first type patch in the current point cloud group is compiled into the code stream.

Regardless of the foregoing implementation manners for encoding the first-type patch in the second point cloud frame, the number of the first-type patch in the current point cloud group can be determined through the second point cloud frame. In this way, for the third point cloud frame in the current point cloud group and the point cloud frames after the third point cloud frame, only the first patch, that is, the first identifier of the first first type patch can be coded It is not necessary to encode the first identifier of the non-first first-type patch into the code stream.

In summary, when the first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group frame. In this way, there is no need to program the length of the point cloud frame of the current point cloud group in the code stream. In addition, for the third point cloud frame in the current point cloud group and the point cloud frames after the third point cloud frame, only the first identifier of the first first type patch can be compiled into the code stream, instead of the non-first point cloud frame. The first identifier of the first type patch is encoded into the code stream, which improves the coding efficiency and reduces the bit overhead of the code stream.

It is worth noting that, in a possible implementation, the first patch in the current point cloud frame can correspond to an identification information, which can also be called an index, and this identification information can uniquely indicate the first patch in the current point cloud frame patch, and the identification information of the first patch can usually be incorporated into the code stream. For other patches in the current point cloud frame, a counter can be used to determine which patch is the current point cloud frame. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.

It should be noted that the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), and the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), And the offset of the patch to be encoded in the 3D space along the normal direction (3d_shift_normal_axis).

As an example, when the first identifier is the first value, the three-dimensional information of the patch to be encoded can be encoded into the code stream according to the three-dimensional information of the reference patch of the patch to be encoded. Specifically, the differential encoding method can be used to encode the three-dimensional information of the patch to be encoded. The 3D information is encoded into the code stream, or directly into the code stream, and the specific encoding method is not limited. The fixed-length encoding method is adopted to encode the first identifier of the patch to be encoded into the code stream.

Wherein, using differential encoding to encode the three-dimensional information of the patch to be encoded into the code stream refers to encoding the difference between the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch of the patch to be encoded into the code stream.

As an example, this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, as shown in Table 1 and Table 2.

Table 1

Table 2

Among them, skip_patch_data_unit in Table 1 is the information of the patch to be encoded that is encoded into the code stream when the first identifier of the patch to be encoded is the first value, which can also be called the data unit of the patch to be encoded.

Wherein, when the first identifier is the second value or the third value, the auxiliary information of the patch to be encoded and the first identifier can be encoded into the code stream. That is, for each patch in the first point cloud frame in the current point cloud group, the auxiliary information and the first identifier of each patch can be encoded into the code stream. For the non-first-type patches in the non-first point cloud frame in the current point cloud group, the auxiliary information and the first identifier of each patch can be encoded into the code stream.

Wherein, when the first identifier is the second value, it indicates that the patch to be encoded is a second type of patch. The second type of patch can be a local matching patch in the current point cloud group. The auxiliary information of the patch to be encoded includes two-dimensional information and three-dimensional information. Information and identification information of the reference patch of the patch to be encoded. When the first identifier is the third value, it indicates that the patch to be coded is a third type of patch. The third type of patch can be a non-matching patch in the current point cloud group. The auxiliary information of the patch to be coded includes two-dimensional information, three-dimensional information, Rotation transformation information and normal coordinate axis information.

Generally, the second type of patch has a matching patch in the reference frame of the current point cloud frame, and the rotation transformation information and the normal coordinate axis information between the second type of patch and its reference patch are the same, while the two-dimensional information It may be different from the three-dimensional information. The third type of patch does not have a matching patch in the reference frame of the current point cloud frame. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first When the value is one, the auxiliary information of the patch to be encoded may include three-dimensional information, but not two-dimensional information, rotation transformation information, and normal coordinate axis information. When the first identifier is the second value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded, but does not include rotation transformation information and normal coordinate axis information. When the first identifier is the third value, the auxiliary information of the patch to be encoded may include two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

It should be noted that the two-dimensional information of the patch to be encoded may include the offset (2d_shift_u) of the patch to be encoded in the current point cloud frame occupancy map along the u axis, and the patch to be encoded in the current point cloud frame occupancy map along the v axis The direction shift (2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v). In addition, the syntax element of the reference patch of the patch to be encoded may be patchIndex, which is used to indicate the identification information of the reference patch; the syntax element may also include the identification information of the reference frame where the reference patch is located: frameIndex. The syntax element of the rotation transformation information of the patch to be encoded may be rotation. Furthermore, the identification information of the reference patch of the patch to be encoded may be used to indicate the reference patch of the patch to be encoded. The normal axis information may be the normal axis index of the patch to be encoded, for example, the normal axis index of the normalAxis may be 0, 1, or 2, for the X axis, Y axis, and Z axis, respectively.

As an example, when the first identifier is the second value, the two-dimensional information and three-dimensional information of the patch to be encoded can be compiled into the code stream according to the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded. The differential encoding method encodes the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream, and a more accurate predictive differential method can also be used to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream. The specific encoding method is not used. limited. The identification information and the first identification of the reference patch of the patch to be encoded are encoded into the code stream using a fixed-length encoding method. When the first identifier is the third value, the fixed-length encoding method can be used to encode the two-dimensional information, three-dimensional information, rotation transformation information, normal axis information and the first identifier of the patch to be encoded into the code stream, or variable The long encoding method encodes the two-dimensional information, three-dimensional information, rotation transformation information, normal coordinate axis information, and first identifier of the patch to be encoded into the code stream, which is not limited in the embodiment of the present application.

Among them, using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream refers to the difference between the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch of the patch to be encoded, and The difference between the three-dimensional information of the coded patch and the three-dimensional information of the reference patch of the patch to be coded is encoded into the code stream.

Based on the above description, for the current point cloud frame, the first identifier of the first first-type patch in the current point cloud frame, the first identifier of each second-type patch, and the first identifier of each third-type patch in the current point cloud frame The logo needs to be programmed into the code stream. The auxiliary information of each patch included in the current point cloud frame also needs to be compiled into the code stream. There may be the following three possible implementation methods for the encoding method of the first identification and auxiliary information.

In the first possible implementation manner, the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches. The auxiliary information of each patch included in the current point cloud frame can be individually packaged and compiled into a code stream. For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream. Then, the auxiliary information of each patch included in the current point cloud frame is packaged one by one and compiled into the code stream.

In the second possible implementation, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately. Code stream, the auxiliary information of each patch included in the current point cloud frame can also be packaged separately and compiled into code stream. For example, the first identifier of the first patch in the current point cloud frame can be compiled into the code stream, and then the auxiliary information of the first patch can be compiled into the code stream. Then, the auxiliary information of other first-type patches in the current point cloud frame is compiled into the code stream one by one. Finally, for the remaining patches, that is, the second type of patch and the third type of patch, the coding can be performed in a way that the first identifier is first compiled, and then the auxiliary information is compiled.

In a third possible implementation manner, the first identifiers of multiple patches included in the current point cloud frame can be packaged together and compiled into the code stream, and the first identifiers of the multiple patches include the first identifier of the patch to be encoded, and Except the patch to be coded, the remaining first identifier of the patch whose value is the second value or the third value among the multiple patches. For the auxiliary information of each patch included in the current point cloud frame, the same type of information in each patch auxiliary information can be packaged together and compiled into the code stream. Different types of information in the auxiliary information of each patch can be individually packaged and compiled into the code stream.

For example, the first identifier of the first patch in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged together and compiled into the code stream. Suppose that the auxiliary information of each patch includes rotation transformation information and normal coordinate axis information, while some auxiliary information of patches includes two-dimensional information or three-dimensional information, and some auxiliary information of patches does not include two-dimensional information or For three-dimensional information, the rotation transformation information and normal coordinate axis information in the auxiliary information of each patch are packaged together and compiled into the code stream. The two-dimensional information and three-dimensional information in the auxiliary information of each patch are individually packaged and compiled into the code stream.

Among them, the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively. The reference patch of the coded patch is included in the reference frame of the current point cloud frame, which is not limited in the embodiment of the present application.

In order to further reduce the bit overhead of the code stream, when encoding is performed in the order shown in Figure 8, for the third type of patch, other methods other than the encoding methods described above can also be used for encoding, for example, if The patch to be coded is the first third-type patch, then the fixed-length coding method may be used for coding, or the variable-length coding method may be used, which is not limited in the embodiment of the application. If the patch to be coded is not the first type 3 patch, at this time, according to the 2D information and 3D information of the first type 3 patch, the 2D information and 3D information of the patch to be coded can be encoded into the code stream using differential encoding. , And other information is also encoded in the manner described above. Among them, according to the two-dimensional information and three-dimensional information of the first third-type patch, using differential encoding to encode the two-dimensional information and three-dimensional information of the patch to be encoded into the code stream means: the two-dimensional information and the first The difference between the two-dimensional information of the third-type patch and the difference between the three-dimensional information of the patch to be encoded and the first three-dimensional patch of the third type are encoded into the code stream.

In this embodiment of the present application, first obtain the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded. Since the number of the first-type patches in each point cloud frame in the current point cloud group can be the same, when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame, Indicates that the patch to be encoded is the first patch of the first type in the current point cloud frame. At this time, the auxiliary information and the first identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, the patch to be encoded is When the current point cloud frame is not the first patch, it indicates that the patch to be coded is the non-first type I patch in the current point cloud frame. At this time, the auxiliary information of the patch to be coded can be encoded into the code stream instead of the waiting The first identifier of the coded patch is encoded into the code stream, so that the format of the code stream can be simplified, the bit overhead of the code stream is further reduced, and the coding efficiency is improved.

Next, the point cloud decoding method corresponding to the point cloud encoding method based on the embodiment shown in FIG. 9 will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the decoder 200 shown in FIG. 6, any of the following point cloud decoding methods may be the point cloud decoding system. What is executed by the decoder 200, more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200. Referring to Figure 11, the decoding method includes:

S1101: When the patch to be decoded is the first patch in the current point cloud frame, analyze the first identifier of the patch to be decoded from the code stream.

Since the first identifier of the first patch in the current point cloud frame is encoded in the code stream, when the point cloud block to be decoded is the first patch in the current point cloud frame, the to be decoded can be parsed from the code stream The first ID of the patch.

In a possible implementation manner, it can be determined whether the code stream includes the identification information of the patch to be decoded, and if the identification information of the patch to be decoded is included, it can be determined that the patch to be decoded is the first patch in the current point cloud frame. If the identification information of the patch to be decoded is not included, it can be determined that the patch to be decoded is the non-first patch in the current point cloud frame. At this time, the number of the patch to be decoded in the current point cloud frame can be determined according to the counter. Among them, the counter starts counting when the first patch of the current point cloud frame is determined.

S1102: When the first identifier is the first value, parse the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.

It should be noted that when the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, that is, the patch to be decoded has a matching patch in all point cloud frames in the current point cloud group. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.

As an example, the number of patches of the first type can be parsed from the code stream by a method that matches the encoding end. For example, fixed-length decoding or variable-length decoding can be used to parse the number of patches of the first type from the code stream.

It can be seen from the embodiment shown in FIG. 9 that, when the first identifier is the first value, it can indicate that the current point cloud frame is the non-first point cloud frame in the current point cloud group. At this time, if the current point cloud frame is the second point cloud frame in the current point cloud group, the patch to be decoded can be decoded through the following two possible implementation methods.

The first possible implementation is the first identification of the first patch in the second point cloud frame in the current point cloud group, and the number of the first type of patch included in each point cloud frame in the current point cloud group Code stream together. At this point, if the current point cloud frame is the second point cloud frame in the current point cloud group, and the patch to be decoded is the first patch in the current point cloud frame, the first patch to be decoded can be parsed from the code stream. Identification, when the first identification is the first value, the number of the first-type patches in the current point cloud group is analyzed from the code stream. Then parse the auxiliary information of the patch to be decoded from the code stream.

In the second possible implementation manner, the first identifiers of the first type patch included in the second point cloud frame in the current point cloud group are all encoded into the code stream. At this time, if the current point cloud frame is in the current point cloud group If the patch to be decoded is the first patch in the second point cloud frame, the first identifier of the patch to be decoded can be parsed from the code stream. When the first identifier is the first value, the patch to be decoded is parsed from the code stream. Supplementary information.

Next, the case where the patch to be decoded is the nth patch in the current point cloud frame is described through S1103 and S1104.

S1103: When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first-type patch in the reference point cloud frame of the current point cloud frame, analyze from the code stream The auxiliary information of the patch to be decoded, and the auxiliary information includes the three-dimensional information of the patch to be decoded.

It can be seen from the embodiment shown in FIG. 9 that, during the encoding process of the first-type patch, the first identifier of the non-first first-type patch is not encoded into the code stream, so when the patch to be decoded is the first in the current point cloud frame n patches, when n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first first-type patch in the current point cloud frame, At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.

Since the auxiliary information of the first type of patch that is encoded into the code stream during the encoding process includes three-dimensional information, but does not include two-dimensional information, rotation transformation information and normal coordinate axis information, when the patch to be decoded is the first type of patch The auxiliary information parsed from the code stream includes the three-dimensional information of the patch to be decoded, but does not include the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded. At this time, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can also be obtained according to the auxiliary information of the reference patch of the patch to be decoded.

As an example, a method matching the encoding end may be used for decoding, for example, a differential decoding method may be used to parse the three-dimensional information of the patch to be decoded from the code stream. For other auxiliary information of the patch to be decoded, for example, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded, the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, and then according to the The identification information of the reference frame determines the reference frame of the current point cloud frame, and then obtains the two-dimensional information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame, and refers to the patch The two-dimensional information, rotation transformation information and normal coordinate axis information of the patch are used as the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded. Among them, the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch. For ease of description, the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame. In this way, the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame The auxiliary information of the patch is determined. In this way, the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc. In addition, the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can be obtained from the decoded point cloud, and the obtained two-dimensional information, The rotation transformation information and the normal coordinate axis information are used as the two-dimensional information, the rotation transformation information and the normal coordinate axis information of the patch to be decoded.

When the patch in each point cloud frame in the current point cloud group is encoded according to the arrangement shown in Figure 8, the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded at the current point Patches with the same position in the cloud frame.

Regarding the method of using differential decoding to parse the 3D information of the patch to be decoded from the code stream, it may specifically be: obtaining the 3D information of the patch whose position is the same as the position of the patch to be decoded in the current point cloud frame from the decoded point cloud. Or, obtain the three-dimensional information of the reference patch of the patch to be decoded from the reference frame of the current point cloud frame. Then, the obtained three-dimensional information is added with the difference of the three-dimensional information parsed from the code stream to obtain the three-dimensional information of the patch to be decoded.

Based on the above description, if the patch to be decoded is the non-first patch in the current point cloud frame, the counter can be used to determine the number of patches in the current point cloud frame to be decoded. In this way, when the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, the patch to be decoded can be determined It is the non-first first-type patch in the current point cloud frame, so that the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded.

Among them, based on the description of S1102, the number of patches of the first type included in each point cloud frame in the current point cloud group can be determined in two ways. In the first possible implementation, after decoding the first identifier of the first patch in the second point cloud frame in the current point cloud group, the first identifier in the current point cloud group can be directly parsed from the code stream. The number of class patches. In the second possible implementation manner, after decoding the first identifiers of all patches of the second point cloud frame in the current point cloud group, the number of patches with the first identifier as the first value can be determined, so as to determine The number of first-type patches included in the second point cloud frame, that is, the number of first-type patches in the current point cloud group. That is, the number of first-type patches included in each point cloud frame in the current point cloud group can be determined through the second point cloud frame.

In a possible situation, the second point cloud frame in the current point cloud group can be used as the reference point cloud frame of the third point cloud frame, and the third point cloud frame can be used as the reference point cloud frame of the fourth point cloud frame Point cloud frame, and so on. In this way, the third point cloud frame and the point cloud frames after the third can determine whether the nth patch in the current point cloud frame is the first type patch according to the number of the first type patch in the reference point cloud frame.

In addition, since the first identifier of the first patch in the current point cloud frame, and the first identifier of the patch whose value is the second or third value other than the first patch can be packaged separately and compiled into the code stream, Can be packaged together and compiled into code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for patches that take the second value or the third value other than the first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of the first patch to be decoded in the current point cloud frame and the first identifier of the patch whose value is the second or third value other than the first patch can be parsed in sequence. For the method of packing together and compiling into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , Wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis. Similarly, when the same type of information in the auxiliary information of each patch included in the current point cloud frame is also packaged and compiled into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, you can The code stream parses the same type of information in the auxiliary information of each patch included in the current point cloud frame. That is, the same type of information in the auxiliary information of each patch included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.

As an example, a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream. For example, a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.

S1104: When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first-type patch in the reference point cloud frame of the current point cloud frame, analyze the to-be-decoded patch from the code stream The first ID of the patch.

Since in the encoding process of the second type patch and the third type patch, the first identifier is encoded into the code stream, so when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current When the point cloud frame refers to the number of the first-type patch in the point cloud frame, it indicates that the patch to be decoded is a second-type patch or a third-type patch. At this time, the first identifier of the patch to be decoded can be parsed from the code stream.

S1105: When the first identifier is the second value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information and three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded.

It should be noted that when the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch, that is, there is a reference patch matching the patch to be decoded in the reference frame of the current point cloud frame. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded, but does not include rotation transformation information and normal coordinate axis information.

Because the auxiliary information of the second type of patch that is encoded into the code stream during the encoding process includes two-dimensional information, three-dimensional information and identification information of the reference patch, but does not include the rotation transformation information and the normal coordinate axis information, so when the patch to be decoded When it is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and normal direction of the patch to be decoded Coordinate axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.

Among them, the identification information of the reference patch of the patch to be decoded can be parsed from the code stream by a method matching the encoding end, for example, a fixed-length decoding method can be used for analysis. For the rotation transformation information and normal coordinate axis information of the patch to be decoded, the reference patch of the reference patch to be decoded can be determined through the identification information of the reference patch to be decoded, and then the rotation transformation information and normal coordinates of the reference patch to be decoded are obtained. Axis information, and use the rotation transformation information and normal coordinate axis information of the reference patch as the rotation transformation information and normal coordinate axis information of the patch to be decoded. For the two-dimensional information and three-dimensional information of the patch to be decoded, as an example, a method that matches the encoding end can be used for decoding. For example, the two-dimensional information and three-dimensional information of the patch to be decoded can be parsed from the code stream in a differential decoding manner. Specifically, the reference patch of the patch to be decoded can be determined by the identification information of the reference patch of the patch to be decoded, and then the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded can be obtained. Then, the obtained two-dimensional information and three-dimensional information are respectively added with the two-dimensional information difference and the three-dimensional information difference parsed from the code stream, so as to obtain the two-dimensional information and the three-dimensional information of the patch to be decoded.

S1106: When the first identifier is the third value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.

It should be noted that when the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch, that is, there is no reference patch matching the patch to be decoded in the current point cloud group. At this point, the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded can be parsed from the code stream.

Among them, the first value, the second value, and the third value may be binary 10, 11, and 00 respectively, or the first value, the second value, and the third value may be binary 11, 10, and 00, etc., respectively. The application embodiment does not limit this.

Based on the above description, the first type patch, the second type patch and the third type patch in each point cloud frame in the current point cloud group can be arranged as shown in Figure 8. At this time, if the patch to be decoded is The first patch in the current point cloud frame, then, after parsing the first identifier of the patch to be decoded from the code stream, you can determine whether the current point cloud frame is the first frame in the current point cloud group according to the value of the first identifier Point cloud. That is, when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is the third value, it can be determined that the current point cloud frame is the first point cloud in the current point cloud group.

As an example, the fixed-length decoding method can be used to parse the two-dimensional information, three-dimensional information, rotation transformation information, and normal axis information of the patch to be decoded from the code stream, or the variable-length decoding method can be used to parse the to-be-decoded patch from the code stream. The two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the decoded patch are not limited in the embodiment of the present application. However, if the auxiliary information of the first third-type patch is coded according to fixed-length coding or variable-length coding, the two-dimensional information and three-dimensional information of the non-first third-type patch are coded by differential coding, and other auxiliary information is coded Encode with fixed-length encoding. Then, in this step, for non-first type III patches, differential decoding can be used to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream, and fixed-length decoding can be used to parse the to-be-decoded patch from the code stream. Decode the rotation transformation information and normal axis information of the patch. Among them, for the non-first third-type patch, the method of using differential decoding to parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream is: parse the two-dimensional information and the first information of the patch to be decoded from the code stream. The difference between the two-dimensional information of the third-type patch is added to the two-dimensional information of the first third-type patch to obtain the two-dimensional information of the patch to be decoded. Analyze the difference between the 3D information of the patch to be decoded and the 3D information of the first third type patch from the code stream, and add the difference to the 3D information of the first third type patch to obtain the 3D information of the patch to be decoded information.

S1107: According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.

It should be noted that after obtaining the auxiliary information of the patch to be decoded, that is, after obtaining the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded, the auxiliary information can be reproduced according to but not limited to these auxiliary information. To construct the patch to be decoded, this application does not limit the specific information needed to reconstruct the patch to be decoded.

In the embodiment of this application, since the first identifier of the first patch in the current point cloud frame is encoded into the code stream, when the patch to be decoded is the first patch in the current point cloud frame, The first identifier of the patch to be decoded is parsed from the code stream. When the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The information includes the three-dimensional information of the patch to be decoded. When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first A first-type patch, at this time, the auxiliary information of the patch to be decoded can be parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded. When the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information and information of the patch to be decoded. The identification information of the reference patch of the patch to be decoded. When the first identifier is the third value, it indicates that the patch to be decoded is a third type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, and Rotation transformation information and normal coordinate axis information. That is, for the first type one patch, the first identifier needs to be parsed from the code stream, while for the non-first type first patch, the first identifier does not need to be parsed from the code stream, and for the first type patch and the second type For patch-like, only part of the auxiliary information needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. In this way, the format of the parsed code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.

Hereinafter, the second point cloud coding method provided by the embodiment of the present application will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the encoder 100 shown in FIG. 2, any of the following point cloud encoding methods may be the point cloud decoding system. The execution by the encoder 100, more specifically, may be executed by the auxiliary information encoding module 109 in the encoder 100. Referring to Figure 12, the method includes:

S1201: Obtain auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded.

When it needs to be explained, the content of S1201 may refer to S901 in the embodiment shown in FIG. 9, and this is not repeated in this embodiment of the application.

S1202: When the first identifier is the first value, encode the auxiliary information of the patch to be encoded, the first identifier, and the second identifier of the patch to be encoded into the code stream, and the patch to be encoded is the first patch in the current point cloud frame; Or, when the first identifier is the first value, the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded are encoded into the code stream, and the patch to be encoded is the non-first patch in the current point cloud frame.

Generally, the first-type patch has patches with matching relationships in all point cloud frames in the current point cloud group, and the two-dimensional information, rotation transformation information and information between the first-type patches in the current point cloud group The normal coordinate axis information is the same, but the three-dimensional information may be different, but in some cases, the three-dimensional information of the first type of patch may also be the same. In this case, when the first identifier is the first value, it may indicate that the patch to be encoded is a first-type patch, and the information encoded into the code stream may also include the second identifier.

It should be noted that the second identifier is used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. The syntax element of the second identification may be override_3d_shift_data_flag. When the second identifier is the fourth value, it can indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. information. When the second identifier is the fifth value, it can indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is different from the three-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include three-dimensional information. The specific encoding method is not limited here. For example, a differential encoding method may be used, or a fixed-length encoding method may be used. The syntax element of the fourth value may be false, and the syntax element of the fifth value may be true.

As an example, this article exemplifies a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier, see Table 1 and Table 3.

table 3

Further, in a possible situation, when the first identifier of the patch to be encoded is the first value, that is, when the patch to be encoded is the first type of patch, the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and The two-dimensional information of the patch to be encoded is different from the two-dimensional information of the reference patch. In this way, when the first identifier is the first value, in addition to the second identifier, the information encoded into the code stream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the code stream. The third-identified syntax element may be override_2d_shift_data_flag. Among them, the second identifier has already been described, so it will not be repeated here.

When the third identifier is the sixth value, it may indicate that the two-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is the same as the two-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded can be Does not include two-dimensional information. When the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be encoded has been encoded into the code stream. That is, the two-dimensional information of the patch to be encoded is not the same as the two-dimensional information of the reference patch of the patch to be encoded. Therefore, it is necessary to encode the two-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may include Two-dimensional information. The syntax element of the sixth value may be false, and the syntax element of the seventh value may be true.

As an example, this article illustrates a partial syntax structure when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the second identifier and the third identifier, see Table 1 and Table 4 .

Table 4

When the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, not only the above-mentioned second and third marks can be used to indicate the coding Whether the information of the stream includes two-dimensional information and three-dimensional information of the patch to be encoded. It can also be indicated in other ways. For example, the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be encoded have been encoded into the code stream, that is, when the first identifier is the first value , It can indicate that the patch to be encoded is the first type of patch, and the information encoded into the code stream can also include a fourth identifier. The fourth identified syntax element may be override_patch_data_flag. When the fourth identifier is the eighth value, it may indicate that the two-dimensional information and three-dimensional information of the patch to be encoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded, and the code stream is not included. When the fourth identifier is a ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be encoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be encoded, and both have been encoded into the code stream. The syntax element of the eighth value can be flase, and the syntax element of the ninth value can be true.

Among them, because the three-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the 3D space along the tangent direction (3d_shift_tangent_axis), the offset of the patch to be encoded in the 3D space along the bitangent direction (3d_shift_bitangent_axis), and The offset of the encoded patch in the 3D space along the normal direction (3d_shift_normal_axis). The two-dimensional information of the patch to be encoded may include the offset of the patch to be encoded in the current point cloud frame occupancy map along the u-axis direction (2d_shift_u), and the offset of the patch to be encoded in the current point cloud frame occupancy map along the v-axis direction ( 2d_shift_v), the width of the patch to be encoded in the current point cloud frame occupancy map (2d_size_u), and the height of the patch to be encoded in the current point cloud frame occupancy map (2d_size_v). Therefore, it is also possible to pass the flag (override_3d_shift_tangent_axis) used to indicate whether 3d_shift_tangent_axis has been coded into the code stream, the flag used to indicate whether 3d_shift_bitangent_axis has been coded into the code stream (override_3d_shift_bitangent_axis) and the flag (override_3d_shift_bitangent_axis) used to indicate whether the 3d_shift_normal_axis has been coded override_3d_shift_normal_axis) to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream, and through the flag (override_2d_shift_u) used to indicate whether 2d_shift_u has been encoded into the code stream, and the flag used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v) An identifier (override_2d_size_u) used to indicate whether 2d_size_u has been encoded into the code stream and an identifier (override_2d_size_v) used to indicate whether 2d_size_v has been encoded into the code stream to indicate whether the two-dimensional information of the patch to be encoded has been encoded into the code stream. Similar to the second identifier, the third identifier, and the fourth identifier, each of the above identifiers can have two values, and the syntax elements of the two values can be true and false. The meaning of each value can refer to the meaning of different values of the second identifier, the third identifier or the fourth identifier, which will not be repeated here.

As an example, this article exemplifies when the first identifier of the patch to be encoded is the first value, and the information encoded into the code stream includes the fourth identifier, and the identifier used to indicate different two-dimensional information and three-dimensional information. Part of the grammatical structure, see Table 1 and Table 5.

table 5

Wherein, when the first identifier is the second value, it may indicate that the patch to be encoded is a second-type patch. That is, there is a reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded.

Wherein, when the first identifier is a third value, it may indicate that the patch to be encoded is a third-type patch. That is, there is no reference patch that matches the patch to be encoded in the reference frame of the current point cloud frame. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information.

It should be noted that the same content in the embodiment of the present application and the embodiment shown in FIG. 9 will not be repeated, and reference may be made to the description in the embodiment shown in FIG. 9.

In this embodiment of the present application, first obtain the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded. Since the number of the first-type patches in each point cloud frame in the current point cloud group can be the same, when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame, Indicates that the patch to be encoded is the first patch of the first type in the current point cloud frame. At this time, the auxiliary information, the first identifier and the second identifier of the patch to be encoded can be encoded into the code stream; when the first identifier is the first value, When the patch to be coded is the non-first patch in the current point cloud frame, it indicates that the patch to be coded is the non-first type 1 patch in the current point cloud frame, and the auxiliary information and the second identifier of the patch to be coded can be coded Instead of coding the first identifier of the patch to be coded into the code stream, the format of the code stream can be simplified. The second identifier is an identifier used to indicate whether the three-dimensional information of the patch to be encoded has been encoded into the code stream. When the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be encoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be encoded is the same as the three-dimensional information of the reference patch of the patch to be encoded, so there is no need to re-encode the three-dimensional information of the patch to be encoded into the code stream, that is, the auxiliary information of the patch to be encoded may not include the three-dimensional information. Information, in this way, can further reduce the bit overhead of the code stream, thereby improving the coding efficiency.

Next, the point cloud decoding method based on the point cloud encoding method of the embodiment shown in FIG. 12 will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the decoder 200 shown in FIG. 6, any of the following point cloud decoding methods may be the point cloud decoding system. What is executed by the decoder 200, more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200. Referring to Figure 13, the decoding method includes:

S1301: When the patch to be decoded is the first patch in the current point cloud frame, analyze the first identifier of the patch to be decoded from the code stream.

It should be noted that the content of S1301 can refer to the content of S1101, which will not be repeated here.

S1302: When the first identifier is the first value, parse the second identifier of the patch to be decoded from the code stream.

It should be noted that the second identifier has been described in S1202 above, so it will not be repeated here.

S1303: When the second identifier is the fourth value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; when the second identifier is the fifth value, parse the auxiliary information of the patch to be decoded from the code stream The auxiliary information includes the three-dimensional information of the patch to be decoded.

It should be noted that when the second identifier is the fourth value, it may indicate that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the 3D information of the patch to be decoded is the same as the 3D information of the reference patch of the patch to be decoded. At this time, the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of the reference patch of the patch to be decoded. Specifically, the auxiliary information of the reference patch of the patch to be decoded may be used as the auxiliary information of the patch to be decoded. When the second identifier is the fifth value, it can indicate that the 3D information of the patch to be decoded has been encoded into the bitstream, and the auxiliary information of the patch to be decoded includes 3D information. At this time, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded. As an example, a differential decoding method can be used to parse the auxiliary information of the patch to be decoded from the code stream. The specific method can refer to the previous article.

In some embodiments, based on the description of step 1202, when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are not the same, the In addition to the second identifier, the information of the codestream may also include a third identifier for indicating whether the two-dimensional information of the patch to be encoded has been encoded into the codestream. That is, when the first identifier is the first value, in addition to parsing the second identifier of the patch to be decoded from the code stream, the third identifier of the patch to be decoded can also be parsed from the code stream. When the third identifier is the sixth value, it can indicate that the two-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the two-dimensional information of the patch to be decoded is the same as the two-dimensional information of the reference patch of the patch to be decoded, so the two-dimensional information of the reference patch of the patch to be decoded can be used as the two-dimensional information of the patch to be decoded. When the third identifier is the seventh value, it can indicate that the two-dimensional information of the patch to be decoded has been encoded into the code stream. That is, the two-dimensional information of the patch to be decoded is different from the two-dimensional information of the reference patch of the patch to be decoded, so the two-dimensional information of the patch to be decoded can be parsed from the code stream. The specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.

In other embodiments, based on the description of step 1202, when the three-dimensional information of the patch to be encoded and the three-dimensional information of the reference patch, and the two-dimensional information of the patch to be encoded and the two-dimensional information of the reference patch are different, not only The above-mentioned second identifier and third identifier can be used to indicate whether the information compiled into the code stream includes the two-dimensional information and three-dimensional information of the patch to be decoded. It can also be indicated in other ways. For example, the fourth identifier can also be used to indicate whether the two-dimensional information and three-dimensional information of the patch to be decoded have been encoded into the code stream. That is, when the first identifier is the first value, the fourth identifier of the patch to be decoded can be parsed from the code stream. When the fourth identifier is the eighth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are the same as the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and the code stream is not encoded, so the to-be decoded The 2D information and 3D information of the reference patch of the patch are used as the 2D information and 3D information of the patch to be decoded. When the fourth identifier is the ninth value, it can indicate that the two-dimensional information and three-dimensional information of the patch to be decoded are different from the two-dimensional information and three-dimensional information of the reference patch of the patch to be decoded, and both have been encoded into the code stream. Parse the two-dimensional information and three-dimensional information of the patch to be decoded from the code stream. The specific decoding method may be a decoding method that matches the encoding method, which will not be repeated here.

In addition, the identification (override_3d_shift_tangent_axis) used to indicate whether 3d_shift_tangent_axis has been coded into the code stream, the flag (override_3d_shift_bitangent_axis) used to indicate whether the 3d_shift_bitangent_axis has been coded into the code stream, and the flag (override_shift_normal_axis) used to indicate whether 3d_shift_normal_axis has been coded into the code stream ) To indicate whether the three-dimensional information of the patch to be decoded has been encoded into the code stream, and the identifier used to indicate whether 2d_shift_u has been encoded into the code stream (override_2d_shift_u), the identifier used to indicate whether 2d_shift_v has been encoded into the code stream (override_2d_shift_v), The flag used to indicate whether 2d_size_u has been encoded into the code stream (override_2d_size_u) and the flag used to indicate whether 2d_size_v has been encoded into the code stream (override_2d_size_v) to indicate whether the two-dimensional information of the patch to be decoded has been encoded into the code stream, and The above-mentioned second identification, third identification and fourth identification are similar and will not be repeated here.

S1304: When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, analyze from the code stream The second identifier of the patch to be decoded.

Since the second identifiers of the non-first first-type patch in the current point cloud frame are all encoded into the bitstream, when the patch to be decoded is the nth patch in the current point cloud frame, n is greater than 1 and greater than the current When the point cloud frame refers to the number of the first-type patch in the point cloud frame, that is, when the patch to be decoded is not the first first-type patch, the second identifier of the patch to be decoded can be parsed from the code stream.

S1305: When the second identifier is the fourth value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; when the second identifier is the fifth value, parse the auxiliary information of the patch to be decoded from the code stream The auxiliary information includes the three-dimensional information of the patch to be decoded.

It should be noted that, for the content of S1305, reference may be made to the content of S1303, which is not repeated here.

S1306: According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.

It should be noted that the content of S1306 can refer to the content of S1107, which will not be repeated here.

In the embodiment of this application, since the first identifier of the first patch in the current point cloud frame is encoded into the code stream, when the point cloud block patch to be decoded is the first patch in the current point cloud frame, from The first identifier of the patch to be decoded is parsed in the code stream. When the first identifier is the first value, it indicates that the patch to be decoded is the first patch of the first type. At this time, the second identifier of the patch to be decoded can be parsed from the code stream. When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and less than or equal to the number of the first type of patch in the reference point cloud frame of the current point cloud frame, it indicates that the patch to be decoded is not the first A first-type patch, at this time, the second identifier of the patch to be decoded can be parsed from the code stream. Wherein, when the second identifier is the fourth value, it indicates that the three-dimensional information of the patch to be decoded is not encoded into the code stream. That is, the three-dimensional information of the patch to be decoded is the same as the three-dimensional information of the reference patch of the patch to be decoded. At this time, the three-dimensional information of the reference patch of the patch to be decoded can be used as the three-dimensional information of the patch to be decoded, without the need to extract from the code stream Parsing the three-dimensional information of the patch to be decoded can simplify the format of the parsing code stream, reduce the bit overhead of the code stream, and improve the efficiency of decoding.

In the following, the third point cloud coding method provided by the embodiment of the present application will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the encoder 100 shown in FIG. 2, any of the following point cloud encoding methods may be the point cloud decoding system. The execution by the encoder 100, more specifically, may be executed by the auxiliary information encoding module 109 in the encoder 100. Referring to Figure 14, the method includes:

S1401: Obtain auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded.

It should be noted that, for the content of S1401, reference may be made to S901 in the embodiment shown in FIG. 9, which will not be repeated in this embodiment of the application.

S1402: When the first identifier is the first value, encode the first identifier of the patch to be encoded into the code stream.

When the first identifier is the first value, it indicates that the patch to be encoded is the first type of patch. In a possible implementation, the first-type patch has patches with matching relationships in all point cloud frames in the current point cloud group, and two-dimensional information between the first-type patches in the current point cloud group , 3D information, rotation transformation information and normal coordinate axis information are all the same. Therefore, in order to reduce the bit overhead of the code stream and improve the coding efficiency, when the first identifier is the first value, the first identifier of the patch to be coded is encoded into the code stream instead of the auxiliary information of the patch to be coded. Code stream.

S1403: When the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into the code stream.

Based on the description of S1402 and S1403, it can be known that no matter whether the first identifier of the patch to be encoded is the first value, the second value or the third value, the first identifier needs to be encoded into the code stream. In this case, the first identifiers of all patches in the current point cloud frame can be packaged into the code stream. Of course, they can also be packaged into the code stream separately.

It should be noted that the same content in the embodiment of the present application and the embodiment shown in FIG. 9 will not be repeated, and reference may be made to the description in the embodiment shown in FIG. 9.

In the embodiment of the present application, first obtain the auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded. Since the two-dimensional information, rotation transformation information and normal coordinate axis information and three-dimensional information among the first-type patches in the current point cloud group can be the same, when the first identifier of the patch to be encoded is the first value, it indicates that the The coded patch is the first type of patch. At this time, the first identifier of the patch to be coded can be encoded into the code stream, instead of coding the auxiliary information of the patch to be coded into the code stream, so that the format of the code stream can be simplified. In addition, the value of the first identifier may be the first value, the second value, and the third value. Different values indicate different types of patches. Therefore, different types of patches can be distinguished by the first identifier, and for different types of patches, the content of the auxiliary information compiled into the code stream may be different, and for the first type of patch and the second type of patch , Only part of the auxiliary information is compiled into the code stream. In this way, the format of the code stream can be simplified, and the bit overhead of the code stream can be further reduced, thereby improving the coding efficiency.

Next, the point cloud decoding method corresponding to the point cloud encoding method based on the embodiment shown in FIG. 14 will be described. It should be noted that, in combination with the point cloud decoding system shown in FIG. 1 and the schematic block diagram of the decoder 200 shown in FIG. 6, any of the following point cloud decoding methods may be the point cloud decoding system. What is executed by the decoder 200, more specifically, may be executed by the auxiliary information decoding module 204 in the decoder 200. Referring to Figure 15, the decoding method includes:

S1501: Parse the first identifier of the patch to be decoded from the code stream.

Because the first identifiers of all patches in the current point cloud frame can be individually packaged and compiled into a code stream, or they can be packaged together and compiled into a code stream. Therefore, for the respective methods of packaging and encoding into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identifier of the patch to be decoded can be parsed from the code stream. Of course, for the non-first patch, the first identifier of the patch to be decoded can also be parsed from the code stream. That is, the first identifier of each patch to be decoded in the current point cloud frame can be analyzed sequentially. For the method of packing together and compiling into the code stream, if the patch to be decoded is the first patch in the current point cloud frame, then the first identification of multiple patches included in the current point cloud frame can be parsed from the code stream , Wherein the multiple first identifiers parsed include the first identifier of the patch to be decoded. That is, the first identifiers of multiple patches included in the current point cloud frame can be obtained through one analysis, without the need for multiple analysis.

As an example, a method of matching with the encoding end may be used to parse the first identifier of the patch to be decoded from the code stream. For example, a fixed-length decoding method can be used to parse the first identifier of the patch to be decoded from the code stream.

S1502: When the first identifier is the first value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded. The auxiliary information of the patch to be decoded includes 2D information, 3D information, rotation transformation information, and method. Axis information.

Because the first type of patch does not encode auxiliary information into the code stream during the encoding process, when the first identifier is the first value, that is, when the patch to be decoded is the first type of patch, the reference patch of the patch to be decoded can be Auxiliary information to obtain 2D information, 3D information, rotation transformation information and normal coordinate axis information of the patch to be decoded.

As an example, the identification information of the reference frame of the current point cloud frame can be parsed from the code stream, so as to determine the reference frame of the current point cloud frame according to the identification information of the reference frame, and then from the reference frame of the current point cloud frame Obtain the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch of the patch to be decoded, and use the 2D information, 3D information, rotation transformation information and normal coordinate axis information of the reference patch as the information to be decoded Patch's two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. Among them, the reference frame of the current point cloud frame may also have a corresponding reference frame, that is, the reference patch in the reference frame of the current point cloud frame also has a corresponding reference patch. For ease of description, the reference frame of the current point cloud frame is recorded Is the first reference frame, and the reference frame corresponding to the reference frame of the current point cloud frame is marked as the second reference frame. In this way, the auxiliary information of the reference patch in the first reference frame can be based on the corresponding reference in the second reference frame The auxiliary information of the patch is determined. In this way, the auxiliary information of the patch to be decoded can be obtained step by step according to the reference frame of the current point cloud frame, the reference frame corresponding to the reference frame of the current point cloud frame, etc. In addition, for the two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded, the position of the patch whose position is the same as that of the patch to be decoded in the current point cloud frame can also be obtained from the decoded point cloud. Two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information, and the obtained two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information are used as the two-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information.

When the patch in each point cloud frame in the current point cloud group is encoded according to the arrangement shown in Figure 8, the reference patch of the patch to be decoded can be the position in the reference frame of the current point cloud frame and the patch to be decoded at the current point Patches with the same position in the cloud frame.

S1503: When the first identifier is the second value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be decoded.

It should be noted that, for the content of S1503, reference may be made to S1105 in the embodiment shown in FIG. 11, which will not be repeated in this embodiment of the application.

Among them, because the auxiliary information of the second type of patch that is incorporated into the code stream during the encoding process includes two-dimensional information, three-dimensional information, and identification information of the reference patch, but does not include rotation transformation information and normal coordinate axis information, so when When the decoded patch is the second type of patch, the auxiliary information parsed from the code stream includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the identification information of the reference patch of the patch to be decoded, but does not include the rotation transformation information and information of the patch to be decoded. Normal axis information. At this time, the rotation transformation information and normal coordinate axis information of the patch to be decoded can also be obtained according to the identification information of the reference patch of the patch to be decoded.

S1504: When the first identifier is the third value, parse the auxiliary information of the patch to be decoded from the code stream. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

It should be noted that, for the content of S1504, reference may be made to S1106 in the embodiment shown in FIG. 11, which is not repeated in this embodiment of the application.

S1505: According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded.

It should be noted that, for the content of S1505, reference may be made to S1107 in the embodiment shown in FIG. 11, which is not repeated in this embodiment of the application.

In this embodiment of the application, the first identifier of the point cloud block patch to be decoded is first parsed from the code stream. When the first identifier is the first value, it indicates that the patch to be decoded is the first type of patch, and the code stream does not include the Decode the auxiliary information of the patch. At this time, the auxiliary information of the patch to be decoded can be determined according to the auxiliary information of the reference patch of the patch to be decoded. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information. When the first identifier is the second value, it indicates that the patch to be decoded is a second type of patch. At this time, the auxiliary information of the patch to be decoded can be parsed from the code stream. The auxiliary information of the patch to be decoded includes two-dimensional information and three-dimensional information. Information and identification information of the reference patch of the patch to be decoded. When the first identifier is the third value, it indicates that the patch to be decoded is of the third type. At this time, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and rotation. Transformation information and normal axis information. That is, for the first type of patch, only the first identifier needs to be parsed from the code stream, and other auxiliary information can be derived or obtained. For the second type of patch, only part of the auxiliary information needs to be parsed from the code stream. Other auxiliary information can also be derived or obtained. In this way, the format of the parsing code stream can be simplified, the bit overhead of the code stream can be reduced, and the decoding efficiency can be improved.

Refer to FIG. 16, which is a schematic structural diagram of a first point cloud encoding device provided by an embodiment of the application. As shown in FIG. 16, the encoding apparatus 1600 may include:

The patch information obtaining module 1601 is used to obtain auxiliary information of the point cloud block to be coded and the first identifier of the patch to be coded;

The auxiliary information encoding module 1602 is used to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the first value, and the patch to be encoded is the first patch in the current point cloud frame; or

The auxiliary information encoding module 1602 is used to encode the auxiliary information of the patch to be encoded into the code stream when the first identifier is the first value, and the patch to be encoded is the non-first patch in the current point cloud frame;

The first identifier is a first value to indicate that the patch to be encoded is a first type of patch, and the auxiliary information of the patch to be encoded includes three-dimensional information.

Among them, the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.

Among them, the auxiliary information encoding module 1602 is also used for:

When the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into the code stream;

The first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;

The first identifier is a third value to indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

Among them, the auxiliary information encoding module 1602 is also used for:

The first identifiers of the multiple patches included in the current point cloud frame are encoded into the code stream, the first identifiers of the multiple patches include the first identifiers of the patch to be encoded, and the first identifiers of the multiple patches are excluding the patch to be encoded The remaining first identifier besides the first identifier is the second value or the third value.

Among them, the first value, the second value and the third value are binary 10, 11 and 00 respectively,

Or, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.

The first identifier of the patch to be encoded is the third value, and the patch to be encoded is the first patch in the current point cloud frame.

Wherein, the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame.

Among them, the first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the third type of patch is a non-matching patch in the current group.

It should be noted that the patch information acquisition module 1601 is used to execute the related content of step S901 of the embodiment shown in FIG. 9, and the auxiliary information encoding module 1602 is used to execute the related content of step S902 and step S903 of the embodiment shown in FIG. 9. As an example, the patch information acquisition module 1601 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2, and the auxiliary information encoding module 1602 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words, The function of the patch information acquisition module 1601 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2, and the function of the auxiliary information generating module 1602 may be implemented by the auxiliary information encoding module 109 in FIG. 2. In one example, the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded, and the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc. Of course, this application is not limited to this. For other details, reference can be made to the prior art or the above explanation of the principle of the encoder shown in FIG. 2, which will not be repeated here.

Referring to FIG. 17, FIG. 17 is a schematic structural diagram of the first point cloud decoding apparatus provided by an embodiment of the application. As shown in FIG. 17, the decoding apparatus 1700 may include:

The auxiliary information decoding module 1701 is used to parse the first identifier of the patch to be decoded from the code stream when the point cloud block patch to be decoded is the first patch in the current point cloud frame;

The auxiliary information decoding module 1701 is further configured to parse auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded;

The auxiliary information decoding module 1701 is also used for when the patch to be decoded is the nth patch in the current point cloud frame, where n is greater than 1 and less than or equal to the number of the first-type patch in the reference point cloud frame of the current point cloud frame Analyze the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded;

The reconstruction module 1702 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

Among them, the current point cloud frame is the Kth point cloud frame in the current group, and K is an integer greater than or equal to 3.

Among them, the auxiliary information decoding module 1701 is also used for:

When the patch to be decoded is the nth patch in the current point cloud frame, and n is greater than 1 and greater than the number of the first type of patch in the reference point cloud frame of the current point cloud frame, parse the patch to be decoded from the code stream First logo

When the first identifier is the second value, parse the auxiliary information of the patch to be decoded from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and identification information of the reference patch of the patch to be decoded;

When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded.

Among them, the auxiliary information decoding module 1701 is also used for:

When the patch to be decoded is the first patch in the current point cloud frame, the first identifiers of multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers parsed include the patch to be decoded A first identifier, and the remaining first identifiers among the plurality of first identifiers except the first identifier of the patch to be decoded are the second value or the third value.

Among them, the first value, the second value and the third value are binary 10, 11 and 00 respectively,

Or, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively.

Wherein, when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is the third value, the current point cloud frame is the first point cloud frame.

Wherein, when the first identifier is the first value, the auxiliary information decoding module 1701 is further configured to:

Obtain the two-dimensional information, rotation transformation information and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded;

The reconstruction module 1702 is also used to:

According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.

Wherein, when the first identifier is the second value, the auxiliary information decoding module 1701 is further configured to:

Obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;

The reconstruction module 1702 is also used to:

According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.

The reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame.

It should be noted that the auxiliary information decoding module 1701 is used to execute the related content of steps S1101-S1106 in the embodiment shown in FIG. 11, and the reconstruction module 1702 is used to execute the related content of step S1107 in the embodiment shown in FIG. As an example, the auxiliary information decoding module 1701 in FIG. 17 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1702 in FIG. 17 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5. In other words The function of the auxiliary information decoding module 1701 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1702 may be implemented by the geometric information reconstruction module 206 of the point cloud in FIG. 6. Of course, this application is not limited to this. For other details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in FIG. 6, which will not be repeated here.

Referring to FIG. 18, FIG. 18 is a schematic structural diagram of the first point cloud encoding device provided by an embodiment of the application. As shown in FIG. 18, the encoding device 1800 may include:

The patch information obtaining module 1801 is used to obtain auxiliary information of the point cloud block to be coded and the first identifier of the patch to be coded;

The auxiliary information encoding module 1802 is used to encode the auxiliary information of the patch to be encoded, the first identification, and the second identification of the patch to be encoded into the code stream when the first identifier is the first value, and the patch to be encoded is the current point cloud frame The first patch in; or

The auxiliary information encoding module 1802 is used to encode the auxiliary information of the patch to be encoded and the second identification of the patch to be encoded into the code stream when the first identifier is the first value. The patch to be encoded is the non-first identifier in the current point cloud frame. Patch;

The first identifier is the first value to indicate that the patch to be encoded is the first type of patch;

The second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream;

Alternatively, the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded has been encoded into the code stream, and the auxiliary information of the patch to be encoded includes three-dimensional information.

It should be noted that the patch information acquisition module 1801 is used to execute the related content of step S1201 in the embodiment shown in FIG. 12, and the auxiliary information encoding module 1802 is used to execute the related content of step S1202 in the embodiment shown in FIG. As an example, the patch information acquisition module 1801 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2, and the auxiliary information encoding module 1802 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words, The function of the patch information acquisition module 1801 may be implemented by the point cloud block information generating module 101 and the packing module 102 in FIG. 2, and the function of the auxiliary information generating module 1802 may be implemented by the auxiliary information encoding module 109 in FIG. 2. In an example, the point cloud block information generation module 101 is used to obtain three-dimensional information, normal coordinate axis information, first identification and second identification of the patch to be encoded, and the packaging module 102 is used to obtain rotation transformation information of the patch to be encoded , Refer to patch identification information and two-dimensional information, etc. Of course, this application is not limited to this. For other details, reference can be made to the prior art or the above explanation of the principle of the encoder shown in FIG. 2, which will not be repeated here.

Refer to FIG. 19, which is a schematic structural diagram of a first point cloud decoding apparatus provided by an embodiment of this application. As shown in FIG. 19, the decoding apparatus 1900 may include:

The auxiliary information decoding module 1901 is used to parse the first identifier of the patch to be decoded from the code stream when the point cloud block patch to be decoded is the first patch in the current point cloud frame;

The auxiliary information decoding module 1901 is further configured to parse the second identifier of the patch to be decoded from the code stream when the first identifier is the first value;

The auxiliary information decoding module 1901 is further configured to determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is the fourth value; when the second identifier is the fifth value, from the code stream Parse the auxiliary information of the patch to be decoded, and the auxiliary information includes the three-dimensional information of the patch to be decoded;

The auxiliary information decoding module 1901 is also used for when the patch to be decoded is the n-th patch in the current point cloud frame, n is greater than 1 and less than or equal to the number of the first-type patch in the reference point cloud frame of the current point cloud frame Parse the second identifier of the patch to be decoded from the code stream;

The auxiliary information decoding module 1901 is further configured to determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded when the second identifier is the fourth value; when the second identifier is the fifth value, from the code stream Parse the auxiliary information of the patch to be decoded, and the auxiliary information includes the three-dimensional information of the patch to be decoded;

The reconstruction module 1902 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

It should be noted that the auxiliary information decoding module 1901 is used to execute the related content of steps S1301-S1305 in the embodiment shown in FIG. 13, and the reconstruction module 1902 is used to execute the related content of step S1306 in the embodiment shown in FIG. 13. As an example, the auxiliary information decoding module 1901 in FIG. 19 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 1902 in FIG. 19 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5. In other words The function of the auxiliary information decoding module 1901 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 1902 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6. Of course, this application is not limited to this. For other details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in FIG. 6, which will not be repeated here.

Referring to FIG. 20, FIG. 20 is a schematic structural diagram of the first point cloud encoding device provided by an embodiment of the application. As shown in FIG. 20, the encoding device 2000 may include:

The patch information acquisition module 2001 is used to acquire auxiliary information of the point cloud block patch to be encoded and the first identifier of the patch to be encoded;

The auxiliary information encoding module 2002 is configured to encode the first identifier of the patch to be encoded into the code stream when the first identifier is the first value;

The auxiliary information encoding module 2002 is further configured to encode the auxiliary information of the patch to be encoded and the first identifier into the code stream when the first identifier is the second value or the third value;

The first identifier is the first value to indicate that the patch to be encoded is the first type of patch;

The first identifier is the second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and identification information of the reference patch of the patch to be encoded;

The first identifier is a third value to indicate that the patch to be encoded is a third type of patch. The auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information.

It should be noted that the patch information acquisition module 2001 is used to execute the related content of step S1401 in the embodiment shown in FIG. 14, and the auxiliary information encoding module 2002 is used to execute the related content of step S1402 and step S1403 in the embodiment shown in FIG. 14. As an example, the patch information acquisition module 2001 may correspond to the combination of the point cloud block information generation module 101 and the packing module 102 in FIG. 2, and the auxiliary information encoding module 2002 may correspond to the auxiliary information encoding module 109 in FIG. 2, in other words, The function of the patch information acquisition module 2001 can be implemented by the point cloud block information generation module 101 and the packing module 102 in FIG. 2, and the function of the auxiliary information generation module 2002 can be implemented by the auxiliary information encoding module 109 in FIG. 2. In one example, the point cloud block information generation module 101 is used to obtain the three-dimensional information, normal coordinate axis information, and first identification of the patch to be encoded, and the packaging module 102 is used to obtain the rotation transformation information of the patch to be encoded and the reference patch Identification information and two-dimensional information, etc. Of course, this application is not limited to this. For other details, reference can be made to the prior art or the above explanation of the principle of the encoder shown in FIG. 2, which will not be repeated here.

Referring to FIG. 21, FIG. 21 is a schematic structural diagram of a first point cloud decoding apparatus provided by an embodiment of this application. As shown in FIG. 21, the decoding apparatus 2100 may include:

The auxiliary information decoding module 2101 is used to parse the first identifier of the point cloud block patch to be decoded from the code stream;

The auxiliary information decoding module 2101 is further configured to determine auxiliary information of the patch to be decoded according to auxiliary information of the reference patch of the patch to be decoded when the first identifier is the first value. The auxiliary information of the patch to be decoded includes two-dimensional information and three-dimensional information , Rotation transformation information and normal coordinate axis information;

The auxiliary information decoding module 2101 is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the second value. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and references to the patch to be decoded. Patch identification information;

The auxiliary information decoding module 2101 is also used to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the third value. The auxiliary information of the patch to be decoded includes 2D information, 3D information, rotation transformation information, and method. To coordinate axis information;

The reconstruction module 2102 is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded.

Wherein, when the first identifier is the second value, the auxiliary information decoding module 2101 is further configured to:

Obtain the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded;

The reconstruction module 2102 is also used to:

According to the two-dimensional information, three-dimensional information, normal coordinate axis information and rotation transformation information of the patch to be decoded, the patch to be decoded is reconstructed.

It should be noted that the auxiliary information decoding module 2101 is used to execute the related content of steps S1501-S1504 in the embodiment shown in FIG. 15, and the reconstruction module 2102 is used to execute the related content of step S1506 in the embodiment shown in FIG. 15. As an example, the auxiliary information decoding module 2101 in FIG. 21 corresponds to the auxiliary information decoding module 204 in FIG. 6, and the reconstruction module 2102 in FIG. 21 corresponds to the geometric information reconstruction module 206 of the point cloud in FIG. 5. In other words The function of the auxiliary information decoding module 2101 may be implemented by the auxiliary information decoding module 204 in FIG. 6, and the function of the reconstruction module 2102 may be implemented by the point cloud geometric information reconstruction module 206 in FIG. 6. Of course, this application is not limited to this. For other details, reference may be made to the prior art or the above explanation of the principle of the decoder shown in FIG. 6, which will not be repeated here.

FIG. 22 is a schematic block diagram of an implementation manner of a decoding device 2200 used in an embodiment of the present application. The acquiring apparatus 2200 may include a processor 2201, a memory 2202, and a bus system 2203. The processor 2201 and the memory 2202 are connected through a bus system 2203. The memory 2202 is used to store instructions. The processor 2201 is used to execute the instructions stored in the memory 2202 to execute various point cloud encoding or decoding methods described in this application. , Especially the method of filtering the current image block based on the block size of the current image block. To avoid repetition, it will not be described in detail here.

In this embodiment of the application, the processor 2201 may be a central processing unit (CPU), and the processor 2201 may also be other general-purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gates. Or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 2202 may include a ROM device or a RAM device. Any other suitable type of storage device can also be used as the memory 2202. The memory 2202 may include code and data 22021 accessed by the processor 2201 using the bus 2203. The memory 2202 may further include an operating system 22023 and an application program 22022. The application program 22022 includes at least one of the point cloud encoding or decoding methods described in this application (especially the point cloud encoding and decoding methods described in this application) that allows the processor 2201 to execute program. For example, the application program 22022 may include applications 1 to N, which further include a point cloud encoding or decoding application (referred to as a point cloud decoding application) that executes the point cloud encoding or decoding method described in this application.

In addition to the data bus, the bus system 2203 may also include a power bus, a control bus, and a status signal bus. However, for clear description, various buses are marked as the bus system 2203 in the figure.

Optionally, the encoding apparatus 2200 may further include one or more output devices, such as a display 2204. In one example, the display 2204 may be a touch-sensitive display that merges the display with a touch-sensitive unit operable to sense touch input. The display 2204 may be connected to the processor 2201 via a bus 2203.

It should be pointed out that the decoding device 2200 can execute the point cloud encoding method in this application, and can also execute the point cloud decoding method in this application.

Those skilled in the art can understand that the functions described in conjunction with the various illustrative logical blocks, modules, and algorithm steps disclosed herein can be implemented by hardware, software, firmware, or any combination thereof. If implemented in software, the functions described by various illustrative logical blocks, modules, and steps can be stored or transmitted as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. The computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium that includes any medium that facilitates the transfer of a computer program from one place to another (for example, according to a communication protocol) . In this manner, computer-readable media may generally correspond to (1) non-transitory tangible computer-readable storage media, or (2) communication media, such as signals or carrier waves. Data storage media can be any available media that can be accessed by one or more computers or one or more processors to retrieve instructions, codes, and/or data structures for implementing the techniques described in this application. The computer program product may include a computer-readable medium.

By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, flash memory, or structures that can be used to store instructions or data Any other media that can be accessed by the computer in the form of desired program code. And, any connection is properly termed a computer-readable medium. For example, if you use coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave to transmit instructions from a website, server, or other remote source, then the coaxial cable Wire, fiber optic cable, twisted pair, DSL or wireless technologies such as infrared, radio and microwave are included in the definition of media. However, it should be understood that the computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other temporary media, but are actually directed to non-transitory tangible storage media. As used herein, magnetic disks and optical disks include compact disks (CDs), laser disks, optical disks, DVDs, and Blu-ray disks, where disks usually reproduce data magnetically, and optical disks use lasers to reproduce data optically. Combinations of the above should also be included in the scope of computer-readable media.

It can be processed by one or more digital signal processors (DSP), general-purpose microprocessors, application-specific integrated circuits (ASIC), field programmable logic arrays (FPGA), or other equivalent integrated or discrete logic circuits, for example To execute instructions. Therefore, the term "processor" as used herein may refer to any of the foregoing structure or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functions described by the various illustrative logical blocks, modules, and steps described herein may be provided in dedicated hardware and/or software modules configured for encoding and decoding, or combined Into the combined codec. Moreover, the technology may be fully implemented in one or more circuits or logic elements. In an example, various illustrative logical blocks, units, and modules in the encoder 100 and the decoder 200 can be understood as corresponding circuit devices or logical elements.

The technology of this application can be implemented in a variety of devices or devices, including wireless handsets, integrated circuits (ICs), or a set of ICs (for example, chipsets). Various components, modules, or units are described in this application to emphasize the functional aspects of the device for performing the disclosed technology, but they do not necessarily need to be implemented by different hardware units. In fact, as described above, various units can be combined with appropriate software and/or firmware in the codec hardware unit, or by interoperating hardware units (including one or more processors as described above). provide.

The above are only exemplary specific implementations of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or changes within the technical scope disclosed in this application. Replacement shall be covered within the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims (46)

A point cloud coding method, characterized in that the method includes: Acquiring auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded; When the first identifier is the first value, encode the auxiliary information of the patch to be encoded and the first identifier into the code stream, where the patch to be encoded is the first patch in the current point cloud frame; or When the first identifier is the first value, encode the auxiliary information of the patch to be encoded into a code stream, where the patch to be encoded is the non-first patch in the current point cloud frame; The first identifier is a first value to indicate that the patch to be encoded is a first-type patch, and the auxiliary information of the patch to be encoded includes three-dimensional information. The method according to claim 1, wherein the current point cloud frame is the Kth point cloud frame in the current group, and the K is an integer greater than or equal to 3. The method according to claim 1 or 2, wherein after said obtaining the auxiliary information of the point cloud block patch to be coded and the first identifier of the patch to be coded, the method further comprises: When the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into a code stream; The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information; The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information. 5. The method according to any one of claims 1 to 3, wherein when the first identifier is a first value, encoding the first identifier into a code stream comprises: The first identifiers of the multiple patches included in the current point cloud frame are encoded into the code stream, the first identifiers of the multiple patches include the first identifiers of the patch to be encoded, and the first identifiers of the multiple patches The remaining first identifier in the identifier except for the first identifier of the patch to be encoded is the second value or the third value. The method according to claim 3 or 4, wherein the first value, the second value and the third value are binary 10, 11 and 00, respectively, Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively. The method according to any one of claims 3-5, wherein the first identifier of the patch to be encoded is a third value, and the patch to be encoded is the first patch in the current point cloud frame. The method according to any one of claims 3-6, wherein the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame. The method according to any one of claims 3-7, wherein the first type of patch is a global matching patch in the current group, and the second type of patch is a local matching patch in the current group, and the The third type of patch is the non-matching patch in the current group. A point cloud decoding method, characterized in that the method includes: When the point cloud block patch to be decoded is the first patch in the current point cloud frame, parse the first identifier of the patch to be decoded from the code stream; When the first identifier is the first value, parse the auxiliary information of the patch to be decoded from the code stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded; When the patch to be decoded is the n-th patch in the current point cloud frame, and the n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame Parse the auxiliary information of the patch to be decoded from the code stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded; According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded. The method according to claim 9, wherein the current point cloud frame is the Kth point cloud frame in the current group, and the K is an integer greater than or equal to 3. The method according to claim 9 or 10, wherein the method further comprises: When the patch to be decoded is the n-th patch in the current point cloud frame, and the n is greater than 1 and greater than the number of first-type patches in the reference point cloud frame of the current point cloud frame, from Parsing the first identifier of the patch to be decoded in the code stream; When the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded. Reference patch identification information for patch; When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information. The method according to any one of claims 9-11, wherein when the point cloud block patch to be decoded is the first patch in the current point cloud frame, parsing the patch to be decoded from the code stream The first identification includes: When the patch to be decoded is the first patch in the current point cloud frame, the first identifiers of the multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers are parsed Includes the first identifier of the patch to be decoded, and the remaining first identifiers among the plurality of first identifiers except the first identifier of the patch to be decoded are the second value or the third value. The method according to claim 11 or 12, wherein the first value, the second value and the third value are binary 10, 11 and 00, respectively, Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively. The method according to any one of claims 11-13, wherein when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is a third value, the The current point cloud frame is the first point cloud frame. The method according to any one of claims 9-14, wherein when the first identifier is a first value, the method further comprises: Acquiring the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; According to the auxiliary information of the patch to be decoded, reconstructing the patch to be decoded includes: The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded. The method according to any one of claims 11-14, wherein when the first identifier is a second value, the method further comprises: Obtaining the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; According to the auxiliary information of the patch to be decoded, reconstructing the patch to be decoded includes: The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded. The method according to any one of claims 11-16, wherein the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame. A point cloud coding method, characterized in that the method includes: Acquiring auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded; When the first identifier is the first value, the auxiliary information of the patch to be encoded, the first identifier, and the second identifier of the patch to be encoded are encoded into the code stream, and the patch to be encoded is the current point The first patch in the cloud frame; or When the first identifier is the first value, the auxiliary information of the patch to be encoded and the second identifier of the patch to be encoded are encoded into the code stream, and the patch to be encoded is the current point cloud frame Not the first patch; The first identifier is a first value to indicate that the patch to be encoded is a first-type patch; The second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; Alternatively, the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded has been incorporated into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information. A point cloud decoding method, characterized in that the method includes: When the point cloud block patch to be decoded is the first patch in the current point cloud frame, parse the first identifier of the patch to be decoded from the code stream; When the first identifier is the first value, parse the second identifier of the patch to be decoded from the code stream; When the second identifier is the fourth value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; when the second identifier is the fifth value, from the code Parse the auxiliary information of the patch to be decoded in the stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded; When the patch to be decoded is the n-th patch in the current point cloud frame, and the n is greater than 1 and less than or equal to the number of first-type patches in the reference point cloud frame of the current point cloud frame Parse the second identifier of the patch to be decoded from the code stream; When the second identifier is the fourth value, determine the auxiliary information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; when the second identifier is the fifth value, from the code Parse the auxiliary information of the patch to be decoded in the stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded; According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded. A point cloud coding method, characterized in that the method includes: Acquiring auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded; When the first identifier is the first value, encode the first identifier of the patch to be encoded into the code stream; When the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into a code stream; The first identifier is a first value to indicate that the patch to be encoded is a first-type patch; The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information; The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information. A point cloud decoding method, characterized in that the method includes: Parse the first identifier of the point cloud block patch to be decoded from the code stream; When the first identifier is the first value, the auxiliary information of the patch to be decoded is determined according to the auxiliary information of the reference patch of the patch to be decoded, and the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, Rotation transformation information and normal coordinate axis information; When the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, and information of the patch to be decoded. Refer to the identification information of the patch; When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream. The auxiliary information of the patch to be decoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal direction. Coordinate axis information; According to the auxiliary information of the patch to be decoded, reconstruct the patch to be decoded. The method of claim 21, wherein when the first identifier is a second value, the method further comprises: Obtaining the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; According to the auxiliary information of the patch to be decoded, reconstructing the patch to be decoded includes: The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded. A point cloud coding device, characterized in that the device comprises: A point cloud block patch information acquisition module, configured to acquire auxiliary information of the patch to be encoded and the first identifier of the patch to be encoded; The auxiliary information encoding module is configured to encode the auxiliary information of the patch to be encoded and the first identification into a code stream when the first identifier is the first value, and the patch to be encoded is in the current point cloud frame The first patch; or The auxiliary information encoding module is configured to encode the auxiliary information of the patch to be encoded into a code stream when the first identifier is a first value, and the patch to be encoded is the non-information in the current point cloud frame. The first patch; The first identifier is a first value to indicate that the patch to be encoded is a first-type patch, and the auxiliary information of the patch to be encoded includes three-dimensional information. The apparatus according to claim 23, wherein the current point cloud frame is the Kth point cloud frame in the current group, and the K is an integer greater than or equal to 3. The device according to claim 23 or 24, wherein the auxiliary information encoding module is further configured to: When the first identifier is the second value or the third value, encode the auxiliary information of the patch to be encoded and the first identifier into a code stream; The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information; The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information. The device according to any one of claims 23-25, wherein the auxiliary information encoding module is further configured to: The first identifiers of the multiple patches included in the current point cloud frame are encoded into the code stream, the first identifiers of the multiple patches include the first identifiers of the patch to be encoded, and the first identifiers of the multiple patches The remaining first identifier in the identifier except for the first identifier of the patch to be encoded is the second value or the third value. The device according to claim 25 or 26, wherein the first value, the second value and the third value are binary 10, 11 and 00 respectively, Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively. The device according to any one of claims 25-27, wherein the first identifier of the patch to be encoded is a third value, and the patch to be encoded is the first patch in the current point cloud frame. The device according to any one of claims 25-28, wherein the reference patch of the patch to be encoded is included in the reference frame of the current point cloud frame. The device according to any one of claims 25-29, wherein the first type of patch is a global matching patch in the current group, the second type of patch is a local matching patch in the current group, and the The third type of patch is the non-matching patch in the current group. A point cloud decoding device, characterized in that the device includes: The auxiliary information decoding module is used to analyze the first identifier of the patch to be decoded from the code stream when the point cloud block patch to be decoded is the first patch in the current point cloud frame; The auxiliary information decoding module is further configured to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is the first value, and the auxiliary information includes the three-dimensional information of the patch to be decoded. information; The auxiliary information decoding module is further configured to: when the patch to be decoded is the n-th patch in the current point cloud frame, and the n is greater than 1 and less than or equal to the reference point cloud frame of the current point cloud frame Parse the auxiliary information of the patch to be decoded from the code stream when the number of the first-type patch in the code stream, and the auxiliary information includes the three-dimensional information of the patch to be decoded; The reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded. The apparatus according to claim 31, wherein the current point cloud frame is the Kth point cloud frame in the current group, and the K is an integer greater than or equal to 3. The device according to claim 31 or 32, wherein the auxiliary information decoding module is further configured to: When the patch to be decoded is the n-th patch in the current point cloud frame, and the n is greater than 1 and greater than the number of first-type patches in the reference point cloud frame of the current point cloud frame, from Parsing the first identifier of the patch to be decoded in the code stream; When the first identifier is the second value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information of the patch to be decoded, and the patch to be decoded. Reference patch identification information for patch; When the first identifier is the third value, the auxiliary information of the patch to be decoded is parsed from the code stream, and the auxiliary information includes the two-dimensional information, three-dimensional information, rotation transformation information and information of the patch to be decoded. Normal axis information. The apparatus according to any one of claims 31-33, wherein the auxiliary information decoding module is further configured to: When the patch to be decoded is the first patch in the current point cloud frame, the first identifiers of the multiple patches included in the current point cloud frame are parsed from the code stream, and the multiple first identifiers are parsed Includes the first identifier of the patch to be decoded, and the remaining first identifiers among the plurality of first identifiers except the first identifier of the patch to be decoded are the second value or the third value. The device according to claim 33 or 34, wherein the first value, the second value and the third value are binary 10, 11 and 00, respectively, Alternatively, the first value, the second value, and the third value are 11, 10, and 00 in binary, respectively. The device according to any one of claims 33-35, wherein when the patch to be decoded is the first patch in the current point cloud frame, and the first identifier is a third value, the The current point cloud frame is the first point cloud frame. The apparatus according to any one of claims 31-36, wherein when the first identifier is a first value, the auxiliary information decoding module is further configured to: Acquiring the two-dimensional information, rotation transformation information, and normal coordinate axis information of the patch to be decoded according to the auxiliary information of the reference patch of the patch to be decoded; The reconstruction module is also used for: The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded. The apparatus according to any one of claims 33-36, wherein when the first identifier is a second value, the auxiliary information decoding module is further configured to: Obtaining the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; The reconstruction module is also used for: The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded. The device according to any one of claims 33-38, wherein the reference patch of the patch to be decoded is included in the reference frame of the current point cloud frame. A point cloud coding device, characterized in that the device comprises: A point cloud block patch information acquisition module, configured to acquire auxiliary information of the point cloud block patch to be coded and the first identifier of the patch to be coded; The auxiliary information encoding module is used to encode the auxiliary information of the patch to be encoded, the first identification and the second identification of the patch to be encoded into the code stream when the first identifier is the first value, so The patch to be encoded is the first patch in the current point cloud frame; or The auxiliary information encoding module is configured to encode the auxiliary information of the patch to be encoded and the second identification of the patch to be encoded into a code stream when the first identifier is a first value, and the patch to be encoded Is the non-first patch in the current point cloud frame; The first identifier is a first value to indicate that the patch to be encoded is a first-type patch; The second identifier is a fourth value to indicate that the three-dimensional information of the patch to be encoded is not encoded into the code stream; Alternatively, the second identifier is a fifth value to indicate that the three-dimensional information of the patch to be encoded has been incorporated into the code stream, and the auxiliary information of the patch to be encoded includes the three-dimensional information. A point cloud decoding device, characterized in that the device includes: The auxiliary information decoding module is used to analyze the first identifier of the patch to be decoded from the code stream when the point cloud block patch to be decoded is the first patch in the current point cloud frame; The auxiliary information decoding module is further configured to parse the second identifier of the patch to be decoded from the code stream when the first identifier is a first value; The auxiliary information decoding module is further configured to determine the auxiliary information of the patch to be decoded according to auxiliary information of the reference patch of the patch to be decoded when the second identifier is a fourth value; when the second identifier When it is the fifth value, parse the auxiliary information of the patch to be decoded from the code stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded; The auxiliary information decoding module is further configured to: when the patch to be decoded is the n-th patch in the current point cloud frame, and the n is greater than 1 and less than or equal to the reference point cloud frame of the current point cloud frame Parse the second identifier of the patch to be decoded from the code stream when the number of the first-type patch in The auxiliary information decoding module is further configured to determine the auxiliary information of the patch to be decoded according to auxiliary information of the reference patch of the patch to be decoded when the second identifier is a fourth value; when the second identifier When it is the fifth value, parse the auxiliary information of the patch to be decoded from the code stream, where the auxiliary information includes the three-dimensional information of the patch to be decoded; The reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded. A point cloud coding device, characterized in that the device comprises: A point cloud block patch information acquisition module, configured to acquire auxiliary information of the point cloud block patch to be coded and the first identifier of the patch to be coded; An auxiliary information encoding module, configured to encode the first identifier of the patch to be encoded into a code stream when the first identifier is a first value; The auxiliary information encoding module is further configured to encode the auxiliary information of the patch to be encoded and the first identifier into a code stream when the first identifier is a second value or a third value; The first identifier is a first value to indicate that the patch to be encoded is a first-type patch; The first identifier is a second value to indicate that the patch to be encoded is a second type of patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, and the identifier of the reference patch of the patch to be encoded information; The first identifier is a third value to indicate that the patch to be encoded is a third-type patch, and the auxiliary information of the patch to be encoded includes two-dimensional information, three-dimensional information, rotation transformation information, and normal coordinate axis information. A point cloud decoding device, characterized in that the device includes: The auxiliary information decoding module is used to parse the first identifier of the point cloud block patch to be decoded from the code stream; The auxiliary information decoding module is further configured to determine the auxiliary information of the patch to be decoded according to auxiliary information of the reference patch of the patch to be decoded when the first identifier is the first value, The auxiliary information includes two-dimensional information, three-dimensional information, rotation transformation information and normal coordinate axis information; The auxiliary information decoding module is further configured to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is a second value, and the auxiliary information of the patch to be decoded includes two-dimensional information , Three-dimensional information and identification information of the reference patch of the patch to be decoded; The auxiliary information decoding module is further configured to parse the auxiliary information of the patch to be decoded from the code stream when the first identifier is a third value, and the auxiliary information of the patch to be decoded includes two-dimensional information , Three-dimensional information, rotation transformation information and normal coordinate axis information; The reconstruction module is used to reconstruct the patch to be decoded according to the auxiliary information of the patch to be decoded. The apparatus of claim 43, wherein when the first identifier is a second value, the auxiliary information decoding module is further configured to: Obtaining the rotation transformation information and normal coordinate axis information of the patch to be decoded according to the identification information of the reference patch of the patch to be decoded; The reconstruction module is also used for: The patch to be decoded is reconstructed according to the two-dimensional information, three-dimensional information, normal coordinate axis information, and rotation transformation information of the patch to be decoded. A computer-readable storage medium, characterized by comprising program code, which when running on a computer, causes the computer to execute any one of claims 1 to 8, or claim 18, or claim 20 The point cloud coding method. A computer-readable storage medium, characterized by comprising program code, which when running on a computer, causes the computer to execute any one of claims 9 to 17, or claim 19, or claim 21 The point cloud decoding method described in any one of to 22.

Images