CN118803252A - Image encoding method, image decoding method, image processing device, and storage medium - Google Patents

Abstract

The embodiment of the application provides an image coding method, which comprises the following steps: acquiring a to-be-predicted coding block and a predicted coding block in an image frame, wherein a region corresponding to the predicted coding block is a reconstructed region; determining a plurality of target templates corresponding to the coding blocks to be predicted; determining a matched reference block corresponding to each target template in the reconstructed region, and fusing at least one matched reference block to obtain a predicted value corresponding to the to-be-predicted coding block; obtaining a residual value according to the original value of the coding block to be predicted and the predicted value; and coding the residual error value and writing coding bits into a code stream, so that the accuracy of an intra-frame prediction result can be improved.

Description

Image encoding method, image decoding method, image processing device, and storage medium

Technical Field

The embodiments of the present application relate to the field of image processing, and in particular to an image encoding method, an image decoding method, an image processing device, a computer-readable storage medium, and a computer program product.

Background Art

With the continuous improvement of video quality and the continuous expansion of video application scenarios, the amount of data for video transmission and video storage has also greatly increased. In order to improve the efficiency of video transmission and storage, video coding technology has been proposed. That is, various coding tools are used to remove spatial redundant information and temporal redundant information in the original video as much as possible, thereby effectively reducing the amount of video data while ensuring video quality.

In the related art, intra-frame prediction technology is used to remove the spatial correlation of video images, that is, to use the pixel information of adjacent reconstructed blocks encoded in the frame to predict the pixel value of the current coding block to be predicted. Among them, the intra-frame template matching prediction (IntraTMP, Intra Template Matching Prediction) technology uses a preset template to perform template matching in a set search area, that is, by searching for a matching template similar to the preset template, and using the corresponding block of the matching template as the prediction value of the current coding block to be predicted. However, in actual applications, the best matching reference block may not be found or multiple best matching reference blocks may be found, and the matching reference block found may not be the best matching reference block, which leads to inaccurate intra-frame prediction results. Therefore, how to improve the accuracy of intra-frame prediction results is a problem that needs to be solved urgently.

Summary of the invention

The embodiments of the present application provide an image encoding method, an image decoding method, an image processing device, a computer-readable storage medium and a computer program product, which are intended to improve the accuracy of intra-frame prediction results and the efficiency of intra-frame prediction.

In a first aspect, an embodiment of the present application provides an image coding method, comprising: obtaining a to-be-predicted coding block and a predicted coding block in an image frame, wherein an area corresponding to the predicted coding block is a reconstructed area; determining multiple target templates corresponding to the to-be-predicted coding block; determining a matching reference block corresponding to each of the target templates in the reconstructed area, and fusing at least one of the matching reference blocks to obtain a prediction value corresponding to the to-be-predicted coding block; obtaining a residual value based on the original value of the to-be-predicted coding block and the predicted value; encoding the residual value, and writing the coded bits into a bitstream.

In a second aspect, an embodiment of the present application provides an image decoding method, comprising: parsing a bit stream to obtain a residual value corresponding to a to-be-predicted decoding block; determining, in a reconstructed area, a matching reference block corresponding to each of the target templates according to a plurality of target templates, and fusing at least one of the matching reference blocks to obtain a prediction value corresponding to the to-be-predicted decoding block; wherein the area corresponding to the predicted decoding block is a reconstructed area; obtaining, according to the prediction value of the to-be-predicted decoding block and the residual value, a decoding value corresponding to the to-be-predicted decoding block; and obtaining a decoding recovery value of the to-be-predicted decoding block according to the decoding value.

In a third aspect, an embodiment of the present application provides an image processing device, comprising: at least one processor; at least one memory for storing at least one program; when at least one of the programs is executed by at least one of the processors, an image encoding method as described in the first aspect, or an image decoding method as described in the second aspect is implemented.

In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program executable by a processor, and when the program executable by the processor is executed by the processor, it is used to implement the image encoding method as described in the first aspect, or the image decoding method as described in the second aspect.

In a fifth aspect, an embodiment of the present application provides a computer program product, comprising a computer program or computer instructions, wherein the computer program or the computer instructions are stored in a computer-readable storage medium, a processor of a computer device reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions, so that the computer device performs the image encoding method as described in the first aspect, or the image decoding method as described in the second aspect.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a block-based hybrid coding framework provided by the present application;

FIG2 is a schematic diagram of the principle of the IntraTMP technology provided by the present application;

FIG3 is a schematic diagram of three templates in the IntraTMP technology provided in this application;

FIG4 is a schematic diagram of a video image transmission system architecture provided by an embodiment of the present application;

FIG5 is a flowchart of an image encoding method provided by an embodiment of the present application;

FIG6 is a flowchart of a method for determining a matching reference block corresponding to each target template provided by an embodiment of the present application;

FIG7 is a schematic diagram of the IntraTMP BV provided in an embodiment of the present application;

FIG8 is a flowchart of a method for determining a matching reference block corresponding to each target template provided by an embodiment of the present application;

FIG9 is a schematic diagram of the IntraTMP-fusion solution flow chart of the encoding end provided in the example of this application;

FIG10 is a schematic diagram of the fusion process of the multi-mode IntraTMP-fusion solution provided in the example of this application;

FIG11 is a template SAD of a location-dependent fusion solution provided in an example of this application;

FIG12 is a flowchart of an image decoding method provided by an embodiment of the present application;

FIG13 is a schematic diagram of the process flow of the IntraTMP-fusion solution provided by the encoding end of the present application example;

FIG14 is a schematic diagram of a grammatical structure provided by an example of this application;

FIG15 is a schematic diagram of a grammatical structure provided by an example of this application;

FIG16 is a schematic diagram of a grammatical structure provided by an example of this application;

FIG. 17 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application.

DETAILED DESCRIPTION

In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

In the embodiments of the present application, the words "further", "exemplarily" or "optionally" are used to indicate examples, illustrations or descriptions, and should not be interpreted as being more preferred or more advantageous than other embodiments or designs. The use of the words "further", "exemplarily" or "optionally" is intended to present related concepts in a specific way.

Today's world is a multimedia society surrounded by images and videos. For example, the fields of advertising, medical treatment, security, entertainment, and conference television that can be seen everywhere all involve media content such as images and videos. At the same time, with the improvement of video quality, the gradual shift from high-definition video to ultra-high-definition video such as 4K and 8K, the increase in various video types such as screen content and panoramic video, as well as the widespread application of network streaming and real-time communication, have greatly increased the amount of data transmitted or stored in the video, and thus put forward higher requirements for the video transmission bandwidth. In order to improve the transmission and storage efficiency of the video, it is necessary to reduce the amount of video data, so video compression technology is proposed. Video compression technology, also known as video coding technology, is a technology that uses various coding tools to remove spatial redundant information and temporal redundant information in the original video as much as possible. It can effectively reduce the amount of video data while ensuring the quality of the video. Therefore, video coding technology is one of the hot research directions in the field of multimedia technology research.

The Joint Video Expert Teams (JVET) of the Moving Pictures Experts Group (MPEG) of the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) and the Video Code Expert Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector ITU-T released the Versatile Video Coding standard (H.266/VVC), which can better support a variety of new video types such as 8K ultra-high definition, screen content, high dynamic and 360° panoramic video, as well as streaming media and real-time communication applications with adaptive bandwidth and resolution. Since then, the Joint Expert Team has continued to explore the next generation of video coding standards and launched another advanced standard (ECM, Enhanced Compression Model) after VVC.

The ECM standard uses the coding framework of H.266/VVC: a block-based hybrid coding framework. Figure 1 shows the block-based hybrid coding framework provided by this application. Referring to Figure 1, the hybrid coding framework mainly includes modules such as intra-frame prediction, inter-frame prediction, transformation, quantization, entropy coding, and loop filtering. Each module covers a variety of new coding technologies. Therefore, the ECM standard requires a large amount of memory, but compared with H.266/VVC, it further improves the coding gain.

The following is a brief introduction to the various modules of the H.266/VVC coding framework: the intra-frame prediction module mainly removes the spatial correlation of video images, and uses the pixel information of adjacent reconstructed blocks encoded within the frame to predict the pixel value of the current predicted coding block, with a low compression rate; the inter-frame prediction module mainly removes the temporal correlation of video images, and uses the inter-frame encoded image as the reference image of the current frame to obtain the motion information of each block in the current frame. One-way and two-way reference images can be used for prediction. Generally speaking, the compression rate is higher than that of intra-frame prediction coding. After the video image passes through the intra-frame/inter-frame prediction module, the prediction value of the current block to be predicted is obtained. The residual data of the image block can be obtained by subtracting the original image block from the prediction block. Transforming the residual data can effectively remove the frequency domain correlation of the image. By transforming from the time domain described in pixel form to the frequency domain, the image energy is concentrated in the low-frequency area, that is, the correlation of the transformation coefficients is small; usually the coefficients after transformation have a large dynamic range, and the quantization module is needed to reduce the dynamic range and achieve better compression; the entropy coding module encodes intra-frame prediction data, motion information, quantization coefficients, coding control and other data into a binary stream for storage or transmission, that is, the output data of the entropy coding is the compressed bitstream of the original video.

In addition, in the hybrid coding framework, in order to build a decoded image cache for inter-frame prediction as a reference image, an image reconstruction process is also implemented, which mainly includes inverse quantization, inverse transformation and loop filtering modules. The specific process is to inverse quantize and inverse transform the quantized transform coefficients to obtain residual data, and then add it to the prediction information to reconstruct the image data. Since video encoding is block-based, the reconstructed image usually has block effects, ringing effects and other phenomena. It is necessary to introduce a loop filtering module to improve the subjective quality and compression efficiency of the image and reduce decoding errors. The loop filter module of H.266/VVC mainly includes the following filtering technologies: Luma Mapping Chroma Scaling (LMCS), Deblocking Filter (DBF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF). Luma mapping and chroma scaling are to redistribute codewords for information within the dynamic range, which can improve compression efficiency; deblocking filtering is to smooth the block boundary, which can effectively reduce and remove the block effect; sample adaptive offset uses different compensation values for different categories of reconstructed pixels, which can improve the ringing effect caused by the loss of high-frequency information and effectively improve the subjective and objective quality of the reconstructed image; adaptive loop filtering is to perform Wiener filtering on the pixels of the reconstructed image, which can effectively reduce decoding errors.

The overall framework process of the encoding end is as follows:

The input video frame (image) is divided into blocks: the frame is first divided into multiple coding tree units (CTUs, Coding Tree Units), each CTU can be divided into 4 coding units (CUs, Coding Units) of the same size according to the quadtree, or can be recursively divided into CUs of different sizes according to the multi-type tree (MTT, Multiple Type Trees), that is, a binary tree or ternary tree structure. CTU contains all color channels, that is, a CTU consists of a luminance CTB (Coding Tree Block) and two chrominance CTBs; CU is the basic unit of video coding, the maximum size of the luminance CU is 128*128, the minimum size is 4*4, the maximum size of the chrominance CU is 64*64, and the minimum size is 2*2; the size of the coding unit CU finally divided is related to the content of the video image.

The intra/inter prediction mode decision determines whether the coding block is to be intra-frame prediction coded or inter-frame prediction coded. The main purpose of intra-frame prediction coding is to remove the spatial correlation of the image; the main purpose of inter-frame prediction coding is to remove the temporal correlation of the image.

The prediction block obtained by intra-frame/inter-frame coding is subtracted from the original coding block to obtain the residual data of the current coding block to be predicted. The residual data is transformed and quantized to remove the frequency domain correlation, realize lossy compression of image residual data, and further improve compression efficiency.

The quantized two-dimensional transform coefficients are Z-scanned and run-length encoded to obtain one-dimensional transform coefficients. Finally, the transform coefficients, coding control information, prediction data, motion vector data and other information are used as the input of the entropy coding module and encoded into a binary stream, that is, the compressed bit stream of the original video, for storage or transmission.

The quantized transform coefficients are dequantized and inversely transformed to obtain the residual data of the reconstructed block, which is then added to the predicted value obtained by intra-frame/inter-frame coding to obtain the reconstructed value of the coding block, thereby obtaining the reconstructed image.

The reconstructed image needs to be filtered through the loop filter module to obtain the final reconstructed image and build a decoded image cache as a reference image for inter-frame prediction.

In fact, the image reconstruction process of the hybrid coding framework is the image decoding process, except that a bitstream parsing step is added at the decoding end.

At the JVET-U meeting in January 2021, the intra-frame template matching prediction (IntraTMP) technology was first proposed and then incorporated into ECM 1.0 at the next meeting. IntraTMP is a special intra-frame prediction coding tool that was originally only applicable to screen content coding (SCC). Although it has a good compression rate gain for natural sequences (Camera Captured Contents), its complexity is too high, which seriously affects the coding timeliness. Later, the template matching search process in IntraTMP was improved, which greatly reduced the complexity and made the technology applicable to the entire sequence content.

FIG2 is a schematic diagram of the principle of the IntraTMP technology provided by the present application. As shown in FIG2, the upper area 220, the left area 230 and the upper left corner area 240 of the current block to be predicted 210 constitute an L-shape template. The encoder uses the L-shape template to perform template matching in the set search area (the reconstructed area of the current frame), searches for the template most similar to this template, that is, the absolute error sum (SAD, Sum of AbsoluteDifferences) between the current coding block template to be predicted and the reference template is the smallest, and the corresponding block 250 of the searched similar template is used as the prediction value of the current coding block to be predicted. The preset search areas are R1, R2, R3 and R4 in FIG2, R1 is inside the CTU where the current coding block to be predicted is located, R2 to R4 are all outside the current CTU, and may include multiple CTUs, that is, R2 is located in the upper left area (Top-left) outside the current CTU, R3 is located in the upper area (Above) outside the current CTU, and R4 is located in the left area (Left) outside the current CTU.

If IntraTMP is enabled in the intra-frame prediction mode of the current coding block to be predicted, the encoder needs to identify the syntax element tmp_enabled_flag of this coding tool and transmit it to the decoding end; the decoder will perform the same operation as the encoder: search for the template that is most similar to the L-shape template of the current coding block to be predicted in the predefined four areas (R1~R4), that is, the template with the smallest SAD, and use its corresponding block as the prediction of the current coding block to be predicted.

In the process of searching for the most similar L-shape template, i.e., the template matching process, the four search areas set are a rough range, and the search width (SearchRange_w) and height (SearchRange_h) of each area are also limited. If the width and height of the current block to be predicted are (BlkW, BlkH) respectively, then the search width/height is 5 times the width/height of the current block to be predicted: (a=5)

SearchRange_w＝a*BlkW

SearchRange_h＝a*BlkH

The specific search process is divided into two steps:

A rough search is performed in the four regions R1 to R4, with a search step length, i.e., a downsampling factor of 2. After the rough search is completed, the best "matching reference block" at this time, i.e., the corresponding block of the "most similar template", is obtained;

A fine search is performed around the best "matching reference block" obtained in the coarse search stage. The search range is min(BlkW, BlkH)/2, and the search step is 1, that is, an integer pixel search. The best matching reference block finally obtained is used as the prediction of the current coding block to be predicted.

At present, the templates used in the IntraTMP technology include three shapes: L-shape, Above and Left. The width of the Above template is the same as the width of the current coding block to be predicted, and the height is the same as the upper area of the L-shape template; the height of the Left template is the same as the height of the current coding block to be predicted, and the width is the same as the left area of the L-shape template. Figure 3 is a schematic diagram of three IntraTMP modes provided by this application. As shown in Figure 3, three different shapes of templates obtain three different IntraTMP modes: L-shape IntraTMP, Above IntraTMP and Left IntraTMP, while the related technology ultimately selects only one of the templates to search for the best matching reference block, that is, the current coding block to be predicted only uses one IntraTMP mode, which may result in the searched best matching reference block not being the best; in addition, for some video content, the best matching reference block may not be found, or multiple best matching reference blocks may be found, etc., and the related technology needs to directly output a best matching reference block according to the template matching process, which results in the inability to output the searched best matching reference block, which seriously affects the performance of the IntraTMP encoding tool.

Based on this, the embodiments of the present application provide an image encoding method, an image decoding method, an image processing device, a computer-readable storage medium and a computer program product, wherein the encoding end obtains the to-be-predicted coding block and the predicted coding block in the image frame, wherein the area corresponding to the predicted coding block is the reconstructed area, and then determines multiple target templates corresponding to the to-be-predicted coding block, and determines the matching reference block corresponding to each target template in the reconstructed area, and then fuses at least one matching reference block to obtain the intra-frame prediction value corresponding to the to-be-predicted coding block, and then obtains the residual value according to the original value of the to-be-predicted coding block and the intra-frame prediction value, and then encodes the residual value, and writes the encoding bit into the bitstream and sends it to the decoding end. Through the above-mentioned image encoding method, the accuracy of the intra-frame prediction result and the efficiency of the intra-frame prediction can be improved.

Fig. 4 is a schematic diagram of a video image transmission system architecture provided by an embodiment of the present application. As shown in Fig. 4, the video image transmission system includes an encoding end 410 and a decoding end 420. After obtaining the video image, the encoding end 410 will encode the video image and then send the bit stream to the decoding end 420. After receiving the bit stream, the decoding end 420 will decode the video image and display it.

In the embodiment of the present application, the encoding end or the decoding end may be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal, an augmented reality (AR) terminal, an Internet of Things (IoT) terminal device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in smart grid, a wireless terminal in transportation safety, a wireless terminal in smart city, a wireless terminal in smart home, a user equipment (UE), a vehicle communication device, a vehicle communication chip, a roadside unit or a communication device in a roadside unit, etc. It may also be a wearable device/wearable smart device, such as a portable device integrated into a user's clothes or accessories, such as a bracelet, glasses, gloves, watches, and clothing.

In order to fully explain the technical solution, the image encoding method of the embodiment of the present application is further described.

Fig. 5 is a flow chart of an image coding method provided in an embodiment of the present application. As shown in Fig. 5, the image coding method includes but is not limited to S100, S200, S300, S400 and S500. It should be noted that the embodiment of the present application does not strictly limit the following steps.

S100: Acquire a block to be predicted and a predicted block in an image frame, wherein an area corresponding to the predicted block is a reconstructed area.

In this step, after the original video is input into the hybrid coding framework of the encoding end, the image frame is divided into multiple CTUs, the size of the CTUs can be set according to actual needs, and then each CTU can be further divided into coding blocks CU according to needs or no longer divided. Obtain the coding block to be predicted and the predicted coding block in the image frame, wherein the predicted coding block is the coding block that has completed the prediction, and its corresponding area is the reconstructed area, and the coding block to be predicted is the coding block that has not completed the prediction. The embodiment of the present application provides an intra-frame prediction method for the coding block to be predicted.

In one embodiment, a frame of image is divided into CTUs of size 128*128, and then each CTU is divided into coding blocks CU according to a quadtree or multi-type tree recursive structure, and the size of CU is 64*64.

In another embodiment, a frame of image is divided into CTUs of size 128*128, and the CTU is no longer divided. That is, in this embodiment, the CU size is the same as the CTU size, both of which are 128*128.

It should be noted that the IntraTMP encoding tool has requirements on the CU size, that is, the width and height of the CU are both less than or equal to 64. Therefore, whether to further divide the CTU to obtain a smaller CU depends on the actual needs of the encoding tool.

S200: Determine multiple target templates corresponding to the to-be-predicted coding block.

In this step, it is first necessary to make an intra-frame or inter-frame prediction mode decision for the current block to be predicted. After it is determined that the intra-frame prediction coding is adopted for the current block to be predicted, a coding tool for the intra-frame prediction is further determined.

In one embodiment, it is determined that the IntraTMP coding tool is enabled for intra prediction of the current coding block to be predicted. The specific determination method is to determine whether the syntax element tmp_enabled_flag is marked as on. The latest ECM standard includes a variety of intra prediction tools, such as: Decoder Side Intra Mode Derivation (DIMD), Template-based Intra Mode Derivation Fusion (TIMD), Spatial Geometric Partitioning Mode (SGPM), Convolutional Cross-Component Intra Prediction Model (CCCM), etc. After determining that the IntraTMP coding tool is enabled for intra prediction of the current coding block to be predicted, the three shapes of target templates corresponding to the current coding block to be predicted can be determined, namely, L-shape template, Above template and Left template.

It should be noted that the embodiment of the present application uses target templates of three shapes as an example for illustration, but in actual use, target templates of other shapes may be used, or less than three or more than three target templates may be used for prediction at the same time, and the embodiment of the present application does not limit this.

S300: determining a matching reference block corresponding to each target template in the reconstructed area, and fusing at least one matching reference block to obtain a prediction value corresponding to the to-be-predicted coding block.

In this step, the matching reference blocks corresponding to each target template are determined in the reconstructed area using the multiple target templates determined in step S200. Since there are multiple target templates, there are multiple matching reference blocks. The matching reference block can be understood as the best "matching reference block" corresponding to the target template. According to different requirements and definitions, the method for determining the best "matching reference block" may be different. After determining the matching reference block corresponding to each target template, it may also be necessary to screen these matching reference blocks to obtain matching reference blocks that meet the fusion conditions, and those matching reference blocks that do not meet the fusion conditions will be discarded. Next, these matching reference blocks that meet the fusion conditions are fused to obtain the intra-frame prediction value corresponding to the coding block to be predicted.

The following specifically describes how to determine the matching reference block corresponding to each target template. FIG6 is a flow chart of a method for determining the matching reference block corresponding to each target template provided in an embodiment of the present application. As shown in FIG6 , the method includes but is not limited to S310 and S320. It should be noted that the embodiment of the present application does not strictly limit the following steps.

S310: Determine at least one reference template corresponding to each target template in the reconstructed area.

In this step, in the reconstructed area, each target template can generally find multiple corresponding reference templates, and there may be multiple search strategies for the reference templates.

S320: Determine a candidate reference block corresponding to each reference template.

In this step, after searching for each reference template, the similarity between the reference template and its target template is calculated, and the similarity is used to screen which candidate reference blocks corresponding to the reference template are matching reference blocks.

S330: Determine a matching reference block for each target template based on the candidate reference blocks.

In this step, the candidate reference blocks obtained in step S320 are further screened to further determine the matching reference blocks.

It should be noted that the matching reference block corresponding to each target template finally obtained can be directly selected from the candidate reference blocks, or can be obtained after further processing of the candidate reference blocks.

In one embodiment, a search with a determined step length is first performed in the reconstructed area to obtain a plurality of candidate reference blocks, and then the matching degrees of the candidate reference blocks are calculated, and one of them is selected as a matching reference block.

In another embodiment, the existing search strategy of IntraTMP is adopted, and a coarse search with a step size of 2 is first performed in the reconstructed area to preliminarily determine the intermediate process matching reference block corresponding to this coarse search process, and then a fine search with a step size of 1 is performed around the determined intermediate process matching reference block. At this time, the search area is limited to the minimum value of half the width or height of the current predicted coding block: min(BlkW, BlkH)/2, and the final matching reference block, that is, the matching reference block corresponding to the target template, is determined.

In another embodiment, after determining the multiple candidate reference blocks corresponding to each target template, a candidate list corresponding to each target template is constructed based on these candidate reference blocks. The candidate list of each target template allows the existence of multiple candidate block vectors IntraTMP BVs, where IntraTMP BV is a vector of a matching reference block pointed to by the current coding block to be predicted. Figure 7 is a schematic diagram of the IntraTMP BV provided in an embodiment of the present application. As shown in Figure 7, different IntraTMP BVs correspond to different reference blocks, which means that each target template can search or save multiple candidate reference blocks, and the candidate IntraTMP BVs in each candidate list are arranged in ascending order according to the template SAD. Then, a rate-distortion decision is made on the candidate reference blocks pointed to by the block vector in each candidate list to determine the matching reference blocks corresponding to each target template.

It should be noted that the absolute error and SAD are measures of similarity between image blocks. It is calculated by taking the absolute difference between each pixel in the original block and the corresponding pixel in the block used for comparison. These differences are added to create a simple measure of block similarity, the L1 norm of the difference image or the Manhattan distance between two image blocks. The sum of absolute differences can be used for various purposes, such as object recognition, generation of disparity maps for stereo images, and motion estimation for video compression. In the embodiment of the present application, the use of calculating the SAD value to measure the similarity between blocks is only an exemplary explanation. In actual application scenarios, other measurement algorithms can also be used, such as the sum of squared differences (SSD) algorithm or the semi-global matching (SGM) algorithm, etc., and the embodiment of the present application is not limited here.

In another embodiment, after determining multiple candidate reference blocks corresponding to each target template, a candidate list corresponding to each target template is constructed based on these candidate reference blocks, and the candidate list of each target template allows the existence of multiple candidate block vectors IntraTMP BVs, where IntraTMP BV is a vector of a matching reference block pointed to by the current coding block to be predicted. The difference from the previous embodiment is that in this embodiment, the candidate reference blocks pointed to by the block vectors in each candidate list are first fused, and the reference blocks obtained after fusion are determined as matching reference blocks corresponding to each target template.

After determining the matching reference blocks corresponding to each target template, these matching reference blocks need to be fused to obtain the intra-frame prediction values corresponding to the reference blocks to be predicted. Before fusion, matching reference blocks that meet the fusion conditions need to be selected, and these matching reference blocks that meet the fusion conditions are called reference blocks to be fused.

The following is a detailed description of how to determine the reference block to be fused.

In one embodiment, all matching reference blocks obtained by matching all target templates are the same. In this case, any matching reference block can be selected as the reference block to be fused. In other words, there is no need for fusion and traditional IntraTMP processing can be used.

In another embodiment, there are at least two different matching reference blocks. In this case, according to a preset algorithm, the degree of difference between the target templates corresponding to the at least two different matching reference blocks is determined, and according to the degree of difference, at least one matching reference block is selected as the reference block to be fused.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, two matching reference blocks are the same and the other is different. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, an absolute error sum algorithm is used to measure the degree of difference. The specific process includes: determining the target templates corresponding to the two different matching reference blocks as the templates to be compared, and then respectively determining the absolute error sums corresponding to the two templates to be compared, and the degree of difference between the absolute error sums corresponding to the two templates to be compared is greater than a preset threshold. In this case, the two matching reference blocks are selected together as the reference blocks to be fused.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, two matching reference blocks are the same and the other is different. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, the absolute error sum algorithm is used to measure the degree of difference. The specific process includes: determining the target templates corresponding to the two different matching reference blocks as the templates to be compared, and then respectively determining the absolute error sums corresponding to the two templates to be compared, and the degree of difference between the absolute error sums corresponding to the two templates to be compared is less than a preset threshold. In this case, a matching reference block is selected as the reference block to be fused. It can be understood that in actual application scenarios, the matching reference block corresponding to the template with a smaller SAD value is often selected as the reference block to be fused, that is, the matching reference block corresponding to the template with a larger SAD value is abandoned and does not participate in the fusion.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, the three matching reference blocks are different from each other. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, an absolute error sum algorithm is used to measure the degree of difference. The specific process includes: among the target templates corresponding to all the different matching reference blocks, a reference template and the remaining multiple non-reference templates are determined, and then the absolute error sum corresponding to the reference template and each non-reference template is determined, and then the absolute error sum corresponding to the reference template and the absolute error sum corresponding to each non-reference template are compared one by one, and it is obtained that the difference degree of the absolute error sum corresponding to the two is greater than a preset threshold. In this case, the matching reference blocks corresponding to the reference template and the non-reference template are used as reference blocks to be fused.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, the three matching reference blocks are different from each other. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, an absolute error sum algorithm is used to measure the degree of difference. The specific process includes: among the target templates corresponding to all the different matching reference blocks, a reference template and the remaining multiple non-reference templates are determined, and then the absolute error sum corresponding to the reference template and each non-reference template is determined, and then the absolute error sum corresponding to the reference template and the absolute error sum corresponding to each non-reference template are compared one by one, and it is obtained that the difference degree of the absolute error sum corresponding to the two is less than a preset threshold. In this case, only the matching reference block corresponding to the reference template is used as the reference block to be fused, that is, the matching reference block corresponding to the non-reference template is abandoned and does not participate in the fusion.

It can be understood that in the above embodiments, by screening out reference blocks to be fused that meet the fusion conditions, the number of matching reference blocks to be fused can be reduced in some cases, which can improve the overall fusion efficiency.

After determining the reference blocks to be fused, the fusion of the reference blocks can be enabled. The following is a detailed description of how to fuse these reference blocks to be fused. FIG8 is a flow chart of a method for determining a matching reference block corresponding to each target template provided in an embodiment of the present application. As shown in FIG8 , the method includes but is not limited to S340 and S350. It should be noted that the embodiment of the present application does not strictly limit the following steps.

S340: Determine a target fusion weight corresponding to each target template according to the intra-frame template fusion weight.

In one embodiment, a fixed target fusion weight may be determined for each target template. For example, a certain preset target fusion weight may be set for each target template. The preset target fusion weight may be a common coefficient value, such as {22/64, 21/64, 21/64} corresponding to the three modes of L-shape, Above and Left.

In another embodiment, the preset target fusion weight may be determined according to the number of samples included in the target template. The number of samples here may be understood as the number of pixels. For example, the target fusion weights corresponding to the three modes of L-shape, Above and Left are: Wherein W represents the width of the L-shape template, that is, the width of the current coding block to be predicted or the Above template, and H represents the height of the L-shape template, that is, the height of the current coding block to be predicted or the Left template.

In another embodiment, the intra-frame template fusion weight can be determined according to the absolute error and the number of samples corresponding to the target template. The specific process includes: respectively determining the target absolute error and the target number of samples included in each target template, and then normalizing the SAD corresponding to each target template according to the SAD of the target template and the target number of samples to obtain the normalized SAD corresponding to each target template, and then summing all normalized SADs to obtain the normalized absolute error sum, and then determining the target fusion weight corresponding to each target template according to the normalized SAD corresponding to each target template and the normalized absolute error sum. Among them, the normalized SAD corresponding to each target template can be obtained according to the ratio of the SAD of the target template to the target number of samples. It can be understood that normalizing the SAD of each target template can make the SAD value not affected by the number of samples contained in the template, thereby ensuring the accuracy of the weight calculation.

In another embodiment, the intra-frame template fusion weight can be determined based on the positional relationship between the pixel to be predicted in the coding block to be predicted and the target template. The specific process includes: determining the target absolute error sum corresponding to each target template, and determining the weight base value corresponding to each target template based on the target absolute error sum, and then determining the target position relationship between the pixel to be predicted and each target template, and determining the weight relative value corresponding to each target template based on the target position relationship, and then determining the target fusion weight corresponding to each target template based on the weight base value and the weight relative value.

S350: performing weighted calculation on the reference blocks to be fused corresponding to all target templates according to the target fusion weight corresponding to each target template, to obtain an intra-frame prediction value corresponding to the coding block to be predicted.

In this step, the target fusion weight corresponding to each target template obtained in step 330 is weightedly calculated with the matching reference block to obtain the intra-frame prediction value corresponding to the coding block to be predicted.

S400: Obtain a residual value according to the original value and the predicted value of the coding block to be predicted.

S500: Encode the residual value and write the encoded bits into the bitstream.

Steps S400 and S500 are not the key points of the embodiment of the present application, and thus will not be described in detail.

According to the image encoding method provided by the embodiment of the present application, when the encoding end transmits the indication information for intra-frame template fusion, there are two main situations: one is implicit fusion, that is, there is no need to transmit the block-level intra-frame template fusion identifier to the decoding end, and the other is non-implicit fusion, that is, it is necessary to transmit the block-level intra-frame template fusion identifier to the decoding end. The following describes in detail the two ways of indicating fusion.

The following is an explanation of implicit fusion:

In one embodiment, when determining a matching reference block, a method is adopted in which one is directly selected from a plurality of candidate reference blocks as a matching reference block corresponding to a target template. The encoding end need not transmit any information to the decoding end, and the decoding end can automatically adopt the same method to determine the matching reference block, thereby completing decoding.

In one embodiment, when determining the matching reference block, a candidate list corresponding to each target template is constructed, and a method of re-deciding the matching reference block based on the candidate reference block pointed to by the block vector in the list is adopted. The encoding end can transmit the block vector index value to the decoding end, and the decoding end can determine the matching reference block based on the block vector index value, thereby completing decoding.

In one embodiment, when determining a matching reference block, a method is adopted for determining a matching reference block by constructing a candidate list corresponding to each target template and fusing the candidate reference blocks pointed to by the block vector in each candidate list. The encoding end can transmit an intra-frame template reference block vector fusion identifier to the decoding end, and the decoding end can determine whether to enable the fusion of the candidate reference blocks based on the intra-frame template reference block vector fusion identifier, thereby completing decoding.

The image coding method using implicit fusion provided in the above embodiment can save transmission bits to a certain extent.

The following first explains non-implicit fusion:

In one embodiment, when determining a matching reference block, a method of directly selecting one from multiple candidate reference blocks as a matching reference block corresponding to a target template is adopted, and the encoding end instructs the decoding end to perform decoding by transmitting a high-level syntax element fusion flag and a block-level intra-frame template fusion flag. The specific process includes: the encoding end sets both the high-level syntax element fusion flag and the block-level intra-frame template fusion flag to an on state, and when the decoding end receives and parses the high-level syntax element fusion flag, it first determines whether the high-level syntax element fusion flag is in an on state, and when the high-level syntax element fusion flag is in an on state, continues to check whether the block-level intra-frame template fusion flag is in an on state, and if the block-level intra-frame template fusion flag is in an on state, the matching reference block is determined and fused in the same manner as the encoding end, thereby completing decoding.

In one embodiment, when determining a matching reference block, a candidate list corresponding to each target template is constructed, and a method of re-deciding the matching reference block for the candidate reference block pointed to by the block vector in the list is adopted. The encoder instructs the decoder to perform decoding by transmitting a high-level syntax element fusion flag, a block-level intra-frame template fusion flag and a block vector index value. The specific process includes: the encoder sets both the high-level syntax element fusion flag and the block-level intra-frame template fusion flag to an on state. When the decoder receives and parses the high-level syntax element fusion flag, it first determines whether the high-level syntax element fusion flag is in an on state, and when the high-level syntax element fusion flag is in an on state, it continues to check whether the block-level intra-frame template fusion flag is in an on state. If the block-level intra-frame template fusion flag is in an on state, the matching reference block is determined and fused according to the block vector index value, thereby completing decoding.

In one embodiment, when determining a matching reference block, a method is adopted in which a candidate list corresponding to each target template is constructed, and the candidate reference blocks pointed to by the block vector in each candidate list are merged to determine the matching reference block. The encoder instructs the decoder to perform decoding by transmitting a high-level syntax element fusion flag, a block-level intra-frame template fusion flag, and an intra-frame template reference block vector fusion flag. The specific process includes: the encoder sets both the high-level syntax element fusion flag and the block-level intra-frame template fusion flag to an on state. When the decoder receives and parses the high-level syntax element fusion flag, it first determines whether the high-level syntax element fusion flag is in an on state, and when the high-level syntax element fusion flag is in an on state, it continues to check whether the block-level intra-frame template fusion flag is in an on state. If the block-level intra-frame template fusion flag is in an on state, it continues to detect whether the intra-frame template reference block vector fusion flag is in an on state. If the intra-template reference block vector fusion flag is in an on state, the decoder first merges the candidate reference blocks to obtain a matching reference block, and then merges the matching reference block to complete decoding.

The image encoding method provided by the embodiment of the present application can improve the accuracy of IntraTMP prediction, solve the problem that the IntraTMP encoding tool cannot be applied to certain video content, and does not increase too many additional transmission bits.

In order to further illustrate the image encoding method provided in the embodiment of the present application, the following example is used for detailed description.

Example 1

This example describes in detail the entire process of image encoding at the image encoding end, and this process is used for encoding in subsequent examples.

FIG9 is a schematic diagram of the IntraTMP-fusion solution flow of the encoding end provided by the example of this application. As shown in the figure, the IntraTMP-fusion flow of the encoding end includes steps S910-S970:. Specifically:

S910: Input a video image and divide it into coding blocks.

Input the original video into the hybrid coding framework, first divide a frame into 128*128 CTUs, then each CTU is divided into coding blocks CU according to the quadtree or multi-type tree recursive structure, or no further division, that is, the CU size is the same as the CTU. However, there are two points worth noting: first, quadtree division cannot be performed after division using the multi-type tree recursive structure; second, the IntraTMP coding tool has requirements for the CU size, that is, the width and height of the CU are both less than or equal to 64, otherwise IntraTMP cannot be enabled.

S920: Determine whether to enable the IntraTMP coding tool for intra-frame prediction of the current block to be predicted and coded.

Make intra/inter prediction mode decision for the current block to be predicted, determine whether the current block to be predicted uses intra or inter prediction coding, and if the current block to be predicted uses intra prediction coding, further determine the coding tool for intra prediction. The latest ECM standard includes a variety of intra prediction tools, such as: decoder side intra mode derivation (DIMD), template-based intra mode derivation fusion (TIMD), spatial geometric partitioning mode (SGPM), convolutional cross-component intra prediction model (CCCM), etc. Determine whether the IntraTMP tool is enabled for the current block to be predicted, that is, whether the syntax element tmp_enabled_flag is marked as on.

Steps S930, S940, and S950 are parallel steps, that is, in actual applications, only one of the methods will be used to determine the matching reference block. However, those skilled in the art will appreciate that simultaneously using two or three of the methods S930, S940, and S950 to determine the matching reference block is also within the protection scope of the embodiments of the present application.

Step S930: Enable the multi-mode IntraTMP-fusion method to determine matching reference blocks of different templates respectively using a conventional search strategy.

Step S940: Activate the multi-mode multi-candidate IntraTMP-fusion method 1, and use a candidate item in each template candidate list to determine matching reference blocks of different templates.

Step S950: Activate the multi-mode multi-candidate IntraTMP-fusion method 2, and determine the matching reference blocks of different templates respectively by fusing the candidates in each template-based candidate list.

Step S960: Fusing the matching reference blocks of different modes in a selected fusion manner to obtain a final matching reference block as a prediction value of the current coding block to be predicted.

Step S970: Transmitting the enabled IntraTMP-fusion method related syntax elements to the decoding end.

Example 2

This example elaborates on an intra-frame prediction method in the image coding process, and calls this intra-frame prediction method a multi-mode IntraTMP-fusion method. In summary, this method first searches for the best "matching reference block" of each of the three IntraTMP modes according to different shape templates: L-shape, Above, and Left, and then fuses the three best "matching reference blocks" in a certain optimal fusion method to obtain the final matching reference block as the prediction of the current coding block to be predicted, thereby improving the prediction accuracy.

The specific implementation process is as follows:

Step 1: Input video image and divide it into coding blocks CU.

Step 2: Determine that the intra-frame prediction of the current block to be predicted and coded uses the IntraTMP coding tool.

Step 3: Enable the multi-mode IntraTMP-fusion method for the current block to be predicted and use a conventional search strategy to determine the best “matching reference block” corresponding to different modes.

After the multi-mode IntraTMP-fusion method is enabled for the current predicted coding block, the best "matching reference blocks" corresponding to the three templates of L-shape, Above and Left are searched respectively. If the best "matching reference block" of a certain mode is not available, the matching reference block is discarded.

The search strategy is consistent with the traditional IntraTMP, that is, a coarse search with a step size of 2 is first performed in the four search areas R1 to R4 to preliminarily determine the best matching reference block at this time, and then a fine search with a step size of 1 is performed around the determined best matching reference block to determine the best "matching reference block". At this time, the search area is limited to the minimum value of half the width or height of the current predicted coding block: min(BlkW,BlkH)/2.

Step 4: Fusing the best matching reference blocks of different modes in a certain optimal fusion method to obtain a final matching reference block as a prediction of the current coding block to be predicted;

Before fusing the best "matching reference blocks" of different modes, it is necessary to determine whether the matching reference blocks meet the fusion conditions. There are mainly three situations:

(1) If these matching reference blocks are the same, there is no need to merge them and they are processed according to the traditional IntraTMP method;

(2) If there are two identical blocks and one different block among these matching reference blocks, it is necessary to compare the template SADs of the two different matching reference blocks. If the ratio of the two is greater than a certain threshold, the two matching reference blocks meet the fusion condition; otherwise, no fusion is required, and one of the matching reference blocks is selected and processed according to the traditional IntraTMP method.

(3) If these matching reference blocks are different, that is, the three matching reference blocks are all different, then the reference block corresponding to the L-shapeIntraTMP template meets the fusion condition by default, and then the SAD of the Above template is compared with the SAD of the upper area of the L-shape template. If the ratio of the two is greater than a certain threshold, the reference block corresponding to the Above template meets the fusion condition, otherwise no fusion is required; and the SAD of the Left template is compared with the SAD of the left area of the L-shape template. If the ratio of the two is greater than a certain threshold, the reference block corresponding to the Left template meets the fusion condition; otherwise no fusion is required.

The best "matching reference blocks" that meet the fusion conditions are fused in a certain fusion method to obtain the final matching reference block, that is, the prediction of the current coding block to be predicted. There are many options for fusion methods, such as: based on predefined fixed weights, based on template matching SAD, based on position dependency, etc.

The fusion process of the best “matching reference blocks” of different modes can be expressed by the following formula:

Wherein, n represents the number of best matching reference blocks for fusion, which can be 2 or 3, indicating the best “matching reference blocks” corresponding to different IntraTMP modes (the default is L-shape, Above, and Left IntraTMP), and _wi represents the weight coefficient of the best matching reference block in the corresponding mode, as shown in FIG10 .

Step 5: Transmit the syntax elements related to the multi-mode IntraTMP-fusion method to the decoding end.

The identifier of the multi-mode IntraTMP-fusion method is intra_tmp_template_fusion_flag, which is located at the CU level. If this identifier is 1, the multi-mode IntraTMP-fusion method is enabled, otherwise the traditional IntraTMP is enabled.

In addition, an additional high-level syntax element fusion_enabled may be added to any one or more of the video parameter set (VPS), sequence parameter set (SPS), picture parameter set (PPS), adaptive parameter set (APS) or SEI (Supplemental enhancement information) to control whether fusion is enabled. For example, if the sps_fusion_enabled fusion flag is added to the SPS, it can be determined whether fusion is turned on at the sequence level. If sps_fusion_enabled is 0, fusion is prohibited for all coding blocks of this sequence, and there is no need to further determine the CU-level flag. If sps_fusion_enabled is 1, it is necessary to enable fusion for each coding block based on the CU-level flag intra_tmp_template_fusion_flag.

These syntax elements need to be transmitted to the decoder in the bitstream. After receiving the relevant identifier, the decoder can determine whether to enable the multi-mode IntraTMP-fusion method and complete the correct decoding process according to the same operation as the encoder.

Example 3:

This example elaborates another intra-frame prediction method in the image coding process, and calls this intra-frame prediction method a multi-mode multi-candidate IntraTMP-fusion method 1. In summary, firstly, candidate lists are constructed for three IntraTMP modes (L-shape, Above and Left) respectively. The candidate list of each mode allows multiple candidate IntraTMPBVs, where IntraTMP BV is a vector pointing from the current coding block to the best corresponding reference block. Refer to Figure 7, that is, different IntraTMP BVs correspond to different reference blocks, which means that each mode can search or save multiple reference matching reference blocks, and the candidate IntraTMP BVs in each candidate list are arranged in ascending order according to the template SAD. Then, one best IntraTMP BV is selected from the candidate lists of different modes respectively, and the index of IntraTMP BV in the candidate list is used to identify it, and the best "matching reference block" in the three modes is obtained; finally, the best "matching reference blocks" of different modes are fused in a certain best fusion mode to obtain the final matching reference block as the prediction of the current coding block to be predicted.

Step 1: Input video image and divide it into coding blocks CU.

Step 2: Determine that the intra-frame prediction of the current block to be predicted and coded uses the IntraTMP coding tool.

Step 3: Enable the multi-mode multi-candidate IntraTMP-fusion method 1 for the current block to be predicted and construct candidate lists of different modes respectively.

After determining that the IntraTMP coding tool has been enabled for the current coding block to be predicted, the multi-mode multi-candidate IntraTMP-fusion method 1 is used for the current coding block to be predicted, and candidate lists under three templates of L-shape, Above and Left are constructed respectively. The search strategy used when constructing the candidate list is consistent with the traditional IntraTMP. The specific process is as follows:

First, a coarse search with a step length of 2 is performed in the four search areas R1 to R4 shown in Figure 2, and the template SAD of the reference block and the current coding block to be predicted is calculated and arranged in ascending order. At the end of the search, the first 25 reference blocks with smaller template SADs are preliminarily determined and represented by IntraTMP BVs; then a fine search with a step length of 1 is performed on the reference blocks pointed to by the 25 IntraTMP BVs one by one, and the template SAD of the reference block and the current coding block to be predicted is also calculated at this time, and arranged in ascending order, and the first 10 IntraTMP BVs are selected to build a candidate list. The reference blocks pointed to by these 10 IntraTMP BVs are multiple candidate matching reference blocks in the corresponding mode. It should be noted that when selecting the first 10 IntraTMP BVs, if there is a reference block pointed to by the IntraTMP BV that is unavailable, this IntraTMP BV should be discarded, and one should be added from the subsequent BVs as a supplement.

Step 4: According to the reference block RDO decision pointed to by the candidate BVs, select a best IntraTMP BV from the candidate lists of different modes respectively, and determine the best matching reference block of the corresponding mode.

After building the candidate lists in different modes, it is necessary to make RDO decisions on the reference blocks pointed to by these 10 IntraTMP BVs, select one of the best IntraTMP BVs, and use the reference block it points to as the best "matching reference block" for the corresponding mode; at the same time, it is necessary to transmit the index intra_tmp_template_candidates1_idx of the best IntraTMP BV in the candidate list in the bitstream to identify the selected best IntraTMP BV, so that the decoding end can select the same IntraTMP BV to complete decoding.

Step 5: The best matching reference block is fused in some optimal fusion mode as the prediction of the current coding block to be predicted.

Before fusing the best "matching reference blocks" of different modes, it is also necessary to determine whether these matching reference blocks meet the fusion conditions. There are mainly three situations, which are the same as step 4 of Example 2 and will not be repeated here.

These matching reference blocks are fused in some optimal fusion method, and the final matching reference block obtained is used as the prediction of the current coding block to be predicted. Please refer to the implementation methods of Examples 5, 6, and 7 below. There are also many options for fusion methods, which will not be repeated here.

Step 6: Transmit the syntax elements related to the multi-mode multi-candidate IntraTMP-fusion method 1 to the decoding end.

The multi-mode multi-candidate IntraTMP-fusion method 1 involves the following syntax elements: mode identifier intra_tmp_template_candidates1_fusion_flag and index identifier of the best IntraTMP BV in different mode candidate lists: intra_tmp_template_candidates1_LA_idx, intra_tmp_template_candidates1_A_idx and intra_tmp_template_candidates1_L_idx. The mode identifier and index identifier are both at the CU level. If the mode identifier is 1, the multi-mode multi-candidate IntraTMP-fusion method 1 is enabled, otherwise the traditional IntraTMP is enabled; the index identifier is to allow the decoding end to determine the best "matching reference block" for different modes.

In addition, an additional high-level syntax element fusion_enabled may be added to any one or more of VPS, SPS, PPS, APS or SEI to control whether fusion is enabled, refer to step 5 of Example 2.

These syntax elements need to be transmitted in the bitstream. After receiving the relevant identifier, the decoder can determine whether to enable the multi-mode multi-candidate IntraTMP-fusion method 1. If enabled, the best "matching reference block" of different modes is first determined according to the three index identifiers, and then the correct decoding process is completed according to the same operation as the encoder.

Example 4:

This example elaborates on another intra-frame prediction method in the image coding process, and calls this intra-frame prediction method a multi-mode multi-candidate IntraTMP-fusion method 2. In summary, firstly, candidate lists are constructed for the three IntraTMP modes (L-shape, Above and Left) respectively. The candidate list of each mode allows multiple candidate IntraTMPBVs, and the candidate IntraTMP BVs in each candidate list are arranged in ascending order according to the template SAD; then, the reference blocks pointed to by multiple IntraTMP BVs in the same candidate list are fused, and the fused reference block is used as the best "matching reference block" of the corresponding mode; finally, the best "matching reference blocks" of different modes are fused in an optimal fusion manner to obtain the final matching reference block as the prediction of the current coding block to be predicted.

Step 1: Input video image and divide it into coding blocks CU.

Step 2: Determine that the intra-frame prediction of the current block to be predicted and coded uses the IntraTMP coding tool.

Step 3: Enable the multi-mode multi-candidate IntraTMP-fusion method 2 for the current block to be predicted and construct candidate lists under different modes respectively.

The candidate list is constructed in the same way as step 3 of the multi-mode multi-candidate IntraTMP-fusion method 1. The only difference is that in the fine search stage, only the first five IntraTMP BVs are selected to construct the candidate list. The reference blocks pointed to by these five IntraTMP BVs are the multiple candidate matching reference blocks in the corresponding mode.

Step 4: Fuse the reference blocks pointed to by the IntraTMP BVs of different mode candidate lists to determine the best matching reference block for the corresponding mode.

The reference blocks pointed to by the five IntraTMP BVs in the candidate lists of different modes are fused separately, and the results are used as the best “matching reference blocks” of the corresponding modes.

Step 5: In some optimal fusion method, the best matching reference blocks are fused to obtain the final matching reference block as the prediction of the current coding block to be predicted.

Before fusing the best "matching reference blocks" of different modes, it is also necessary to determine whether these matching reference blocks meet the fusion conditions. There are mainly three situations, which are the same as step 4 of Example 2 and will not be repeated here.

These matching reference blocks are fused in some optimal fusion method, and the final matching reference block obtained is used as the prediction of the current coding block to be predicted. Please refer to the implementation methods of Examples 5, 6, and 7 below. There are also many options for fusion methods, which will not be repeated here.

Step 6: Transmit syntax elements related to the multi-mode multi-candidate IntraTMP-fusion method 2 to the decoding end.

The multi-mode multi-candidate IntraTMP-fusion method 2 involves the following syntax elements: mode identifier intra_tmp_template_candidates2_fusion_flag and the fusion identifier of the candidate IntraTMP BVs intra_tmp_template_bv_fusion_flag. The mode identifier and the candidate fusion identifier are both at the CU level. If the mode identifier is 1, the multi-mode multi-candidate IntraTMP-fusion method 2 is enabled, otherwise the traditional IntraTMP is enabled; the candidate fusion identifier is to instruct the decoding end to fuse the mode candidates and determine the best "matching reference block" for different modes.

In addition, an additional high-level syntax element fusion_enabled may be added to any one or more of VPS, SPS, PPS, APS or SEI to control whether fusion is enabled, refer to step 5 of Example 2.

These syntax elements need to be transmitted in the bitstream. After receiving the relevant identifier, the decoder can determine whether to enable the multi-mode multi-candidate IntraTMP-fusion method 2. If enabled, the candidate list is constructed according to the same operation as the encoder, and the fusion of the candidate items pointing to the reference blocks is performed, and the fusion of the best "matching reference blocks" in different modes is realized to complete the correct decoding process.

Example 5:

This example describes in detail the process of determining the weight corresponding to each target template in the intra-frame prediction method.

In this example, fixed-weight-based fusion is to fuse the best matching reference blocks in three modes according to predefined fixed weights, and finally obtain the best matching reference block for predicting the current coding block to be predicted. The fixed-weight method does not require tedious weight calculations and can greatly reduce complexity.

There are two ways to set fixed weights:

The fixed weight is determined based on the number of reference samples of the template. The weights corresponding to the three modes of L-shape, Above and Left are: Where W represents the width of the L-shape template, that is, the width of the current coding block to be predicted or the Above template, and H represents the height of the L-shape template, that is, the height of the current coding block to be predicted or the Left template;

Use common coefficient values as fixed weights. For example, the weights corresponding to the three modes L-shape, Above, and Left are {22/64, 21/64, 21/64}.

Example 6:

This example describes in detail the process of determining the weight corresponding to each target template in the intra-frame prediction method.

In this example, the fusion based on template matching SAD determines the weight according to the template SAD of the three templates (L-shape, Above, Left) between the current coding block to be predicted and the reference block in the search area. Let the SAD of L-shape IntraTMP be Sad_LA, the SAD of Above IntraTMP be Sad_A, and the SAD of Left IntraTMP be Sad_L. The number of template samples of the three modes are n_LA, n_A, and n_L, respectively. Then, the template SADs of the three modes are normalized based on the number of their respective reference samples, because the number of template samples will affect the SAD value. The normalization process is as follows:

SAD1＝Sad_LA/n_LA

SAD2＝Sad_A/n_A

SAD3＝Sad_L/n_L

After normalization, the weight of each template can be determined by the following formula:

Where n represents the number of best matching reference blocks for fusion, which can be 2 or 3, represents the normalized template SAD under different modes, and represents the weight coefficient under the corresponding mode. In addition, the calculation of weights involves division operations. In order to reduce the complexity, a table lookup method is used instead of a division operation.

Example 7:

This example describes in detail the process of determining the weight corresponding to each target template in the intra-frame prediction method.

In this example, the fusion based on position dependence takes into account the influence of the distance between the predicted pixel position and the template. If the position of a certain predicted pixel is closer to the area above the current block to be predicted, that is, closer to the Above template, the weight of the reference block corresponding to the Above template needs to be assigned a larger coefficient value, and the weight coefficient of the area above the L-shape template needs to be reduced synchronously, which means that the predicted pixel is more dependent on the Above template; on the contrary, that is, the position of a certain predicted pixel is farther from the area above the current block to be predicted, the weight of the reference block corresponding to the Above template needs to be assigned a smaller coefficient value, and the weight coefficient of the area above the L-shape template needs to be increased synchronously. Similarly, if the position of a certain predicted pixel is closer to the left area of the current block to be predicted, that is, closer to the Left template, the weight of the reference block corresponding to the Left template needs to be assigned a larger coefficient value, and the weight coefficient of the area on the left of the L-shape template needs to be reduced synchronously; on the contrary, that is, the position of a certain predicted pixel is farther from the left area of the current block to be predicted, the weight of the reference block corresponding to the Left template needs to be assigned a smaller coefficient value, and the weight coefficient of the area on the left of the L-shape template needs to be increased synchronously.

It should be noted that the position of the predicted pixel is two-dimensional, that is, it includes the horizontal position (x, that is, the position from the left boundary of the current coding block to be predicted) and the vertical position (y, that is, the position from the upper boundary of the current coding block to be predicted). This means that for a certain predicted pixel, it is necessary to consider the influence of vertical distance and horizontal distance on the weight at the same time, that is, it is necessary to discuss the degree to which the predicted pixel depends on the Above template and the Left template respectively, determine the corresponding weight coefficients of different templates of L-shape, Above and Left, and finally select the best fusion method to realize the fusion of reference blocks corresponding to different modes (usually three modes) as the prediction of the current coding block to be predicted.

The specific steps are as follows:

Step 1: Determine the template SAD of different modes (usually three modes: L-shape, Above, and Left IntraTMP);

When searching for the best "matching reference block" of different IntraTMP modes, the SAD between the reference matching reference block template of each mode and the current coding block template to be predicted can be obtained, as shown in Figure 11, let a0 represent the template SAD of the upper area of the L-shape IntraTMP mode, b0 represent the template SAD of the left area of the L-shape IntraTMP mode, a1 represent the template SAD of the AboveIntraTMP mode, and b1 represent the template SAD of the Left IntraTMP mode.

First, according to the SAD of the templates in different modes above, the basic weight values of the L-shape, Above and Left templates are obtained as the weight base values based on location-dependent fusion:

Step 2: Determine the weight values corresponding to different modes based on the predicted pixel positions;

If the predicted pixel position is (x, y), where x represents the distance between the pixel position and the left boundary of the current block to be predicted, that is, the distance from the Left template area, and y represents the distance between the pixel position and the upper boundary of the current block to be predicted, that is, the distance from the Above template area, then the weights that depend on the L-shape, Above, and Left templates are:

Where BlkH and BlkW represent the height and width of the current coding block to be predicted, respectively. Δi is a predefined range related to the deviation between the base weight value and the position-dependent weight part, such as the deviation between w _LAbase and w _LAloc . The larger the Δi value, the greater the weight of the current coding block to be predicted that depends on the position.

Considering that different IntraTMP modes are affected by the basic weight, the position-dependent weights in the three different modes (L-shape, Above, and Left IntraTMP modes) are:

Step 3: Calculate the fusion of reference blocks in different modes and corresponding weight coefficient values as the prediction of the current coding block to be predicted.

The reference blocks corresponding to the three modes and the corresponding weight coefficient values are fused to achieve the prediction of the current coding block to be predicted:

p _fusion ＝w _LA *p _L-shape +w _A *p _Above +w _L *p _Left

In the aforementioned embodiments of the present application, the image encoding process performed by the encoding end is described. Correspondingly, in the following embodiments of the present application, the image decoding process performed by the decoding end will be described.

Fig. 12 is a flow chart of an image decoding method provided by an embodiment of the present application. As shown in Fig. 12, the image encoding method includes but is not limited to S600, S700, S800, and S900. It should be noted that the embodiment of the present application does not strictly limit the following steps.

S600: Parse the bitstream to obtain a residual value corresponding to the to-be-predicted decoding block.

In this step, the decoder reads the code stream (bit stream), processes the network adaptation layer (NAL, Network Abstraction Layer) to read the network adaptation layer unit (NALU, Network Abstraction Layer Unit), and checks whether the NALU is normal. If it is normal, it means that a new frame of picture is input into the code stream and the new image needs to be decoded; otherwise, it means that no new image is input. When a new image to be decoded is read, the residual value for image reconstruction can be obtained. The process of obtaining the residual value is not the focus of this application, so it will not be described here.

The decoder performs loop filtering and a series of decomposition operations on the decoded frame, and performs different types of decoding according to different NAL types; when the decoder decodes a frame of an image, it first traverses the slice and then traverses each CTU, and each CTU needs to be divided to obtain coding blocks CUs. Before decoding each CU, it is necessary to check whether the current CU is in Skip mode, palette mode, intra-frame or inter-frame prediction mode; after determining that the current CU uses intra-frame prediction mode, check whether the CU-level flag of the traditional IntraTMP mode is enabled. After determining that the current CU enables the traditional IntraTMP mode, that is, tmp_enabled_flag = 1; then check which IntraTMP-fusion method is enabled for the current CU, which can be determined by the relevant syntax elements of the method. After determining that the IntraTMP encoding tool is enabled for the intra-frame prediction of the current coding block to be predicted, the three shapes of target templates corresponding to the current coding block to be predicted can be determined, namely L-shape template, Above template and Left template.

It should be noted that the embodiment of the present application uses target templates of three shapes as an example for illustration, but in actual use, target templates of other shapes may be used, or less than three or more than three target templates may be used for prediction at the same time, and the embodiment of the present application does not limit this.

S700: According to multiple target templates, determine a matching reference block corresponding to each template in the reconstructed area, and fuse at least one reference block to obtain a prediction value corresponding to the predicted decoding block; wherein the area corresponding to the predicted decoding block is the reconstructed area.

In this step, the matching reference blocks corresponding to each target template are determined in the reconstructed area using the determined multiple target templates. Since there are multiple target templates, there are multiple matching reference blocks. The matching reference block can be understood as the best "matching reference block" corresponding to the target template. According to different requirements and definitions, the method for determining the best "matching reference block" may be different. After determining the matching reference block corresponding to each target template, it may also be necessary to screen these matching reference blocks to obtain matching reference blocks that meet the fusion conditions, and those matching reference blocks that do not meet the fusion conditions will be discarded. Next, these matching reference blocks that meet the fusion conditions are fused to obtain the intra-frame prediction value corresponding to the coding block to be predicted.

It should be noted that the process of determining the matching reference block and the process of fusing the matching reference blocks at the decoding end are the same as those at the encoding end, and therefore, the following embodiment is used for a brief description.

First, in the reconstructed area, each target template can generally find multiple reference templates corresponding to it, and there can be multiple search strategies for reference templates. Then, after searching for each reference template, the similarity between the reference template and its target template is calculated, and the similarity is used to screen which candidate reference blocks corresponding to the reference template are matching reference blocks. Finally, by further screening the candidate reference blocks, a matching reference block is determined for each target template. It should be noted that the matching reference block corresponding to each target template obtained in the end can be directly selected from the candidate reference blocks, or it can be obtained after further processing of the candidate reference blocks.

In one embodiment, a search with a determined step length is first performed in the reconstructed area to obtain a plurality of candidate reference blocks, and then the matching degrees of the candidate reference blocks are calculated, and one of them is selected as a matching reference block.

In another embodiment, the existing search strategy of IntraTMP is adopted, and a coarse search with a step size of 2 is first performed in the reconstructed area to preliminarily determine the intermediate process matching reference block corresponding to this coarse search process, and then a fine search with a step size of 1 is performed around the determined intermediate process matching reference block. At this time, the search area is limited to the minimum value of half the width or height of the current predicted coding block: min(BlkW, BlkH)/2, and the final matching reference block, that is, the matching reference block corresponding to the target template, is determined.

In another embodiment, after determining a plurality of candidate reference blocks corresponding to each target template, a candidate list corresponding to each target template is constructed based on these candidate reference blocks, and the candidate list of each target template allows the existence of multiple candidate block vectors IntraTMP BVs, where IntraTMP BV is a vector of a matching reference block pointed to by the current coding block to be predicted. Different IntraTMP BVs correspond to different reference blocks, which means that each target template can search or save multiple candidate reference blocks, and the candidate IntraTMP BVs in each candidate list are arranged in ascending order according to the template SAD. Then, a rate-distortion decision is made for the candidate reference blocks pointed to by the block vector in each candidate list to determine the matching reference blocks corresponding to each target template.

It should be noted that the absolute error and SAD are measures of similarity between image blocks. It is calculated by taking the absolute difference between each pixel in the original block and the corresponding pixel in the block used for comparison. These differences are added to create a simple measure of block similarity, the L1 norm of the difference image or the Manhattan distance between two image blocks. The sum of absolute differences can be used for various purposes, such as object recognition, generation of disparity maps for stereo images, and motion estimation for video compression. In the embodiment of the present application, the use of calculating the SAD value to measure the similarity between blocks is only an exemplary explanation. In actual application scenarios, other measurement algorithms can also be used, such as the sum of squared differences (SSD) algorithm or the semi-global matching (SGM) algorithm, etc., and the embodiment of the present application is not limited here.

In another embodiment, after determining multiple candidate reference blocks corresponding to each target template, a candidate list corresponding to each target template is constructed based on these candidate reference blocks, and the candidate list of each target template allows the existence of multiple candidate block vectors IntraTMP BVs, where IntraTMP BV is a vector of a matching reference block pointed to by the current coding block to be predicted. The difference from the previous embodiment is that in this embodiment, the candidate reference blocks pointed to by the block vectors in each candidate list are first fused, and the fused reference blocks are determined as matching reference blocks corresponding to each target template.

After determining the matching reference blocks corresponding to each target template, these matching reference blocks need to be fused to obtain the intra-frame prediction values corresponding to the reference blocks to be predicted. Before fusion, matching reference blocks that meet the fusion conditions need to be selected, and these matching reference blocks that meet the fusion conditions are called reference blocks to be fused.

The following is a detailed description of how to determine the reference block to be fused.

In one embodiment, all matching reference blocks obtained by matching all target templates are the same. In this case, any matching reference block can be selected as the reference block to be fused. In other words, there is no need for fusion and traditional IntraTMP processing can be used.

In another embodiment, there are at least two different matching reference blocks. In this case, according to a preset algorithm, the degree of difference between the target templates corresponding to the at least two different matching reference blocks is determined, and according to the degree of difference, at least one matching reference block is selected as the reference block to be fused.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, two matching reference blocks are the same and the other is different. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, an absolute error sum algorithm is used to measure the degree of difference. The specific process includes: determining the target templates corresponding to the two different matching reference blocks as the templates to be compared, and then respectively determining the absolute error sums corresponding to the two templates to be compared, and the degree of difference between the absolute error sums corresponding to the two templates to be compared is greater than a preset threshold. In this case, the two matching reference blocks are selected together as the reference blocks to be fused.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, two matching reference blocks are the same and the other is different. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, the absolute error sum algorithm is used to measure the degree of difference. The specific process includes: determining the target templates corresponding to the two different matching reference blocks as the templates to be compared, and then respectively determining the absolute error sums corresponding to the two templates to be compared, and the degree of difference between the absolute error sums corresponding to the two templates to be compared is less than a preset threshold. In this case, a matching reference block is selected as the reference block to be fused. It can be understood that in actual application scenarios, the matching reference block corresponding to the template with a smaller SAD value is often selected as the reference block to be fused, that is, the matching reference block corresponding to the template with a larger SAD value is abandoned and does not participate in the fusion.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, the three matching reference blocks are different from each other. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, an absolute error sum algorithm is used to measure the degree of difference. The specific process includes: among the target templates corresponding to all the different matching reference blocks, a reference template and the remaining multiple non-reference templates are determined, and then the absolute error sum corresponding to the reference template and each non-reference template is determined, and then the absolute error sum corresponding to the reference template and the absolute error sum corresponding to each non-reference template are compared one by one, and it is obtained that the difference degree of the absolute error sum corresponding to the two is greater than a preset threshold. In this case, the matching reference blocks corresponding to the reference template and the non-reference template are used as reference blocks to be fused.

In a more specific embodiment, among the three matching reference blocks corresponding to the three target templates, the three matching reference blocks are different from each other. In order to compare the degree of difference between the target templates corresponding to the different matching reference blocks, an absolute error sum algorithm is used to measure the degree of difference. The specific process includes: among the target templates corresponding to all the different matching reference blocks, a reference template and the remaining multiple non-reference templates are determined, and then the absolute error sum corresponding to the reference template and each non-reference template is determined, and then the absolute error sum corresponding to the reference template and the absolute error sum corresponding to each non-reference template are compared one by one, and it is obtained that the difference degree of the absolute error sum corresponding to the two is less than a preset threshold. In this case, only the matching reference block corresponding to the reference template is used as the reference block to be fused, that is, the matching reference block corresponding to the non-reference template is abandoned and does not participate in the fusion.

It can be understood that in the above embodiments, by screening out reference blocks to be fused that meet the fusion conditions, the number of matching reference blocks to be fused can be reduced in some cases, which can improve the overall fusion efficiency.

After determining the reference blocks to be fused, the fusion of the reference blocks can be started. The following is a detailed description of how to fuse these reference blocks to be fused.

First, the target fusion weight corresponding to each target template is determined according to the intra-frame template fusion weight.

In one embodiment, a fixed target fusion weight may be determined for each target template. For example, a certain preset target fusion weight may be set for each target template. The preset target fusion weight may be a common coefficient value, such as {22/64, 21/64, 21/64} corresponding to the three modes of L-shape, Above and Left.

In another embodiment, the preset target fusion weight may be determined according to the number of samples included in the target template. The number of samples here may be understood as the number of pixels. For example, the target fusion weights corresponding to the three modes of L-shape, Above and Left are: Where W represents the width of the L-shape template, that is, the width of the current decoding block to be predicted or the Above template, and H represents the height of the L-shape template, that is, the height of the current decoding block to be predicted or the Left template.

In another embodiment, the intra-frame template fusion weight can be determined according to the absolute error and the number of samples corresponding to the target template. The specific process includes: respectively determining the target absolute error and the target number of samples included in each target template, and then normalizing the SAD corresponding to each target template according to the SAD of the target template and the target number of samples to obtain the normalized SAD corresponding to each target template, and then summing all normalized SADs to obtain the normalized absolute error sum, and then determining the target fusion weight corresponding to each target template according to the normalized SAD corresponding to each target template and the normalized absolute error sum. Among them, the normalized SAD corresponding to each target template can be obtained according to the ratio of the SAD of the target template to the target number of samples. It can be understood that normalizing the SAD of each target template can make the SAD value not affected by the number of samples contained in the template, thereby ensuring the accuracy of the weight calculation.

In another embodiment, the intra-frame template fusion weight can be determined based on the positional relationship between the pixel to be predicted in the decoded block to be predicted and the target template. The specific process includes: determining the target absolute error sum corresponding to each target template, and determining the weight base value corresponding to each target template based on the target absolute error sum, and then determining the target positional relationship between the pixel to be predicted and each target template, and determining the weight relative value corresponding to each target template based on the target positional relationship, and then determining the target fusion weight corresponding to each target template based on the weight base value and the weight relative value.

Next, according to the target fusion weight corresponding to each target template, weighted calculation is performed on the reference blocks to be fused corresponding to all target templates to obtain the intra-frame prediction value corresponding to the decoded block to be predicted.

S800: Obtain a decoding value corresponding to a decoding block to be predicted according to a prediction value and a residual value of a reference block to be predicted.

S900: Obtain a decoding recovery value of a to-be-predicted decoding block according to the decoding value.

Steps S800 and S900 are not the key points of the embodiment of the present application, and thus will not be described in detail.

According to the image encoding method provided by the embodiment of the present application, when the encoder transmits the indication information for intra-frame template fusion, there are two main situations: one is implicit fusion, that is, there is no need to transmit the block-level intra-frame template fusion identifier to the decoding end, and the other is to transmit the block-level intra-frame template fusion identifier to the decoding end. The following describes in detail the two ways of indicating fusion.

The following is an explanation of implicit fusion:

In one embodiment, when determining a matching reference block, a method is adopted in which one is directly selected from a plurality of candidate reference blocks as a matching reference block corresponding to a target template. The encoding end need not transmit any information to the decoding end, and the decoding end can automatically adopt the same method to determine the matching reference block, thereby completing decoding.

In one embodiment, when determining the matching reference block, a candidate list corresponding to each target template is constructed, and a method of re-deciding the matching reference block based on the candidate reference block pointed to by the block vector in the list is adopted. The encoding end can transmit the block vector index value to the decoding end, and the decoding end can determine the matching reference block based on the block vector index value, thereby completing decoding.

In one embodiment, when determining a matching reference block, a method is adopted for determining a matching reference block by constructing a candidate list corresponding to each target template and fusing the candidate reference blocks pointed to by the block vector in each candidate list. The encoding end can transmit an intra-frame template reference block vector fusion identifier to the decoding end, and the decoding end can determine whether to enable the fusion of the candidate reference blocks based on the intra-frame template reference block vector fusion identifier, thereby completing decoding.

The image coding method using implicit fusion provided in the above embodiment can save transmission bits to a certain extent.

The following first explains non-implicit fusion:

In one embodiment, when determining a matching reference block, a method of directly selecting one from multiple candidate reference blocks as a matching reference block corresponding to a target template is adopted, and the encoding end instructs the decoding end to perform decoding by transmitting a high-level syntax element fusion flag and a block-level intra-frame template fusion flag. The specific process includes: the encoding end sets both the high-level syntax element fusion flag and the block-level intra-frame template fusion flag to an on state, and when the decoding end receives and parses the high-level syntax element fusion flag, it first determines whether the high-level syntax element fusion flag is in an on state, and when the high-level syntax element fusion flag is in an on state, continues to check whether the block-level intra-frame template fusion flag is in an on state, and if the block-level intra-frame template fusion flag is in an on state, the matching reference block is determined and fused in the same manner as the encoding end, thereby completing decoding.

In one embodiment, when determining a matching reference block, a candidate list corresponding to each target template is constructed, and a method of re-deciding the matching reference block for the candidate reference block pointed to by the block vector in the list is adopted. The encoder instructs the decoder to perform decoding by transmitting a high-level syntax element fusion flag, a block-level intra-frame template fusion flag and a block vector index value. The specific process includes: the encoder sets both the high-level syntax element fusion flag and the block-level intra-frame template fusion flag to an on state. When the decoder receives and parses the high-level syntax element fusion flag, it first determines whether the high-level syntax element fusion flag is in an on state, and when the high-level syntax element fusion flag is in an on state, it continues to check whether the block-level intra-frame template fusion flag is in an on state. If the block-level intra-frame template fusion flag is in an on state, the matching reference block is determined and fused according to the block vector index value, thereby completing decoding.

In one embodiment, when determining a matching reference block, a method is adopted in which a candidate list corresponding to each target template is constructed, and the candidate reference blocks pointed to by the block vector in each candidate list are merged to determine the matching reference block. The encoder instructs the decoder to perform decoding by transmitting a high-level syntax element fusion flag, a block-level intra-frame template fusion flag, and an intra-frame template reference block vector fusion flag. The specific process includes: the encoder sets both the high-level syntax element fusion flag and the block-level intra-frame template fusion flag to an on state. When the decoder receives and parses the high-level syntax element fusion flag, it first determines whether the high-level syntax element fusion flag is in an on state, and when the high-level syntax element fusion flag is in an on state, it continues to check whether the block-level intra-frame template fusion flag is in an on state. If the block-level intra-frame template fusion flag is in an on state, it continues to detect whether the intra-frame template reference block vector fusion flag is in an on state. If the intra-template reference block vector fusion flag is in an on state, the decoder first merges the candidate reference blocks to obtain a matching reference block, and then merges the matching reference block to complete decoding.

The image decoding method provided by the embodiment of the present application can improve the accuracy of IntraTMP prediction, solve the problem that the IntraTMP encoding tool cannot be applied to certain video content, and does not increase too many additional transmission bits.

Example 8

This example describes in detail the entire process of image decoding at the image decoding end, and this process is used for decoding in subsequent examples.

FIG13 is a schematic diagram of the IntraTMP-fusion solution flow of the encoding end provided by the example of this application. As shown in the figure, the IntraTMP-fusion flow of the encoding end includes steps S1310-S1350. Specifically:

S1310: Parse the bitstream and determine the IntraTMP-fusion decoding type of the current block to be predicted and decoded.

In this step, the decoder reads the code stream (bit stream), processes the network adaptation layer (NAL, Network Abstraction Layer) to read the network adaptation layer unit (NALU, Network Abstraction Layer Unit), and checks whether the NALU is normal. If it is normal, it means that a new frame of picture is input into the code stream and the new image needs to be decoded; otherwise, it means that no new image is input. When a new image to be decoded is read, the residual value for image reconstruction can be obtained. The process of obtaining the residual value is not the focus of this application, so it will not be described here.

The decoder performs loop filtering and a series of decomposition operations on the decoded frame, and performs different types of decoding according to different NAL types; when the decoder decodes a frame of an image, it first traverses the slice and then traverses each CTU, and each CTU needs to be divided to obtain coding blocks CUs. Before decoding each CU, it is necessary to check whether the current CU is in Skip mode, palette mode, intra-frame or inter-frame prediction mode; after determining that the current CU uses intra-frame prediction mode, check whether the CU-level flag of the traditional IntraTMP mode is enabled. After determining that the current CU enables the traditional IntraTMP mode, that is, tmp_enabled_flag = 1; then check which IntraTMP-fusion method is enabled for the current CU, which can be determined by the relevant syntax elements of the method. After determining that the IntraTMP encoding tool is enabled for the intra-frame prediction of the current coding block to be predicted, the three shapes of target templates corresponding to the current coding block to be predicted can be determined, namely L-shape template, Above template and Left template.

It should be noted that the embodiment of the present application uses target templates of three shapes as an example for illustration, but in actual use, target templates of other shapes may be used, or less than three or more than three target templates may be used for prediction at the same time, and the embodiment of the present application does not impose any restrictions on this.

S1320: Multi-mode IntraTMP-fusion method: Use conventional search strategies to determine matching reference blocks corresponding to different modes.

S1330: Multi-mode multi-candidate IntraTMP-fusion method 1: Determine the matching reference block of the corresponding mode by the candidate index identifiers of different modes.

S1340: Multi-mode multi-candidate IntraTMP-fusior method 2: Fusion of candidates from different modes to determine matching reference blocks of corresponding modes.

Steps S1320, S1330, and S1340 are parallel steps, that is, in actual applications, only one of the methods will be used to determine the matching reference block. However, those skilled in the art will appreciate that simultaneously using two or three of the methods S1320, S1330, and S1340 to determine the matching reference block is also within the protection scope of the embodiments of the present application.

S1350: Fusing the matching reference blocks of different modes in a fusion manner at the encoding end to serve as a prediction value of the current decoding block to be predicted.

The fusion method at the decoding end can refer to the embodiment of the encoding end in this application, so it will not be described here in detail.

Example 9

This example provides an intra-frame prediction method for a video decoding process, which is the decoding process of the multi-mode IntraTMP-fusion method of Example 2, and describes the syntax elements in the bitstream related to the method.

The decoder first determines whether there are high-level syntax elements in the bitstream, indicating whether fusion is enabled for the current sequence or image; if the high-level syntax elements are enabled, it further checks whether the CU-level flag of the multi-mode IntraTMP-fusion method is enabled, and the multi-mode IntraTMP-fusion method is considered to be a subset of the traditional IntraTMP mode, that is, only when the traditional IntraTMP is enabled for the current predicted decoding block, is it decided whether the multi-mode IntraTMP-fusion is enabled. The syntax structure is shown in Figure 14.

Semantic explanation:

intra_tmp_template_fusion_flag: equal to 1 indicates that the current coding block enables the multi-mode IntraTMP-fusion method, and equal to 0 indicates that the current coding block does not enable the multi-mode IntraTMP-fusion method.

The decoding end uses the same search method as the encoding end to derive the prediction block. The specific implementation process is as follows:

Step 2: Use conventional search strategies to determine the best matching reference blocks corresponding to different modes.

When the CU flag intra_tmp_template_fusion_flag is 1, decoding is performed according to the multi-mode IntraTMP-fusion method, and the search strategy refers to step 3 of Example 2, otherwise decoding is performed according to the traditional IntraTMP mode.

Step 3: The best “matching reference blocks” of different modes are fused in the fusion mode of the encoding end to determine the prediction of the current decoding block.

The check before fusion is the same as that at the encoding end. It is also necessary to determine whether the three best "matching reference blocks" meet the fusion conditions, refer to step 4 of Example 2; the fusion method at the decoding end is determined according to the fusion method selected by the encoding end.

Example 10

This embodiment provides an intra-frame prediction method for a video decoding process, which is the decoding process of the multi-mode multi-candidate IntraTMP-fusion method 1 in Example 3, and describes the syntax elements related to the method in the bitstream.

The decoder first determines whether there are high-level syntax elements in the bitstream, indicating whether fusion is enabled for the current sequence or image; under the premise that the high-level is enabled, it further checks whether the CU-level flag of the multi-mode multi-candidate IntraTMP-fusion method 1 is enabled, and the multi-mode multi-candidate IntraTMP-fusion method 1 is considered to be a subset of the traditional IntraTMP mode, that is, only when the traditional IntraTMP is enabled for the current predicted decoding block, it is determined whether the multi-mode multi-candidate IntraTMP-fusion method 1 is enabled; after determining to enable, the best "matching reference block" pointed to by each mode is determined according to the index flag of the best IntraTMP BV in the candidate list of different modes. The syntax structure is shown in Figure 15:

Semantic explanation:

intra_tmp_template_candidates1_fusion_flag: equal to 1 indicates that the current coding block enables multi-mode multi-candidate IntraTMP-fusion method 1, and equal to 0 indicates that the current coding block does not enable this method.

intra_tmp_template_candidates1_LA_idx: indicates the best IntraTMP BV index value of the current coding block in the L-shapeIntraTMP mode candidate list.

intra_tmp_template_candidates1_A_idx: indicates the best IntraTMP BV index value of the current coding block in the Above IntraTMP mode candidate list.

intra_tmp_template_candidates1_L_idx: indicates the best IntraTMP BV index value of the current coding block in the Left IntraTMP mode candidate list.

The decoding end uses the same implementation operation as the encoding end to derive the prediction block. The specific implementation process is as follows:

Step 2: Construct candidate lists for each mode respectively, and determine the best matching reference blocks of different modes by the candidate index identifiers of each mode.

When the CU flag intra_tmp_template_candidates1_fusion_flag is 1, decoding is completed according to the multi-mode multi-candidate IntraTMP-fusion method 1. First, a candidate list is constructed. The construction process refers to step 3 of the multi-mode multi-candidate IntraTMP-fusion method 1 in Example 3; otherwise, decoding is performed according to the traditional IntraTMP mode.

The reference block pointed to by the best IntraTMP BV index identifier in each mode candidate list determines the best "matching reference block" for the corresponding mode.

Step 3: The best “matching reference blocks” of different modes are fused in the fusion mode of the encoding end to determine the prediction of the current decoding block.

The check before fusion is the same as that at the encoding end. It is also necessary to determine whether the three best "matching reference blocks" meet the fusion conditions, refer to step 4 of Example 2; the fusion method at the decoding end is determined according to the fusion method selected by the encoding end.

Example 11

This embodiment provides an intra-frame prediction method for a video decoding process, which is the decoding process of the multi-mode multi-candidate IntraTMP-fusion method 2 in Example 4, and describes the syntax elements related to the method in the bitstream.

The decoder first determines whether there are high-level syntax elements in the bitstream, indicating whether fusion is enabled for the current sequence or image; under the premise that the high-level is enabled, it further checks whether the CU-level flag of the multi-mode multi-candidate IntraTMP-fusion method 2 is enabled, and the multi-mode multi-candidate IntraTMP-fusion method 2 is considered to be a subset of the traditional IntraTMP mode, that is, only when the traditional IntraTMP is enabled for the current predicted decoding block, it is determined whether the multi-mode multi-candidate IntraTMP-fusion method 2 is enabled; after determining that it is enabled, the best "matching reference block" of different modes is determined according to the fusion flag of the candidate IntraTMP BVs. The syntax structure is shown in Figure 16:

Semantics:

intra_tmp_template_candidates2_fusion_flag: equal to 1 indicates that the current coding block enables multi-mode multi-candidate IntraTMP-fusion method 2, and equal to 0 indicates that the current coding block does not enable this method.

intra_tmp_template_bv_fusion_flag: equal to 1 indicates that the current coding block needs to fuse the candidate IntraTMP BVs in the multi-mode candidate list as the best "matching reference block" of the corresponding mode; equal to 0 indicates that the current coding block does not need to fuse the candidate IntraTMP BVs.

The decoding end uses the same implementation operation as the encoding end to derive the prediction block. The specific implementation process is as follows:

Step 2: Construct candidate lists for each mode respectively, and fuse the reference blocks pointed to by candidates of different modes to determine the best matching reference blocks of different modes.

When the CU flag intra_tmp_template_candidates2_fusion_flag is 1, decoding is completed according to the multi-mode multi-candidate IntraTMP-fusion method 2. First, a candidate list is constructed. The construction process refers to step 3 of the multi-mode multi-candidate IntraTMP-fusion method 2 in Example 4; otherwise, decoding is performed according to the traditional IntraTMP mode.

The candidate IntraTMP BVs fusion flag intra_tmp_template_bv_fusion_flag determines whether the current CU needs to fuse the candidates in the multi-mode candidate list to obtain the best "matching reference block" for the corresponding mode.

Step 3: The best “matching reference blocks” of different modes are fused in the fusion mode of the encoding end to determine the prediction of the current decoding block;

The check before fusion is the same as that at the encoding end. It is also necessary to determine whether the three best "matching reference blocks" meet the fusion conditions, refer to step 4 of Example 2; the fusion method at the decoding end is determined according to the fusion method selected by the encoding end.

FIG17 is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. As shown in FIG17 , the image processing device includes a processor 1710 and a memory 1720. The number of the processor 1710 and the memory 1720 may be one or more. FIG17 takes one processor 1710 and one memory 1720 as an example. The processor 1710 and the memory 1720 in the device may be connected via a bus or other means. FIG17 takes the connection via a bus as an example.

An embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions for executing an image encoding method as provided in an encoding end embodiment of the present application, or an image decoding method as provided in a decoding end embodiment of the present application.

An embodiment of the present application also provides a computer program product, including a computer program or computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions, so that the computer device executes the image encoding method provided in the encoding end embodiment of the present application, or the image decoding method provided in the decoding end embodiment of the present application.

The system architecture and application scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules/units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

In hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. By way of illustration, both applications running on a computing device and a computing device can be components. One or more components may reside in a process or an execution thread, and a component may be located on a computer or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may communicate, for example, through local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems through signals).

The above describes some embodiments of the present application with reference to the accompanying drawings, but does not limit the scope of the present application. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application shall be within the scope of the present application.

Claims (28)

1. An image encoding method, comprising: Acquire a block to be predicted and a predicted block in an image frame, wherein an area corresponding to the predicted block is a reconstructed area; Determine a plurality of target templates corresponding to the coding block to be predicted; Determine a matching reference block corresponding to each of the target templates in the reconstructed area, and fuse at least one of the matching reference blocks to obtain a prediction value corresponding to the to-be-predicted coding block; Obtaining a residual value according to the original value of the to-be-predicted coding block and the predicted value; The residual value is encoded, and the encoded bits are written into a bitstream. 2. The image encoding method according to claim 1, wherein determining a matching reference block corresponding to each target template in the reconstructed area comprises: Determine at least one reference template corresponding to each of the target templates in the reconstructed area; Determine a candidate reference block corresponding to each reference template; A matching reference block is determined for each target template based on the candidate reference blocks. 3. The image encoding method according to claim 2, wherein determining a matching reference block for each target template based on the candidate reference blocks comprises: A candidate reference block is selected from the candidate reference blocks as a matching reference block corresponding to the target template. 4. The image encoding method according to claim 2, wherein determining a matching reference block for each target template based on the candidate reference blocks comprises: Constructing a candidate list corresponding to each target template according to at least one candidate reference block corresponding to the target template; A rate-distortion decision is performed on each candidate reference block pointed to by the block vector in the candidate list to determine a matching reference block corresponding to each target template. 5. The image encoding method according to claim 4, characterized in that the method further comprises: Determine the block vector index value of the matching reference block corresponding to each of the target templates as the intra-frame template fusion information corresponding to the to-be-predicted coding block; The block vector index value is encoded, and the encoded bits are written into a bitstream. 6. The image encoding method according to claim 2, wherein determining a matching reference block for each target template based on the candidate reference blocks comprises: Constructing a candidate list corresponding to each target template according to at least one candidate reference block corresponding to the target template; The candidate reference blocks pointed to by the block vectors in each of the candidate lists are merged to determine a matching reference block corresponding to each of the target templates. 7. The image encoding method according to claim 6, characterized in that the method further comprises: Determine the intra-frame template reference block vector fusion identifier as the intra-frame template fusion information corresponding to the to-be-predicted coding block; The intra-frame template reference block vector fusion identifier is encoded, and the encoded bits are written into a bitstream. 8. The image coding method according to claim 1, wherein the step of fusing at least one of the matching reference blocks to obtain a prediction value corresponding to the block to be predicted comprises: Selecting at least one of the matching reference blocks as a reference block to be fused; All the reference blocks to be merged are merged to obtain the intra-frame prediction value corresponding to the coding block to be predicted. 9. The image encoding method according to claim 8, characterized in that the selecting at least one of the matching reference blocks as the reference block to be fused comprises: When all the matching reference blocks are the same, select any matching reference block as the reference block to be fused; In the case that there are at least two different matching reference blocks, the degree of difference between the reference templates corresponding to the at least two different matching reference blocks is determined according to a preset algorithm, and at least one of the matching reference blocks is selected as the reference block to be fused according to the degree of difference. 10. The image encoding method according to claim 9, wherein there are two different matching reference blocks, and the preset algorithm includes an absolute error sum algorithm; determining the difference between the reference templates corresponding to at least two different matching reference blocks according to the preset algorithm, and selecting at least one matching reference block as a reference block to be fused according to the difference, comprises: Determining the reference templates corresponding to the two matching reference blocks as templates to be compared; Determine the sum of absolute errors corresponding to the two templates to be compared; When the difference between the sum of absolute errors corresponding to the two templates to be compared is greater than a preset threshold, selecting the two matching reference blocks as reference blocks to be fused; When the difference between the sums of absolute errors corresponding to the two templates to be compared is less than or equal to a preset threshold, one of the matching reference blocks is selected as a reference block to be fused. 11. The image encoding method according to claim 9, characterized in that there are at least three matching reference blocks that are different from each other, and the preset algorithm includes an absolute error sum algorithm; determining the difference between the reference templates corresponding to at least two different matching reference blocks according to the preset algorithm, and selecting at least one matching reference block as a reference block to be fused according to the difference, comprises: Determine a reference template and the remaining plurality of non-reference templates among the reference templates corresponding to all the different matching reference blocks; Determine the sum of absolute errors corresponding to the reference template and each of the non-reference templates; The absolute error sum corresponding to the reference template is compared with the absolute error sum corresponding to each of the non-reference templates one by one. When the difference between the absolute error sums corresponding to the two is greater than a preset threshold, the matching reference block corresponding to the reference template and the non-reference template is used as the reference block to be fused; when the difference between the absolute error sums corresponding to the two is less than or equal to the preset threshold, the matching reference block corresponding to the reference template is used as the reference block to be fused. 12. The image encoding method according to claim 8, wherein the step of fusing at least one of the matching reference blocks to obtain an intra-frame prediction value corresponding to the block to be predicted comprises: Determine the target fusion weight corresponding to each target template according to the intra-frame template fusion weight; According to the target fusion weight corresponding to each of the target templates, weighted calculation is performed on the matching reference blocks corresponding to all the target templates to obtain the intra-frame prediction value corresponding to the to-be-predicted coding block. 13. The image encoding method according to claim 12, wherein the intra-frame template fusion weight comprises a preset target fusion weight; and the process of determining the preset target fusion weight comprises: Determining a preset target fusion weight according to the number of samples included in the target template; or, A predetermined preset target fusion weight is set for each target template. 14. The image encoding method according to claim 12, characterized in that the intra-frame template fusion weight is determined according to the absolute error sum and sample quantity corresponding to the target template; the target fusion weight corresponding to each target template is determined according to the intra-frame template fusion weight, comprising: Determine the target absolute error corresponding to each target template and the number of target samples included in each target template; According to the absolute error sum of the target template and the number of target samples, the absolute error sum corresponding to each target template is normalized to obtain the normalized absolute error sum corresponding to each target template, and the normalized absolute error sum is obtained according to all the normalized absolute error sums; The target fusion weight corresponding to each target template is determined according to the normalized absolute error corresponding to each target template and the sum of the normalized absolute errors. 15. The image coding method according to claim 12, characterized in that the intra-frame template fusion weight is determined according to the positional relationship between the pixel to be predicted in the to-be-predicted coding block and the target template; and the target fusion weight corresponding to each target template is determined according to the intra-frame template fusion weight, comprising: Determine a target absolute error sum corresponding to each of the target templates, and determine a weight base value corresponding to each of the target templates according to the target absolute error sum; Determining a target position relationship between the pixel to be predicted and each of the target templates, and determining a relative weight value corresponding to each of the target templates according to the target position relationship; The target fusion weight corresponding to each target template is determined according to the weight base value and the weight relative value. 16 . The image encoding method according to claim 1 , wherein the target template comprises at least one of an L-shape template, an Above template, and a Left template. 17. The image encoding method according to any one of claims 1 to 15, characterized in that the method further comprises: Setting the high-level syntax element fusion flag to on; The high-level syntax element fusion flag set to the on state is encoded, and the encoded bits are written into the bitstream. 18. The image encoding method according to claim 17, characterized in that the method further comprises: Set the block-level intra-frame template fusion flag to on; The block-level intra-frame template fusion flag set to the on state is encoded, and the encoded bits are written into the bitstream. 19. The image encoding method according to any one of claims 1 to 15, characterized in that the method further comprises: Set the block-level intra-frame template fusion flag to on; The block-level intra-frame template fusion flag set to the on state is encoded, and the encoded bits are written into the bitstream. 20. An image decoding method, comprising: Parse the bitstream to obtain the residual value corresponding to the decoded block to be predicted; According to the multiple target templates, a matching reference block corresponding to each target template is determined in the reconstructed area, and at least one matching reference block is fused to obtain a prediction value corresponding to the to-be-predicted decoding block; wherein the area corresponding to the predicted decoding block is the reconstructed area; Obtaining a decoding value corresponding to the to-be-predicted decoding block according to the prediction value of the to-be-predicted decoding block and the residual value; A decoding recovery value of the to-be-predicted decoding block is obtained according to the decoding value. 21. The image decoding method according to claim 20, wherein determining the matching reference block corresponding to each target template in the reconstructed area comprises: Determine at least one reference template corresponding to each of the target templates in the reconstructed area; Determine a candidate reference block corresponding to each reference template; A matching reference block is determined for each target template based on the candidate reference blocks. 22. The image decoding method according to claim 21, characterized in that the step of determining a matching reference block for each target template based on the candidate reference blocks comprises: A candidate reference block is selected from the candidate reference blocks corresponding to each of the target templates as a matching reference block corresponding to the target template. 23. The image decoding method according to claim 21, wherein determining a matching reference block for each target template based on the candidate reference blocks comprises: Constructing a candidate list corresponding to each target template according to at least one candidate reference block corresponding to the target template; A rate-distortion decision is performed on each candidate reference block pointed to by the block vector in the candidate list to determine a matching reference block corresponding to each target template. 24. The image decoding method according to claim 21, wherein determining a matching reference block for each target template based on the candidate reference blocks comprises: Constructing a candidate list corresponding to each target template according to at least one candidate reference block corresponding to the target template; The candidate reference blocks pointed to by the block vectors in each of the candidate lists are merged to determine a matching reference block corresponding to each of the target templates. 25. The image decoding method according to claim 20, wherein the step of fusing at least one of the matching reference blocks to obtain a prediction value corresponding to the to-be-predicted decoding block comprises: Selecting at least one of the matching reference blocks as a reference block to be fused; All the reference blocks to be merged are merged to obtain the intra-frame prediction value corresponding to the decoded block to be predicted. 26. An image processing device, comprising: at least one processor; at least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the image encoding method according to any one of claims 1 to 19, or the image decoding method according to any one of claims 20 to 32 is implemented. 27. A computer-readable storage medium, characterized in that a processor-executable program is stored therein, and when the processor-executable program is executed by the processor, it is used to implement the image encoding method as described in any one of claims 1 to 19, or the image decoding method as described in any one of claims 20 to 25. 28. A computer program product, comprising a computer program or a computer instruction, characterized in that the computer program or the computer instruction is stored in a computer-readable storage medium, a processor of a computer device reads the computer program or the computer instruction from the computer-readable storage medium, and the processor executes the computer program or the computer instruction, so that the computer device executes the image encoding method as described in any one of claims 1 to 19, or the image decoding method as described in any one of claims 20 to 25.