CN116250240A - Image coding method, image decoding method and related device - Google Patents

Abstract

An image encoding method, an image decoding method and a related device, wherein the image encoding method comprises the following steps: determining intra-frame prediction filtering indication information of a current coding block; if the fact that the current coding block needs to use the first intra-frame prediction filtering mode is determined according to the intra-frame prediction filtering indication information, a first use identification bit of the first intra-frame prediction filtering mode of the current coding block is set to be allowed to be used; writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode and the first use identification bit into a code stream; and according to the first intra-frame prediction filtering mode, predicting the current coding block to obtain a prediction block, coding the prediction block, and writing the coding result into the code stream. The method provides selection for operations such as smoothing processing or local blurring for intra prediction, and for parts of image textures which do not need to be sharpened, so that predicted pixels are smoother, predicted blocks are closer to an original image, and finally, the coding efficiency is improved.

Description

Image coding method, image decoding method and related device technical field

The present application relates to the technical field of electronic equipment, and in particular to an image encoding method, an image decoding method, and related devices.

Background technique

Digital video capabilities can be incorporated into a wide range of devices, including digital television, digital broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, tablet computers, electronic books Readers, digital cameras, digital recording devices, digital media players, video game devices, video game consoles, cellular or satellite radiotelephones, video conferencing devices, video streaming devices, and more.

Digital video devices implement video compression techniques such as those described in Moving Picture Experts Group (MPEG)-2, MPEG-4, ITU-TH.263, ITU-TH.264/MPEG-4 Part 10 Advanced Video Coding Decoding (advanced video coding, AVC), ITU-TH.265 high efficiency video coding (high efficiency video coding, HEVC) standards defined in the standard and those video compression techniques described in the extensions of the standard, so that more efficient transmit and receive digital video information. Video devices may transmit, receive, encode, decode, and/or store digital video information more efficiently by implementing these video codec techniques.

With the rapid increase of Internet video, despite the continuous evolution of digital video compression technology, higher requirements are still placed on the video compression ratio.

Contents of the invention

The embodiment of the present application provides an image encoding method, an image decoding method, and related devices, in order to provide options for operations such as smoothing or local blurring for intra prediction. For the part of the image texture that does not need to be sharpened, use this The technology makes the predicted pixels smoother, and the predicted blocks are closer to the original image, which will eventually improve the coding efficiency.

In the first aspect, the embodiment of the present application provides an image coding method, including:

Determine the intra-frame prediction filtering indication information of the current coding block, the intra-frame prediction filtering indication information includes first indication information and second indication information, and the first indication information is used to indicate whether the first intra-frame prediction filtering mode is allowed to be used , the second indication information is used to indicate whether the second intra-frame prediction filtering mode is allowed to be used, and the first intra-frame prediction filtering mode is an intra-frame prediction filtering IPF mode;

If it is determined according to the intra-frame prediction filtering indication information that the current coding block needs to use the first intra-frame prediction filtering mode, set the first use flag of the first intra-frame prediction filtering mode of the current coding block bit is set to the first value;

Writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first use flag into a code stream;

According to the first intra-frame prediction filtering mode, the current coded block is predicted to obtain a predicted block, the predicted block is coded, and the coded result is written into a code stream.

Compared with the existing technology, the solution of this application provides options for intra-frame prediction that require smoothing or local blurring. For the part of the image texture that does not need to be sharpened, using this technology makes the prediction pixels smoother and the prediction block more accurate. Closer to the original image, which will eventually improve coding efficiency.

In a second aspect, an embodiment of the present application provides an image decoding method, including:

Parsing the code stream to determine the intra-frame prediction filtering indication information and the first use flag of the current decoding block, the intra-frame prediction indication information includes the first indication information and the second indication information, and the first indication information is used to indicate whether The first intra-frame prediction filtering mode is allowed to be used, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is an intra-frame prediction filtering IPF mode, and the second A use flag bit is the use flag bit of the first intra-frame prediction filtering mode;

According to the intra-frame prediction filtering indication information and the first use flag, determine to use the first frame rate prediction filtering mode to predict the current decoding block, and obtain a prediction block of the current decoding block.

Compared with the existing technology, the solution of this application provides options for intra-frame prediction that require smoothing or local blurring. For the part of the image texture that does not need to be sharpened, using this technology makes the prediction pixels smoother and the prediction block more accurate. Closer to the original image, which will eventually improve coding efficiency.

In a third aspect, the embodiment of the present application provides an image encoding device, including:

A determining unit, configured to determine intra-frame prediction filtering indication information of the current coding block, where the intra-frame prediction filtering indication information includes first indication information and second indication information, and the first indication information is used to indicate whether the first indication information is allowed to be used. Intra-frame prediction filtering mode, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is intra-frame prediction filtering IPF mode;

a setting unit, configured to set the first intra prediction filtering mode of the current coding block to The first use identification bit of is set to the first value;

A transmission unit, configured to write the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first usage flag into a code stream;

A superposition unit, configured to determine to use the first frame rate predictive filter mode to predict the current decoded block according to the intra-frame predictive filter indication information and the first use flag, to obtain the current decoded block prediction block.

In a fourth aspect, the embodiment of the present application provides an image decoding device, including:

The parsing unit is configured to determine the intra-frame prediction filtering indication information and the first use flag of the current decoding block, the intra-frame prediction indication information includes first indication information and second indication information, and the first indication information is used to indicate Whether to allow the use of the first intra-frame prediction filtering mode, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is the intra-frame prediction filtering IPF mode, the The first usage flag is the usage flag of the first intra prediction filtering mode;

A determining unit, configured to determine to use the first frame rate predictive filtering mode to predict the current decoding block according to the intra-frame prediction filtering indication information and the first use flag, and obtain the current decoding block prediction block.

In a fifth aspect, an embodiment of the present application provides an encoder, including: a processor and a memory coupled to the processor; the processor is configured to execute the method described in the first aspect above.

In a sixth aspect, an embodiment of the present application provides a decoder, including: a processor and a memory coupled to the processor; the processor is configured to execute the method described in the second aspect above.

In a seventh aspect, an embodiment of the present application provides a terminal, where the terminal includes: one or more processors, a memory, and a communication interface; the memory and the communication interface are connected to the one or more processors; The terminal communicates with other devices through the communication interface, and the memory is used to store computer program codes, the computer program codes include instructions, and when the one or more processors execute the instructions, the terminal executes The method as described in the first aspect or the second aspect.

In the eighth aspect, the embodiment of the present application provides a computer-readable storage medium, and instructions are stored in the computer-readable storage medium, and when the instructions are run on a computer, the computer executes the above-mentioned first aspect or the second aspect. method described in the aspect.

In a ninth aspect, the embodiments of the present application provide a computer program product containing instructions, which, when run on a computer, cause the computer to execute the method described in the first aspect or the second aspect.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic block diagram of a coding tree unit in an embodiment of the present application;

FIG. 2 is a schematic block diagram of a CTU and a coding block CU in an embodiment of the present application;

FIG. 3 is a schematic block diagram of a color format in an embodiment of the present application;

Fig. 4 is the schematic diagram of IPF in the embodiment of the present application;

FIG. 5 is a schematic diagram of intra-frame prediction filtering in an embodiment of the present application;

FIG. 6 is a schematic block diagram of a video decoding system in an embodiment of the present application;

FIG. 7 is a schematic block diagram of a video encoder in an embodiment of the present application;

FIG. 8 is a schematic block diagram of a video decoder in an embodiment of the present application;

FIG. 9 is a schematic flowchart of an image encoding method in an embodiment of the present application;

FIG. 10 is a schematic flowchart of an image decoding method in an embodiment of the present application;

FIG. 11A is a schematic diagram of a filling of the prediction block in the embodiment of the present application;

FIG. 11B is another schematic diagram of filling the prediction block in the embodiment of the present application;

FIG. 12 is a block diagram of a functional unit of an image encoding device in an embodiment of the present application;

FIG. 13 is a block diagram of another functional unit of an image encoding device in an embodiment of the present application;

FIG. 14 is a block diagram of a functional unit of an image decoding device in an embodiment of the present application;

FIG. 15 is a block diagram of another functional unit of an image decoding device in an embodiment of the present application.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

It can be understood that the terms "first", "second", etc. used in the present invention may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, a first client could be termed a second client, and, similarly, a second client could be termed a first client, without departing from the scope of the present invention. Both the first client and the second client are clients, but they are not the same client.

Firstly, the terms used in the embodiments of this application are introduced.

For image division, in order to more flexibly represent video content, High Efficiency Video Coding standard (HEVC) technology defines coding tree unit (coding tree unit, CTU), coding block (Coding Unit, CU) , a prediction unit (Prediction Unit, PU) and a transformation unit (Transform Unit, TU). CTU, CU, PU, and TU are image blocks.

Coding tree unit CTU, an image is composed of multiple CTUs, and a CTU usually corresponds to a square image area, including luma pixels and chrominance pixels in this image area (or may only contain luma pixels, or may only contain Chroma pixels); the CTU also contains syntax elements, which indicate how to divide the CTU into at least one coding unit (coding unit, CU), and a method for decoding each coding block to obtain a reconstructed image. As shown in FIG. 1 , an image 10 is composed of multiple CTUs (including CTU A, CTU B, CTU C, etc.). The coding information corresponding to a certain CTU includes luminance values and/or chrominance values of pixels in the square image area corresponding to the CTU. In addition, the coding information corresponding to a certain CTU may further include syntax elements, which indicate how to divide the CTU into at least one CU, and a method of decoding each CU to obtain a reconstructed image. The image area corresponding to one CTU may include 64×64, 128×128 or 256×256 pixels. In one example, a CTU of 64×64 pixels includes a rectangular pixel matrix consisting of 64 columns of 64 pixels each, and each pixel includes a luma component and/or a chrominance component. A CTU can also correspond to a rectangular image area or an image area of other shapes, and an image area corresponding to a CTU can also be an image area in which the number of pixels in the horizontal direction is different from the number of pixels in the vertical direction, for example, including 64×128 pixels .

Coding block CU, as shown in Figure 2, a coding tree unit CTU can be further divided into coding blocks CU, usually corresponding to an A×B rectangular area in the image, including A×B luma pixels or/and its corresponding chrominance Pixels, A is the width of the rectangle, B is the height of the rectangle, A and B can be the same or different, the value of A and B is usually an integer power of 2, such as 128, 64, 32, 16, 8, 4. Wherein, the width involved in the embodiment of the present application refers to the length along the X-axis direction (horizontal direction) in the two-dimensional rectangular coordinate system XoY shown in Figure 1, and the height refers to the two-dimensional rectangular coordinate system XoY shown in Figure 1 The length along the Y-axis direction (vertical direction). The reconstructed image of a CU can be obtained by adding the predicted image and the residual image. The predicted image is generated by intra-frame prediction or inter-frame prediction. Specifically, it can be composed of one or more prediction blocks (prediction blocks, PB). The residual image is obtained by The transform coefficients are generated by inverse quantization and inverse transform processing, and may specifically be composed of one or more transform blocks (transform block, TB). Specifically, a CU includes encoding information, and the encoding information includes information such as prediction mode and transformation coefficient. According to the encoding information, corresponding decoding processes such as prediction, inverse quantization, and inverse transformation are performed on the CU to generate a reconstructed image corresponding to the CU. The relationship between coding tree units and coding blocks is shown in FIG. 3 .

The prediction unit PU is the basic unit of intra prediction and inter prediction. The motion information defining the image block includes inter-frame prediction direction, reference frame, motion vector, etc. The image block being encoded is called the current coding block (CCB), and the image block being decoded is called the current decoding block. block (current decoding block, CDB), for example, when an image block is being predicted, the current coding block or the current decoding block is the prediction block; when the residual processing is being performed on an image block, the current coding block or the current decoding block is Transform blocks. The picture in which the current coding block or the current decoding block is located is called the current frame. In the current frame, the image block located on the left or upper side of the current block (based on the coordinate system in Figure 1, the left side refers to the negative direction of the X axis, and the upper side refers to the positive direction of the Y axis) may be inside the current frame and has After the encoding/decoding process is completed, reconstructed images are obtained, which are called reconstructed blocks; information such as the encoding mode and reconstructed pixels of the reconstructed blocks are available. A frame that has been encoded/decoded before the current frame is encoded/decoded is called a reconstructed frame. When the current frame is a unidirectional predictive frame (P frame) or a bidirectional predictive frame (B frame), it has one or two reference frame lists respectively, the two lists are called L0 and L1, and each list contains at least one reconstruction frame, called the reference frame of the current frame. The reference frame provides reference pixels for the inter prediction of the current frame.

The transform unit TU processes the residual of the original image block and the predicted image block.

A pixel (also known as a pixel point) refers to a pixel point in an image, such as a pixel point in a coding block, a pixel point in a luminance component pixel block (also called a luminance pixel), or a pixel point in a chrominance component pixel block (Also known as chroma pixels) and so on.

Sample (also known as pixel value, sample value) refers to the pixel value of a pixel point. The pixel value specifically refers to brightness (that is, gray scale value) in the luminance component domain, and the pixel value specifically refers to Chroma values (ie, color and saturation), according to different processing stages, a pixel sample specifically includes an original sample, a predicted sample, and a reconstructed sample.

Direction Description: Horizontal direction, for example: in the two-dimensional Cartesian coordinate system XoY as shown in Figure 1, along the X-axis direction, vertical direction, for example: in the two-dimensional Cartesian coordinate system XoY shown in Figure 1, along the negative direction of the Y-axis .

Intra-frame prediction generates a predicted image of the current block based on the spatially adjacent pixels of the current block. An intra prediction mode corresponds to a method of generating a predicted image. The division of the intra prediction unit includes 2N×2N division method and N×N division method. The 2N×2N division method does not divide the image block; the N×N division method divides the image block into four sub-images of equal size. piece.

Usually, digital video compression technology works on video sequences whose color encoding method is YCbCr, also called YUV, and the color format is 4:2:0, 4:2:2 or 4:4:4. Among them, Y represents the brightness (Luminance or Luma), that is, the gray scale value, Cb represents the blue chroma component, Cr represents the red chroma component, U and V represent the chroma (Chrominance or Chroma), which are used to describe color and saturation Spend. In terms of color format, 4:2:0 means that every 4 pixels have 4 luminance components and 2 chroma components (YYYYCbCr), 4:2:2 means that every 4 pixels have 4 luminance components and 4 chroma components Component (YYYYCbCrCbCr), and 4:4:4 means full-pixel display (YYYYCbCrCbCrCbCrCbCr), Figure 3 shows the distribution of each component in different color formats, where the white circle is the Y component, and the gray circle is the UV component.

The intra-frame prediction part in digital video coding and decoding mainly refers to the adjacent block image information of the current frame to predict the current coding unit block. process, and transmit the residual information to the decoder. After receiving and parsing the code stream, the decoding end obtains the residual information through steps such as inverse transformation and inverse quantization, and superimposes the residual information on the predicted image block predicted by the decoding end to obtain the reconstructed image block. In this process, intra-frame prediction usually predicts the current coding block with the help of respective angle mode and non-angle mode to obtain the prediction block, and selects the optimal prediction mode of the current coding unit according to the rate-distortion information calculated by the prediction block and the original block , and then write the prediction mode into the code stream to the decoder. The decoding end analyzes the prediction mode, predicts the predicted image of the current decoded block, and superimposes the residual pixels written into the code stream to obtain the reconstructed image.

After the development of digital video coding and decoding standards in the past, the non-angle mode remains relatively stable, and there are mean mode and flat mode; the angle mode continues to increase with the evolution of digital video coding and decoding standards. Take the international digital video coding standard H series as an example , the H.264/AVC standard only has 8 angle prediction modes and 1 non-angle prediction mode; H.265/HEVC is extended to 33 angle prediction modes and 2 non-angle prediction modes; and the latest general video coding standard H .266/VVC adopts 67 prediction modes, among which 2 non-angle prediction modes are reserved, and the angle modes are extended from 33 to 65 in H.265. Undoubtedly, with the increase of the angle mode, the intra-frame prediction will be more accurate, and it will be more in line with the current society's demand for high-definition and ultra-high-definition video development. Not only the international standard, but the domestic digital audio and video coding standard AVS3 also continues to expand the angle mode and non-angle mode. The development of ultra-high-definition digital video puts forward higher requirements for intra-frame prediction, and we cannot just rely on simply increasing the angle prediction mode. , extending the wide angle to commit coding efficiency. Therefore, the domestic digital audio and video coding standard AVS3 adopts the intra prediction filter technology (IPF, intra prediction filter). The correlation between the current coding units, the intra-frame prediction filtering technology improves the pixel prediction accuracy through point-to-point filtering, which can effectively enhance the spatial correlation, thereby improving the intra-frame prediction accuracy. The IPF technology takes the prediction mode from upper right to lower left in AVS3 as an example, as shown in Figure 4, where URB indicates the boundary pixel of the left adjacent block close to the current coding unit, and MRB indicates the boundary pixel of the upper adjacent block close to the current coding unit. The boundary pixel of the coding unit, filter direction indicates the filtering direction. The direction of the prediction mode is from upper right to lower left, and the generated prediction value of the current coding unit mainly uses the reference pixel points of the adjacent block in the upper MRB row, that is, the prediction pixel of the current coding unit does not refer to the reconstructed pixel of the adjacent block on the left, However, the current coding unit and the reconstruction block on the left are spatially adjacent. If only the upper MRB pixels are referred to instead of the left URB pixels, the spatial correlation is likely to be lost and the prediction effect will be poor.

The IPF technology is applied to all prediction modes of intra-frame prediction, and is a filtering method to improve the accuracy of intra-frame prediction. IPF technology is mainly implemented through the following processes:

a) IPF technology judges the current prediction mode of the coding unit, and divides it into horizontal angle prediction mode, vertical angle prediction mode and non-angle prediction mode;

b) According to different types of prediction modes, the IPF technology uses different filters to filter the input pixels;

c) According to the different distances from the current pixel to the reference pixel, the IPF technology uses different filter coefficients to filter the input pixels;

The input pixels of the IPF technology are predicted pixels obtained in each prediction mode, and the output pixels are the final predicted pixels after IPF.

The IPF technology has an enable flag ipf_enable_flag, a binary variable, a value of '1' indicates that intra-frame prediction filtering can be used; a value of '0' indicates that intra-frame prediction filtering should not be used. At the same time, IPF technology also uses the identification bit ipf_flag, a binary variable, the value of '1' indicates that intra-frame prediction filtering should be used; the value of '0' indicates that intra-frame prediction filtering should not be used, if the identification bit ipf_flag does not exist in the code stream , the default is 0.

The syntax element IPF_flag is as follows:

The above-mentioned IPF technique classifies prediction modes 0, 1 and 2 as non-angular prediction modes, and uses a first three-tap filter to filter the prediction pixels;

Classify prediction modes 3 to 18, prediction modes 34 to 50 as vertical angle prediction modes, and use the first two-tap filter to filter the prediction pixels;

Prediction modes 19 to 32 and prediction modes 51 to 65 are classified as horizontal angle prediction modes, and the second two-tap filter is used to filter the prediction pixels.

The above-mentioned first three-tap filter applicable to IPF technology, the filtering formula is as follows:

P'(x,y)=f(x)·P(-1,y)+f(y)·P(x,-1)+(1-f(x)-f(y))·P( x,y)

The above-mentioned first two-tap filter applicable to IPF technology, the filtering formula is as follows:

P'(x,y)=f(x)·P(-1,y)+(1-f(x))·P(x,y)

The above-mentioned second two-tap filter applicable to IPF technology, the filtering formula is as follows:

P'(x,y)=f(y)·P(x,-1)+(1-f(y))·P(x,y)

In the above formula, P'(x, y) is the final prediction value of the pixel at (x, y) position of the current chroma prediction block, and f(x) and f(y) are the reconstructed pixels referring to the left adjacent block P(-1,y) and P(x,-1) are the left reconstruction pixel in row y and the upper focus pixel in column x, respectively. P(x,y) is the original predicted pixel value in the current chroma component prediction block. Wherein, the values of x and y do not exceed the value ranges of the width and height of the current coding unit block.

The values of the above-mentioned horizontal filter coefficients and vertical filter coefficients are related to the size of the current coding unit block and the distance from the prediction pixel in the current prediction block to the reconstruction pixel on the left and the reconstruction pixel on the upper side. The values of the above-mentioned horizontal filter coefficients and vertical filter coefficients are also related to the size of the current coding block, and are divided into different filter coefficient groups according to the size of the current coding unit block.

Table 1 shows the filter coefficients of IPF technology.

Table 1 Intra-frame chroma prediction filter coefficients

Figure 5 shows the schematic diagrams of three filtering situations of intra prediction filtering, which only refer to the upper reference pixel to filter the predicted value in the current coding unit; only refer to the left reference pixel to predict the measured value of the current coding unit performing filtering; and filtering the predicted value in the current coding unit block with reference to the upper and left reference pixels.

FIG. 6 is a block diagram of an example video decoding system 1 described in the embodiment of this application. As used herein, the term "video coder" refers generally to both video encoders and video decoders. In this application, the term "video coding" or "coding" may generally refer to video encoding or video decoding. The video encoder 100 and the video decoder 200 of the video decoding system 1 are used to implement the image encoding method proposed in this application.

As shown in FIG. 6 , a video coding system 1 includes a source device 10 and a destination device 20 . Source device 10 generates encoded video data. Accordingly, source device 10 may be referred to as a video encoding device. Destination device 20 may decode the encoded video data generated by source device 10 . Accordingly, destination device 20 may be referred to as a video decoding device. Various implementations of source device 10, destination device 20, or both may include one or more processors and memory coupled to the one or more processors. The memory may include, but is not limited to, RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures accessible by the computer, as described herein.

Source device 10 and destination device 20 may include a variety of devices, including desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, telephone handsets such as so-called "smart" phones, etc. , televisions, cameras, display devices, digital media players, video game consoles, in-vehicle computers, or the like.

Destination device 20 may receive encoded video data from source device 10 via link 30 . Link 30 may include one or more media or devices capable of moving encoded video data from source device 10 to destination device 20 . In one example, link 30 may include one or more communication media that enable source device 10 to transmit encoded video data directly to destination device 20 in real time. In this example, source device 10 may modulate the encoded video data according to a communication standard, such as a wireless communication protocol, and may transmit the modulated video data to destination device 20 . The one or more communication media may include wireless and/or wired communication media, such as a radio frequency (RF) spectrum or one or more physical transmission lines. The one or more communication media may form part of a packet-based network, such as a local area network, a wide area network, or a global network (eg, the Internet). The one or more communication media may include routers, switches, base stations, or other equipment that facilitates communication from source device 10 to destination device 20 . In another example, encoded data may be output from output interface 140 to storage device 40 .

The image coding and decoding technology of the present application can be applied to video coding and decoding to support various multimedia applications, such as over-the-air TV broadcasting, cable TV transmission, satellite TV transmission, streaming video transmission (for example, via the Internet), for storing in data storage Encoding of video data on a medium, decoding of video data stored on a data storage medium, or other applications. In some examples, the video coding system 1 can be used to support one-way or two-way video transmission to support applications such as video streaming, video playback, video broadcasting, and/or video telephony.

The video coding system 1 illustrated in FIG. 6 is merely an example, and the techniques of this application are applicable to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between an encoding device and a decoding device. . In other instances, data is retrieved from local storage, streamed over a network, and so on. A video encoding device may encode and store data to memory, and/or a video decoding device may retrieve and decode data from memory. In many instances, encoding and decoding are performed by devices that do not communicate with each other but merely encode and/or retrieve data from memory and decode data to memory.

In the example of FIG. 6 , source device 10 includes video source 120 , video encoder 100 and output interface 140 . In some examples, output interface 140 may include a conditioner/demodulator (modem) and/or a transmitter. Video source 120 may include a video capture device (e.g., a video camera), a video archive containing previously captured video data, a video feed interface for receiving video data from a video content provider, and/or a computer for generating video data A graphics system, or a combination of such sources of video data.

Video encoder 100 may encode video data from video source 120 . In some examples, source device 10 transmits the encoded video data directly to destination device 20 via output interface 140 . In other examples, the encoded video data may also be stored on storage device 40 for later access by destination device 20 for decoding and/or playback.

In the example of FIG. 6 , the destination device 20 includes an input interface 240 , a video decoder 200 and a display device 220 . In some examples, input interface 240 includes a receiver and/or a modem. Input interface 240 may receive encoded video data via link 30 and/or from storage device 40 . The display device 220 may be integrated with the destination device 20 or may be external to the destination device 20 . In general, display device 220 displays decoded video data. The display device 220 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

Although not shown in FIG. 6 , in some aspects video encoder 100 and video decoder 200 may be integrated with an audio encoder and decoder, respectively, and may include appropriate multiplexer-demultiplexer units. or other hardware and software to handle the encoding of both audio and video in a common data stream or in separate data streams.

Video encoder 100 and video decoder 200 may each be implemented as any of a variety of circuits such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), Field Programmable Gate Array (FPGA), discrete logic, hardware, or any combination thereof. If the application is implemented partially in software, a device may store instructions for the software in a suitable non-transitory computer-readable storage medium and may execute the instructions in hardware using one or more processors Thereby implementing the technology of the present application. Any of the foregoing (including hardware, software, a combination of hardware and software, etc.) may be considered to be one or more processors. Each of video encoder 100 and video decoder 200 may be included in one or more encoders or decoders, either of which may be integrated as a combined encoder in a corresponding device /decoder (codec) part.

Fig. 7 is an example block diagram of a video encoder 100 described in the embodiment of the present application. The video encoder 100 is used to output the video to the post-processing entity 41 . Post-processing entity 41 represents an example of a video entity that may process encoded video data from video encoder 100, such as a media-aware network element (MANE) or a splicing/editing device. In some cases, post-processing entity 41 may be an instance of a network entity. In some video encoding systems, post-processing entity 41 and video encoder 100 may be parts of a single device, while in other cases the functionality described with respect to post-processing entity 41 may be provided by the same device including video encoder 100 implement. In a certain example, post-processing entity 41 is an instance of storage device 40 of FIG. 1 .

In the example of FIG. 7 , video encoder 100 includes prediction processing unit 108 , filter unit 106 , memory 107 , summer 112 , transformer 101 , quantizer 102 , and entropy encoder 103 . The prediction processing unit 108 includes an inter predictor 110 and an intra predictor 109 . For image block reconstruction, the video encoder 100 also includes an inverse quantizer 104 , an inverse transformer 105 and a summer 111 . Filter unit 106 represents one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 106 is shown in FIG. 7 as an in-loop filter, in other implementations filter unit 106 may be implemented as a post-loop filter. In an example, the video encoder 100 may further include a video data storage and a segmentation unit (not shown in the figure).

Video encoder 100 receives video data and stores the video data in a video data memory. The segmentation unit divides the video data into several image blocks, and these image blocks can be further divided into smaller blocks, such as image block segmentation based on quadtree structure or binary tree structure. Prediction processing unit 108 may select one of a plurality of possible coding modes for the current image block, such as one of a plurality of intra coding modes or one of a plurality of inter coding modes. Prediction processing unit 108 may provide the resulting intra-, inter-coded blocks to summer 112 to produce a residual block, and to summer 111 to reconstruct the encoded block used as a reference image. The intra predictor 109 in the prediction processing unit 108 may perform intra predictive coding of the current image block relative to one or more adjacent coding blocks in the same frame or slice as the current block to be coded, so as to remove spatial redundancy. Remain. The inter predictor 110 within the prediction processing unit 108 may perform inter predictive encoding of the current image block relative to one or more prediction blocks in one or more reference images to remove temporal redundancy. The prediction processing unit 108 provides the information indicating the selected intra or inter prediction mode of the current image block to the entropy encoder 103 so that the entropy encoder 103 encodes the information indicating the selected inter prediction mode.

After the prediction processing unit 108 generates the prediction block of the current image block via inter prediction/intra prediction, the video encoder 100 forms a residual image block by subtracting the prediction block from the current image block to be encoded. Summer 112 represents one or more components that perform this subtraction operation. The residual video data in the residual block may be contained in one or more TUs and applied to the transformer 101 . A transformer 101 transforms the residual video data into residual transform coefficients using a transform such as a discrete cosine transform (DCT) or a conceptually similar transform. Transformer 101 may convert residual video data from a pixel value domain to a transform domain, such as a frequency domain.

Transformer 101 may send the resulting transform coefficients to quantizer 102 . Quantizer 102 quantizes the transform coefficients to further reduce the bit rate. In some examples, quantizer 102 may then perform a scan of the matrix including the quantized transform coefficients. Alternatively, entropy encoder 103 may perform a scan.

After quantization, the entropy encoder 103 entropy encodes the quantized transform coefficients. For example, the entropy encoder 103 may perform context-adaptive variable-length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), syntax-based context-adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE ) encoding or another entropy encoding method or technique. After entropy encoding by entropy encoder 103 , the encoded codestream may be transmitted to video decoder 200 , or archived for later transmission or retrieval by video decoder 200 . The entropy encoder 103 may also perform entropy encoding on syntax elements of the current image block to be encoded.

The inverse quantizer 104 and inverse transformer 105 apply inverse quantization and inverse transformation respectively to reconstruct the residual block in the pixel domain, eg for later use as a reference block of a reference image. The summer 111 adds the reconstructed residual block to the prediction block generated by the inter predictor 110 or the intra predictor 109 to generate a reconstructed image block. The filter unit 106 may be applied to the reconstructed image blocks to reduce artifacts, such as block artifacts. This reconstructed image block is then stored in memory 107 as a reference block that may be used by inter predictor 110 as a reference block for inter prediction of blocks in subsequent video frames or images.

The video encoder 100 divides an input video into several coding tree units, and each coding tree unit is further divided into several rectangular or square coding blocks. When the current encoding block selects the intra prediction mode for encoding, the calculation and traversal of several prediction modes are performed on the luminance component of the current encoding block, and the optimal prediction mode is selected according to the rate-distortion cost, and several prediction modes are performed on the chrominance component of the current encoding block. The calculation of the prediction mode traverses and selects the optimal prediction mode according to the rate-distortion cost. Afterwards, the residual between the original video block and the predicted block is calculated, and the residual is subsequently transformed, quantized, entropy encoded, etc. to form an output code stream, and the other path is inversely transformed, inversely quantized, and loop filtered to form a reconstructed sample As reference information for subsequent video compression.

The specific implementation of the current IPF technology in the video encoder 100 is as follows.

The input digital video information is divided into several coding tree units at the coding end, and each coding tree unit is divided into several or rectangular or square coding units, and each coding unit performs intra-frame prediction process to calculate the predicted block.

In the current code unit,

① If the allowable flag of IPF is '1', perform all the following steps;

② If the permission flag of IPF is '0', only a1), b1), f1 and g1) steps are performed.

a1) Intra prediction first traverses all prediction modes, calculates the predicted pixels in each intra prediction mode, and calculates the rate-distortion cost based on the original pixels;

b1) Select the optimal prediction mode of the current coding unit according to the minimum rate-distortion cost principle of all the above-mentioned prediction modes, and record the optimal prediction mode information and the corresponding rate-distortion cost information;

c1) Traversing all intra prediction modes again, this process starts the IPF technology, first calculates the prediction pixels in each intra prediction mode, and obtains the prediction block of the current coding unit;

d1) Perform IPF on the predicted block of the current coding unit, select the corresponding filter according to the current prediction mode, and select the corresponding filter coefficient group according to the size of the current coding unit. The specific corresponding relationship can be found in Table 1;

e1) Calculate the rate-distortion cost information of each prediction mode according to the final prediction pixel and original pixel obtained by the above-mentioned IPF technology, and record the prediction mode and corresponding cost value of the minimum rate-distortion cost information;

f1) If the IPF allows the identification bit to be '0', then write the prediction mode index recorded in b1) into the code stream to the decoder;

If the IPF allows the flag to be '1', compare the minimum cost value recorded in b1) with the minimum cost value recorded in e1),

If the rate-distortion cost in b1) is smaller, then encode the prediction mode index recorded in b1) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and use the flag position of the current coding unit of IPF to identify the position No , indicating that the IPF technology is not used, and the code stream is also written to the decoder;

If the rate-distortion in e1) is smaller, then encode the prediction mode index recorded in e1) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, set the current coding unit identification position of IPF to true, Indicates that IPF technology is used, and the code stream is also written to the decoder.

g) Afterwards, the prediction value is superimposed on the residual information after operations such as transformation and quantization to obtain the reconstructed block of the current coding unit as reference information of the subsequent coding unit.

The intra predictor 109 may also provide the information indicating the selected intra prediction mode of the currently encoded block to the entropy encoder 103 so that the entropy encoder 103 encodes the information indicating the selected intra prediction mode.

FIG. 8 is an example block diagram of a video decoder 200 described in the embodiment of the present application. In the example of FIG. 8 , video decoder 200 includes entropy decoder 203 , prediction processing unit 208 , inverse quantizer 204 , inverse transformer 205 , summer 211 , filter unit 206 and memory 207 . The prediction processing unit 208 may include an inter predictor 210 and an intra predictor 209 . In some examples, video decoder 200 may perform a decoding process that is substantially reciprocal to the encoding process described with respect to video encoder 100 from FIG. 7 .

During decoding, video decoder 200 receives from video encoder 100 an encoded video bitstream representing image blocks of an encoded video slice and associated syntax elements. The video decoder 200 may receive video data from the network entity 42, and optionally, store the video data in a video data storage (not shown in the figure). The video data memory may store video data, such as an encoded video code stream, to be decoded by the components of the video decoder 200 . The video data stored in the video data storage may be obtained, for example, from the storage device 40, from a local video source such as a camera, via wired or wireless network communication of video data, or by accessing a physical data storage medium. The video data memory may serve as a decoded picture buffer (CPB) for storing encoded video data from an encoded video bitstream.

Network entity 42 may be, for example, a server, MANE, video editor/slicer, or other such device for implementing one or more of the techniques described above. Network entity 42 may or may not include a video encoder, such as video encoder 100 . Before network entity 42 sends the encoded video codestream to video decoder 200 , network entity 42 may implement portions of the techniques described in this application. In some video decoding systems, network entity 42 and video decoder 200 may be part of a separate device, while in other cases the functionality described with respect to network entity 42 may be performed by the same device including video decoder 200 .

The entropy decoder 203 of the video decoder 200 entropy-decodes the codestream to generate quantized coefficients and some syntax elements. Entropy decoder 203 forwards the syntax elements to prediction processing unit 208 . Video decoder 200 may receive syntax elements at a video slice level and/or a tile level. When a video slice is decoded as an intra-decoded (I) slice, the intra predictor 209 of the prediction processing unit 208 is based on the signaled intra prediction mode and data from previously decoded blocks of the current frame or picture And the prediction block of the image block of the current video slice is generated. When a video slice is decoded as an inter-decoded (ie, B or P) slice, the inter predictor 210 of the prediction processing unit 208 may determine, based on the syntax elements received from the entropy decoder 203 , the An inter-frame prediction mode for decoding the current image block of the video slice, based on the determined inter-frame prediction mode, the current image block is decoded (for example, inter-frame prediction is performed).

The inverse quantizer 204 inverse quantizes, ie dequantizes, the quantized transform coefficients provided in the code stream and decoded by the entropy decoder 203 . The inverse quantization process may include using quantization parameters calculated by the video encoder 100 for each image block in the video slice to determine the degree of quantization that should be applied and likewise determining the degree of inverse quantization that should be applied. An inverse transformer 205 applies an inverse transform, such as an inverse DCT, an inverse integer transform, or a conceptually similar inverse transform process, to the transform coefficients to produce a residual block in the pixel domain.

After the inter predictor 210 generates a prediction block for the current image block or a sub-block of the current image block, the video decoder 200 predicts The blocks are summed to obtain a reconstructed block, ie a decoded image block. Summer 211 represents the component that performs this summing operation. A loop filter (either in the decoding loop or after the decoding loop) may also be used to smooth pixel transitions or otherwise improve video quality, if desired. Filter unit 206 may represent one or more loop filters, such as a deblocking filter, an adaptive loop filter (ALF), and a sample adaptive offset (SAO) filter. Although filter unit 206 is shown in FIG. 8 as an in-loop filter, in other implementations filter unit 206 may be implemented as a post-loop filter.

The image decoding method specifically executed by the video decoder 200 includes: after parsing, inverse transformation and inverse quantization of the input code stream, the prediction mode index of the current coding block is obtained. If the prediction mode index of the chrominance component of the current coding block is an enhanced two-step cross-component prediction mode, then according to the index value, only the reconstructed samples from the upper or left adjacent pixels of the current coding block are selected to calculate the linear model According to the linear model, the original prediction block of the chrominance component of the current coding block is obtained, downsampled, and the prediction correction based on the correlation of the boundary adjacent pixels in the orthogonal direction is performed on the downsampled prediction block to obtain the final color The final prediction block for the degree component. One path of the subsequent code stream is used as reference information for subsequent video decoding, and one path is post-filtered to output video signals.

At present, the specific implementation of the IPF technology at the video decoder 200 is as follows.

The decoding end obtains the code stream and analyzes to obtain the digital video sequence information, and obtains the IPF permission flag of the current video sequence, the intra prediction coding mode of the current decoding unit and the IPF usage flag of the current decoding unit.

In the current decoding unit,

① If the allowable flag of IPF is '1', perform all the following steps;

②If the allowed flag of IPF is '0', only a2), b2) and e2) steps are performed:

a2) Obtain the code stream information, analyze the residual information of the current decoding unit, and obtain the residual information in the time domain through the process of inverse transformation and inverse quantization;

b2) Parse the code stream and obtain the prediction mode index of the current decoding unit, and calculate the prediction block of the current decoding unit according to the adjacent reconstruction block and the prediction mode index;

c2) Parsing and obtaining the use flag of IPF, if the use flag of IPF is '0', then no additional operation is performed on the current prediction block; if the use flag of IPF is '1', then execute d2);

d2) Select the corresponding filter according to the prediction mode classification information of the current decoding unit, select the corresponding filter coefficient group according to the size of the current decoding unit, and then filter each pixel in the prediction block to obtain the final prediction block;

e2) Obtaining the reconstructed block of the current decoding unit by superimposing and restoring the residual information of the predicted block, and outputting it after post-processing;

It should be understood that other structural variations of the video decoder 200 can be used to decode the encoded video code stream. For example, the video decoder 200 can generate an output video stream without being processed by the filter unit 206; or, for some image blocks or image frames, the entropy decoder 203 of the video decoder 200 does not decode the quantized coefficients, correspondingly It does not need to be processed by the inverse quantizer 204 and the inverse transformer 205 .

Among the above-mentioned intra-frame prediction technologies, the existing IPF technology can effectively improve the coding efficiency of intra-frame prediction, greatly enhance the spatial correlation of intra-frame prediction, and solve the problem of using only a single reference pixel row or column, while ignoring the influence of certain pixels on the predicted value. However, when the intra prediction process needs to be smoothed, neither the IPF technology nor the current intra prediction mode can solve similar problems well. The pixel-by-pixel filtering based on the reference pixel can improve the correlation between the prediction block and the reference block. Unable to resolve smoothing inside prediction blocks.

The prediction block calculated according to a single prediction mode usually shows a better prediction effect in images with clearer textures, so the residual error will become smaller and less, and the coding efficiency will be improved. However, in image blocks with blurred textures, over-sharp prediction may lead to increased and larger residuals, poor prediction performance, and reduced coding efficiency.

In view of the above problems, the embodiment of the present application proposes an IPF technology based on smoothing processing for some image blocks that need to be smoothed, and directly filters the predicted blocks obtained according to the intra prediction mode.

FIG. 9 is a schematic flowchart of an image encoding method in an embodiment of the present application. The image encoding method can be applied to the source device 10 in the video decoding system 1 shown in FIG. 6 or the video encoder 100 shown in FIG. 7 . The process shown in FIG. 9 is described by taking the video encoder 100 shown in FIG. 7 as an example for execution. As shown in Figure 9, the image encoding method provided in the embodiment of the present application includes:

Step 110, determine the intra-frame prediction filtering indication information of the current coding block, the intra-frame prediction filtering indication information includes first indication information and second indication information, the first indication information is used to indicate whether to allow the use of the first intra-frame A predictive filtering mode, the second indication information is used to indicate whether to allow the use of a second intra-frame predictive filtering mode, and the first intra-frame predictive filtering mode is an intra-frame predictive filtering IPF mode;

Step 120: If it is determined according to the intra-frame prediction filtering indication information that the current coding block needs to use the first intra-frame prediction filtering mode, set the first intra-frame prediction filtering mode of the current coding block to A use flag is set to a first value, and the first value is used to indicate the use of the first intra-frame prediction filtering mode;

Step 130, writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode and the first usage flag into a code stream;

Step 140: According to the first intra-frame prediction filtering mode, predict the current coded block to obtain a predicted block, code the predicted block, and write the coded result into a code stream.

In some embodiments, the method further includes: superimposing the prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization to obtain a reconstructed reconstruction block as the prediction of the next coding block Reference block.

The specific implementation of the technical solution 1 in the intra-frame prediction part of the encoding end is as follows:

The encoder obtains encoding information, including the intra-frame predictive filtering permission flag and the intra-frame predictive smoothing filter (hereinafter referred to as IPS) permission flag of this technical solution, etc. After obtaining the image information, the image is divided into several CTUs, and further subdivided into several CUs, and each independent CU performs intra-frame prediction;

During the intra prediction process,

① If both the IPF permission flag and the IPS permission flag are '1', perform all the following steps;

② If the IPF allows the flag to be '1' and the IPS allows the flag to be '0', only a3), b3), c3), d3), e3), and i3), j3) are executed;

③ If the IPF allowed flag is '0', and the IPS allowed flag is '1', and the current CU area is greater than or equal to 64 and less than 4096, only execute a3), b3), f3), g3), h3), and i3), j3);

④ If both the IPF permission flag and the IPS permission flag are '0', only a3), b3) and i3), j3) are executed:

a3) The current coding unit traverses all intra-frame prediction modes, calculates the prediction block in each prediction mode, and calculates the rate-distortion cost information of the current prediction mode according to the original block;

b3) According to the minimum rate-distortion cost principle of all the above-mentioned prediction modes, select the optimal prediction mode of the current coding unit, and record the optimal prediction mode information and the corresponding rate-distortion cost information;

c3) Perform a second traversal on all intra-frame prediction modes. This process starts the IPF technology, and first calculates the predicted pixels in each intra-frame prediction mode to obtain the predicted block of the current coding unit;

d3) Perform IPF filtering on the prediction block of the current coding unit, select the filter corresponding to it according to the current prediction mode, and select the corresponding filter coefficient group according to the size of the current coding unit, the specific corresponding relationship can be found in Table 1;

e3) Calculate the rate-distortion cost information of each prediction mode according to the final prediction pixel and the original pixel obtained by the above-mentioned IPF technology, and record the prediction mode and the corresponding cost value of the minimum rate-distortion cost information;

f3) The third traversal is performed on all intra prediction modes. This process starts the IPS technology, and first calculates the prediction pixels in each intra prediction mode to obtain the prediction block of the current coding unit;

g3) Performing IPS twice on the prediction block of the current coding unit to obtain the final prediction block;

h3) Calculate the rate-distortion cost information of each prediction mode according to the final prediction pixel and the original pixel obtained by the above-mentioned IPS technology, and record the prediction mode and corresponding cost value of the minimum rate-distortion cost information;

i3) If the IPF allows the flag to be '0' and the IPS allows the flag to be '0', then write the prediction mode index recorded in b3) into the code stream to the decoder;

If the IPF allows the flag to be '1' and the IPS allows the flag to be '0', then compare the minimum cost value recorded in b3) with the minimum cost value recorded in e3),

If the rate-distortion cost in b3) is smaller, encode the prediction mode index recorded in b3) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and use the flag position '0' for the current coding unit of IPF , indicating that the IPF technology is not used, and the code stream is also written to the decoder;

If the rate-distortion in e3) is smaller, encode the prediction mode index recorded in e3) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoding end, and use the flag position '1' for the current coding unit of IPF, Indicates that IPF technology is used, and the code stream is also written to the decoder;

If the IPF allows the flag to be '0' and the IPS allows the flag to be '1', then compare the minimum cost value recorded in b3) with the minimum cost value recorded in h),

If the rate-distortion cost in b3) is smaller, encode the prediction mode index recorded in b3) as the optimal prediction mode of the current coding unit and write the code stream to the decoder, and set the IPS usage flag position of the current coding unit to '0' ' (that is, the second value), indicating that this technology is not used, and the code stream is also written to the decoder;

If the rate-distortion in h3) is smaller, encode the prediction mode index recorded in h3) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPS use flag position of the current coding unit to '1' , indicating that this technology is used, and the code stream is also written to the decoder;

If the IPF allows the flag to be '1' and the IPS allows the flag to be '1', then compare the minimum cost values recorded in b3), e3) and h3),

If the rate-distortion cost in b3) is smaller, then encode the prediction mode index recorded in b3) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and use the IPS usage flag and IPF of the current coding unit Use the flag position '0', which means that none of them are used, and the code stream is also written to the decoder;

If the rate-distortion in e3) is smaller, encode the prediction mode index recorded in e3) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPF usage flag position of the current coding unit to '1' And the IPS flag is not transmitted, which means that IPF technology is used instead of IPS technology, and the code stream is also written to the decoding end;

If the rate-distortion in h3) is smaller, encode the prediction mode index recorded in h3) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPF usage flag position of the current coding unit to '0' And IPS uses the flag position '1', indicating that IPS technology is used instead of IPF technology, and the code stream is also written to the decoder.

j3) Afterwards, the predicted block is superimposed with the residual after inverse transformation and inverse quantization to obtain a reconstructed coding unit block, which is used as a prediction reference block of the next coding unit.

The technical solution 2 is specifically implemented in the intra-frame prediction part of the encoding end as follows:

The encoder obtains encoding information, including the intra-frame predictive filtering permission flag and the intra-frame predictive smoothing filter (hereinafter referred to as IPS) permission flag of this technical solution, etc. After obtaining the image information, the image is divided into several CTUs, and further subdivided into several CUs, and each independent CU performs intra-frame prediction;

During the intra prediction process,

① If both the IPF permission flag and the IPS permission flag are '1', perform all the following steps;

② If the IPF allows the flag to be '1' and the IPS allows the flag to be '0', only a4), b4), c4), d4), e4), and i4), j4) are executed;

③ If the IPF allowed flag is '0', and the IPS allowed flag is '1', and the current CU area is greater than or equal to 64 and less than 4096, only execute a4), b4), f4), g4), h4), and i4), j4);

④ If both the IPF permission flag and the IPS permission flag are '0', only execute a4), b4) and i4), j4):

a4) The current coding unit traverses all intra-frame prediction modes, calculates the prediction block in each prediction mode, and calculates the rate-distortion cost information of the current prediction mode according to the original block;

b4) According to the principle of minimum rate-distortion cost of all the above-mentioned prediction modes, select the optimal prediction mode of the current coding unit, and record the optimal prediction mode information and the corresponding rate-distortion cost information;

c4) Perform a second traversal on all intra-frame prediction modes. This process starts the IPF technology, and first calculates the predicted pixels in each intra-frame prediction mode to obtain the predicted block of the current coding unit;

d4) Perform IPF filtering on the prediction block of the current coding unit, select the corresponding filter according to the current prediction mode, and select the corresponding filter coefficient group according to the size of the current coding unit, the specific corresponding relationship can be checked in Table 1;

e4) Calculate the rate-distortion cost information of each prediction mode according to the final prediction pixel and original pixel obtained by the above-mentioned IPF technology, and record the prediction mode and corresponding cost value of the minimum rate-distortion cost information;

f4) A third traversal is performed on all intra prediction modes. This process starts the IPS technology, and first calculates the prediction pixels in each intra prediction mode to obtain the prediction block of the current coding unit;

g4) performing an IPS on the prediction block of the current coding unit to obtain the final prediction block;

h4) Calculate the rate-distortion cost information of each prediction mode according to the final prediction pixel and the original pixel obtained by the above-mentioned IPS technology, and record the prediction mode and corresponding cost value of the minimum rate-distortion cost information;

i4) If the IPF allows the flag to be '0' and the IPS allows the flag to be '0', then write the prediction mode index recorded in b4) into the code stream to the decoder;

If the IPF allows the flag to be '1' and the IPS allows the flag to be '0', then compare the minimum cost value recorded in b4) with the minimum cost value recorded in e),

If the rate-distortion cost in b4) is smaller, encode the prediction mode index recorded in b4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and use the flag position '0' for the current coding unit of IPF , indicating that the IPF technology is not used, and the code stream is also written to the decoder;

If the rate-distortion in e4) is smaller, encode the prediction mode index recorded in e4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoding end, and use the flag position '1' for the current coding unit of IPF, Indicates that IPF technology is used, and the code stream is also written to the decoder;

If the IPF allows the flag to be '0' and the IPS allows the flag to be '1', then compare the minimum cost value recorded in b4) with the minimum cost value recorded in h),

If the rate-distortion cost in b4) is smaller, encode the prediction mode index recorded in b4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPS usage flag position of the current coding unit to '0' '(second value), which means that this technology is not used, and the code stream is also written to the decoder;

If the rate-distortion in h4) is smaller, encode the prediction mode index recorded in h4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPS use flag position of the current coding unit to '1' , indicating that this technology is used, and the code stream is also written to the decoder;

If the IPF allows the flag to be '1' and the IPS allows the flag to be '1', then compare the minimum cost values recorded in b4), e4) and h4),

If the rate-distortion cost in b4) is smaller, then encode the prediction mode index recorded in b4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and use the IPS usage flag and IPF of the current coding unit Use the flag position '0', which means that none of them are used, and the code stream is also written to the decoder;

If the rate-distortion in e4) is smaller, encode the prediction mode index recorded in e4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPF usage flag position of the current coding unit to '1' And the IPS flag is not transmitted, which means that IPF technology is used instead of IPS technology, and the code stream is also written to the decoding end;

If the rate-distortion in h4) is smaller, encode the prediction mode index recorded in h4) as the optimal prediction mode of the current coding unit and write it into the code stream to the decoder, and set the IPF usage flag position of the current coding unit to '0' And IPS uses the flag position '1', indicating that IPS technology is used instead of IPF technology, and the code stream is also written to the decoder.

j4) Afterwards, the predicted block is superimposed with the residual after inverse transformation and inverse quantization to obtain a reconstructed coding unit block, which is used as a prediction reference block of the next coding unit.

Corresponding to the image encoding method described in FIG. 9 , FIG. 10 is a schematic flowchart of an image decoding method in an embodiment of the present application, and the image decoding method can be applied to the target device in the video decoding system 1 shown in FIG. 6 20 or the video decoder 200 shown in FIG. 8 . The flow shown in FIG. 10 is described by taking the video encoder 200 shown in FIG. 8 as an example for execution. As shown in Figure 10, the image decoding method provided in the embodiment of the present application includes:

Step 210, analyze the code stream, and determine the intra-frame prediction filtering indication information and the first use identification bit of the current decoding block, the intra-frame prediction indication information includes the first indication information and the second indication information, and the first indication information uses In order to indicate whether to allow the use of the first intra-frame prediction filtering mode, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is the intra-frame prediction filtering IPF mode, The first usage flag is the usage flag of the first intra prediction filtering mode;

Step 220: Determine to use the first frame rate predictive filtering mode to predict the current decoded block according to the intra-frame predictive filter indication information and the first usage flag, and obtain a predictive block of the current decoded block .

The specific process of the intra-frame prediction at the decoding end of the technical solution 1 is as follows:

The decoder obtains the code stream, analyzes the code stream to obtain the IPF permission flag and the IPS permission flag of the current video sequence, parses the code stream, and performs inverse transformation and inverse quantization on the obtained residual information.

During the intra prediction decoding process,

① If the allowable flag of IPF and the flag of IPS are both '1', perform all the following steps;

② If the allow flag of IPF is '1' and the flag of IPS is '0', only execute a5), b5), c5), d5) and g5) steps;

③ If the allowed flag of IPF is '0' and the flag of IPS is '1', and the current CU area is greater than or equal to 64 and less than 4096, only execute steps a5), b5), e5), f5) and g5) ;

4. If the allowed identification bit of IPF and the identification bit of IPS are '0', only carry out a5), b5) and g5) steps;

a5) Obtain the code stream and decode it to obtain the residual information, and obtain the residual information in the time domain through processes such as inverse transformation and inverse quantization;

b5) Analyzing the code stream to obtain the prediction mode of the current decoding unit, and calculating the prediction block according to the prediction mode of the current decoding unit and adjacent reconstruction blocks;

c5) parsing and obtaining the use identification bit of IPF,

If the use flag of the IPF is '0', no additional operation is performed on the current prediction block, and step d5) is skipped;

If the usage flag of IPF is '1', execute d5);

d5) Select the corresponding filter according to the prediction mode classification information of the current decoding unit, select the corresponding filter coefficient group according to the size of the current decoding unit, and then filter each pixel in the prediction block to obtain the prediction block;

e5) Obtain the use identification bit of IPF,

If the use flag of IPF is '1', then skip the remaining process of this step, and skip f5) step;

If the use identification bit of IPF is '0', then parse and obtain the use identification bit of IPS.

If the use flag of the IPS is '0', no additional operations are performed on the current prediction block;

If the use flag of IPS is '1', then execute f5);

f5) using the IPS device to filter the input prediction block twice to obtain the filtered prediction block of the current decoding unit;

g5) Obtain the reconstructed block of the current decoding unit by superimposing and restoring the residual information of the predicted block, and output it after post-processing;

The specific process of the intra-frame prediction at the decoding end of the technical solution 2 is as follows:

The decoder obtains the code stream, analyzes the code stream to obtain the IPF permission flag and the IPS permission flag of the current video sequence, parses the code stream, and performs inverse transformation and inverse quantization on the obtained residual information.

During the intra prediction decoding process,

① If the allowable flag of IPF and the flag of IPS are both '1', perform all the following steps;

② If the allowed flag of IPF is '1' and the flag of IPS is '0', only a6), b6), c6), d6) and g6) steps are performed;

③ If the allowed flag of IPF is '0' and the flag of IPS is '1', and the current CU area is greater than or equal to 64 and less than 4096, only execute steps a6), b6), e6), f6) and g6) ;

4. If the allowed identification bit of IPF and the identification bit of IPS are '0', only carry out a6), b6) and g6) steps;

a6) Obtain the code stream and decode it to obtain the residual information, and obtain the residual information in the time domain through processes such as inverse transformation and inverse quantization;

b6) Analyzing the code stream to obtain the prediction mode of the current decoding unit, and calculating the prediction block according to the prediction mode of the current decoding unit and adjacent reconstruction blocks;

c6) parsing and obtaining the use identification bit of IPF,

If the use flag of the IPF is '0', no additional operation is performed on the current prediction block, and step d6) is skipped;

If the usage flag of IPF is '1', execute d6);

d6) Select the corresponding filter according to the prediction mode classification information of the current decoding unit, select the corresponding filter coefficient group according to the size of the current decoding unit, and then filter each pixel in the prediction block to obtain the prediction block;

e6) Obtain the use identification bit of IPF,

If the use flag of IPF is '1', then skip the remaining process of this step, and skip f6) step;

If the use identification bit of IPF is '0', then parse and obtain the use identification bit of IPS.

If the use flag of the IPS is '0', no additional operations are performed on the current prediction block;

If the usage flag of IPS is '1', then execute f6);

f6) Use the IPS device to perform a filter on the input prediction block to obtain the filtered current decoding unit prediction block;

g6) Obtain the reconstructed block of the current decoding unit by superimposing and restoring the residual information of the predicted block, and output it after post-processing;

This technical solution acts on the intra-frame prediction part in the coding and decoding framework. When the IPS technology is used to filter the current coding unit or decoding unit, the current block needs to be filled first, and the steps are as follows:

a7) If the left and upper reference pixels outside the current prediction block are available, that is, there are reconstructed pixels on the left and upper sides, then the left column and the upper row are filled with reconstructed pixels;

b7) If the reference pixels on the left or upper side outside the current prediction block are not available, that is, there are no reconstructed pixels on the left or upper side, then the side without reconstructed pixels uses the row or column closest to that side of the current prediction block filling;

c7) filling the right adjacent column outside the current prediction block with the predicted value of the rightmost column of the current prediction block;

d7) filling the lower adjacent row outside the current prediction block with the lowest predicted value of the current prediction block;

Fill the upper right pixel outside the current prediction block with the rightmost pixel that has been filled on the upper side outside the current prediction block, and use the rightmost pixel that has been filled on the lower side outside the current prediction block for the lower right pixel outside the current prediction block The side pixels are filled, and the bottom left pixel outside the current prediction block is filled with the bottommost pixel of the filled amount on the left side outside the current prediction block.

FIG. 11A shows a schematic diagram of filling a prediction block, where Pred.Pixel represents a pixel of a prediction block, and Recon.Pixel represents a filled pixel.

FIG. 11B shows another schematic diagram of filling the prediction block, where Pred.Pixel represents the pixel of the prediction block, and Recon.Pixel represents the filled pixel.

The above IPS technology uses a simplified Gaussian convolution kernel to filter the prediction block. The filter has 9 taps and 9 different filter coefficients, as shown below:

filter_coefficients represents the filter coefficients.

Each prediction pixel in the prediction block is filtered, and the filtering formula is as follows:

P'(x,y)=c ₁ ·P(x-1,y-1)+c ₂ ·P(x,y-1)+c ₁ ·P(x+1,y-1)+c ₁ P(x-1,y)+c ₃ P(x,y)+c ₂ P(x+1,y)+c ₁ P(x-1,y+1)+c ₂ P (x,y+1)+c ₁ P(x+1,y+1)

In the above formula, P′(x, y) is the final prediction value at the current coding unit (x, y), c ₁ , c ₂ and c ₃ are the coefficients in the above filter respectively, and in the above approximate Gaussian convolution Among the kernel coefficients, c ₁ is 0075, c ₂ is 0.124, and c ₃ is 0.204. P(x,y) and others such as P(x-1,y-1) are the predicted values at the current coding unit (x,y) and (x-1,y-1), where x and y are taken The range of values does not exceed the width and height of the current coding unit block.

The coefficients of the convolution kernel used in the above-mentioned IPS technology can be approximated as integers and the sum of all coefficients is a power of 2, which can avoid both the floating point calculation of the computer and the division operation, greatly reducing the computational complexity, as shown below:

The sum of the above filter coefficients is 64, that is, the calculated predicted value needs to be shifted to the right by 6 bits.

The IPS technology in the above technical solution 2 uses a simplified Gaussian convolution kernel to filter the prediction block. The filter has a total of 25 taps and 6 different filter coefficients, as shown below:

Each prediction pixel in the prediction block is filtered, and the filtering formula is as follows:

P'(x,y)=c ₁ ·P(x-2,y-2)+c ₂ ·P(x-1,y-2)+c ₃ ·P(x,y-2)+c ₂ P(x+1,y-2)+c ₁ P(x+2,y-2)+c ₂ P(x-2,y-1)+c ₄ P(x-1,y -1)+c ₅ P(x,y-1)+c ₄ P(x+1,y-1)+c ₂ P(x+2,y-1)+c ₃ P(x -2,y)+c ₅ P(x-1,y)+c ₆ P(x,y)+c ₅ P(x+1,y)+c ₃ P(x+2,y )+c ₂ P(x-2,y+1)+c ₄ P(x-1,y+1)+c ₅ P(x,y+1)+c ₄ P(x+1 ,y+1)+c ₂ P(x+2,y+1)+c ₁ P(x-2,y+2)+c ₂ P(x-1,y+2)+c ₃ P(x,y+2)+c ₂ P(x+1,y+2)+c ₁ P(x+2,y+2)

In the above formula, P′(x, y) is the final prediction value at the current coding unit (x, y), and c ₁ , c ₂ , c ₃ , c ₄ , c ₅ and c ₆ are the In the coefficients of the approximate Gaussian convolution kernel mentioned above, c ₁ is 0.0030, c ₂ is 0.0133, c ₃ is 0.0219, c ₄ is 0.0596, c ₅ is 0.0983, and c ₆ is 0.1621. P(x,y) and others such as P(x-1,y-1) are the predicted values at the current coding unit (x,y) and (x-1,y-1), where x and y are taken The range of values does not exceed the width and height of the current coding unit block.

The coefficients of the convolution kernel used in the above-mentioned IPS technology can be approximated as integers and the sum of all coefficients is a power of 2, which can avoid both the floating point calculation of the computer and the division operation, greatly reducing the computational complexity, as shown below:

The sum of the above filter coefficients is 1024, that is, the calculated predicted value needs to be shifted to the right by 10 bits.

The IPS technology in the above technical solution 2 can also use a simplified Gaussian convolution kernel to filter the prediction block. The filter has a total of 13 taps and 4 different filter coefficients, as shown below:

Each prediction pixel in the prediction block is filtered, and the filtering formula is as follows:

The filtering formula is as follows:

P'(x,y)=c ₁ ·P(x,y-2)+c ₂ ·P(x-1,y-1)+c ₃ ·P(x,y)+c ₂ ·P(x +1,y-1)+c ₁ P(x-2,y)+c ₃ P(x-1,y)+c ₄ P(x,y)+c ₃ P(x+1 ,y)+c ₁ P(x+2,y)+c ₂ P(x-1,y+1)+c ₃ P(x,y+1)+c ₂ P(x+1 ,y+1)+c ₁ P(x,y+2)

In the above formula, P′(x, y) is the final prediction value at the current coding unit (x, y), c ₁ , c ₂ , c ₃ and c ₄ are the coefficients in the above filter respectively, and in the above approximation Among the Gaussian convolution kernel coefficients, c ₁ is 13, c ₂ is 18, c ₃ is 25, and c ₄ is 32. P(x,y) and others such as P(x-1,y-1) are the predicted values at the current coding unit (x,y) and (x-1,y-1), where x and y are taken The range of values does not exceed the width and height of the current coding unit block.

The sum of the above filter coefficients is 256, that is, the calculated predicted value needs to be shifted to the right by 8 bits.

This technical solution will be applicable to the encoding and decoding part of intra-frame prediction, providing options for smoothing or local blurring for intra-frame prediction. For the part of the image texture that does not need to be sharpened, use this technology to make the predicted pixels smoother. The predicted block is closer to the original image, which will eventually improve the coding efficiency.

Technical solution 1 is tested on HPM7.0, the official simulation platform of AVS, and the intra-frame prediction block is smoothed and filtered. Under the full intra-frame test conditions and random access conditions, the test results are shown in Table 2 and Table 3.

Table 2 All Intra test results

class Y u V 4K -0.61% -0.67% -0.84% 1080P -0.45% -0.78% -0.48% 720P -0.22% -0.08% -0.66% average performance -0.42% -0.51% -0.66%

Table 3 Random Access test results

class Y u V 4K -0.25% -0.37% -0.57% 1080P -0.22% -0.41% -0.64% 720P -0.22% -0.01% -0.73% average performance -0.23% -0.26% -0.65%

It can be seen from Table 2 and Table 3 that this scheme has a good performance improvement under the two test conditions.

Under AI test conditions, the luminance component has a BDBR saving of 0.42%, and the UV component has a BDBR saving of 0.51% and 0.66%, respectively. It can be clearly seen that it has higher performance and effectively improves the coding efficiency of the encoder.

From the point of view of various resolutions, this solution has a large coding performance improvement on 4K resolution video, which will benefit the development of ultra-high-definition video in the future, save more bit rates for ultra-high-resolution videos, and save more Multiple bandwidths.

This solution proposes that during the intra prediction process, the prediction block calculated by the intra prediction mode is smoothed and filtered to improve the accuracy of the intra prediction and effectively improve the coding efficiency. The details are as follows:

1. Proposed smoothing and filtering of intra-coded prediction blocks;

2. It is proposed that in the case of IPS technology and IPF, when the allowable flags of the two technologies are both '1', IPF and IPS will not be determined to be used at the same time in the same coding unit;

3. It is proposed that when the encoder decides to use IPF technology for the current coding unit, the IPS flag is not transmitted, and the decoder does not need to parse the IPS usage flag;

When the encoder decides not to use IPF technology for the current coding unit, it needs to transmit the IPS flag, and the decoder needs to parse the IPS usage flag;

4. The IPS convolution kernel is proposed as a simplified 9-tap filter Gaussian convolution kernel;

5. It is proposed to approximate the value of the convolution kernel of floating-point numbers, round the filter coefficients to avoid floating-point calculations, and replace the division operations with shift operations to save computing resources and reduce complexity when the sum of filter coefficients is an exponential power of 2;

6. The filter coefficient of the 9-tap integer Gaussian convolution kernel is proposed, the first filter coefficient is 5, the second filter coefficient is 8, and the third filter coefficient is 12. The predicted value after filtering needs to be shifted to the right by 6 bits;

7. Propose 25-tap and 13-tap filters.

This application can also be further expanded from the following directions.

Extension 1: The 9-tap Gaussian convolution kernel in this technical solution is replaced with a filter convolution kernel with more taps to achieve a better smoothing filtering effect.

Extension solution 2: the brightness component and the chrominance component in this technical solution respectively use independent identification bits to indicate whether the IPS is used or not.

Extension 3: Limit the scope of use in this technical solution, and do not use IPS technology for units with a small prediction block area, so as to reduce transmission identification bits and computational complexity.

Extension 4: Limit the scope of use in this technical solution, and filter the prediction mode of the current coding unit. If it is the mean value mode, the IPS technology is not used to reduce the number of transmission identification bits and computational complexity.

An embodiment of the present application provides an image encoding device, and the image encoding device may be a video decoder or a video encoder. Specifically, the image coding device is configured to perform the steps performed by the video decoder in the above decoding method. The image encoding device provided in the embodiment of the present application may include modules corresponding to corresponding steps.

The embodiment of the present application may divide the functional modules of the image encoding device according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. The division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

In the case of dividing each functional module corresponding to each function, FIG. 12 shows a possible structural diagram of the image encoding device involved in the above embodiment. As shown in FIG. 12 , the image encoding device 12 includes a determination unit 120 , a setting unit 121 , a transmission unit 122 , and a superposition unit 123 .

The determining unit 120 is configured to determine intra-frame prediction filtering indication information of the current coding block, the intra-frame prediction filtering indication information includes first indication information and second indication information, and the first indication information is used to indicate whether the second indication information is allowed to be used. An intra prediction filtering mode, the second indication information is used to indicate whether the second intra prediction filtering mode is allowed to be used, and the first intra prediction filtering mode is an intra prediction filtering IPF mode;

The setting unit 121 is configured to set the first intra prediction filtering mode of the current coding block if it is determined according to the intra prediction filtering indication information that the current coding block needs to use the first intra prediction filtering mode. The first use flag of the mode is set to the first value;

A transmission unit 122, configured to write the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first usage flag into a code stream;

The superposition unit 123 is a superposition unit, configured to determine to use the first frame rate prediction filtering mode to predict the current decoded block according to the intra-frame prediction filtering indication information and the first usage flag, to obtain the current The predicted block for the decoded block.

Wherein, all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding function module, and will not be repeated here. Of course, the image encoding device provided in the embodiment of the present application includes but is not limited to the above-mentioned modules, for example, the image encoding device may further include a storage unit. The storage unit can be used to store program codes and data of the image encoding device.

In the case of using an integrated unit, a schematic structural diagram of the image encoding device provided in the embodiment of the present application is shown in FIG. 13 . In FIG. 13 , the image encoding device 13 includes: a processing module 130 and a communication module 131 . The processing module 130 is used to control and manage the actions of the image encoding device, for example, to execute the steps performed by the determination unit 120, the setting unit 121, the transmission unit 122, and the superposition unit 123, and/or other methods for performing the techniques described herein. process. The communication module 131 is used to support the interaction between the image encoding device and other devices. As shown in FIG. 13 , the image coding device may further include a storage module 132, which is used to store program codes and data of the image coding device, for example, store content stored in the above storage unit.

Wherein, the processing module 130 may be a processor or a controller, such as a central processing unit (Central Processing Unit, CPU), a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), ASIC, FPGA or other programmable Logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, a combination of one or more microprocessors, a combination of DSP and a microprocessor, and so on. The communication module 131 may be a transceiver, an RF circuit, or a communication interface and the like. The storage module 132 may be a memory.

Wherein, all relevant content of each scenario involved in the above method embodiment may be referred to the function description of the corresponding functional module, and will not be repeated here. The above-mentioned image coding device may execute the above-mentioned image coding method, and the image coding device may specifically be a video image coding device or other equipment with a video coding function.

The present application also provides a video encoder, including a non-volatile storage medium, and a central processing unit, the non-volatile storage medium stores an executable program, and the central processing unit and the non-volatile storage media connection, and execute the executable program to implement the image coding method of the embodiment of the present application.

An embodiment of the present application provides an image decoding device, and the image decoding device may be a video decoder or a video decoder. Specifically, the image decoding device is configured to perform the steps performed by the video decoder in the above decoding method. The image decoding device provided in the embodiment of the present application may include modules corresponding to corresponding steps.

The embodiment of the present application may divide the functional modules of the image decoding device according to the above method example, for example, each functional module may be divided corresponding to each function, or two or more functions may be integrated into one processing module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software function modules. The division of modules in the embodiment of the present application is schematic, and is only a logical function division, and there may be other division methods in actual implementation.

Fig. 14 shows a possible structural diagram of the image decoding device involved in the above-mentioned embodiment in the case of dividing each functional module corresponding to each function. As shown in FIG. 14 , the image decoding device 14 includes an analysis unit 140 and a determination unit 141 .

The parsing unit is configured to determine the intra-frame prediction filtering indication information and the first use flag of the current decoding block, the intra-frame prediction indication information includes first indication information and second indication information, and the first indication information is used to indicate Whether to allow the use of the first intra-frame prediction filtering mode, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is the intra-frame prediction filtering IPF mode, the The first usage flag is the usage flag of the first intra prediction filtering mode;

A determining unit, configured to determine to use the first frame rate predictive filtering mode to predict the current decoding block according to the intra-frame prediction filtering indication information and the first use flag, and obtain the current decoding block prediction block.

Wherein, all relevant content of each step involved in the above-mentioned method embodiment can be referred to the function description of the corresponding function module, and will not be repeated here. Of course, the image decoding device provided in the embodiment of the present application includes but is not limited to the above-mentioned modules, for example, the image decoding device may further include a storage unit. The storage unit can be used to store program codes and data of the image decoding device.

In the case of using an integrated unit, a schematic structural diagram of an image decoding device provided in an embodiment of the present application is shown in FIG. 15 . In FIG. 15 , the image decoding device 15 includes: a processing module 150 and a communication module 151 . The processing module 150 is used to control and manage the actions of the image decoding device, for example, execute the steps performed by the analysis unit 140 and the determination unit 141, and/or other processes for performing the techniques described herein. The communication module 151 is used to support the interaction between the image decoding device and other devices. As shown in FIG. 13 , the image decoding device may further include a storage module 152, which is used to store program codes and data of the image decoding device, for example, store the content stored in the above storage unit.

Wherein, the processing module 150 may be a processor or a controller, such as a central processing unit (Central Processing Unit, CPU), a general purpose processor, a digital signal processor (Digital Signal Processor, DSP), ASIC, FPGA or other programmable Logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, a combination of one or more microprocessors, a combination of DSP and a microprocessor, and so on. The communication module 151 may be a transceiver, an RF circuit, or a communication interface and the like. The storage module 152 may be a memory.

Wherein, all relevant content of each scenario involved in the above method embodiment may be referred to the function description of the corresponding functional module, and will not be repeated here. The above-mentioned image decoding device may execute the above-mentioned image decoding method, and the image decoding device may specifically be a video image decoding device or other equipment with a video decoding function.

The present application also provides a video decoder, including a non-volatile storage medium, and a central processing unit, the non-volatile storage medium stores an executable program, and the central processing unit and the non-volatile storage media connection, and execute the executable program to implement the image decoding method of the embodiment of the present application.

The present application also provides a terminal, which includes: one or more processors, memory, and a communication interface. The memory and the communication interface are coupled with one or more processors; the memory is used to store computer program codes, the computer program codes include instructions, and when the one or more processors execute the instructions, the terminal executes the image encoding and/or the embodiment of the present application or image decoding method. The terminal here can be a video display device, a smart phone, a portable computer and other devices that can process or play video.

Another embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium includes one or more program codes, the one or more programs include instructions, when the processor in the decoding device executes the program When encoding, the decoding device executes the image encoding method and the image decoding method of the embodiment of the present application.

In another embodiment of the present application, a computer program product is also provided, the computer program product includes computer-executable instructions stored in a computer-readable storage medium; at least one processor of the decoding device can read from the computer The readable storage medium reads the computer-executable instructions, and at least one processor executes the computer-executable instructions so that the terminal implements the image encoding method and the image decoding method of the embodiments of the present application.

In the above embodiments, all or part of the implementation may be implemented by software, hardware, firmware or any combination thereof. When implemented using a software program, it may appear in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present application will be generated in whole or in part.

The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.) means.

The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, or a magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, a Solid State Disk (SSD)).

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be Incorporation or may be integrated into another device, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may be one physical unit or multiple physical units, that is, it may be located in one place, or may be distributed to multiple different places . Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the software product is stored in a storage medium Among them, several instructions are included to make a device (which may be a single-chip microcomputer, a chip, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk and other various media that can store program codes. .

The above is only a specific implementation of the application, but the protection scope of the application is not limited thereto, and any changes or replacements within the technical scope disclosed in the application should be covered within the protection scope of the application . Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims (30)

An image coding method, characterized in that, comprising: Determine the intra-frame prediction filtering indication information of the current coding block, the intra-frame prediction filtering indication information includes first indication information and second indication information, and the first indication information is used to indicate whether the first intra-frame prediction filtering mode is allowed to be used , the second indication information is used to indicate whether the second intra-frame prediction filtering mode is allowed to be used, and the first intra-frame prediction filtering mode is an intra-frame prediction filtering IPF mode; If it is determined according to the intra-frame prediction filtering indication information that the current coding block needs to use the first intra-frame prediction filtering mode, set the first use flag of the first intra-frame prediction filtering mode of the current coding block bit is set to the first value; Writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first use flag into a code stream; According to the first intra-frame prediction filtering mode, the current coded block is predicted to obtain a predicted block, the predicted block is coded, and the coded result is written into a code stream. The method according to claim 1, wherein the determining according to the intra-frame prediction filtering indication information that the current coding block needs to use the first intra-frame prediction filtering mode includes: Detecting that the first indication information indicates that the first intra-frame prediction filtering mode is allowed to be used, and the second indication information indicates that the second intra-frame prediction filtering mode is allowed to be used; Traversing multiple intra-frame prediction modes for the current coding block, respectively calculating the first prediction block of the current coding block in each intra-frame prediction mode; For each of the intra prediction modes, calculate according to the first prediction block and the original block of the current coding block to obtain the first rate-distortion cost information of the current coding block in each of the prediction modes ; Determine the first intra prediction mode with the smallest first rate-distortion cost information and the corresponding first rate-distortion cost information as the first generation value; For each of the intra-frame prediction modes, select a filter corresponding to each of the intra-frame prediction modes, and select a filter corresponding to each of the intra-frame prediction modes according to the size of the current coding block A filter coefficient group, performing IPF filtering on the prediction block of the current coding block according to the filter and the filter coefficient group, to obtain the first prediction block of the current coding block in each intra prediction mode Two prediction blocks; calculate the second rate-distortion cost information of each prediction mode according to the second prediction block and the original block of the current coding block; record the second intra-frame with the minimum second rate-distortion cost information The prediction mode and the corresponding second rate-distortion cost information are used as the second generation value; For each of the intra prediction modes, perform intra prediction smoothing IPS filtering on the prediction block of the current coding block to obtain the first prediction block of the current coding block in each of the intra prediction modes Three prediction blocks; calculate the third rate-distortion cost information of each prediction mode according to the third prediction block and the original block of the current coding block; determine the third frame of the smallest third rate-distortion cost information The intra-prediction mode and the corresponding third rate-distortion cost information are used as the third-generation value; If it is detected that the cost value with the smallest numerical value among the first generation value, the second generation value, and the third generation value is the second generation value, it is determined that the current encoding block needs to use the first generation value. Intra prediction filtering mode. The method according to claim 2, wherein the writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first use identification bit into the code stream includes: writing Writing the intra-frame prediction filtering indication information, the second intra-frame prediction mode, and the first usage flag into a code stream; The method further includes: superimposing the second prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization to obtain a reconstructed reconstruction block as a prediction reference block of the next coding block . The method according to claim 2, further comprising: If it is detected that the cost value with the smallest numerical value among the first generation value, the second generation value, and the third generation value is the first generation value, it is determined that the current coding block does not need to use the first generation value. an intra prediction filtering mode and said second intra prediction filtering mode; Set both the first use flag of the first intra-frame prediction filtering mode and the second use flag of the second intra-frame prediction filtering mode of the current coding block to not use; Writing the intra-frame prediction filtering indication information, the first intra-frame prediction mode, the first usage flag and the second usage flag into a code stream; According to the first intra-frame prediction mode, predict the current coded block to obtain a predicted block, code the predicted block, and write the coded result into a code stream. The method according to claim 2, further comprising: If it is detected that the cost value with the smallest numerical value among the first generation value, the second generation value, and the third generation value is the third generation value, it is determined that the current encoding block needs to use the second generation value. an intra prediction filtering mode without using the first intra prediction filtering mode; Set the first usage flag of the first intra-frame prediction filtering mode of the current coding block to a second value, and set the second usage flags of the second intra-frame prediction filtering mode to use; Writing the intra-frame prediction filtering indication information, the third intra-frame prediction mode, the first usage flag and the second usage flag into a code stream; According to the third intra-frame prediction mode, the current coding block is predicted to obtain a prediction block, the prediction block is coded, and the coded result is written into a code stream. The method according to claim 1, wherein the determining according to the intra-frame prediction filtering indication information that the current coding block needs to use the first intra-frame prediction filtering mode comprises: Detecting that the first indication information indicates that the first intra-frame prediction filtering mode is allowed to be used, and the second indication information indicates that the second intra-frame prediction filtering mode is not allowed to be used; Traversing multiple intra-frame prediction modes for the current coding block, respectively calculating the first prediction block of the current coding block in each intra-frame prediction mode; For each of the intra prediction modes, calculate according to the first prediction block and the original block of the current coding block to obtain the first rate-distortion cost information of the current coding block in each of the prediction modes ; Determine the first intra prediction mode with the smallest first rate-distortion cost information and the corresponding first rate-distortion cost information as the first generation value; For each of the intra-frame prediction modes, select a filter corresponding to each of the intra-frame prediction modes, and select a filter corresponding to each of the intra-frame prediction modes according to the size of the current coding block A filter coefficient group, performing IPF filtering on the prediction block of the current coding block according to the filter and the filter coefficient group, to obtain the first prediction block of the current coding block in each intra prediction mode Two prediction blocks; calculate the second rate-distortion cost information of each prediction mode according to the second prediction block and the original block of the current coding block; record the second intra-frame with the minimum second rate-distortion cost information The prediction mode and the corresponding second rate-distortion cost information are used as the second generation value; If it is detected that the cost value with the smallest value among the first generation value and the second generation value is the second generation value, it is determined that the current encoding block needs to use the first intra-frame prediction filtering mode. The method according to claim 6, wherein the writing the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first usage identification bit into the code stream comprises: writing Writing the intra-frame prediction filtering indication information, the first intra-frame prediction mode, and the first usage flag into a code stream; The method further includes: superimposing the second prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization to obtain a reconstructed reconstruction block as a prediction reference block of the next coding block . The method according to claim 6, further comprising: If it is detected that the cost value with the smallest value among the first generation value and the second generation value is the first generation value, it is determined that the current encoding block does not need to use the first intra-frame prediction filtering mode; Setting the first usage flag of the first intra prediction filtering mode of the current coding block to a second value; Writing the intra-frame prediction filtering indication information, the first intra-frame prediction mode, and the first usage flag into a code stream; According to the first intra-frame prediction mode, predict the current coded block to obtain a predicted block, code the predicted block, and write the coded result into a code stream. The method according to claim 1, further comprising: Detecting that the first indication information indicates that the first intra-frame prediction filtering mode is not allowed to be used, and the second indication information indicates that the second intra-frame prediction filtering mode is not allowed to be used; Traversing multiple intra-frame prediction modes for the current coding block, respectively calculating the first prediction block of the current coding block in each intra-frame prediction mode; For each of the intra prediction modes, calculate according to the first prediction block and the original block of the current coding block to obtain the first rate-distortion cost information of the current coding block in each of the prediction modes ; Determine the first intra prediction mode with the smallest first rate-distortion cost information and the corresponding first rate-distortion cost information as the first generation value; Writing the intra-frame prediction filtering indication information, the first intra-frame prediction mode, the first usage flag and the second usage flag into a code stream; According to the first intra-frame prediction mode, predict the current coded block to obtain a predicted block, code the predicted block, and write the coded result into a code stream. The method according to claim 1, further comprising: It is detected that the first indication information indicates that the first intra-frame prediction filtering mode is not allowed to be used, and the second indication information indicates that the second intra-frame prediction filtering mode is allowed to be used, and the area of the current decoding block Greater than or equal to 64 and less than 4096; For each of the intra prediction modes, calculate according to the first prediction block and the original block of the current coding block to obtain the first rate-distortion cost information of the current coding block in each of the prediction modes ; Determine the first intra prediction mode with the smallest first rate-distortion cost information and the corresponding first rate-distortion cost information as the first generation value; For each of the intra prediction modes, perform intra prediction smoothing IPS filtering on the prediction block of the current coding block to obtain the first prediction block of the current coding block in each of the intra prediction modes Three prediction blocks; calculate the third rate-distortion cost information of each prediction mode according to the third prediction block and the original block of the current coding block; determine the third frame of the smallest third rate-distortion cost information The intra-prediction mode and the corresponding third rate-distortion cost information are used as the third-generation value; If it is detected that the cost value with the smallest value among the first generation value and the third generation value is the first generation value, it is determined that the current coding block does not need to use the first intra-frame prediction filtering mode; The first use flag of the first intra-frame prediction filtering mode of the current coding block is set to a second value; the intra-frame prediction filtering indication information, the first intra-frame prediction mode, the first Using the identification bit to write into the code stream; according to the first intra-frame prediction mode, predicting the current coding block to obtain a prediction block, encoding the prediction block, and writing the coding result into the code stream; If it is detected that the cost value with the smallest value among the first generation value and the third generation value is the third generation value, it is determined that the current coding block needs to use the second intra-frame prediction filtering mode; The second usage flag of the second intra-frame prediction filtering mode of the current coding block is set to use; the intra-frame prediction filtering indication information, the third intra-frame prediction mode, and the second usage flag bit into the code stream; superimposing the third prediction block of the current coding block and the residual block obtained after inverse transformation and inverse quantization to obtain a reconstructed reconstruction block as a prediction reference block of the next coding block . The method according to claim 2 or 6, wherein the IPF filter adopts a 9-tap filter, and the filter is a Gaussian convolution kernel. The method according to claim 11, characterized in that the sum of the filter coefficients of the 9-tap filter is an exponential power of 2. The method according to claim 11, wherein the first filter coefficient of the 9-tap filter is 5, the second filter coefficient is 8, and the third filter coefficient is 12, and the predicted value after filtering is shifted to the right 6 digits. The method according to claim 2 or 6, wherein the IPF filter adopts a 25-tap filter or a 13-tap filter. An image decoding method, characterized in that, comprising: Parsing the code stream to determine the intra-frame prediction filtering indication information and the first use flag of the current decoding block, the intra-frame prediction indication information includes the first indication information and the second indication information, and the first indication information is used to indicate whether The first intra-frame prediction filtering mode is allowed to be used, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is an intra-frame prediction filtering IPF mode, and the second A use flag bit is the use flag bit of the first intra-frame prediction filtering mode; According to the intra-frame prediction filtering indication information and the first use flag, determine to use the first frame rate prediction filtering mode to predict the current decoding block, and obtain a prediction block of the current decoding block. The method according to claim 15, wherein, according to the intra-frame prediction filtering indication information and the first use flag, it is determined to use the first frame rate prediction filtering mode for the current decoding block Perform prediction to obtain a prediction block of the current decoding block, including: It is detected that the first indication information indicates that the first intra-frame prediction filtering mode is allowed to be used, and the second indication information indicates that the use of the second intra-frame prediction filtering mode is allowed or not allowed; Obtaining the code stream and decoding to obtain the residual block of the current decoding block; Analyzing the code stream to obtain a prediction mode of the current decoding block, and calculating an original prediction block of the current decoding block according to the prediction mode and adjacent reconstructed blocks; If the first use flag indicates to use the first intra-frame prediction filtering mode; Select the corresponding filter according to the prediction mode of the current decoding block, select the corresponding filter coefficient group according to the size of the current decoding block, and select the corresponding filter coefficient group according to the filter and the filter coefficient group. The original prediction block of the current decoding block is subjected to IPF filtering to obtain the prediction block of the decoding block. The method according to claim 15, further comprising: If the first indication information indicates that the first intra-frame prediction filtering mode is not allowed to be used, and the second indication information indicates that the second intra-frame prediction filtering mode is allowed to be used, and the area of the current decoding block is larger than Equal to 64 and less than 4096; Obtaining the code stream and decoding to obtain the residual block of the current decoding block; Analyzing the code stream to obtain a prediction mode of the current decoding block, and calculating an original prediction block of the current decoding block according to the prediction mode and adjacent reconstructed blocks; Detecting that the first usage flag indicates that the first intra-frame prediction filtering mode is not used; Parsing the code stream and obtaining a second usage flag of the current decoding block, where the second usage flag is an intra prediction smoothing filtering IPS usage flag; Detecting that the second use flag indicates that a second intra-frame prediction filtering mode is used; Perform intra-frame prediction smoothing IPS filtering on the original prediction block of the current decoding block to obtain a prediction block of the current decoding block. The method according to claim 17, further comprising: Detecting that the second use flag indicates that the second intra-frame prediction filtering mode is not used; Determine the original prediction block of the current decoding block as the prediction block of the current decoding block. The method according to any one of claims 15-18, wherein the method further comprises: and superimposing the prediction block on the residual information to obtain a reconstructed block of the current decoding block. The method according to claim 16, wherein the IPF filter adopts a 9-tap filter, and the filter is a Gaussian convolution kernel. The method according to claim 20, characterized in that the sum of the filter coefficients of the 9-tap filter is an exponential power of 2. The method according to claim 20, wherein the first filter coefficient of the 9-tap filter is 5, the second filter coefficient is 8, and the third filter coefficient is 12, and the predicted value after filtering is shifted to the right 6 digits. The method according to claim 16, wherein the IPF filter adopts a 25-tap filter or a 13-tap filter. An image encoding device, characterized in that it includes: A determining unit, configured to determine intra-frame prediction filtering indication information of the current coding block, where the intra-frame prediction filtering indication information includes first indication information and second indication information, and the first indication information is used to indicate whether the first indication information is allowed to be used. Intra-frame prediction filtering mode, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is intra-frame prediction filtering IPF mode; a setting unit, configured to set the first intra prediction filtering mode of the current coding block to The first use identification bit of is set to the first value; A transmission unit, configured to write the intra-frame prediction filtering indication information, the first intra-frame prediction filtering mode, and the first usage flag into a code stream; A superposition unit, configured to determine to use the first frame rate predictive filter mode to predict the current decoded block according to the intra-frame predictive filter indication information and the first use flag, to obtain the current decoded block prediction block. An image decoding device, characterized in that it comprises: The parsing unit is configured to determine the intra-frame prediction filtering indication information and the first use flag of the current decoding block, the intra-frame prediction indication information includes first indication information and second indication information, and the first indication information is used to indicate Whether to allow the use of the first intra-frame prediction filtering mode, the second indication information is used to indicate whether to allow the use of the second intra-frame prediction filtering mode, the first intra-frame prediction filtering mode is the intra-frame prediction filtering IPF mode, the The first usage flag is the usage flag of the first intra prediction filtering mode; A determining unit, configured to determine to use the first frame rate predictive filtering mode to predict the current decoding block according to the intra-frame prediction filtering indication information and the first use flag, and obtain the current decoding block prediction block. An encoder, comprising a non-volatile storage medium and a central processing unit, wherein the non-volatile storage medium stores an executable program, and the central processing unit is connected to the non-volatile storage medium , when the central processing unit executes the executable program, the encoder executes the image encoding method according to any one of claims 1-14. A decoder, comprising a non-volatile storage medium and a central processing unit, wherein the non-volatile storage medium stores an executable program, and the central processing unit is connected to the non-volatile storage medium , when the central processing unit executes the executable program, the decoder executes the image decoding method according to any one of claims 15-23. A terminal, characterized in that the terminal includes: one or more processors, memory and communication interface; the memory and the communication interface are connected to the one or more processors; the terminal passes through the The communication interface communicates with other devices, the memory is used to store computer program code, the computer program code includes instructions, When the one or more processors execute the instructions, the terminal executes the image coding method according to any one of claims 1-14 and/or the image according to any one of claims 15-23 decoding method. A computer program product containing instructions, characterized in that, when the computer program product is run on a terminal, the terminal is made to execute the image coding method according to any one of claims 1-14 and/or the image coding method according to any one of claims 1-14 The image decoding method described in any one of 15-23 is required. A computer-readable storage medium, including instructions, characterized in that, when the instructions are run on a terminal, the terminal is made to execute the image coding method according to any one of claims 1-14 and/or the image coding method according to any one of claims 1-14 The image decoding method described in any one of 15-23 is required.

Images