CN119299702A - Intra-frame encoding and decoding method, device, equipment and storage medium based on neighborhood features - Google Patents

Abstract

The embodiment of the disclosure provides an intra-frame encoding and decoding method and device based on neighborhood characteristics, electronic equipment and a storage medium. The method comprises the steps of carrying out gradient feature analysis on a first neighborhood of a block to be coded, carrying out preliminary screening on a prediction model to determine a prediction model set, carrying out prediction on a second neighborhood through a third neighborhood based on the prediction model set to determine distortion values corresponding to the prediction models, determining a target prediction model according to the distortion values, and then coding the block to be coded. According to the method, on one hand, the primary screening of the prediction model is carried out through the gradient characteristics of the first neighborhood to form a prediction mode set, and on the other hand, the reference neighborhood range is enlarged through the second neighborhood and the third neighborhood to determine a more matched prediction model. Meanwhile, the encoding end does not need to encode the prediction model information by the method, but predicts the prediction mode based on the implicit of the decoding end, so that the code rate occupied by the prediction mode encoding information is saved, and the encoding code rate of the intra-frame prediction mode is greatly reduced.

Description

Intra-frame encoding and decoding method, device, equipment and storage medium based on neighborhood features

Technical Field

The present disclosure relates to the field of image processing technology, and in particular to an intra-frame encoding method, decoding method, device, electronic device, storage medium and computer program product based on neighborhood features.

Background Art

In video coding, intra frames (I frames) do not rely on adjacent reference frames and can be encoded and decoded independently. Intra prediction is a key technology for encoding I frames and directly determines the compression efficiency of I frames. Intra prediction is a method of predicting unknown information based on texture correlation within the same image frame. It uses the encoded information in the image frame as a priori to remove spatial information redundancy in the image.

In the intra-frame prediction process, based on the reconstructed pixels in the neighborhood of the block to be predicted (usually the area to its left and above) as reference pixels, the pixels to be predicted are projected to the reference pixels along a given angular direction. The density of the angular direction of the projection and the accuracy of the reference pixels determine the accuracy of the intra-frame prediction. In the latest video codec standard H.266/VVC, it has been expanded to 65 prediction modes (corresponding to different angular directions). Therefore, how to determine the prediction mode with the smallest prediction distortion among multiple prediction modes in the intra-frame prediction process has become a key technical issue in this field.

Summary of the invention

The embodiments of the present disclosure provide an intra-frame encoding method, a decoding method, an apparatus, an electronic device, a storage medium, and a computer program product based on neighborhood features.

According to a first aspect of an embodiment of the present disclosure, a neighborhood feature-based intra-frame coding method is provided, comprising: performing gradient feature analysis on reference pixels in a first neighborhood of a block to be coded, and determining a prediction model set; the prediction model set includes at least two prediction models; based on the prediction model set, predicting reference pixels in a second neighborhood through reference pixels in a third neighborhood, and determining distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is a neighborhood of the block to be coded; the third neighborhood is a neighborhood of the second neighborhood; determining a target prediction model according to the distortion value; and encoding the block to be coded according to the target prediction model.

In some exemplary embodiments of the present disclosure, the step of performing gradient feature analysis on reference pixels in a first neighborhood of a block to be encoded to determine a prediction model set includes: determining at least one sliding line in the first neighborhood; obtaining the gradient strength and gradient direction of each reference pixel on the sliding line; performing statistics on the gradient strength and gradient direction to generate a gradient histogram corresponding to the sliding line; determining the prediction model set from a set of pending prediction models based on the gradient histogram; wherein the set of pending prediction models includes at least two pending prediction models; and each of the pending prediction models corresponds to the gradient direction, respectively.

In some exemplary embodiments of the present disclosure, determining at least one sliding line in the first neighborhood includes: performing gradient feature analysis on the reference pixel by sliding a sliding window in the first neighborhood; and determining a center line of the sliding window as the sliding line.

In some exemplary embodiments of the present disclosure, the at least one sliding line includes a first sliding line and a second sliding line; the statistics of the gradient strength and gradient direction to generate a gradient histogram corresponding to the sliding line includes: statistics of the gradient strength and gradient direction of the first sliding line to generate a first gradient histogram corresponding to the first sliding line; statistics of the gradient strength and gradient direction of the second sliding line to generate a second gradient histogram corresponding to the second sliding line.

In some exemplary embodiments of the present disclosure, determining the prediction model set from the set of pending prediction models based on the gradient histogram includes: determining a first prediction model set from the set of pending prediction models based on the first gradient histogram; determining a second prediction model set from the set of pending prediction models based on the second gradient histogram; and determining the prediction model set based on the first prediction model set and the second prediction model set.

In some exemplary embodiments of the present disclosure, the first prediction model set includes M prediction models, and the second prediction model set includes N prediction models; wherein M is greater than N; and the first sliding line is closer to the block to be encoded than the second sliding line.

In some exemplary embodiments of the present disclosure, determining the prediction model set from the set of pending prediction models based on the gradient histogram includes: superimposing the first gradient histogram and the second gradient histogram to obtain a superimposed gradient histogram; and determining the prediction model set from the set of pending prediction models based on the superimposed gradient histogram.

In some exemplary embodiments of the present disclosure, based on the prediction model set, reference pixels in the second neighborhood are predicted by reference pixels in the third neighborhood to determine the distortion values corresponding to each prediction model in the prediction model set, including: determining the second neighborhood according to the block to be encoded; determining the third neighborhood according to the second neighborhood; predicting the reference pixels in the second neighborhood according to the reference pixels in the third neighborhood based on each prediction model in the prediction model set, to obtain predicted pixels corresponding to the second neighborhood; calculating the distortion between the predicted pixels of the second neighborhood and the corresponding reference pixels of the second neighborhood, to obtain the distortion values corresponding to each prediction model in the prediction model set.

In some exemplary embodiments of the present disclosure, the target prediction model includes at least a first prediction model and a second prediction model; the first prediction model and the second prediction model are selected from the prediction model set; encoding the block to be encoded according to the target prediction model includes: encoding the block to be encoded according to the first prediction model to obtain a first prediction block; encoding the block to be encoded according to the second prediction model to obtain a second prediction block; generating a coding block of the block to be encoded based on the first prediction block and the second prediction block.

In some exemplary embodiments of the present disclosure, generating the coding block of the block to be encoded according to the first prediction block and the second prediction block includes: determining the weight values corresponding to the first prediction model and the second prediction model according to the distortion value and/or gradient strength corresponding to the prediction model; and performing weighted prediction on the first prediction block and the second prediction block based on the weight value to obtain the coding block.

According to a second aspect of an embodiment of the present disclosure, a method for intra-frame decoding based on neighborhood features is provided, characterized in that it includes: performing gradient feature analysis on reference pixels in a first neighborhood of a block to be decoded to determine a prediction model set; the prediction model set includes at least two prediction models; based on the prediction model set, predicting reference pixels in a second neighborhood through reference pixels in a third neighborhood to determine distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is a neighborhood of the block to be decoded; the third neighborhood is a neighborhood of the second neighborhood; determining a target prediction model according to the distortion value; and decoding the block to be decoded according to the target prediction model.

According to a third aspect of an embodiment of the present disclosure, there is provided an intra-frame encoding device based on neighborhood features, comprising: a first gradient analysis module, configured to perform gradient feature analysis on reference pixels in a first neighborhood of a block to be encoded, and determine a prediction model set; the prediction model set includes at least two prediction models; a first distortion calculation module, configured to predict reference pixels in a second neighborhood through reference pixels in a third neighborhood based on the prediction model set, and determine distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is a neighborhood of the block to be encoded; the third neighborhood is a neighborhood of the second neighborhood; a first prediction model determination module, configured to determine a target prediction model according to the distortion value; and an encoding module, configured to encode the block to be encoded according to the target prediction model.

According to a fourth aspect of an embodiment of the present disclosure, there is provided an intra-frame decoding device based on neighborhood features, comprising: a second gradient analysis module, configured to perform gradient feature analysis on reference pixels in a first neighborhood of a block to be decoded, and determine a prediction model set; the prediction model set includes at least two prediction models; a second distortion calculation module, configured to predict reference pixels in a second neighborhood through reference pixels in a third neighborhood based on the prediction model set, and determine distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is a neighborhood of the block to be decoded; the third neighborhood is a neighborhood of the second neighborhood; a second prediction model determination module, configured to determine a target prediction model according to the distortion value; and a decoding module, configured to decode the block to be decoded according to the target prediction model.

According to the fifth aspect of an embodiment of the present disclosure, an electronic device is provided, characterized in that it includes: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to implement any one of the intra-frame encoding method based on neighborhood features or the intra-frame decoding method based on neighborhood features.

According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any one of the intra-frame encoding methods based on neighborhood features or the intra-frame decoding methods based on neighborhood features.

According to the seventh aspect of the embodiments of the present disclosure, a computer program product is provided, comprising a computer program/instruction, characterized in that when the computer program/instruction is executed by a processor, it implements any one of the intra-frame encoding methods based on neighborhood features or the intra-frame decoding methods based on neighborhood features.

The intra-frame encoding and decoding method based on neighborhood features provided by the embodiment of the present disclosure performs gradient feature analysis on the first neighborhood of the coding block, preliminarily screens the prediction model to determine a prediction model set; based on the prediction model set, the second neighborhood is predicted through the third neighborhood to determine the distortion value corresponding to each prediction model; the target prediction model is determined according to the distortion value, and then the coding block is encoded. Through this method, on the one hand, the prediction model is preliminarily screened through the gradient features of the first neighborhood to form a prediction model set; on the other hand, the reference neighborhood range is expanded through the second and third neighborhoods to determine a more matching prediction model. At the same time, through this method, the encoding end does not need to encode the prediction model information, but implicitly predicts the prediction mode based on the decoding end, which saves the code rate occupied by the prediction mode encoding information and greatly reduces the encoding code rate of the intra-frame prediction mode.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

FIG. 1 is a schematic diagram showing an exemplary system architecture to which the method according to an embodiment of the present disclosure can be applied.

Fig. 2 is a flow chart showing an intra-frame coding method based on neighborhood features according to an exemplary embodiment.

Fig. 3 is a flowchart 1 showing a gradient feature analysis process according to an exemplary embodiment.

Fig. 4 is a second flowchart of a gradient feature analysis process according to an exemplary embodiment.

Fig. 5 is a schematic diagram showing a gradient feature analysis process according to an exemplary embodiment.

Fig. 6 is a flowchart showing a distortion calculation process according to an exemplary embodiment.

Fig. 7 is a schematic diagram showing a distortion calculation process according to an exemplary embodiment.

Fig. 8 is a flowchart showing a process of encoding a block to be encoded according to an exemplary embodiment.

Fig. 9 is a flowchart showing an intra-frame decoding method based on neighborhood features according to an exemplary embodiment.

Fig. 10 is a block diagram showing an intra-frame encoding apparatus based on neighborhood features according to an exemplary embodiment.

Fig. 11 is a block diagram showing an intra-frame decoding apparatus based on neighborhood features according to an exemplary embodiment.

FIG. 12 is a schematic diagram showing the structure of an electronic device suitable for implementing the exemplary embodiments of the present disclosure according to an exemplary embodiment.

DETAILED DESCRIPTION

Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

The features, structures or characteristics described in the present disclosure may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the present disclosure.

The accompanying drawings are only schematic diagrams of the present disclosure, and the same reference numerals in the drawings represent the same or similar parts, so their repeated description will be omitted. Some block diagrams shown in the accompanying drawings do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in at least one hardware module or integrated circuit, or implemented in different networks and/or processor devices and/or microcontroller devices.

The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and steps, nor must they be executed in the order described. For example, some steps can be decomposed, and some steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

In this specification, the terms "a", "an", "the", "said" and "at least one" are used to indicate the presence of at least one element/component/etc.; the terms "comprising", "including" and "having" are used to express an open-ended inclusion and mean that additional elements/components/etc. may exist in addition to the listed elements/components/etc.; the terms "first", "second" and "third" etc. are used only as labels and are not intended to limit the quantity of their objects.

FIG. 1 is a schematic diagram showing an exemplary system architecture to which the method according to an embodiment of the present disclosure can be applied.

As shown in Fig. 1, the system architecture may include a server 101, a network 102, a terminal device 103, a terminal device 104, and a terminal device 105. The network 102 is used to provide a medium for a communication link between the terminal device 103, the terminal device 104, or the terminal device 105 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

The server 101 may be a server that provides various services, such as a background management server that provides support for devices operated by users using the terminal device 103, the terminal device 104, or the terminal device 105. The background management server may analyze and process the received request and other data, and feed back the processing results to the terminal device 103, the terminal device 104, or the terminal device 105.

Terminal device 103, terminal device 104 and terminal device 105 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a wearable smart device, a virtual reality device, an augmented reality device, etc., but are not limited thereto.

It should be understood that the number of terminal devices 103, terminal devices 104, terminal devices 105, networks 102 and servers 101 in Figure 1 are merely schematic. Server 101 can be a physical server, a server cluster composed of multiple servers, or a cloud server. According to actual needs, it can have any number of terminal devices, networks and servers.

Below, each step of the method in the exemplary embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments.

Fig. 2 is a flow chart of a method for intra-frame coding based on neighborhood features according to an exemplary embodiment. The method provided in the embodiment of Fig. 2 is applied to a client and can be executed by any electronic device, such as the terminal device in Fig. 1, or the server in Fig. 1, or the terminal device and the server in Fig. 1 jointly, but the present disclosure does not limit this.

In step S210, gradient feature analysis is performed on reference pixels in a first neighborhood of a block to be encoded to determine a prediction model set; the prediction model set includes at least two prediction models.

In the related art, the two most critical issues in intra-frame prediction are the establishment of a prediction model and the encoding of a prediction mode. The establishment of a prediction model is to determine a matching prediction mode by analyzing reference information. The encoding of a prediction mode refers to encoding the prediction mode information selected by the encoding end. Since the decoding end lacks the prediction mode information selected by the encoding end, it is necessary to encode the prediction mode information after the prediction mode is determined at the encoding end, and the decoding end performs the corresponding prediction process according to the decoded prediction mode information. Although an increase in prediction modes can improve prediction efficiency, it will also increase the bit rate occupied by encoding the prediction mode information. Therefore, how to encode the prediction mode also determines the final coding efficiency. The disclosed embodiment provides an intra-frame coding method based on neighborhood features to solve the above-mentioned related technical problems.

In the embodiment of the present disclosure, since there is usually a strong correlation between the pixels within a certain range and the current pixel to be predicted, in order to utilize this correlation, feature analysis can be performed on the pixels within the neighborhood range, and the results of the feature analysis can be applied to the prediction process of the current block to increase the prediction prior and improve the prediction efficiency. Based on this, the present disclosure first determines the first neighborhood related to the current block to be encoded. The neighborhood refers to the area adjacent to the block to be encoded. Since the video encoding and decoding standard usually encodes and decodes from the upper left corner to the lower right corner of the image frame. Therefore, the reconstructed reference pixels are usually located on the left and upper sides of the block to be encoded. Referring to Figure 5, the present disclosure selects a certain number of rows and columns of reference pixels adjacent to the left and upper sides of the block to be encoded as the first neighborhood based on the block to be encoded. It should be noted that according to different video encoding and decoding algorithms, the selected first neighborhood can also be selected based on other methods, which is not limited by the present disclosure. Similarly, the present disclosure does not limit the number of rows and columns selected by the first neighborhood. The reference pixels in the first neighborhood are pixels that have been reconstructed based on the encoding algorithm.

In the disclosed embodiment, based on the selected first neighborhood, a gradient feature analysis is performed on the reference pixels in the first neighborhood. The gradient feature analysis is performed based on the gradient features of each pixel. Through the gradient feature analysis, the correlation between pixels can be analyzed to determine the texture direction of the neighborhood area. Considering the continuity between the neighborhood texture and the texture of the current block to be encoded, the texture direction of the current block to be encoded can be predicted.

As mentioned above, the current video coding standards provide a variety of prediction modes to choose from. For example, the H.266/VVC standard has been expanded to 65 prediction modes. Each prediction mode corresponds to a different texture direction angle. According to the texture direction angle of the block to be encoded, a matching prediction mode can be selected to obtain an accurate prediction result. The various pending prediction modes available for selection in these standards form a pending prediction mode set.

In the disclosed embodiment, by performing the gradient feature analysis on the reference pixels in the first neighborhood, the texture direction of the first neighborhood is obtained, and then a number of prediction modes with a high degree of matching with the texture direction are selected from the pending prediction mode set to form a prediction model set. The prediction mode set includes at least two prediction models.

It should be pointed out that the gradient features of the neighborhood and the gradient features of the current block to be encoded are not necessarily completely consistent. They can only reflect the possibility of the texture direction of the current block to be encoded, rather than an absolutely one-to-one correspondence. Therefore, there is a certain error in determining the final target prediction model based solely on the gradient features of the neighborhood, which may reduce the accuracy of the final prediction result and thus affect the coding efficiency. In the present disclosure, the target prediction mode is not directly selected based on the gradient features of the neighborhood, but a number of possible prediction modes are selected based on the gradient features of the neighborhood, which is equivalent to a preliminary screening of the prediction mode to form a prediction mode set for more accurate prediction mode selection in subsequent steps.

In step S220, based on the prediction model set, reference pixels in the second neighborhood are predicted by reference pixels in the third neighborhood to determine the distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is the neighborhood of the block to be encoded; and the third neighborhood is the neighborhood of the second neighborhood.

In the embodiment of the present disclosure, refer to the description of the neighborhood in the aforementioned step S210. Referring to Figure 7, in the present disclosure, a certain number of rows and columns of reference pixels adjacent to the left and upper sides of the block to be encoded 701 are selected as the second neighborhood 702. Similarly, a certain number of rows and columns of reference pixels adjacent to the left and upper sides of the second neighborhood 702 are selected as the third neighborhood 703. The second neighborhood 702 is a neighborhood for reference to the block to be encoded 701. The third neighborhood 703 is a neighborhood for reference to the second neighborhood 702. Referring to the aforementioned description, the area range selected by the second neighborhood 702 and the third neighborhood 703 and the number of rows and columns selected can be set differently according to actual conditions, and the present disclosure does not limit them. At the same time, as the neighborhood of the block to be encoded, the range of the first neighborhood and the second neighborhood may be the same or different.

In the embodiment of the present disclosure, the reference pixels in the second neighborhood 702 and the third neighborhood 703 are pixels that have been reconstructed based on the coding algorithm. Based on the reference pixels in the second neighborhood 702 and the third neighborhood 703, the pixels in the block to be coded are predicted and coded and reconstructed.

In the disclosed embodiment, since the third neighborhood is a neighborhood used as a reference for the second neighborhood. Therefore, the reference pixels in the second neighborhood can be predicted through the reference pixels in the third neighborhood based on the prediction mode. In the disclosed embodiment, based on the prediction model set initially screened in the aforementioned step S210, the reference pixels in the second neighborhood are predicted through the reference pixels in the third neighborhood according to several prediction models in the prediction model set, to obtain predicted pixels corresponding to the second neighborhood. The predicted pixels are in a one-to-one correspondence with each of the reference pixels that have been reconstructed in the second neighborhood.

In the disclosed embodiment, the distortion between the predicted pixel of the second neighborhood and the corresponding reference pixel in the second neighborhood can be calculated to obtain the corresponding distortion value. Since different predicted pixels can be obtained by using the reference pixel in the third neighborhood based on different prediction models, the distortion values corresponding to different prediction modes can also be calculated based on the above process.

In an exemplary embodiment, in the above distortion calculation process, a variety of distortion algorithms may be used, such as SATD (Sum of Absolute Transformed Differences), SAD (Sum of Absolute Differences), SSE (Sum of Squared Errors), etc. The present disclosure does not limit the specific distortion algorithm used.

In the embodiment of the present disclosure, the continuity of the local texture of the spatial domain of the image is taken into account, and the relevant distortion value is calculated not only based on the second neighborhood adjacent to the block to be encoded, but also based on the third neighborhood adjacent to the second neighborhood, so as to further participate in the selection of the prediction mode. By expanding the range of the reference neighborhood, the prediction mode can be selected with reference to information in a wider range. However, since the angle direction corresponding to the prediction mode is too dense, if this method is directly used to select the prediction mode, there may be prediction distortion due to the excessive number of angles, resulting in low accuracy of the determined prediction mode. At the same time, the algorithm has a large amount of calculation, and if this method is used for all prediction modes, it may bring unnecessary calculation overhead. Based on this, in the present disclosure, the set of pending prediction models is first screened through the aforementioned step S210, and several prediction models are selected as the prediction model set. The selection process of step S220 is performed based on the prediction model set, which can well make up for the shortcomings of the algorithm and reduce the calculation complexity of the related algorithm.

At the same time, since the algorithm provided by step S220 is no longer based on the distortion calculation of the original block and the predicted block, but is converted to the distortion calculation between the reconstructed reference pixels in the second neighborhood and the predicted pixels of the second neighborhood by the third neighborhood, the final target prediction mode is determined based on the distortion value. Based on this, explicit prediction mode encoding information is not required at the decoding end to know the prediction mode adopted, but the prediction mode adopted by the block to be predicted can be known based on the reference pixels in the second neighborhood and the third neighborhood during the decoding process, based on the same process as above. Therefore, based on this implicit prediction, the explicit identification of the prediction mode is avoided, the code rate occupied by the prediction mode encoding information is saved, and the encoding code rate of the intra-frame prediction mode is greatly reduced.

In step S230, a target prediction model is determined according to the distortion value.

In the disclosed embodiment, according to the distortion values corresponding to different prediction modes obtained in the aforementioned step S220, the distortion values of each prediction mode may be sorted to select a suitable prediction model from the prediction model set and determine it as the target prediction mode.

In an exemplary embodiment, the prediction model with the smallest distortion value may be determined as the target prediction model according to the sorting of distortion values.

In an exemplary embodiment, multiple prediction models may be selected from the prediction model set according to the distortion value sorting to be determined as the target prediction model.

In step S240, the block to be encoded is encoded according to the target prediction model.

In the disclosed embodiment, the block to be encoded is predicted according to the selected target prediction mode to obtain a corresponding prediction block, and then encoding is performed based on the prediction block to complete the encoding process of the block to be encoded.

In the embodiment of the present disclosure, by repeating the above encoding process for the block to be encoded, the encoding of the entire video frame can be completed.

The intra-frame encoding and decoding method based on neighborhood features provided by the embodiment of the present disclosure performs gradient feature analysis on the first neighborhood of the coding block, preliminarily screens the prediction model to determine a prediction model set; based on the prediction model set, the second neighborhood is predicted through the third neighborhood to determine the distortion value corresponding to each prediction model; the target prediction model is determined according to the distortion value, and then the coding block is encoded. Through this method, on the one hand, the prediction model is preliminarily screened through the gradient features of the first neighborhood to form a prediction model set; on the other hand, the reference neighborhood range is expanded through the second and third neighborhoods to determine a more matching prediction model. At the same time, through this method, the encoding end does not need to encode the prediction model information, but implicitly predicts the prediction mode based on the decoding end, which saves the code rate occupied by the prediction mode encoding information and greatly reduces the encoding code rate of the intra-frame prediction mode.

Fig. 3 is a flowchart 1 of a gradient feature analysis process according to an exemplary embodiment. Fig. 5 is a schematic diagram of a gradient feature analysis process according to an exemplary embodiment. In the disclosed embodiment, based on the intra-frame coding method based on neighborhood features shown in Fig. 2, the aforementioned step S210 may include the following steps.

In step S310, at least one sliding line is determined in the first neighborhood.

In the disclosed embodiment, at least one sliding line is determined in the first neighborhood of the block to be encoded. Gradient feature analysis of each reference pixel is performed based on the sliding line. The sliding line may be one or more.

In an exemplary embodiment, a sliding window is slid in the first field to perform feature analysis on the relevant reference pixels. Based on the sliding trajectory of the sliding window, a center line of the sliding window is determined as the sliding line.

In an exemplary embodiment, the upper three rows and left three columns of pixels adjacent to the current block to be encoded are used as reference pixels of the first neighborhood. A 3X3 sliding window is defined and slid along the first neighborhood, and a sliding line can be determined based on the center line of the sliding window.

In an exemplary embodiment, referring to FIG5 , the upper four rows and left four columns of pixels adjacent to the current block to be encoded are used as reference pixels of the first neighborhood. A 3×3 sliding window is defined, and two sliding tracks may be obtained by sliding along the first neighborhood. Based on the two sliding tracks, two sliding lines may be determined according to the center line of the sliding window.

It should be noted that the above embodiments are only exemplary embodiments of the present disclosure. According to actual application needs, the first neighborhood ranges of different numbers of rows and columns can be determined, and sliding windows of different sizes can also be determined. Based on the first neighborhoods of different ranges and sliding windows of different sizes, multiple different sliding lines can be determined, all of which should be considered to be within the protection scope of the present disclosure.

In step S320, the gradient intensity and gradient direction of each reference pixel on the sliding line are obtained.

In the disclosed embodiment, according to the sliding line, the gradient features of the reference pixels on each sliding line are respectively extracted to obtain the corresponding gradient strength and gradient direction.

In an exemplary embodiment, a sliding window is slid along the sliding line in the first neighborhood. For each reference pixel slid to, the horizontal gradient Gx and vertical gradient Gy of the pixel are obtained based on the sliding window using the Sobel operator. Based on the horizontal gradient Gx and the vertical gradient Gy, the gradient intensity I and gradient direction D corresponding to the pixel are calculated. Among them, the gradient intensity is used to indicate the amplitude of the change between the pixel and the adjacent pixels. The gradient direction is used to indicate the direction of change of the pixel gradient. The calculation formula is as follows:

I＝|Gx|+|Gy|;

D = arctan (Gy/Gx).

In step S330, the gradient intensity and gradient direction are counted to generate a gradient histogram corresponding to the sliding line.

In the disclosed embodiment, by taking statistics on the gradient strength and gradient direction, a corresponding gradient histogram (Histogram of Gradient, HoG) can be generated. The horizontal axis of the gradient histogram corresponds to the gradient direction. As mentioned above, each prediction model corresponds to an angular direction. Based on this, the aforementioned gradient direction is converted to the angular direction of the prediction mode closest to it. The horizontal axis coordinates of the gradient histogram correspond to the angular directions of each pending prediction mode. For example, the H.266/VVC standard includes 65 prediction modes. Corresponding to the prediction modes provided by the standard, the horizontal axis coordinates of the gradient histogram correspond to the angular directions of the 65 prediction modes. The vertical axis of the gradient histogram corresponds to the gradient strength. The gradient strengths of each pixel point corresponding to the same angular direction are accumulated to obtain the gradient strength value corresponding to the angular direction.

In step S340, the prediction model set is determined from the set of pending prediction models according to the gradient histogram, wherein the set of pending prediction models includes at least two pending prediction models, each of which corresponds to the gradient direction.

In the disclosed embodiment, the gradient histogram can be used to count the gradient strength values corresponding to different angular directions in the first neighborhood. According to the gradient histogram, the gradient strength values corresponding to each pending prediction model in the pending prediction model set can be known. The pending prediction model set is a prediction model set provided by the corresponding video coding standard. The pending prediction model set includes at least two pending prediction models. Each pending prediction model corresponds to a gradient direction, respectively. The larger the gradient strength value corresponding to the pending prediction model, the greater the gradient change intensity in the angular direction in the first neighborhood. By comparing the gradient strength values of each pending prediction model, a predetermined number of prediction modes can be selected therefrom to form the prediction model set.

In an exemplary embodiment, since the direct current prediction mode (DC mode) and the planar prediction mode (Planar mode) are added to the H.265/HEVC video codec standard, these two non-angle-direction-related prediction modes. Based on this, on the basis of the aforementioned determined prediction model set, two special prediction modes, the DC mode and the Planar mode, can also be added to form the prediction model set.

Fig. 4 is a second flowchart of a gradient feature analysis process according to an exemplary embodiment. Fig. 5 is a schematic diagram of a gradient feature analysis process according to an exemplary embodiment. In the disclosed embodiment, based on the gradient feature analysis process shown in Fig. 3, the gradient feature analysis process may include the following steps.

In step S410, at least one sliding line is determined in the first neighborhood, wherein the at least one sliding line includes: a first sliding line and a second sliding line.

In the embodiment of the present disclosure, multiple sliding lines can be determined in the first neighborhood. The multiple sliding lines may include: a first sliding line and a second sliding line. The first sliding line and the second sliding line may be any two sliding lines in the multiple sliding lines. Of course, more than two sliding lines can also be determined in the first neighborhood. The embodiment of the present disclosure only uses two sliding lines as an example for illustration.

In an exemplary embodiment, as shown in FIG5 , the upper four rows and left four columns of pixels adjacent to the current block to be encoded are used as reference pixels of the first neighborhood. A 3×3 sliding window is defined, and two sliding tracks may be formed by sliding along the first neighborhood. Based on the two sliding tracks, two sliding lines may be determined according to the center line of the sliding window, namely the first sliding line and the second sliding line.

In step S420, the gradient strength and gradient direction of each reference pixel on the first sliding line and the second sliding line are obtained respectively.

In the embodiment of the present disclosure, the gradient strength and gradient direction of each reference pixel on the sliding line are obtained according to the first sliding line and the second sliding line. The two sliding lines perform the pixel gradient feature analysis process independently. Since the gradient feature analysis process is similar to the aforementioned step S320, it will not be repeated here.

In step S430, the gradient strength and gradient direction are counted to generate a first gradient histogram and a second gradient histogram.

In the disclosed embodiment, for the gradient strength and gradient direction of each reference pixel obtained by the first sliding line, a first gradient histogram HoG1 corresponding to the first sliding line can be generated. For the gradient strength and gradient direction of each reference pixel obtained by the second sliding line, a second gradient histogram HoG2 corresponding to the second sliding line can be generated.

It should be noted that the horizontal axes of the first gradient histogram HoG1 and the second gradient histogram HoG2 correspond to the angle directions of each pending prediction mode, and therefore are consistent. The vertical axes of the first gradient histogram HoG1 and the second gradient histogram HoG2 independently count the gradient intensity values of each reference pixel on the corresponding sliding line.

In step S440, the prediction model set is determined from the pending prediction model set according to the first gradient histogram and the second gradient histogram.

In the disclosed embodiment, a plurality of prediction models are determined from the set of pending prediction models according to the first gradient histogram and the second gradient histogram to form the set of prediction models.

In an exemplary embodiment, the first prediction model set is determined from the pending prediction model set according to the first gradient histogram. The second prediction model set is determined from the pending prediction model set according to the second gradient histogram. The process of determining the first prediction model set and the second prediction model set is similar to the aforementioned step S340 and will not be repeated here. After obtaining the corresponding first prediction model set and second prediction model set based on two sliding lines respectively, the prediction model set is determined according to the first prediction model set and the second prediction model set. Since some prediction modes in the first prediction model set and the second prediction model set may overlap, the prediction model set can be determined based on the union of the first prediction model set and the second prediction model set.

In an exemplary embodiment, different numbers of prediction models can be selected from the first prediction model set and the second prediction model set. The first prediction model set includes M prediction models. The first prediction model set includes N prediction models. Among them, M is greater than N. The first sliding line corresponding to the first prediction model set is closer to the block to be encoded than the second sliding line corresponding to the second prediction model set. As shown in Figure 5, the two sliding lines are at different distances from the block to be encoded. Since it can be considered that the closer the distance to the block to be encoded, the higher its correlation with the block to be encoded. Therefore, more prediction models can be selected based on closer sliding lines, and fewer prediction models can be selected based on farther sliding lines. For example, according to the first gradient histogram HoG1, the top 6 prediction models are selected to form the first prediction model set, and according to the second gradient histogram HoG2, the top 4 prediction models are selected to form the second prediction model set.

In an exemplary embodiment, since the horizontal axes of the first gradient histogram HoG1 and the second gradient histogram HoG2 correspond to the angular directions of each pending prediction mode. Therefore, the first gradient histogram HoG1 and the second gradient histogram HoG2 can be superimposed, and the gradient intensity values of the corresponding horizontal coordinates can be superimposed to obtain a superimposed gradient histogram. The superimposed gradient histogram can comprehensively reflect the gradient characteristics of each reference pixel counted on the two sliding lines. According to the superimposed gradient histogram, the prediction model set is determined from the pending prediction model set. The determination process of the prediction model set is similar to the aforementioned step S340, which will not be repeated here.

The intra-frame encoding and decoding method based on neighborhood features provided by the embodiment of the present disclosure does not directly select the target prediction mode based on the gradient features of the neighborhood, but selects several possible prediction modes based on the gradient features of the neighborhood, which is equivalent to a preliminary screening of the prediction mode to form a prediction mode set for more accurate prediction mode selection in subsequent steps. In this way, the error problem of determining the final target prediction model solely by relying on the gradient features of the neighborhood can be avoided. At the same time, the method supports determining multiple sliding lines for the first neighborhood, and analyzing the gradient features of the neighborhood respectively, thereby greatly expanding the reference range of the gradient feature analysis and improving the accuracy of the final prediction model.

Fig. 6 is a flow chart of a distortion calculation process according to an exemplary embodiment. Fig. 7 is a schematic diagram of a distortion calculation process according to an exemplary embodiment. In the disclosed embodiment, based on the intra-frame coding method based on neighborhood features shown in Fig. 2, the aforementioned step S220 may include the following steps.

In step S610, the second neighborhood is determined according to the block to be encoded.

In the embodiment of the present disclosure, as shown in FIG7 , a preset area range adjacent to the block to be encoded 701 is determined as the second neighborhood 702. As mentioned above, a certain number of rows and columns of reference pixels adjacent to the left and upper sides of the block to be encoded 701 can be selected as the second neighborhood 702. The reference pixels in the second neighborhood 702 are all pixels that have been reconstructed based on the encoding algorithm.

In step S620, the third neighborhood is determined according to the second neighborhood.

In the embodiment of the present disclosure, as shown in FIG7 , the preset area range adjacent to the second neighborhood 702 is determined as the third neighborhood 703. As mentioned above, a certain number of rows and columns of reference pixels adjacent to the left and upper sides of the second neighborhood 702 can be selected as the third neighborhood 703. The reference pixels in the third neighborhood 703 are all pixels that have been reconstructed based on the encoding algorithm.

In step S630, based on each prediction model in the prediction model set, the reference pixels in the second neighborhood are predicted according to the reference pixels in the third neighborhood to obtain predicted pixels corresponding to the second neighborhood.

In the disclosed embodiment, since the third neighborhood is a neighborhood used as a reference for the second neighborhood. Therefore, the reference pixels in the second neighborhood can be predicted based on the prediction mode through the reference pixels in the third neighborhood. The prediction process is the same as predicting the current block to be encoded based on the reference pixels in the second neighborhood. The continuity of the local texture in the spatial domain of the image is taken into account, so the prediction distortion of the block to be encoded and the second neighborhood can be derived based on the prediction distortion of the second neighborhood and the third neighborhood.

In the disclosed embodiment, based on the prediction model set obtained by the preliminary screening, reference pixels in the second neighborhood are predicted by several prediction models in the prediction model set through reference pixels in the third neighborhood, to obtain predicted pixels corresponding to the second neighborhood. The predicted pixels are in one-to-one correspondence with each of the reconstructed reference pixels in the second neighborhood.

In step S640, the distortion between the predicted pixel of the second neighborhood and the corresponding reference pixel of the second neighborhood is calculated to obtain the distortion value corresponding to each prediction model in the prediction model set.

In the disclosed embodiment, the distortion between the predicted pixel of the second neighborhood and the corresponding reference pixel in the second neighborhood can be calculated to obtain the corresponding distortion value. Since different predicted pixels can be obtained by using the reference pixel in the third neighborhood based on different prediction models, the distortion values corresponding to different prediction modes can also be calculated based on the above process.

In an exemplary embodiment, in the above distortion calculation process, a variety of distortion algorithms may be used, such as SATD (Sum of Absolute Transformed Differences), SAD (Sum of Absolute Differences), SSE (Sum of Squared Errors), etc. The present disclosure does not limit the specific distortion algorithm used.

In an exemplary embodiment, RT is a reconstructed reference pixel template in the second neighborhood, _and ^pTm is a prediction result of each prediction mode corresponding to the corresponding pixel template in the second neighborhood. Wherein, m represents different prediction modes. Based on the corresponding ^RT and ^pTm _, SATD can be used as a distortion evaluation criterion to calculate the distortion value corresponding to each prediction model in the prediction model set, and then sort the prediction models.

In the disclosed embodiment, the continuity of the local texture in the spatial domain of the image is taken into account, and the relevant distortion value is calculated not only based on the second neighborhood adjacent to the block to be encoded, but also based on the third neighborhood adjacent to the second neighborhood, so as to further participate in the selection of the prediction mode. By expanding the range of the reference neighborhood, the prediction mode can be selected with reference to information in a wider range. However, since the angle direction corresponding to the prediction mode is too dense, if this method is directly used to select the prediction mode, there may be prediction distortion due to the excessive number of angles, resulting in low accuracy of the determined prediction mode. At the same time, the algorithm has a large amount of calculation, and if this method is used for all prediction modes, it may bring unnecessary calculation overhead. Based on this, in the present disclosure, the above-mentioned set of pending prediction models is first screened, and several prediction models are selected as the prediction model set, and then the above-mentioned distortion calculation process is performed based on the prediction model set, which can well make up for the shortcomings of the algorithm and reduce the calculation complexity of the related algorithms.

At the same time, since the distortion calculation process is no longer based on the original block and the predicted block, but is converted to the distortion calculation between the reconstructed reference pixels in the second neighborhood and the predicted pixels of the third neighborhood to the second neighborhood, the final target prediction mode is determined based on the distortion value. Based on this, explicit prediction mode encoding information is not required at the decoding end to know the prediction mode adopted, but the prediction mode adopted by the block to be predicted can be known based on the reference pixels in the second neighborhood and the third neighborhood during the decoding process, based on the same process as above. Therefore, based on this implicit prediction, the explicit identification of the prediction mode is avoided, the code rate occupied by the prediction mode encoding information is saved, and the encoding code rate of the intra-frame prediction mode is greatly reduced.

Fig. 8 is a flow chart showing a process of encoding a block to be encoded according to an exemplary embodiment. In the embodiment of the present disclosure, based on the intra-frame encoding method based on neighborhood features shown in Fig. 2, the aforementioned step S240 may include the following steps.

In step S810, the block to be encoded is encoded according to the first prediction model to obtain a first prediction block.

As mentioned above, multiple prediction models can be selected from the prediction model set according to the distortion value sorting to determine the target prediction model. Based on this, the target prediction model can at least include: a first prediction model and a second prediction model. The first prediction model and the second prediction model can be any prediction model selected from the prediction model set.

In the disclosed embodiment, the block to be encoded is predicted according to the selected first prediction model to obtain a corresponding first prediction block.

In step S820, the block to be encoded is encoded according to the second prediction model to obtain a second prediction block.

In the disclosed embodiment, the block to be encoded is predicted according to the selected second prediction model to obtain a corresponding second prediction block.

In step S830, a coding block of the block to be coded is generated according to the first prediction block and the second prediction block.

In the embodiment of the present disclosure, a merging prediction is performed based on the first prediction block and the second prediction block to generate a coding block of the block to be coded. By this method, it is not limited to predicting and coding the block to be coded based on a single prediction model, but predicting and coding the block to be coded based on multiple prediction models.

In an exemplary embodiment, different weight values can be set for the selected multiple prediction models. Based on the weight values, the first prediction block and the second prediction block are weighted predicted to obtain the coding block. By setting different weight values, the weight ratio between the multiple prediction models can be adjusted, so as to perform weighted prediction coding more accurately. See the formula as follows:

P＝a·P1+b·P2

Among them, P is the coding block, P1 is the first prediction block, P2 is the second prediction block, a is the weight value of the first prediction model, and b is the weight value of the second prediction model.

In an exemplary embodiment, the weight values of the first prediction model and the second prediction model may be preset weight values.

In an exemplary embodiment, the weight values of the first prediction model and the second prediction model can be determined according to the corresponding distortion values obtained by the above calculation. The smaller the distortion value of the prediction model, the larger the corresponding weight value; the larger the distortion value of the prediction model, the smaller the corresponding weight value. For example, the weight value of the prediction model is set to the inverse of its distortion value.

In an exemplary embodiment, the weight values of the first prediction model and the second prediction model can be determined according to the corresponding gradient strength values obtained by the above calculation. The smaller the gradient strength value of the prediction model, the smaller the corresponding weight value; the larger the gradient strength value of the prediction model, the larger the corresponding weight value. For example, a linear proportional relationship between the weight value of the prediction model and its corresponding gradient strength value can be set.

In an exemplary embodiment, the weight values of the first prediction model and the second prediction model can be determined according to the corresponding distortion value and gradient strength value obtained by the above calculation. The weight value can be determined by designing a functional relationship between the weight value and its corresponding distortion value and gradient strength value. The specific functional relationship is not specifically limited here.

The intra-frame encoding and decoding method based on neighborhood features provided by the embodiment of the present disclosure uses multiple prediction models to form a target prediction model, thereby reducing the possibility of a single prediction model generating prediction errors and improving the accuracy of the final prediction block. At the same time, the method provides a variety of prediction model weight value setting schemes, which can be applied to a variety of application scenarios.

Fig. 9 is a flow chart of a method for intra-frame decoding based on neighborhood features according to an exemplary embodiment. The method provided in the embodiment of Fig. 9 is applied to a client and can be executed by any electronic device, such as the terminal device in Fig. 1 above, or the server in Fig. 1, or the terminal device and the server in Fig. 1 jointly, but the present disclosure does not limit this.

In step S910, gradient feature analysis is performed on reference pixels in a first neighborhood of a block to be decoded to determine a prediction model set; the prediction model set includes at least two prediction models.

In step S920, based on the prediction model set, reference pixels in the second neighborhood are predicted by reference pixels in the third neighborhood to determine the distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is the neighborhood of the block to be decoded; and the third neighborhood is the neighborhood of the second neighborhood.

In step S930, a target prediction model is determined according to the distortion value.

In step S940, the block to be decoded is decoded according to the target prediction model.

In the embodiment of the present disclosure, the decoding end can use the same intra-frame prediction processing flow as the encoding end to decode the encoded video frame. The decoder first determines the first neighborhood of the block to be decoded. The determination rule of the neighborhood range is the same as that of the encoding end, so the gradient feature analysis can be performed based on the reference pixels in the same first neighborhood of the encoding end, and then the prediction model set is determined. Since the same strategy is adopted as the encoding end, the prediction model set selected at the decoding end is also the same as the encoding end. Similarly, the decoding end selects the second neighborhood and the third neighborhood based on the same strategy as the encoding end. In addition, the algorithm is no longer based on the original block and the predicted block for distortion calculation, but is converted to the distortion calculation between the reconstructed reference pixels in the second neighborhood and the predicted pixels of the third neighborhood for the second neighborhood, and then the final target prediction mode is determined based on the distortion value. The block to be decoded is decoded by the target prediction model, and it can be decoded into the original video block. In the present disclosure, by adopting the same algorithm strategy as the encoding end at the decoding end, the decoding end can completely restore the reverse process of the encoding end, thereby realizing decoding. At the same time, the design structure of the terminal video codec is greatly simplified, and the implementation cost of the related video codec algorithm is reduced. Based on this design concept, the contents disclosed in the aforementioned embodiments related to the encoding end can all be referred to as the contents of the decoding end embodiments and will not be repeated here.

In the disclosed embodiment, explicit prediction mode coding information is not required at the decoding end to learn the prediction mode used, but the prediction mode used by the block to be predicted can be learned based on the reference pixels in the second neighborhood and the third neighborhood during the decoding process, based on the same process as above. Therefore, based on this implicit prediction, the explicit identification of the prediction mode is avoided, the bit rate occupied by the prediction mode coding information is saved, and the encoding bit rate of the intra-frame prediction mode is greatly reduced. At the same time, since the same decoding strategy is adopted as the encoding end, the design structure of the terminal video codec can be greatly simplified, and the implementation cost of the related video codec algorithm can be reduced.

The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

Fig. 10 is a block diagram of an intra-frame encoding device based on neighborhood features according to an exemplary embodiment. Referring to Fig. 10 , the intra-frame encoding device 1000 based on neighborhood features may include: a first gradient analysis module 1010, a first distortion calculation module 1020, a first prediction model determination module 1030 and an encoding module 1040.

The first gradient analysis module 1010 is configured to perform gradient feature analysis on reference pixels in a first neighborhood of the block to be encoded to determine a prediction model set; the prediction model set includes at least two prediction models.

The first distortion calculation module 1020 is configured to predict the reference pixels in the second neighborhood through the reference pixels in the third neighborhood based on the prediction model set, and determine the distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is the neighborhood of the block to be encoded; and the third neighborhood is the neighborhood of the second neighborhood.

The first prediction model determination module 1030 is configured to determine a target prediction model according to the distortion value.

The encoding module 1040 is configured to encode the block to be encoded according to the target prediction model.

In some exemplary embodiments of the present disclosure, the first gradient analysis module 1010 is further configured to determine at least one sliding line in the first neighborhood; obtain the gradient strength and gradient direction of each reference pixel on the sliding line; perform statistics on the gradient strength and gradient direction to generate a gradient histogram corresponding to the sliding line; determine the prediction model set from the set of pending prediction models according to the gradient histogram; wherein the set of pending prediction models includes at least two pending prediction models; and each of the pending prediction models corresponds to the gradient direction, respectively.

In some exemplary embodiments of the present disclosure, the first gradient analysis module 1010 is further configured to perform gradient feature analysis on the reference pixel by sliding a sliding window in the first neighborhood; and determine a center line of the sliding window as the sliding line.

In some exemplary embodiments of the present disclosure, the first gradient analysis module 1010 is further configured such that the at least one sliding line includes a first sliding line and a second sliding line; the statistics of the gradient strength and gradient direction to generate a gradient histogram corresponding to the sliding line includes: statistics of the gradient strength and gradient direction of the first sliding line to generate a first gradient histogram corresponding to the first sliding line; statistics of the gradient strength and gradient direction of the second sliding line to generate a second gradient histogram corresponding to the second sliding line.

In some exemplary embodiments of the present disclosure, the first gradient analysis module 1010 is also configured to determine a first prediction model set from the set of pending prediction models based on the first gradient histogram; determine a second prediction model set from the set of pending prediction models based on the second gradient histogram; and determine the prediction model set based on the first prediction model set and the second prediction model set.

In some exemplary embodiments of the present disclosure, the first gradient analysis module 1010 is also configured so that the first prediction model set includes M prediction models, and the first prediction model set includes N prediction models; wherein M is greater than N; and the first sliding line is closer to the block to be encoded than the second sliding line.

In some exemplary embodiments of the present disclosure, the first gradient analysis module 1010 is further configured to superimpose the first gradient histogram and the second gradient histogram to obtain a superimposed gradient histogram; and determine the prediction model set from the pending prediction model set based on the superimposed gradient histogram.

In some exemplary embodiments of the present disclosure, the first distortion calculation module 1020 is further configured to determine the second neighborhood according to the block to be encoded; determine the third neighborhood according to the second neighborhood; predict the reference pixels in the second neighborhood according to the reference pixels in the third neighborhood based on each prediction model in the prediction model set, to obtain predicted pixels corresponding to the second neighborhood; calculate the distortion between the predicted pixels of the second neighborhood and the corresponding reference pixels of the second neighborhood, to obtain the distortion values corresponding to each prediction model in the prediction model set.

In some exemplary embodiments of the present disclosure, the encoding module 1040 is also configured that the target prediction model includes at least a first prediction model and a second prediction model; the first prediction model and the second prediction model are selected from the prediction model set; the block to be encoded is encoded according to the first prediction model to obtain a first prediction block; the block to be encoded is encoded according to the second prediction model to obtain a second prediction block; and a coding block of the block to be encoded is generated based on the first prediction block and the second prediction block.

In some exemplary embodiments of the present disclosure, the encoding module 1040 is also configured to determine the weight values corresponding to the first prediction model and the second prediction model according to the distortion value and/or gradient strength corresponding to the prediction model; based on the weight value, perform weighted prediction on the first prediction block and the second prediction block to obtain the encoding block.

Fig. 11 is a block diagram of an intra-frame decoding device based on neighborhood features according to an exemplary embodiment. Referring to Fig. 11 , the intra-frame decoding device based on neighborhood features 1100 may include: a second gradient analysis module 1110, a second distortion calculation module 1120, a second prediction model determination module 1130 and a decoding module 1140.

The second gradient analysis module 1110 is configured to perform gradient feature analysis on reference pixels in a first neighborhood of the block to be decoded to determine a prediction model set; the prediction model set includes at least two prediction models.

The second distortion calculation module 1120 is configured to predict the reference pixels in the second neighborhood through the reference pixels in the third neighborhood based on the prediction model set, and determine the distortion values corresponding to each prediction model in the prediction model set; wherein the second neighborhood is the neighborhood of the block to be decoded; and the third neighborhood is the neighborhood of the second neighborhood.

The second prediction model determination module 1130 is configured to determine a target prediction model according to the distortion value.

The decoding module 1140 is configured to decode the block to be decoded according to the target prediction model.

Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

The electronic device 1200 according to this embodiment of the present disclosure is described below with reference to Fig. 12. The electronic device 1200 shown in Fig. 12 is only an example and should not bring any limitation to the functions and scope of use of the embodiment of the present disclosure.

As shown in FIG12 , the electronic device 1200 is presented in the form of a general-purpose computing device. The components of the electronic device 1200 may include, but are not limited to: the at least one processing unit 1210, the at least one storage unit 1220, a bus 1230 connecting different system components (including the storage unit 1220 and the processing unit 1210), and a display unit 1240.

The storage unit stores program codes, which can be executed by the processing unit 1210, so that the processing unit 1210 executes the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 1210 can execute the steps shown in FIG. 2.

For another example, the electronic device can implement the steps shown in FIG. 2 .

The storage unit 1220 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 1221 and/or a cache memory unit 1222 , and may further include a read-only memory unit (ROM) 1223 .

The storage unit 1220 may also include a program/utility 1224 having a set (at least one) of program modules 1225, such program modules 1225 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

Bus 1230 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 1200 may also communicate with one or more external devices 1270 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1200, and/or communicate with any device that enables the electronic device 1200 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input/output (I/O) interface 1250. In addition, the electronic device 1200 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and/or public networks, such as the Internet) via a network adapter 1260. As shown, the network adapter 1260 communicates with other modules of the electronic device 1200 via a bus 1230. It should be understood that, although not shown in the figure, other hardware and/or software modules may be used in conjunction with the electronic device 1200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiment of the present disclosure.

In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to complete the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

In an exemplary embodiment, a computer program product is also provided, including a computer program/instruction, and when the computer program/instruction is executed by a processor, the method in the above embodiment is implemented.

Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (15)

1. An intra-frame encoding method based on a neighborhood feature, comprising:

performing gradient feature analysis on reference pixels in a first neighborhood of a block to be coded to determine a prediction model set, wherein the prediction model set comprises at least two prediction models;

based on the prediction model set, predicting reference pixels in a second neighborhood through reference pixels in a third neighborhood, and determining distortion values corresponding to each prediction model in the prediction model set, wherein the second neighborhood is the neighborhood of the block to be encoded;

determining a target prediction model according to the distortion value;

And encoding the block to be encoded according to the target prediction model.

2. The method of claim 1, wherein the performing gradient feature analysis on the reference pixels in the first neighborhood of the block to be encoded to determine the set of prediction models comprises:

Determining at least one sliding line in the first neighborhood;

Acquiring gradient strength and gradient direction of each reference pixel on the sliding line;

Counting the gradient strength and the gradient direction to generate a gradient histogram corresponding to the sliding line;

Determining a prediction model set from a set of undetermined prediction models according to the gradient histogram;

the set of the undetermined prediction models comprises at least two undetermined prediction models, and each undetermined prediction model corresponds to the gradient direction.

3. The method of claim 2, wherein the at least one sliding line comprises a first sliding line and a second sliding line;

The step of counting the gradient intensity and the gradient direction to generate a gradient histogram corresponding to the sliding line comprises the following steps:

Counting the gradient intensity and the gradient direction of the first sliding line, and generating a first gradient histogram corresponding to the first sliding line;

and counting the gradient intensity and the gradient direction of the second sliding line, and generating a second gradient histogram corresponding to the second sliding line.

4. A method according to claim 3, wherein said determining said set of predictive models from a set of predictive models to be determined from said gradient histogram comprises:

determining a first prediction model set from the undetermined prediction model sets according to the first gradient histogram;

Determining a second prediction model set from the undetermined prediction model sets according to the second gradient histogram;

And determining the prediction model set according to the first prediction model set and the second prediction model set.

5. The method of claim 4, wherein the step of determining the position of the first electrode is performed,

The first prediction model set comprises M prediction models, the first prediction model set comprises N prediction models, M is larger than N, and the first sliding line is closer to the block to be encoded than the second sliding line.

6. A method according to claim 3, wherein said determining said set of predictive models from a set of predictive models to be determined from said gradient histogram comprises:

Superposing the first gradient histogram and the second gradient histogram to obtain a superposed gradient histogram;

and determining the prediction model set from the undetermined prediction model set according to the superimposed gradient histogram.

7. The method according to claim 1, wherein the predicting, based on the prediction model set, the reference pixel in the second neighborhood by the reference pixel in the third neighborhood, and determining the distortion value corresponding to each prediction model in the prediction model set, includes:

determining the second neighborhood according to the block to be coded;

Determining the third neighborhood according to the second neighborhood;

predicting the reference pixels in the second neighborhood according to the reference pixels in the third neighborhood based on each prediction model in the prediction model set respectively to obtain prediction pixels corresponding to the second neighborhood;

and calculating the distortion between the predicted pixels of the second neighborhood and the corresponding reference pixels of the second neighborhood to obtain distortion values corresponding to each prediction model in the prediction model set.

8. The method of claim 1, wherein the target prediction model comprises at least a first prediction model and a second prediction model, the first prediction model and the second prediction model selected from the set of prediction models;

The encoding the block to be encoded according to the target prediction model includes:

Coding the block to be coded according to the first prediction model to obtain a first prediction block;

Coding the block to be coded according to the second prediction model to obtain a second prediction block;

and generating the coding block of the block to be coded according to the first prediction block and the second prediction block.

9. The method of claim 8, wherein generating the encoded block of the block to be encoded from the first and second prediction blocks comprises:

determining weight values corresponding to the first prediction model and the second prediction model according to the distortion values and/or gradient strength corresponding to the prediction models;

And carrying out weighted prediction on the first prediction block and the second prediction block based on the weight value to obtain the coding block.

10. An intra-frame decoding method based on a neighborhood feature, comprising:

performing gradient feature analysis on reference pixels in a first neighborhood of a block to be decoded to determine a prediction model set, wherein the prediction model set comprises at least two prediction models;

based on the prediction model set, predicting reference pixels in a second neighborhood through reference pixels in a third neighborhood, and determining distortion values corresponding to each prediction model in the prediction model set, wherein the second neighborhood is the neighborhood of the block to be decoded;

determining a target prediction model according to the distortion value;

and decoding the block to be decoded according to the target prediction model.

11. An intra-coding apparatus based on neighborhood characteristics, comprising:

the first gradient analysis module is configured to perform gradient feature analysis on reference pixels in a first neighborhood of a block to be coded and determine a prediction model set, wherein the prediction model set comprises at least two prediction models;

the first distortion calculation module is configured to predict reference pixels in a second neighborhood through reference pixels in a third neighborhood based on the prediction model set, and determine distortion values corresponding to each prediction model in the prediction model set;

A first predictive model determination module configured to determine a target predictive model from the distortion value;

and the encoding module is configured to encode the block to be encoded according to the target prediction model.

12. An intra decoding apparatus based on a neighborhood feature, comprising:

the second gradient analysis module is configured to perform gradient feature analysis on reference pixels in a first neighborhood of the block to be decoded and determine a prediction model set, wherein the prediction model set comprises at least two prediction models;

The second distortion calculation module is configured to predict reference pixels in a second neighborhood through reference pixels in a third neighborhood based on the prediction model set, and determine distortion values corresponding to each prediction model in the prediction model set;

A second predictive model determination module configured to determine a target predictive model from the distortion value;

and the decoding module is configured to decode the block to be decoded according to the target prediction model.

13. An electronic device, comprising:

A processor;

a memory for storing the processor-executable instructions;

Wherein the processor is configured to execute the executable instructions to implement the neighborhood feature based intra-coding method of any of claims 1 to 9 or the neighborhood feature based intra-decoding method of claim 10.

14. A computer readable storage medium, which when executed by a processor of an electronic device, causes the electronic device to perform the neighborhood feature-based intra-coding method of any one of claims 1 to 9 or the neighborhood feature-based intra-decoding method of claim 10.

15. A computer program product comprising computer program/instructions which, when executed by a processor, implement the neighborhood feature based intra-coding method of any one of claims 1 to 9 or the neighborhood feature based intra-decoding method of claim 10.