CN113473135B - A nonlinear texture-oriented intra prediction method, device and medium - Google Patents

Abstract

The present disclosure relates to a nonlinear texture-oriented intra prediction method, device and medium, the method is used in an intra prediction module in the field of image and video coding, and the method includes: determining a current intra prediction mode; predicting a non-linear texture in the intra prediction module using predictive modeling comprising a quadratic function; the position of the reference pixel is derived from the result of predicting the nonlinear texture and the predicted pixel value is generated from the interpolation of the reference pixel. The method solves the problem of high-efficiency prediction modeling of the image and video coding and decoding standard intra-frame prediction module facing nonlinear texture content. The method and the device can generate the high-fidelity prediction signal close to the original signal, reduce the prediction residual error and improve the coding efficiency.

Description

A nonlinear texture-oriented intra prediction method, device and medium

technical field

The present disclosure relates to the technical field of intra-frame prediction, and more specifically, the present disclosure relates to an intra-frame prediction method, device and medium for nonlinear texture.

Background technique

In image and video coding, intra-frame prediction is mainly used to reduce the spatial redundancy of signals. Mainstream video codec standards (such as VVC, AVS3, etc.) define a series of intra prediction angle modes in the intra prediction module to generate prediction content. However, the current angle prediction mode can only generate linear textures, and cannot accurately and efficiently model nonlinear textures. Therefore, the present invention proposes a non-linear texture-oriented intra-frame prediction algorithm to improve the efficiency of intra-frame prediction and improve the coding efficiency.

There are 67 types of intra prediction modes in VVC, including DC mode, Planar mode and other 65 angle prediction modes. The intra prediction angle mode defines the direction of the intra prediction, that is, the reference pixel is projected to the corresponding position of the prediction block through this direction to form the prediction block. Taking FIG. 1 as an example, the angle α corresponds to the tangent angle of a certain intra-frame prediction direction, and the intensity of pixels at positions passed by it is the same. Therefore, for the prediction pixel p[x0][y0], its corresponding reference pixel position c can be derived by the following formula, and its pixel intensity can be generated by multi-tap filter interpolation.

c＝x ₀ -tanα*y ₀ (1)

p[x ₀ ][y ₀ ]=f[0]*p[c][0]+f[1]*p[c][1]+f[2]*p[c][2]+f [3]*p[c][3] (2).

Contents of the invention

In order to solve the technical problem that the image and video coding standards in the prior art cannot generate non-linear texture content close to the original signal in the intra prediction module.

In order to achieve the above technical purpose, the present disclosure provides an intra prediction method oriented to nonlinear texture, including:

Determine the current intra prediction mode;

predicting non-linear textures using predictive modeling involving quadratic functions in said intra prediction module;

The position of the reference pixel is derived according to the result of predicting the nonlinear texture, and the predicted pixel value is generated according to the interpolation of the reference pixel.

Further, the intra prediction mode specifically includes:

Normal prediction mode close to reference pixels and extended prediction mode far away from reference pixels.

Further, the predictive modeling is a model using a quadratic function or a model using a linear combination of a linear function and a quadratic function.

Further, the prediction of nonlinear texture using predictive modeling including quadratic function in the intra prediction module specifically includes:

The predictive modeling involving quadratic functions to predict non-linear textures is expressed using two angular prediction modes.

Further, the two angle prediction modes belong to the vertical prediction mode set or the horizontal prediction mode set.

Further, the deriving the reference pixel according to the result of predicting the nonlinear texture specifically includes:

When the two angle prediction modes belong to the vertical prediction mode set,

use formula

Determine the reference pixel position c;

Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block;

x ₀ represents the abscissa, y ₀ represents the ordinate, and h represents the height of the intra prediction block;

use formula

p[x ₀ ][y ₀ ]=f[0]*p[c][0]+f[1]*p[c][1]+f[2]*p[c][2]+f [3]*p[c][3]

determining the position of the intra-prediction pixel;

Among them, p[x0][y0] represents the intra prediction pixel;

When the two angle prediction modes belong to the horizontal prediction mode set,

use formula

Determine the reference pixel position c;

Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block;

x ₀ represents the abscissa, y ₀ represents the ordinate, and w represents the width of the intra prediction block;

use formula

p[x ₀ ][y ₀ ]=f[0]*p[0][c]+f[1]*p[0][c+1]+f[2]*p[0][c+ 2]+f[3]*p[0][c+3]

determining the position of the intra-prediction pixel;

Among them, p[x0][y0] represents an intra prediction pixel.

Further, the generating the predicted pixel value according to the position of the reference pixel specifically adopts Gaussian interpolation or cubic spline interpolation method to generate the predicted pixel value according to the position of the reference pixel.

Further, the combination method of the angle prediction mode is based on a model of statistical laws, and also based on a data-driven method based on deep learning or machine learning.

To achieve the above technical purpose, the present disclosure can also provide a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, it is used to realize the steps of the above nonlinear texture-oriented intra prediction method.

In order to achieve the above-mentioned technical purpose, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and operable on the processor. The processor executes the computer program to realize the above-mentioned non-linear texture-oriented The steps of the intra prediction method.

The beneficial effects of the disclosure are:

The disclosure solves the problem of high-efficiency predictive modeling for non-linear texture content of standard intra-frame predictive modules of image and video codecs. The disclosure can generate a high-fidelity prediction signal close to the original signal, reduce prediction residuals, and improve coding efficiency.

Description of drawings

FIG. 1 shows a schematic diagram of an intra prediction method in the prior art;

FIG. 2 shows a schematic flow diagram of Embodiment 1 of the present disclosure;

FIG. 3 shows a schematic diagram of an intra prediction method according to Embodiment 1 of the present disclosure;

FIG. 4 shows a schematic structural diagram of Embodiment 3 of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. It should be understood, however, that these descriptions are exemplary only, and are not intended to limit the scope of the present disclosure. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concept of the present disclosure.

Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. The figures are not drawn to scale, with certain details exaggerated and possibly omitted for clarity of presentation. The shapes of the various regions and layers shown in the figure, as well as their relative sizes and positional relationships are only exemplary, and may deviate due to manufacturing tolerances or technical limitations in practice, and those skilled in the art will Regions/layers with different shapes, sizes, and relative positions can be additionally designed as needed.

Embodiment one:

as shown in picture 2:

The present disclosure provides an intra prediction method oriented to nonlinear texture, including:

Determine the current intra prediction mode;

predicting non-linear textures using predictive modeling involving quadratic functions in said intra prediction module;

The reference pixel is derived according to the result of predicting the nonlinear texture, and the predicted pixel value is generated according to the position of the reference pixel.

Further, the intra prediction mode specifically includes:

Normal prediction mode close to reference pixels and extended prediction mode far away from reference pixels.

Further, the predictive modeling is a model using a quadratic function or a model using a linear combination of a linear function and a quadratic function.

Further, the prediction of nonlinear texture using predictive modeling including quadratic function in the intra prediction module specifically includes:

The predictive modeling involving quadratic functions to predict non-linear textures is expressed using two angular prediction modes.

Further, the two angle prediction modes belong to the vertical prediction mode set or the horizontal prediction mode set.

Further, the deriving the reference pixel according to the result of predicting the nonlinear texture specifically includes:

When the two angle prediction modes belong to the vertical prediction mode set,

use formula

Determine the reference pixel position c;

Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block;

x ₀ represents the abscissa, y ₀ represents the ordinate, and h represents the height of the intra prediction block;

use formula

p[x ₀ ][y ₀ ]=f[0]*p[c][0]+f[1]*p[c][1]+f[2]*p[c][2]+f [3]*p[c][3]

determining the position of the intra-prediction pixel;

Among them, p[x0][y0] represents the intra prediction pixel;

When the two angle prediction modes belong to the horizontal prediction mode set,

use formula

Determine the reference pixel position c;

Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block;

x ₀ represents the abscissa, y ₀ represents the ordinate, and w represents the width of the intra prediction block;

use formula

p[x ₀ ][y ₀ ]=f[0]*p[0][c]+f[1]*p[0][c+1]+f[2]*p[0][c+ 2]+f[3]*p[0][c+3]

determining the position of the intra-prediction pixel;

Among them, p[x0][y0] represents an intra prediction pixel.

Further, the generating the predicted pixel value according to the position of the reference pixel specifically adopts Gaussian interpolation or cubic spline interpolation method to generate the predicted pixel value according to the position of the reference pixel.

Further, the combination method of the angle prediction mode is based on a model of statistical laws, and also based on a data-driven method based on deep learning or machine learning.

The nonlinear texture-oriented intra-frame prediction method proposed by the present invention is shown in FIG. 3 . By virtue of the form of a quadratic function, the present invention represents this non-linear texture using two angular prediction modes. In Fig. 3, both angle prediction modes belong to the vertical mode set. Specifically, the prediction mode close to the reference pixel is called the normal prediction mode, and its angle α corresponds to the tangent direction when the quadratic function enters the prediction block; the prediction mode far away from the reference pixel is called the extended prediction mode, and its angle β corresponds to the The tangent direction of the quadratic function as it leaves the prediction block. Points on the same quadratic function have the same pixel intensity. Therefore, for the prediction pixel p[x0][y0], its corresponding reference pixel position c can be derived by the following formula, and its pixel intensity can be generated by multi-tap filter interpolation.

When the two angle prediction modes belong to the vertical mode set, the position of the reference pixel and the derivation process of the pixel intensity are shown in the following formula.

p[x ₀ ][y ₀ ]=f[0]*p[c][0]+f[1]*p[c][1]+f[2]*p[c][2]+f [3]*p[c][3] (2)

Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block;

x ₀ represents the abscissa, y ₀ represents the ordinate, and h represents the height of the intra prediction block;

Correspondingly, when the two angle prediction modes belong to the horizontal mode set, the position of the reference pixel and the derivation process of the pixel intensity are shown in the following formula.

p[x ₀ ][y ₀ ]=f[0]*p[0][c]+f[1]*p[0][c+1]+f[2]*p[0][c+ 2]+f[3]*p[0][c+3] (4)

In the present disclosure, the two angle prediction modes should both belong to the vertical mode set or both belong to the horizontal mode set. In order to reduce the number of searches, conventional prediction modes can only be selected from the set of most probable modes (Most Probable Modes, MPM). Further, for each conventional prediction mode, only the four extended prediction modes with the highest occurrence frequency based on statistical laws are searched. The present invention can achieve an average coding performance gain of 0.1% in the VTM10.0 intra-frame coding mode (All Intra, AI).

Table 1 This disclosure is based on the coding performance of VTM10.0

Embodiment two:

The present disclosure can also provide a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is used to implement the steps of the above-mentioned intra prediction method oriented to nonlinear texture.

The computer storage medium of the present disclosure may be implemented using semiconductor memory, magnetic core memory, magnetic drum memory, or magnetic disk memory.

Semiconductor memory, mainly used in computers, mainly has two types of semiconductor memory elements: Mos and bipolar. Mos components are highly integrated, the process is simple but the speed is slow. Bipolar components are complex in process, high in power consumption, low in integration but fast in speed. After the advent of NMos and CMos, Mos memory began to play a major role in semiconductor memory. NMos is fast, for example, the access time of Intel's 1K-bit SRAM is 45ns. CMos consumes less power, and the 4K-bit CMos static memory access time is 300ns. The above-mentioned semiconductor memories are all random access memories (RAM), that is, they can be read and written into new content randomly during the working process. The semiconductor read-only memory (ROM) can be read randomly but cannot be written in during the working process, and it is used to store solidified programs and data. ROM is divided into non-rewritable fuse-type read-only memory ─ ─ PROM and rewritable read-only memory EPROM two.

Magnetic core memory has the characteristics of low cost and high reliability, and has more than 20 years of actual use experience. Before the mid-1970s, magnetic core memory was widely used as the main memory. Its storage capacity can reach more than 10 bits, and the fastest access time is 300ns. The typical magnetic core memory capacity in the world is 4MS ~ 8MB, and the access cycle is 1.0 ~ 1.5μs. After the rapid development of semiconductor storage replaced the magnetic core memory as the main memory, the magnetic core memory can still be used as a large-capacity expansion memory.

Drum memory, a magnetically recorded external memory. Due to its fast information access speed and stable and reliable work, although its capacity is small, it is gradually being replaced by disk storage, but it is still used as an external memory for real-time process control computers and medium and large computers. In order to meet the needs of small and microcomputers, ultra-small magnetic drums have appeared, which are small in size, light in weight, high in reliability, and easy to use.

Disk storage, a type of magnetically recorded external storage. It has the advantages of magnetic drum and magnetic tape storage, that is, its storage capacity is larger than that of magnetic drum, and its access speed is faster than that of magnetic tape storage, and it can be stored offline. Therefore, disks are widely used as large storage devices in various computer systems. capacity of external memory. Disks are generally divided into two categories: hard disks and floppy disks.

There are many types of hard disk storage. Structurally, it can be divided into interchangeable type and fixed type. Interchangeable disk platters can be exchanged, and fixed disk platters are fixed. There are two kinds of replaceable and fixed disks: multi-chip combination and single-chip structure, and both can be divided into fixed head type and movable head type. The capacity of the fixed head type disk is small, the recording density is low and the access speed is high, but the cost is high. The moving head type disk has a high recording density (up to 1000-6250 bits/inch), so it has a large capacity, but its access speed is lower than that of a fixed head disk. The storage capacity of disk products can reach hundreds of megabytes, the bit density is 6 250 bits per inch, and the track density is 475 tracks per inch. Among them, the multi-chip interchangeable disk storage has a large off-body capacity because the disk group can be replaced, and has a large capacity and high speed, and can store large-capacity intelligence data. It is widely used in online information retrieval systems and database management systems.

Embodiment three:

The present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the above-mentioned intra prediction method oriented to nonlinear texture is realized. step.

Fig. 4 is a schematic diagram of the internal structure of an electronic device in one embodiment. As shown in FIG. 4, the electronic device includes a processor, a storage medium, a memory and a network interface connected through a system bus. Wherein, the storage medium of the computer device stores an operating system, a database, and computer-readable instructions, and the database can store control information sequences. When the computer-readable instructions are executed by the processor, the processor can implement a non-linear Intra prediction method for textures. The processor of the electrical device is used to provide computing and control capabilities, and supports the operation of the entire computer device. Computer-readable instructions may be stored in the memory of the computer device, and when the computer-readable instructions are executed by the processor, the processor may execute an intra-frame prediction method oriented to nonlinear texture. The network interface of the computer device is used for connecting and communicating with the terminal. Those skilled in the art can understand that the structure shown in Figure 4 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation to the computer equipment on which the solution of the application is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

The electronic devices include, but are not limited to, smart phones, computers, tablet computers, wearable smart devices, artificial intelligence devices, mobile power supplies, and the like.

In some embodiments, the processor can be composed of integrated circuits, for example, it can be composed of a single packaged integrated circuit, or it can be composed of multiple integrated circuits with the same function or different functions, including one or more central Processor (Central Processing unit, CPU), microprocessor, digital processing chip, graphics processor and a combination of various control chips, etc. The processor is the control core (Control Unit) of the electronic equipment, and uses various interfaces and lines to connect the various components of the entire electronic equipment, by running or executing programs or modules stored in the memory (such as executing remote data read and write programs, etc.), and call the data stored in the memory to execute various functions of the electronic device and process data.

The bus may be a peripheral component interconnect (PCI for short) bus or an extended industry standard architecture (EISA for short) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. The bus is configured to implement communication between the memory and at least one processor.

Figure 4 only shows an electronic device with components, and those skilled in the art can understand that the structure shown in Figure 4 does not constitute a limitation to the electronic device, and may include fewer or more components than shown in the figure , or combinations of certain components, or different arrangements of components.

For example, although not shown, the electronic device may also include a power supply (such as a battery) for supplying power to each component. Preferably, the power supply may be logically connected to the at least one processor through a power management device, thereby realizing Charge management, discharge management, and power management functions. The power supply may also include one or more DC or AC power supplies, recharging devices, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components. The electronic device may also include various sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be repeated here.

Further, the electronic device may also include a network interface. Optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which are usually used to communicate between the electronic device and A communication link is established between other electronic devices.

Optionally, the electronic device may further include a user interface. The user interface may be a display (Display) or an input unit (such as a keyboard (Keyboard)). Optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, Organic Light-Emitting Diode) touch device, and the like. Wherein, the display may also be properly referred to as a display screen or a display unit, and is used for displaying information processed in the electronic device and for displaying a visualized user interface.

Further, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required by at least one function, etc.; use of the created data, etc.

In the several embodiments provided by the present invention, it should be understood that the disclosed devices, devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each unit may physically exist separately, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or in the form of hardware plus software function modules.

The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. The scope of the present disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims (6)

1. A non-linear texture-oriented intra-frame prediction method, used in an intra-frame prediction module in the field of image and video coding, is characterized in that, comprising: Determine the current intra prediction mode; predicting non-linear textures using predictive modeling involving quadratic functions in said intra prediction module; The position of the reference pixel is derived according to the result of predicting the nonlinear texture, and the predicted pixel value is generated according to the reference pixel interpolation; The derivation of reference pixels according to the result of predicting nonlinear texture specifically includes: When two angle prediction modes belong to the vertical prediction mode set, use formula Determine the reference pixel position c; Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block; x ₀ represents the abscissa, y ₀ represents the ordinate, and h represents the height of the intra prediction block; use formula p[x ₀ ][y ₀ ]＝[0]*[c][0]+[1]*[c][1]+[2]*[c][2]+[3]*p[] [3] Determine the position of the intra prediction pixel; Among them, p[x0][0] represents the intra prediction pixel; When the two angle prediction modes belong to the horizontal prediction mode set, use formula Determine the reference pixel position c; Wherein, angle α corresponds to the tangent direction when the quadratic function enters the prediction block, and angle β corresponds to the tangent direction when the quadratic function leaves the prediction block; x ₀ represents the abscissa, y ₀ represents the ordinate, and w represents the width of the intra prediction block; use formula p[x ₀ ][y ₀ ]＝[0]*[0][c]+[1]*[0][c+1]+[2]*[0][c+2]+[3] *p[0][+3] determining the position of the intra-prediction pixel; Among them, p[x0][0] represents an intra prediction pixel. 2. The method according to claim 1, wherein the intra-frame prediction mode specifically comprises: Normal prediction mode close to reference pixels and extended prediction mode far away from reference pixels. 3. The method according to claim 1, wherein the generating the predicted pixel value according to the position of the reference pixel specifically adopts Gaussian interpolation or cubic spline interpolation method to generate the predicted pixel value according to the position of the reference pixel. 4. The method according to claim 3, characterized in that, the combination method of the angle prediction mode is based on a model of statistical laws, and is also based on a data-driven method based on deep learning or machine learning. 5. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the computer program, the computer program described in any one of claims 1 to 4 is implemented. The steps corresponding to the nonlinear texture-oriented intra prediction method. 6. A computer storage medium on which computer program instructions are stored, wherein the program instructions are used to implement the non-linear texture-oriented frame described in any one of claims 1 to 4 when executed by a processor The steps corresponding to the intra-prediction method.