CN112203085A - Image processing method, device, terminal and storage medium - Google Patents

Abstract

The present disclosure provides an image processing method, apparatus, terminal, and storage medium. Among them, some embodiments of the image processing method include: determining a reference block of a current image block, wherein the current image block is located in a current frame, and the reference block is located in a reference frame of the current frame; determining a luminance reconstruction value of a reference block and a chrominance reconstruction value of the reference block; performing illumination compensation on the brightness reconstruction value of the reference block to obtain a brightness compensation value of the reference block; and determining a brightness predicted value and a chroma predicted value of the current image block, wherein the brightness predicted value of the current image block is a brightness compensation value of the reference block, and the chroma predicted value of the current image block is a chroma reconstruction value of the reference block. The method can reduce the data processing amount in the video coding and video decoding processes, reduce the complexity and simultaneously improve the coding compression efficiency.

Description

Image processing method, device, terminal and storage medium

technical field

The present disclosure relates to the technical field of image processing, and in particular, to an image processing method, device, terminal, and storage medium.

Background technique

A video contains a large number of image frames, each image frame contains tens of thousands or hundreds of thousands of pixels, and each pixel is generally represented by 24 bits, so a video needs to occupy a lot of storage space and transmission bandwidth. To reduce the space and bandwidth occupied by the video, the video is usually compressed when the video is transmitted, that is, the video is encoded, and the video is decompressed before the video is played, that is, the video is decoded. Generally speaking, video compression techniques include intra-frame compression and inter-frame compression. Intra-frame compression is used to remove spatial redundancy in image frames, and inter-frame compression is used to remove temporal redundancy of images.

SUMMARY OF THE INVENTION

In order to solve the existing problems, the present disclosure provides an image processing method, device, terminal and storage medium.

The present disclosure adopts the following technical solutions.

In some embodiments, the present disclosure provides an image processing method, comprising:

Determine the reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame;

determining the luma reconstruction value of the reference block and the chroma reconstruction value of the reference block;

Perform illumination compensation on the brightness reconstruction value of the reference block to obtain the brightness compensation value of the reference block;

Determine the luma predicted value and the chrominance predicted value of the current image block, wherein the luma predicted value of the current image block is the luma compensation value of the reference block, and the chrominance predicted value of the current image block is the chrominance reconstruction value of the reference block.

In some embodiments, the method further includes: determining the value of the switch flag bit carried in the current frame to be the first value.

In some embodiments, the value of the switch flag bit carried by the current frame is determined according to the difference between the luminance component of the current frame and the luminance component of the reference frame, and the difference between the chrominance component of the current frame and the chrominance component of the reference frame .

In some embodiments, when the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag carried in the current frame is the first value, wherein the first change value is the current The difference between the luminance component of the frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the chrominance component of the reference frame.

In some embodiments, the luminance prediction value of the current image block is determined according to formula (1):

为当前图像块的亮度预测值，

为参考块的亮度重构值，α^Y为亮度分量的颜色通道缩放系数，β^Y为亮度分量的偏移量。in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the reference block, α ^Y is the color channel scaling factor of the luminance component, and β ^Y is the offset of the luminance component.

In some embodiments, the determining the luma predicted value and the chrominance predicted value of the current image block includes: if the value of the switch flag carried in the current frame is a first value, the luma of the current image block The predicted value is the luminance compensation value of the reference block, and the chrominance predicted value of the current image block is the chrominance reconstruction value of the reference block.

If the value of the switch flag bit carried in the current frame is the second value, the luma predicted value and the chrominance predicted value of the current image block are respectively equal to the luma reconstruction value and the chrominance reconstruction value of the reference block; or, the current image The luma predicted value and the chrominance predicted value of the block are respectively equal to the luma compensation value and the chrominance compensation value of the reference block, and the chrominance compensation value is a value obtained by performing illumination compensation on the chrominance reconstruction value of the reference frame.

In some embodiments, the present disclosure provides an image processing method, comprising:

Determine the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, where the current image block is located in the current frame, the motion vector of the chrominance component is different from the co-located motion vector, and the co-located motion vector is the motion vector of the luminance component, or , the co-located motion vector is the motion vector obtained by scaling the motion vector of the luminance component according to the luminance sampling rate and the chrominance sampling rate;

Determine the luminance reference block of the luminance component of the current image block and the chrominance reference block of the chrominance component of the current image block according to the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, wherein the luminance reference block and the chrominance reference block The block is located in the reference frame of the current frame;

determining the luma reconstruction value of the luma reference block and the chroma reconstruction value of the chroma reference block;

Perform illumination compensation on the luminance reconstruction value of the luminance reference block to obtain the luminance compensation value of the luminance reference block;

Perform illumination compensation on the chrominance reconstruction value of the chrominance reference block to obtain the chrominance compensation value of the chrominance reference block;

Determine the luma predicted value and the chrominance predicted value of the current image block, where the luma predicted value of the current image block is the luma compensation value of the luma reference block, and the chrominance predicted value of the current image block is the chrominance compensation value of the chrominance reference block .

In some embodiments, the motion vector of the chroma reference block is the same as the motion vector of the chroma component of the current image block.

In some embodiments, the motion vector of the chroma component of the current image block is (0,0).

In some embodiments, the motion vector of the chrominance component of the current image block is selected from the vector candidate list, and the motion vector of the chrominance component is recorded by encoding the index of the motion vector of the chrominance component in the vector candidate list.

In some embodiments, formula (2) is used to obtain the luminance prediction value of the current image block:

为当前图像块的亮度预测值，

为参考块的亮度重构值，α^Y为亮度分量的颜色通道缩放系数，β^Y为亮度分量的偏移量；in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the reference block, α ^Y is the color channel scaling factor of the luminance component, and β ^Y is the offset of the luminance component;

and / or,

Use formula (3) and formula (4) to obtain the chrominance prediction value of the current image block:

为当前图像块的色度预测值的U分量，

为所述当前图像块的色度预测值的V分量，

为色度参考块的色度重构值的U分量，

为色度参考块的色度重构值的V分量，α^U为U分量的颜色通道缩放系数，α^V为V分量的颜色通道缩放系数，β^U为U分量的偏移量，β^V为V分量的偏移量。in,

is the U component of the chroma prediction value of the current image block,

is the V component of the chrominance prediction value of the current image block,

is the U component of the chrominance reconstruction value of the chrominance reference block,

is the V component of the chrominance reconstruction value of the chrominance reference block, α ^U is the color channel scaling coefficient of the U component, α ^V is the color channel scaling coefficient of the V component, β ^U is the offset of the U component, and β ^V is The offset of the V component.

In some embodiments, determining the motion vector of the luminance component and the motion vector of the chrominance component of the current image block includes:

Determine the value of the switch flag bit carried by the current frame;

If the value of the switch flag of the current frame is the first value, the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the luminance compensation value of the chrominance reference block. chrominance reconstruction value;

If the value of the switch flag of the current frame is the second value, the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the The chrominance compensation value of the chrominance reference block.

In some embodiments, the value of the switch flag bit carried by the current frame is determined according to the difference between the luminance component of the current frame and the luminance component of the reference frame, and the difference between the chrominance component of the current frame and the chrominance component of the reference frame .

In some embodiments, when the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag carried in the current frame is the first value, wherein the first change value is the current The difference between the luminance component of the frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the chrominance component of the reference frame.

In some embodiments, the present disclosure provides an image processing apparatus, including:

A determining unit for determining the reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame;

a determining unit, further configured to determine the luminance reconstruction value of the reference block and the chrominance reconstruction value of the reference block;

a processing unit, configured to perform illumination compensation on the luminance reconstruction value of the reference block to obtain the luminance compensation value of the reference block;

The determining unit is further configured to determine the luminance prediction value and the chrominance prediction value of the current image block, wherein the luminance prediction value of the current image block is the luminance compensation value of the reference block, and the chrominance prediction value of the current image block is the color value of the reference block. degree reconstruction value.

In some embodiments, the present disclosure provides an image processing apparatus, comprising:

A determination module, configured to determine the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, wherein the current image block is located in the current frame, the motion vector of the chrominance component is different from the co-located motion vector, the The co-located motion vector is the motion vector of the luminance component, or the co-located motion vector is a motion vector obtained by scaling the motion vector of the luminance component according to the luminance sampling rate and the chrominance sampling rate;

The determining module is further configured to determine the luminance reference block of the luminance component of the current image block and the chrominance component of the current image block according to the motion vector of the luminance component and the motion vector of the chrominance component of the current image block The chrominance reference block, wherein the luma reference block and the chrominance reference block are located in the reference frame of the current frame;

The determining module is further configured to determine the luminance reconstruction value of the luminance reference block and the chrominance reconstruction value of the chrominance reference block;

a processing module, configured to perform illumination compensation on the luminance reconstruction value of the luminance reference block to obtain the luminance compensation value of the luminance reference block;

a processing module, further configured to perform illumination compensation on the chrominance reconstruction value of the chrominance reference block to obtain the chrominance compensation value of the chrominance reference block;

The determining module is further configured to determine the luminance prediction value and the chrominance prediction value of the current image block, wherein the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the luminance prediction value of the current image block is a luminance compensation value of the luminance reference block. The chrominance prediction value is the chrominance compensation value of the chrominance reference block.

In some embodiments, the present disclosure provides a terminal comprising: at least one memory and at least one processor;

The memory is used for storing program codes, and the processor is used for calling the program codes stored in the memory to execute the above method.

In some embodiments, the present disclosure provides a storage medium for storing program codes for performing the above-described methods.

The image processing method provided by the embodiments of the present disclosure reduces the amount of data processing in the process of video encoding and video decoding by only performing illumination compensation on the luminance component in the inter-frame prediction process (reducing the step of performing illumination compensation on the chrominance component). , reducing the complexity of video encoding and decoding. The methods proposed in the embodiments of the present disclosure can also improve the accuracy of chrominance component prediction, thereby reducing residual errors and reducing the coding rate, thus improving the coding compression performance.

Description of drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent when taken in conjunction with the accompanying drawings and with reference to the following detailed description. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and elements are not necessarily drawn to scale.

FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure.

FIG. 2 is a schematic diagram of a current image block and a reference block corresponding to the current image block according to an embodiment of the present disclosure.

FIG. 3 is a flowchart of another image processing method according to an embodiment of the present disclosure.

FIG. 4 is a composition diagram of an image processing apparatus according to an embodiment of the present disclosure.

FIG. 5 is a composition diagram of another image processing apparatus according to an embodiment of the present disclosure.

FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.

Detailed ways

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for the purpose of A more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the protection scope of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in sequence and/or in parallel. Furthermore, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this regard.

As used herein, the term "including" and variations thereof are open-ended inclusions, ie, "including but not limited to". The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below.

It should be noted that concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or interdependence.

It should be noted that the modification of "a" mentioned in the present disclosure is illustrative rather than restrictive, and those skilled in the art should understand that unless the context clearly indicates otherwise, it should be understood as "one or more".

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of these messages or information.

With the rise of short video applications, more and more users shoot and share short videos through mobile terminals such as mobile phones. Such User Generated Content (UGC, User Generated Content) videos are significantly different from traditionally shot videos. UGC videos generally Non-professional users use non-professional equipment to shoot in non-professional lighting scenes, so the light in the same video changes significantly. UGC videos are shot with various equipment, scenes, and video content, and UGC videos usually go through special effects and filters. Rendering; UGC videos are usually shot by users through handheld terminal devices such as mobile phones and tablets, and then uploaded to the video platform after being compressed. UGC video will be encoded, compressed and uploaded after shooting. Improving the video compression efficiency will help save traffic bandwidth and computing power consumption.

In some technologies, in order to solve the problem of local illumination changes between adjacent frames in the time domain when inter-frame compression is performed on the video, in the motion compensation stage of inter-frame prediction, the luminance component and color of the coding unit currently undergoing inter-frame coding are calculated. The luminance components are separately compensated for local illumination, and the luma coding unit (luma block) shares the motion vector with the chroma coding unit (chroma block), however, the illumination compensation for both the luma coding unit and the chroma coding unit may reduce the chroma Predictive performance of the components, and will increase the complexity of the coding.

In order to at least partially solve the above problem, an image processing method is proposed in this embodiment of the present disclosure. The image in this embodiment may be an image in a video. The solution provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

As shown in FIG. 1 , FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure, including the following steps.

S11: Determine the reference block of the current image block.

Specifically, the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame. In some embodiments, the image may be an image in a video, and the video may be a UGC video. The image processing method proposed in the present disclosure can be used for video The inter-frame prediction process of compression or decompression. The video includes multiple image frames. The current frame can be any image frame in the video that needs to be inter-predicted. For example, the current frame can be a P image or a B image. The current frame includes multiple image frames. Coding unit, the current image block here can be any coding unit. In the process of inter-frame prediction, the current frame will correspond to a reference frame, that is, the image used for prediction, which is also commonly referred to as a reference image ( Reference Frame), the reference frame can be, for example, the previous image frame or the next image frame adjacent to the current frame time domain, the reference frame of the current frame also includes a plurality of coding units, and the coding unit corresponding to the current image block is used as a reference Block, the displacement from the reference block to the current image block is usually called a motion vector (Motion Vector, MV), and the difference between the current image block and the reference block is usually called the prediction residual (Prediction Residual). The process of the reference block corresponding to the block is usually called motion estimation (Motion Estimation). Algorithms, three-step search methods, etc., are not limited.

S12: Determine the luminance reconstruction value of the reference block and the chrominance reconstruction value of the reference block.

S13: Perform illumination compensation on the luminance reconstruction value of the reference block to obtain the luminance compensation value of the reference block.

Specifically, the luminance compensation value is a luminance value obtained by performing illumination compensation on the luminance reconstruction value.

S14: Determine the luminance prediction value and the chrominance prediction value of the current image block.

Specifically, the luminance prediction value of the current image block is the luminance compensation value of the reference block, and the chrominance prediction value of the current image block is the chrominance reconstruction value of the reference block. In this embodiment, a color space of luminance and chrominance may be used for the color, that is, the YUV color space, Y represents the luminance, that is, the grayscale value, and U and V represent the chromaticity. Illumination compensation is to solve the problem of local illumination changes between adjacent frames in the time domain, and acts on the motion compensation stage of inter-frame prediction. In the prior art, when determining the luminance prediction value and chrominance prediction value of the current frame, the same In the illumination compensation method, the luminance reconstruction value and the chrominance reconstruction value of the reference block corresponding to the current image block are respectively subjected to illumination compensation as the luminance prediction value and chrominance prediction value of the current image block. , when determining the brightness prediction value of the current image block, the value obtained after performing illumination compensation on the brightness reconstruction value of the reference block corresponding to the current image block is used as the brightness prediction value of the current image block, and the reference block corresponding to the current image block is directly used. The chrominance reconstruction value of the current image block is used as the chrominance prediction value of the current image block, and there is no need to perform illumination compensation on the chrominance prediction value. Compared with the prior art, the image processing method in this embodiment reduces the amount of The step of performing illumination compensation on the chrominance prediction value reduces the amount of data processing in the process of video encoding and video decoding, and reduces the complexity, and the chrominance of the current image block may be unchanged relative to the chrominance of the reference block, Performing illumination compensation on the chrominance predicted value in the prior art will reduce the prediction accuracy of the chrominance component. For example, when the video in this embodiment is a continuously shot video (such as a UGC video), the chrominance component of the current image block and The chrominance components of the reference block are similar, and the illumination compensation of the chrominance prediction value is not performed at this time, which reduces the amount of data processing, reduces the complexity of encoding and decoding, improves the encoding efficiency, and the prediction of the chrominance components of the current image block is more accurate.

In some embodiments of the present disclosure, formula (1) is used to obtain the luminance prediction value of the current image block:

为当前图像块的亮度预测值，

为当前图像块对应的参考块的亮度重构值，α^Y为亮度分量的颜色通道缩放系数，β^Y为亮度分量的偏移量。in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the reference block corresponding to the current image block, α ^Y is the color channel scaling coefficient of the luminance component, and β ^Y is the offset of the luminance component.

Specifically, in some embodiments, in the process of inter-frame prediction, a linear illumination transformation model is used to perform illumination compensation on the luminance prediction value of the reference block corresponding to the current image block, wherein α ^Y and β ^Y can be based on the current image in the current frame. The adjacent reference pixels of the block and the adjacent reference pixels of the corresponding reference block are derived by linear regression. For example, please refer to FIG. 2. 2a in FIG. 2 is the current image block and the adjacent reference pixels of the current image block, FIG. 2 2b is the reference block corresponding to the current image block and the adjacent reference pixels of the reference block, the solid circle in Fig. 2 is the adjacent reference pixels, the adjacent reference pixels of the current image block and the adjacent reference pixels of the corresponding reference block can be used. Square the error to get the parameters α ^Y , β ^Y .

其中

是当前图像块的色度分量中U分量的预测值，

是当前图像块的色度分量中U分量的重构值，

是当前图像块对应的参考帧中参考块的色度分量中U分量的重构值，

是当前图像块对应的参考帧中参考块的色度分量中V分量的重构值。通过这种方式，不但降低了数据处理的复杂度，提高了编码解码效率，并且还提高了色度预测值的准确度，还可提高压缩性能。In the prior art, illumination compensation is usually performed simultaneously on the three color components (Y, U, V) of the coding unit that needs illumination compensation. However, according to the characteristics of UGC video, there will be illumination changes between image frames of the video. , but the chromaticity does not change or the change is small, that is, the luminance changes but the chromaticity basically does not change. Therefore, in some embodiments of the present disclosure, only local illumination compensation is performed on the luminance component Y of the current image block. The chrominance components (U, V) do not perform illumination compensation, and directly use the chrominance components of the reference block corresponding to the current image block for prediction, that is

in

is the predicted value of the U component in the chrominance component of the current image block,

is the reconstructed value of the U component in the chroma component of the current image block,

is the reconstructed value of the U component in the chrominance component of the reference block in the reference frame corresponding to the current image block,

is the reconstructed value of the V component in the chrominance component of the reference block in the reference frame corresponding to the current image block. In this way, not only the complexity of data processing is reduced, the coding and decoding efficiency is improved, but also the accuracy of the chrominance prediction value is improved, and the compression performance is also improved.

In some embodiments of the present disclosure, the determining the luma predicted value and the chrominance predicted value of the current image block includes: determining whether the current frame is a target frame or a non-target frame, and if the current image frame is a target frame, the The luminance prediction value of the current image block is the luminance compensation value of the reference block, and the chrominance prediction value of the current image block is the chrominance reconstruction value of the reference block.

If the current frame is a non-target frame, the luma predicted value and chrominance predicted value of the current image block are respectively equal to the luma reconstruction value and chrominance reconstructed value of the reference block; or, the luma predicted value and chrominance predicted value of the current image block , respectively equal to the luminance compensation value and the chrominance compensation value of the reference block, and the chrominance compensation value is the value obtained by performing illumination compensation on the chrominance reconstruction value of the reference frame.

Specifically, in this embodiment, the image is divided into a target frame and a non-target frame. When the current frame is a target frame, only illumination compensation is performed on the brightness prediction value, and when the current frame is a non-target frame. When determining the luma predicted value and chrominance predicted value of the image block in the non-target frame, there are two ways. One is to directly use the luma reconstructed value and the chrominance reconstructed value of the reference block as the luma predicted value and the chrominance predicted value. The predicted value, and the other is the value after illumination compensation is performed using the luminance reconstruction value and the chrominance reconstruction value of the reference block, that is, the luminance compensation value and the chrominance compensation value. It should be noted that the non-target frame may adopt any one of the above two methods for determining the luma predicted value and the chrominance predicted value, and the methods adopted by each non-target frame may be different.

In some embodiments, before determining the luma predicted value and the chrominance predicted value of the current image block, the method further includes: determining the value of the switch flag bit carried in the current frame to be the first value.

Specifically, in this embodiment, a switch flag is set in each image frame, and the switch flag can be used as a frame-level switch to determine whether to perform illumination compensation on the chrominance prediction value of the reference block. The switch flag of the current frame is: When it is equal to the first value, the chrominance prediction value of the current image block is the chrominance reconstruction value of the reference block, and the value of the switch flag bit can be set randomly. For example, the switch flag bit of one frame selected in every two frames is set to the first When the value of the switch flag of the current frame is the second value, the luminance compensation value and the chrominance compensation value of the reference block of the reference frame can be used as the luminance prediction value and chrominance value of the current image block, respectively. The predicted value, the chrominance compensation value is the chrominance value for illumination compensation of the chrominance reconstruction value of the reference block. The switch flag bit is set in the file, the switch flag bit of the target frame can be set to 1, and the switch flag bit of the non-target frame can be set to 0, so as to determine the type of the image frame during the inter-frame prediction process. When the image processing method proposed in the present disclosure is used for video coding, it is first determined whether each image frame is a target image or a non-target image, and the corresponding switch flag bit is set for each image frame according to the result. During video decoding, the value of the switch flag bit of the image frame is first identified, and how to obtain the luma predicted value and the chrominance predicted value of the current image block is determined according to the value of the switch flag bit.

In some embodiments of the present disclosure, the switch flag is only set in the target frame of the video; specifically, in this embodiment, no identification information is set in the non-target frame, so as long as the identification information is identified, the current frame can be determined The chrominance prediction value of the image block is equal to the chrominance reconstruction value of the reference block. Similarly, in some embodiments of the present disclosure, the switch flag is only set in the non-target frame of the video. By switching the flag bit, it can be determined that the chrominance prediction value of the image block of the current frame is equal to the chrominance reconstruction value of the reference block.

In some embodiments of the present disclosure, the value of the switch flag bit carried by the current frame is based on the difference between the luminance component of the current frame and the luminance component of the reference frame, and the chrominance component of the current frame. is determined by the difference with the chrominance components of the reference frame. Specifically, the difference between the luminance component and chrominance component of the current frame and the luminance component and chrominance component of the reference frame in the present disclosure indicates the magnitude of change of the luminance component and the chrominance component relative to the reference frame. If they are similar, the luma component and the chrominance component can be calculated in a similar way, otherwise, the two should be calculated in different ways.

In some embodiments of the present disclosure, when the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag bit carried in the current frame is the first value, wherein the The first change value is the difference between the luminance component of the current frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the chrominance component of the reference frame. difference. Specifically, the first change value is the absolute value of the difference between the luminance components of the current frame and the reference frame. Similarly, the second change value is the absolute value of the difference between the chrominance components of the current frame and the reference frame.

Specifically, this embodiment clarifies the criterion for determining whether to perform illumination compensation only on the luminance reconstruction value, that is, how to determine whether an image frame should use the illumination compensation for only the luminance reconstruction value proposed in the embodiment of the present disclosure, without color compensation This embodiment can be used in the video encoding process. For any currently encoded image frame, the difference between the first change value and the second change value of the image frame is determined, and then The difference value is compared with the difference threshold value, and according to the comparison result, whether the image frame adopts the solution proposed in the present disclosure is set, and when the difference value is not greater than the difference threshold value, it indicates the difference between the luminance component and the chrominance component of the image frame currently being encoded. The change trend is similar. At this time, the processing methods of the luminance component and chrominance component of the current frame currently being encoded should be similar. At this time, illumination compensation should be performed on the predicted values of the luminance component and chrominance component, or no illumination compensation should be performed. When When the difference is greater than the difference threshold, it means that the change trend of the luminance component and the chrominance component is quite different. The two should not be processed in a similar way. Usually, the chrominance of the image frame does not change, so only the luminance component should be used. Illumination compensation is performed on the predicted value of the chrominance component, but not on the predicted value of the chrominance component.

In order to better illustrate the image processing method proposed by the embodiments of the present disclosure, a specific embodiment is provided below by taking the image processing method proposed in the present disclosure applied to a video encoding end as an example.

计算，当前图像块的色度预测值采用公式：

此处三个公式中的参数的含义与之前描述的相同，即对于当前图像块对应的参考块的亮度重构值进行线性光照补偿后作为当前图像块的亮度预测值，直接使用当前图像块对应的参考块的色度重构值作为当前图像块的色度预测值。即对于当前图像块只对亮度预测值进行光照补偿，对色度预测值不进行光照补偿，对于视频的非目标帧，则对亮度预测值和色度预测值都进行光照补偿。In the video coding process, the image frames of the video are divided into equal-sized, non-overlapping maximum coding units. Then, with the largest coding unit as a node, different types of recursive tree divisions can be performed, for example, a quad tree, a binary tree, and a ternary tree, etc., to form a coding unit. A coding unit is a basic unit of video coding, and each coding unit may contain one luminance block (Y) and two chrominance blocks (UV). Video coding performance comes from the removal of data redundancy. Inter-frame mode prediction can effectively remove temporal redundancy. The content between the frames before and after the video is similar, and the change of illumination will greatly affect the efficiency of inter-frame coding. When performing inter-frame prediction, frame-level histogram statistics can be used. Compare the cumulative difference between the current frame and the reference frame. If the cumulative difference of the changes of the luminance component between the current frame and the reference frame is similar to the cumulative difference of the changes of the chrominance components, the current frame is a non-target frame. On the contrary, if the cumulative difference between the changes of the luminance component of the current frame and the reference frame is far different from that of the chrominance component, the current frame is the target frame. Whether the current frame is a target frame or a non-target frame is determined by setting a flag (switch flag bit) in each image frame, for example, a frame-level switch (lic_sep_flag) can be used to identify whether the current frame is a target frame. lic_sep_flag can be determined by the histogram statistics of all frames in the current coded frame and the reference frame list and passed as a frame-level flag bit. If the current frame is a non-target frame, lic_sep_flag is 0, and if the current frame is a target frame, lic_sep_flag is 1. The method for determining the luminance prediction value and the chrominance prediction value is described in detail below. In this embodiment, assuming that the current frame is the target frame, the formula is used for the luminance prediction value of the current image block:

To calculate, the chrominance prediction value of the current image block adopts the formula:

The meanings of the parameters in the three formulas here are the same as those described above, that is, the luminance reconstruction value of the reference block corresponding to the current image block is subjected to linear illumination compensation as the luminance prediction value of the current image block, and the corresponding value of the current image block is directly used. The chrominance reconstruction value of the reference block is used as the chrominance prediction value of the current image block. That is, for the current image block, only illumination compensation is performed for the luminance prediction value, and illumination compensation is not performed for the chrominance prediction value. For non-target frames of the video, illumination compensation is performed for both the luminance prediction value and the chrominance prediction value.

When the image processing method proposed by the embodiment of the present disclosure is used for the video decoding end, in the video decoding process, for the current frame, the switch flag bit of the current frame is first acquired, and the value of the switch flag bit is the first one according to whether the value of the switch flag bit is the first one. The value determines whether the current frame is a target frame or a non-target frame, and then determines the luma predicted value and the chrominance predicted value of the current frame, wherein the method for determining the luma predicted value and the chrominance predicted value is the same as the above-described embodiment for video coding. The same is not repeated here.

Inter-frame prediction needs to be performed when decoding and encoding video, and motion vector corresponding to the coding unit needs to be obtained when performing inter-frame prediction. ), because the luminance sampling rate and the chrominance sampling rate are different, the number of luminance blocks and chrominance blocks in a coding unit is not necessarily the same. In the prior art, it is determined that the coding block is to obtain the motion vector of the luminance component first, The motion vector of the luminance component is scaled according to the luminance sampling rate and the chrominance sampling rate to obtain a co-located motion vector, which is used as the motion vector of the chrominance component. Take the luminance sampling rate as half of the chrominance sampling rate in the horizontal and vertical directions as an example , in the 420-sampling format, a coding block includes one luminance block (Y component) and two chrominance blocks (UV components), and the motion vector of the luminance component needs to be multiplied by 2 to obtain the motion vector of the chrominance component, that is, for luminance The motion vector of the component is scaled according to the sampling rate to obtain the co-located motion vector. However, the co-located motion vector is not necessarily the motion vector that best matches the chroma block of the current coding block. The prediction accuracy of the components is reduced, and the chroma is distorted.

In some embodiments of the present disclosure, an image processing method is also proposed. As shown in FIG. 3 , the method in this embodiment includes:

S21: Determine the motion vector of the luminance component and the motion vector of the chrominance component of the current image block.

Specifically, the current image block is located in the current frame, the motion vector of the chrominance component is different from the co-located motion vector, and the co-located motion vector is the motion vector of the luminance component, or the co-located motion vector is based on the luminance sampling rate and the chrominance sampling rate. The motion vector amount obtained by scaling the motion vector; in some embodiments, when the luminance sampling rate and the chrominance sampling rate are the same, the co-located motion vector is equal to the motion vector of the luminance component. The image processing method in this embodiment can be used in an inter-frame prediction process of video decoding or video encoding. The current frame can be an image frame in a video currently being encoded or decoded, and the current image block can be any encoding in the current frame. unit. In some embodiments, the motion vector of the chrominance component is different from the co-located motion vector, which is a motion vector obtained by scaling the motion vector of the luma component according to the luma sampling rate and the chroma sampling rate. The method for determining the motion vector of the luminance component in this embodiment may adopt the method in the prior art, which is not limited.

S22: Determine a luminance reference block of the luminance component of the current image block and a chrominance reference block of the chrominance component of the current image block according to the motion vector of the luminance component and the motion vector of the chrominance component of the current image block.

Specifically, the current frame corresponds to a reference frame, and the luminance reference block and the chrominance reference block are located in the reference frame corresponding to the current frame.

S23: Perform illumination compensation on the luminance reconstruction value of the luminance reference block to obtain the luminance compensation value of the luminance reference block.

Specifically, the luminance compensation value is a luminance value obtained by performing illumination compensation on the luminance reconstruction value of the luminance reference block.

S24: Perform illumination compensation on the chrominance reconstruction value of the chrominance reference block to obtain the chrominance compensation value of the chrominance reference block.

Specifically, the chrominance compensation value is a chrominance value obtained by performing illumination compensation on the chrominance reconstruction value of the chrominance reference block.

S25: Determine the luminance prediction value and the chrominance prediction value of the current image block,

Specifically, the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the chrominance compensation value of the chrominance reference block. In some embodiments, the method proposed by the embodiments of the present disclosure is used in the inter-frame prediction process. When performing the inter-frame prediction of the luminance component and the chrominance component on the coding unit, the motion vector of the luminance component is obtained first, and then the chrominance component is sampled according to the The luminance component is scaled with the luminance sampling rate and the luminance sampling rate to obtain the co-located motion vector (when the luminance sampling rate and chrominance sampling rate are the same, the motion vector of the luminance component is equal to the co-located motion vector), and the co-located motion vector is used as the chrominance motion vector of the coding unit. , this method defaults that the motion mode of the luma block is the same as that of the chroma block, but in practice, the motion mode of the luma block and the chroma block in the image is not necessarily the same, especially when the video is shot. When the illumination changes significantly, the above method will lead to unsatisfactory chrominance prediction values, resulting in chrominance distortion in the encoding or decoding process, and affecting the user's experience. Therefore, in this embodiment, the motion vector of the chrominance component is different from the co-located motion vector, Thus, the problem of chrominance distortion caused by improper selection of the motion vector of the chrominance component is avoided.

In some embodiments of the present disclosure, the motion vector of the chrominance reference block is the same as the motion vector of the chrominance component of the current image block. Specifically, in this embodiment, the corresponding motion vector is determined solely for the chrominance component of the current image block instead of using the co-located motion vector, and the method for determining the chrominance reference block may use the Full Search (FS) method.

In some embodiments of the present disclosure, the motion vector of the chroma component of the current image block is (0,0). Specifically, in this embodiment, the motion vector of the chrominance component of the current image block is set to (0,0) by default, which can save the coding overhead of the motion vector and the computing power consumption during coding and decoding.

In some embodiments of the present disclosure, the motion vector of the chrominance component of the current image block is selected from the vector candidate list, and the motion vector of the chrominance component is recorded by encoding the index of the motion vector of the chrominance component in the vector candidate list. Specifically, in this embodiment, a vector candidate list is separately established for the chrominance component, thereby improving the calculation speed of the motion vector of the chrominance component, and the vector candidate list may be a candidate list of Skip mode or Merge mode.

In some embodiments of the present disclosure, the current image block corresponds to a luminance reference block in the reference frame corresponding to the target frame; the luminance prediction value of the current image block is equal to the luminance reconstruction value of the luminance reference block after illumination compensation is performed. Specifically, the reference frame corresponding to the target frame may be an adjacent frame of the target frame, for example, the previous frame or the next frame of the target frame in the time domain, and the luminance reference block may be the luminance component of the reference frame and the current image block. For the closest coding block, the luminance reconstruction value corresponding to the luminance reference block is subjected to illumination compensation to obtain the luminance prediction value of the current image block.

In some embodiments of the present disclosure, the current image block corresponds to a chrominance reference block in the reference frame corresponding to the target frame; the chrominance prediction value of the current image block is equal to the chrominance reconstruction value of the chrominance reference block after illumination compensation The value of , or the chroma prediction value of the current image block is equal to the chroma reconstruction value of the chroma reference block. Specifically, the chrominance prediction value of the current image block can also be obtained by means of illumination compensation, or the chrominance reconstruction value of the chrominance reference block can be directly used, thereby reducing the amount of data processing.

In some embodiments of the present disclosure, formula (2) is used to obtain the luminance prediction value of the current image block:

为当前图像块的亮度预测值，

为亮度参考块的亮度重构值，α^y为亮度分量的颜色通道缩放系数，β^Y为亮度分量的偏移量。具体的，在本实施例中，在帧间预测的过程中采用线性光照变换模型对亮度参考块的亮度重构值进行光照补偿，其中α^Y和β^Y可以根据目标帧中当前图像块的邻近参考像素和对应的参考块的邻近参考像素以线性回归的方式导出。in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the luminance reference block, α ^y is the color channel scaling factor of the luminance component, and β ^Y is the offset of the luminance component. Specifically, in this embodiment, in the process of inter-frame prediction, a linear illumination transformation model is used to perform illumination compensation on the luminance reconstruction value of the luminance reference block, wherein α ^Y and β ^Y can be determined according to the proximity of the current image block in the target frame. The reference pixels and neighboring reference pixels of the corresponding reference block are derived in a linear regression fashion.

In some embodiments of the present disclosure, the following formula (3) and formula (4) are used to obtain the chrominance prediction value of the current image block:

为当前图像块的色度预测值的U分量，为所述当前图像块的色度预测值的V分量，

为色度参考块的色度重构值的U分量，

为色度参考块的色度重构值的V分量，α^U为U分量的颜色通道缩放系数，α^V为V分量的颜色通道缩放系数，β^U为U分量的偏移量，β^V为V分量的偏移量。在一些实施例中，采用线性光照变换模型对参考块的重构值进行光照补偿，其中α^U、α^V、β^U、β^V可以根据目标帧中当前图像块的邻近参考像素和对应的参考块的邻近参考像素以线性回归的方式导出。in,

is the U component of the chrominance predicted value of the current image block, is the V component of the chrominance predicted value of the current image block,

is the U component of the chrominance reconstruction value of the chrominance reference block,

is the V component of the chrominance reconstruction value of the chrominance reference block, α ^U is the color channel scaling coefficient of the U component, α ^V is the color channel scaling coefficient of the V component, β ^U is the offset of the U component, and β ^V is The offset of the V component. In some embodiments, a linear illumination transformation model is used to perform illumination compensation on the reconstructed value of the reference block, wherein α ^U , α ^V , β ^U , β ^V can be based on the adjacent reference pixels and corresponding reference pixels of the current image block in the target frame Neighboring reference pixels of the block are derived by linear regression.

In some embodiments of the present disclosure, the determining the luma predicted value and the chrominance predicted value of the current image block includes: determining whether the current frame is a target frame or a non-target frame, and if the current image frame is a target frame, the The luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the chrominance reconstruction value of the chrominance reference block.

If the current frame is a non-target frame, the luma predicted value and the chrominance predicted value of the current image block are equal to the luma reconstructed value of the luma reference block and the chrominance reconstructed value of the chrominance reference block, respectively; or, the luma predicted value of the current image block Value and chrominance prediction value, respectively equal to the luminance compensation value of the luminance reference block and the chrominance compensation value of the chrominance reference block. The chrominance compensation value is the chrominance reconstruction value of the chroma reference block of the reference frame after illumination compensation value of . .

Specifically, in this embodiment, an image frame is divided into a target frame and a non-target frame. When determining the luminance prediction value and the chrominance prediction value of the coding block in the non-target frame, there may be two methods. One is to directly Use the luminance reconstruction value and chrominance reconstruction value of the reference block corresponding to the coding block in the non-target frame, and the other is to use the luminance reconstruction value and chrominance reconstruction value of the reference block corresponding to the coding block in the non-target frame respectively. The value after lighting compensation. It should be noted that the non-target frame may adopt any one of the above two methods for determining the luma predicted value and the chrominance predicted value, and the methods adopted by each non-target frame may be different.

In some embodiments of the present disclosure, determining the motion vector of the luminance component and the motion vector of the chrominance component of the current image block includes:

Determine the value of the switch flag bit carried by the current frame;

If the value of the switch flag of the current frame is the first value, the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the luminance reference value. the chroma reconstruction value of the block;

If the value of the switch flag of the current frame is the second value, the luminance prediction value of the current image block is the luminance compensation value of the reference block, and the chrominance prediction value of the current image block is the luminance compensation value of the current image block. The chroma compensation value of the reference block.

Specifically, in some embodiments, a switch flag is set in each image frame, and the switch flag is used as a frame-level switch to determine the calculation method of the chrominance prediction value and the luminance prediction value of the current image block, for example, it can be used in the video The switch flag is set in the header file, and the first value can be 1 and the second value can be 0, so as to determine the calculation method of the luma predicted value and the chrominance predicted value of the current image block during the inter prediction process. When the image processing method proposed in the present disclosure is used for video coding, it is first determined that each image frame is the value of the switch flag, and then the luma predicted value and the chrominance predicted value are determined, wherein the luma predicted value is equal to the luma compensation value of the reference block , and the chrominance prediction value may be a chrominance compensation value or a chrominance reconstruction value according to different value results.

In some embodiments, the value of the switch flag bit carried by the current frame is based on the difference between the luminance component of the current frame and the luminance component of the reference frame, and the difference between the chrominance component of the current frame and the Determined by the difference of the chrominance components of the reference frame. Specifically, when the difference between the difference between the luminance component and the difference between the chrominance component of the current frame and the reference frame is small, it indicates that the two change trends are the same, and the same processing method can be used; otherwise, different processing methods should be used.

In some embodiments of the present disclosure, when the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag bit carried in the current frame is the first value, wherein the The first change value is the difference between the luminance component of the current frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the chrominance component of the reference frame. difference.

Specifically, this embodiment clarifies the standard for determining the value of the switch flag bit, that is, how to determine whether the chrominance prediction value of an image block should be subjected to illumination compensation. This embodiment can be used in the video encoding process. For any current For the encoded image frame, determine the difference between the first change value and the second change value of the image frame, then compare the difference with the difference threshold, and set whether the image frame is a target frame or a non-target frame according to the comparison result frame, when the difference is not greater than the difference threshold, it indicates that the luminance and chrominance components of the currently coded image frame have similar trends. At this time, illumination compensation is performed on the predicted values of the luminance and chrominance components or no illumination is performed. Compensation, when the difference is greater than the difference threshold, it means that the change trends of the luminance component and the chrominance component are quite different, and the two should not adopt similar processing methods, so the motion vector of the chrominance component and the motion vector of the same position should not be the same.

In order to better illustrate the image processing method proposed by the embodiments of the present disclosure, a specific embodiment is presented below by taking the image processing method proposed in the present disclosure applied to a video encoding end as an example.

In the video coding process, the image frames of the video are divided into equal-sized, non-overlapping maximum coding units. Then, with the largest coding unit as a node, different types of recursive tree divisions can be performed, for example, a quad tree, a binary tree, and a ternary tree, etc., to form a coding unit. A coding unit is a basic unit of video coding, and each coding unit may contain one luminance block (Y) and two chrominance blocks (UV). Video coding performance comes from the removal of data redundancy. Inter-frame mode prediction can effectively remove temporal redundancy. The content between the frames before and after the video is similar, and the change of illumination will greatly affect the efficiency of inter-frame coding. When performing inter-frame prediction, frame-level histogram statistics can be used. Compare the cumulative difference between the current frame and the reference frame. If the cumulative difference of the changes of the luminance component between the current frame and the reference frame is similar to the cumulative difference of the changes of the chrominance components, the current frame is a non-target frame. On the contrary, if the cumulative difference between the changes of the luminance component of the current frame and the reference frame is far different from that of the chrominance component, the current frame is the target frame. Whether the current frame is a target frame or a non-target frame is determined by setting a switch flag bit in each image frame, for example, a frame-level switch (lic_sep_flag) can be used as a switch flag bit. lic_sep_flag can be determined by the histogram statistics of all frames in the current coded frame and the reference frame list and passed as a frame-level flag bit. If the current frame is a non-target frame, set lic_sep_flag to 0, and if the current frame is a non-target frame, set lic_sep_flag to 1. The method for determining the luma predicted value and the chrominance predicted value will be described in detail below. In this embodiment, when the lic_sep_flag of the current frame is 1, for the luma predicted value of the current image block, the surrounding pixels of the luma reference block indicated by the motion vector MV_Y are used. A linear illumination compensation model (α ^Y , β ^Y ) is derived from the surrounding reference pixels of the current image block. The brightness prediction value of the brightness block after illumination compensation will be calculated by formula (2):

The motion vector information of the chrominance components U and V can be derived as (0, 0) Alternatively, the motion vector of the chrominance components can be obtained from the Merge/Skip candidate list, and then the linear illumination compensation model (α) is derived in the same way as the luma component prediction. ^U , β ^U ) and (α ^V , β ^V ), and calculated according to formulas (3) and (4):

Wherein, β ^V and/or β ^V can be deduced as 1 by default, and β ^V and/or β ^V can be deduced as 0 by default.

When the image processing method proposed by the embodiment of the present disclosure is used for the video decoding end, in the video decoding process, for the current image frame, the value of the switch flag bit of the current image frame is obtained first, and the color is determined according to the value of the switch flag bit. Whether the predicted value of the chrominance component is equal to the chrominance reconstructed value of the chrominance reconstructed value of the chrominance reference block or the chrominance compensation value, the luma predicted value is equal to the luma compensation value of the luma reference block.

An image processing apparatus is proposed in an embodiment of the present disclosure, as shown in FIG. 4 , including:

a determining unit 31, configured to determine a reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame;

The determining unit 31 is further configured to determine the luminance reconstruction value of the reference block and the chrominance reconstruction value of the reference block

a processing unit 32, configured to perform illumination compensation on the luminance reconstruction value of the reference block to obtain the luminance compensation value of the reference block;

The determining unit 31 is further configured to determine a luminance prediction value and a chrominance prediction value of the current image block, wherein the luminance prediction value of the current image block is a luminance compensation value of the reference block, and the current image block is a luminance compensation value of the reference block. The chroma predicted value of the block is the chroma reconstructed value of the reference block.

An embodiment of the present disclosure also proposes an image processing apparatus, as shown in FIG. 5 , including:

The determining module 41 is configured to determine the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, wherein the current image block is located in the current frame, and the motion vector of the chrominance component is different from the co-located motion vector, so The co-located motion vector is the motion vector of the luminance component, or the co-located motion vector is a motion vector obtained by scaling the motion vector of the luminance component according to the luminance sampling rate and the chrominance sampling rate;

The determining module 41 is further configured to determine the difference between the luminance reference block of the luminance component of the current image block and the chrominance component of the current image block according to the motion vector of the luminance component and the motion vector of the chrominance component of the current image block. a chroma reference block, wherein the luma reference block and the chroma reference block are located in a reference frame of the current frame;

The determining module 41 is further configured to determine the luminance reconstruction value of the luminance reference block and the chrominance reconstruction value of the chrominance reference block;

a processing module 42, configured to perform illumination compensation on the luminance reconstruction value of the luminance reference block, to obtain the luminance compensation value of the luminance reference block;

The processing module 42 is further configured to perform illumination compensation on the chrominance reconstruction value of the chrominance reference block to obtain the chrominance compensation value of the chrominance reference block;

The determination module 41 is further configured to determine the luminance prediction value and the chrominance prediction value of the current image block, wherein the luminance prediction value of the current image block is the luminance compensation value of the reference block, and the luminance prediction value of the current image block is The chrominance prediction value is the chrominance compensation value of the reference block.

For the embodiments of the apparatus, since they basically correspond to the method embodiments, reference may be made to the partial descriptions of the method embodiments for related parts. The apparatus embodiments described above are merely illustrative, wherein the modules described as separate modules may or may not be separate. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

In the above, the method and apparatus of the present disclosure have been described based on the embodiments and application examples. In addition, the present disclosure also provides a terminal and a storage medium, which are described below.

Referring next to FIG. 6 , it shows a schematic structural diagram of an electronic device (eg, a terminal device or a server) 800 suitable for implementing an embodiment of the present disclosure. Terminal devices in the embodiments of the present disclosure may include, but are not limited to, such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (eg, mobile terminals such as in-vehicle navigation terminals), etc., and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in the figure is only an example, and should not impose any limitation on the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 6 , an electronic device 800 may include a processing device (eg, a central processing unit, a graphics processor, etc.) 801 that may be loaded into random access according to a program stored in a read only memory (ROM) 802 or from a storage device 808 Various appropriate actions and processes are executed by the programs in the memory (RAM) 803 . In the RAM 803, various programs and data necessary for the operation of the electronic device 800 are also stored. The processing device 801 , the ROM 802 , and the RAM 803 are connected to each other through a bus 804 . An input/output (I/O) interface 805 is also connected to bus 804 .

Typically, the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 807 of a computer, etc.; a storage device 808 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 809. Communication means 809 may allow electronic device 800 to communicate wirelessly or by wire with other devices to exchange data. While the figures show electronic device 800 having various means, it should be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network via the communication device 809 , or from the storage device 808 , or from the ROM 802 . When the computer program is executed by the processing device 801, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or a combination of any of the above. More specific examples of computer readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), fiber optics, portable compact disk read only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave with computer-readable program code embodied thereon. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted using any suitable medium including, but not limited to, electrical wire, optical fiber cable, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some embodiments, the client and server can communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium (eg, a communications network) interconnected. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (eg, the Internet), and peer-to-peer networks (eg, ad hoc peer-to-peer networks), as well as any currently known or future development network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or may exist alone without being assembled into the electronic device.

The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to execute the aforementioned method of the present disclosure.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages—such as Java, Smalltalk, C++, but also conventional Procedural programming language - such as the "C" language or similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (eg, using an Internet service provider through Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more logical functions for implementing the specified functions executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or operations , or can be implemented in a combination of dedicated hardware and computer instructions.

The units involved in the embodiments of the present disclosure may be implemented in a software manner, and may also be implemented in a hardware manner. Among them, the name of the unit does not constitute a limitation of the unit itself under certain circumstances.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical Devices (CPLDs) and more.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, an image processing method is provided, including:

Determine the reference block corresponding to the current image block of the target frame in the video in the reference frame corresponding to the target frame;

Perform illumination compensation on the luminance reconstruction value of the reference block corresponding to the current image block to obtain the luminance prediction value of the current image block, and use the chrominance reconstruction value of the reference block corresponding to the current image block as the chrominance prediction value of the current image block.

According to one or more embodiments of the present disclosure, in some embodiments, the present disclosure provides an image processing method, comprising:

Determine the reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame;

determining the luma reconstruction value of the reference block and the chroma reconstruction value of the reference block;

Perform illumination compensation on the brightness reconstruction value of the reference block to obtain the brightness compensation value of the reference block;

Determine the luma predicted value and the chrominance predicted value of the current image block, wherein the luma predicted value of the current image block is the luma compensation value of the reference block, and the chrominance predicted value of the current image block is the chrominance reconstruction value of the reference block.

In some embodiments, the method further includes: determining the value of the switch flag bit carried in the current frame to be the first value.

In some embodiments, the value of the switch flag bit carried by the current frame is determined according to the difference between the luminance component of the current frame and the luminance component of the reference frame, and the difference between the chrominance component of the current frame and the chrominance component of the reference frame .

In some embodiments, when the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag carried in the current frame is the first value, wherein the first change value is the current The difference between the luminance component of the frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the chrominance component of the reference frame.

In some embodiments, the luminance prediction value of the current image block is determined according to formula (1):

为当前图像块的亮度预测值，

为参考块的亮度重构值，α^Y为亮度分量的颜色通道缩放系数，α^Y为亮度分量的偏移量。in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the reference block, α ^Y is the color channel scaling factor of the luminance component, and α ^Y is the offset of the luminance component.

In some embodiments, the determining the luma predicted value and the chrominance predicted value of the current image block includes: determining whether the current frame is a target frame or a non-target frame, if the current image frame is a target frame, the current image block The luma predicted value of the reference block is the luma compensation value, and the chrominance predicted value of the current image block is the chrominance reconstruction value of the reference block.

If the current frame is a non-target frame, the luma predicted value and chrominance predicted value of the current image block are respectively equal to the luma reconstruction value and chrominance reconstructed value of the reference block; or, the luma predicted value and chrominance predicted value of the current image block , respectively equal to the luminance compensation value and the chrominance compensation value of the reference block, and the chrominance compensation value is the value obtained by performing illumination compensation on the chrominance reconstruction value of the reference frame.

In some embodiments, the present disclosure provides an image processing method, comprising:

Determine the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, where the current image block is located in the current frame, the motion vector of the chrominance component is different from the co-located motion vector, and the co-located motion vector is the motion vector of the luminance component, or , the co-located motion vector is the motion vector obtained by scaling the motion vector of the luminance component according to the luminance sampling rate and the chrominance sampling rate;

The luminance reference block of the luminance component of the current image block and the chrominance reference block of the chrominance component of the current image block are determined according to the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, wherein the luminance reference block and the chrominance reference block are The reference block is located in the reference frame of the current frame;

determining the luma reconstruction value of the luma reference block and the chroma reconstruction value of the chroma reference block;

Perform illumination compensation on the luminance reconstruction value of the luminance reference block to obtain the luminance compensation value of the luminance reference block;

Perform illumination compensation on the chrominance reconstruction value of the chrominance reference block to obtain the chrominance compensation value of the chrominance reference block;

Determine the luma predicted value and the chrominance predicted value of the current image block, where the luma predicted value of the current image block is the luma compensation value of the luma reference block, and the chrominance predicted value of the current image block is the chrominance compensation value of the chrominance reference block .

In some embodiments, the present disclosure provides an image processing method where the motion vector of a chrominance reference block is the same as the motion vector of a chrominance component of a current image block.

In some embodiments, the present disclosure provides an image processing method, wherein the motion vector of the chrominance component of the current image block is (0,0).

In some embodiments, the present disclosure provides an image processing method, the motion vector of the chrominance component of the current image block is determined according to the vector candidate list, and the index of the motion vector of the chrominance component in the vector candidate list is used to indicate the color The motion vector of the degree component.

In some embodiments, the present disclosure provides an image processing method, which adopts formula (2) to obtain the luminance prediction value of the current image block:

为所述当前图像块的亮度预测值，

为所述亮度参考块的亮度重构值，α^Y为所述亮度分量的颜色通道缩放系数，β^Y为所述亮度分量的偏移量；in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the luminance reference block, α ^Y is the color channel scaling coefficient of the luminance component, and β ^Y is the offset of the luminance component;

and / or,

Use formula (3) and formula (4) to obtain the chrominance prediction value of the current image block:

为所述当前图像块的色度预测值的U分量，

为所述当前图像块的色度预测值的V分量，

为所述色度参考块的色度重构值的U分量，

为所述色度参考块的色度重构值的V分量，α^U为所述U分量的颜色通道缩放系数，α^V为所述V分量的颜色通道缩放系数，β^U为所述U分量的偏移量，β^V为所述V分量的偏移量。in,

is the U component of the chrominance prediction value of the current image block,

is the V component of the chrominance prediction value of the current image block,

is the U component of the chrominance reconstruction value of the chrominance reference block,

is the V component of the chrominance reconstruction value of the chrominance reference block, α ^U is the color channel scaling coefficient of the U component, α ^V is the color channel scaling coefficient of the V component, and β ^U is the U component , β ^V is the offset of the V component.

In some embodiments, the present disclosure provides an image processing method for determining a motion vector of a luminance component and a motion vector of a chrominance component of a current image block, including:

Determine the value of the switch flag bit carried by the current frame;

If the value of the switch flag of the current frame is the first value, the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the chrominance reconstruction of the chrominance reference block. value;

If the value of the switch flag of the current frame is the second value, the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the chrominance compensation value of the chrominance reference block. .

In some embodiments, the present disclosure provides an image processing method. The value of a switch flag carried by the current frame is based on the difference between the luminance component of the current frame and the luminance component of the reference frame, and the chrominance component of the current frame and the reference frame. The difference between the chroma components of the frame is determined.

In some embodiments, the present disclosure provides an image processing method, in the case that the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag bit carried in the current frame is the first value , wherein the first change value is the difference between the luminance component of the current frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the chrominance component of the reference frame.

In some embodiments, the present disclosure provides an image processing apparatus, including:

A determining unit for determining the reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame;

a determining unit, further configured to determine the luminance reconstruction value of the reference block and the chrominance reconstruction value of the reference block;

a processing unit, configured to perform illumination compensation on the luminance reconstruction value of the reference block to obtain the luminance compensation value of the reference block;

The determining unit is further configured to determine the luminance prediction value and the chrominance prediction value of the current image block, wherein the luminance prediction value of the current image block is the luminance compensation value of the reference block, and the chrominance prediction value of the current image block is the color value of the reference block. degree reconstruction value.

In some embodiments, the present disclosure provides an image processing apparatus, comprising:

The determination module is used to determine the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, wherein the current image block is located in the current frame, the motion vector of the chrominance component is different from the co-located motion vector, and the co-located motion vector is the luminance component The motion vector of , or the co-located motion vector is the motion vector obtained by scaling the motion vector of the luminance component according to the luminance sampling rate and the chrominance sampling rate;

The determining module is further configured to determine the luminance reference block of the luminance component of the current image block and the chrominance reference block of the chrominance component of the current image block according to the motion vector of the luminance component and the motion vector of the chrominance component of the current image block, wherein the luminance The reference block and the chroma reference block are located in the reference frame of the current frame;

a determining module, further configured to determine the luminance reconstruction value of the luminance reference block and the chrominance reconstruction value of the chrominance reference block;

a processing module, configured to perform illumination compensation on the luminance reconstruction value of the luminance reference block to obtain the luminance compensation value of the luminance reference block;

The processing module is also used to perform illumination compensation on the chrominance reconstruction value of the chrominance reference block to obtain the chrominance compensation value of the chrominance reference block;

The determining module is further configured to determine the luminance prediction value and the chrominance prediction value of the current image block, wherein the luminance prediction value of the current image block is the luminance compensation value of the luminance reference block, and the chrominance prediction value of the current image block is the chrominance reference value. The chroma compensation value for the block.

According to one or more embodiments of the present disclosure, there is provided a terminal including: at least one memory and at least one processor;

Wherein, the at least one memory is used for storing program codes, and the at least one processor is used for calling the program codes stored in the at least one memory to execute any one of the methods described above.

According to one or more embodiments of the present disclosure, there is provided a storage medium for storing a program code for executing the above-described method.

The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of the disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, and should also cover, without departing from the above-mentioned disclosed concept, the technical solutions formed by the above-mentioned technical features or Other technical solutions formed by any combination of its equivalent features. For example, a technical solution is formed by replacing the above features with the technical features disclosed in the present disclosure (but not limited to) with similar functions.

Additionally, although operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although the above discussion contains several implementation-specific details, these should not be construed as limitations on the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or logical acts of method, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (8)

1. an image processing method, is characterized in that, comprises: determining a reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in a reference frame of the current frame; determining a luma reconstruction value for the reference block and a chroma reconstruction value for the reference block; performing illumination compensation on the luminance reconstruction value of the reference block to obtain the luminance compensation value of the reference block; Determine the luma predicted value and the chrominance predicted value of the current image block, wherein the luma predicted value of the current image block is the luma compensation value of the reference block, and the chrominance predicted value of the current image block is the The chroma reconstruction value of the reference block. 2. The image processing method according to claim 1, further comprising: It is determined that the value of the switch flag bit carried in the current frame is the first value. 3. The image processing method according to claim 2, wherein the value of the switch flag bit carried by the current frame is based on the difference between the luminance component of the current frame and the luminance component of the reference frame, and The difference between the chrominance components of the current frame and the chrominance components of the reference frame is determined. 4. The image processing method according to claim 3, wherein, when the difference between the first change value and the second change value is greater than the difference threshold, the value of the switch flag bit carried by the current frame is The first value, where the first change value is the difference between the luminance component of the current frame and the luminance component of the reference frame, and the second change value is the difference between the chrominance component of the current frame and the reference frame. difference between the chrominance components of the reference frame. 5. The image processing method according to claim 1, wherein the predicted value of the luminance of the current image block is determined according to formula (1):

in,

is the luminance prediction value of the current image block,

is the luminance reconstruction value of the reference block, α ^Y is the color channel scaling factor of the luminance component, and β ^Y is the offset of the luminance component. 6. An image processing device, comprising: a determining unit, configured to determine a reference block of the current image block, wherein the current image block is located in the current frame, and the reference block is located in the reference frame of the current frame; The determining unit is further configured to determine the luminance reconstruction value of the reference block and the chrominance reconstruction value of the reference block; a processing unit, configured to perform illumination compensation on the luminance reconstruction value of the reference block to obtain the luminance compensation value of the reference block; The determining unit is further configured to determine a luminance predicted value and a chrominance predicted value of the current image block, wherein the luminance predicted value of the current image block is a luminance compensation value of the reference block, and the current image block The chrominance predicted value of is the chrominance reconstructed value of the reference block. 7. A terminal, characterized in that, comprising: at least one memory and at least one processor; Wherein, the at least one memory is used for storing program codes, and the at least one processor is used for calling the program codes stored in the at least one memory to execute the method according to any one of claims 1 to 5. 8. A storage medium for storing program codes for performing the method of any one of claims 1 to 5.

Images