CN113632481B - Delay reduction method and device for chroma residual scaling - Google Patents

Abstract

A video decoding method and device are disclosed. According to one approach, a chroma residual scaling factor is derived based on adjacent predicted or reconstructed luma samples of a collocated luma block, where the adjacent predicted or reconstructed luma samples of the collocated luma block correspond to the adjacent predicted or reconstructed luma samples along the collocated luma Samples from M samples at the top boundary of the block and N samples along the left boundary of the collocated luma block. Applies chroma scaling to the chroma residual samples of the chroma residual block according to the derived chroma residual scaling factor. In another approach, the chroma residual scaling factor is derived based on one or more reconstructed luma samples juxtaposed outside the luma processing data unit. In another approach, the chroma residual scaling factors are signaled in or parsed out of the APS (Adaptive Parameter Set) of the data stream.

Description

Delay reduction method and device for chroma residual scaling

[Cross-reference to related applications]

The present invention requires the U.S. Provisional Patent Application No. 62/818,799 submitted on March 15, 2019, the U.S. Provisional Patent Application No. 62/822,866 submitted on March 23, 2019, and the U.S. Provisional Patent Application No. 62/837,773 submitted on April 24, 2019. U.S. provisional patent application, U.S. provisional patent application with application number No. 62/863,333 filed on June 19, 2019, U.S. provisional patent application with application number No. 62/866,710 filed on June 26, 2019, application filed on July 4, 2019 Priority to U.S. Provisional Patent Application No. 62/870,757. The contents of this U.S. Provisional Patent Application are incorporated by reference into this specification.

【Technical field】

The present invention relates to video codecs for color video data in which luminance mapping is applied to the luminance component. In particular, this disclosure discloses techniques for deriving and/or signaling one or more chroma scaling factors for chroma residual scaling.

【Background technique】

Versatile Video Codec (VVC) is an emerging video codec standard developed by the Joint Video Experts Group, which consists of ITU-T Study Group 16 Video Codec Expert Group and ISO/IEC JTC1SC29/WG11 (Moving Pictures Experts Group (Moving Picture Experts Group, abbreviated as MPEG)). VVC is based on the HEVC (High Efficiency Video Codec) video standard with improved and new codec tools. For example, the reshap process is a new codec tool adopted in VTM-4.0 (VVC Test Model version 4.0). The reshaping process is also called LMCS (LumaMapping and Chroma Scaling). When reshaping is applied, video samples are encoded, decoded and reconstructed in the reshape domain before loop filtering. Convert the reconstructed samples from the reshaped domain to the original domain by using inverse reshaping. The loop-filtered original domain reconstructed samples are stored in the decoded picture buffer. For Inter mode, the motion compensation (MC) predictor is converted into the reshaping domain by using forward reshaping. Figure 1 shows an example of the reshaping process on the decoder side.

As shown in Figure 1, the data stream is derived from the CABAC (Context Adaptive Binary Arithmetic Coding and Decoding) decoder (i.e. CABAC ^-1 ), inverse quantization (i.e. Q ^-1 ) and inverse transform (T ^-1 ) in block 110 The reconstructed brightness residual Y _res . The reconstructed luminance residual is provided to luminance reconstruction block 120 to generate a reconstructed luminance signal. For intra mode, the predictor is from intra prediction block 130. For inter mode, the predictor comes from motion compensation block 140. Since reshaping is applied to the luma signal on the encoder side, forward reshaping 150 is used for the predictor from the motion compensation block 140 before it is provided to the reconstruction block 120 . Inverse reshaping 160 is applied to the reconstructed luminance signal from reconstruction block 120 to recover an un-shaped reconstructed luminance signal. A loop filter 170 is then applied to the unshaped reconstructed luminance signal before the signal is stored in a decoded picture buffer (DPB) 180.

When reshaping is applied, chroma residual scaling is also applied. Chroma residual scaling compensates for the interaction between the luma and chroma signals, as shown in Figure 2. In Figure 2, the upper part corresponds to luma decoding and the lower part corresponds to chroma decoding.

Chroma residual scaling is applied at the TU level at the encoder side and decoder side according to the following equations respectively:

Encoder side: C _ResScale ＝C _Res *C _Scale ＝C _Res /C _ScaleInv (1)

Decoder side: C _Res =C _ResScale /C _Scale =C _ResScale *C _ScaleInv (2)

In the above equation, C _Res is the original chroma residual signal and C _ResScale is the scaled chroma residual signal. C _Scale is the scaling factor calculated for the inter mode predictor using FwdLUT (i.e. forward look-up table) and converted to its reciprocal C _ScaleInv for multiplication instead of division on the decoder side, This reduces implementation complexity. The scaling operations on the encoder and decoder sides are implemented through fixed-point integer arithmetic (fixed-point integerarithmetic) through the following formula:

c'＝sign(c)*((abs(c)*s+2 ^{CSCALE_FP_PREC-1} )>>

CSCALE_FP_PREC) (3)

In the above equation, c is the chroma residual, s is the chroma residual scaling factor in cScaleInv[pieceIdx], pieceIdx is determined by the corresponding average luminance value of the TU, and CSCALE_FP_PREC is a constant used to specify the accuracy. To derive the scaling factor, the predictor for the entire TU is used. The value of C_ScaleInv is calculated according to the following steps:

If it is intra mode, the average value of the intra prediction brightness value is calculated; if it is the inter mode, the average value of the inter prediction brightness value of the forward reshaping is calculated. In other words, the average brightness value acgY′ _TU is calculated in the reshaped domain.

Find the index idx where avgY′ _TU belongs to the inverse mapping PWL.

C _ScaleInv =cScaleInv[idx]

The step of deriving the chroma scaling factor C _ScaleInv is performed by block 210 in Figure 2, which is input to chroma residual scaling block 250. The derived chroma scaling factor C _ScaleInv is used to transform the scaled chroma residuals via CABAC (Context Adaptive Binary Arithmetic Coding) decoding (i.e. CABAC ^-1 ), inverse quantization (i.e. Q ^-1 ) and inverse transform ( T ^-1 ) to rebuild. Reconstruction block 220 reconstructs the chroma signal by adding predictors to the reconstructed chroma residual. For intra mode, the predictor is from intra prediction block 230. For inter mode, the predictor comes from motion compensation block 240. A loop filter 270 is then applied to the reconstructed chroma signal before it is stored in a chroma decoded picture buffer (DPB) 280 .

Figure 3 shows an example of brightness mapping. In Figure 3A, a 1:1 mapping is shown, where the output (i.e., the reshaped brightness) is the same as the input. Since the histogram of luminance samples is usually not flat, using intensity reshaping can help improve performance in terms of RDO (rate-distortion optimization). Computes statistics for luma samples for an image region (such as a picture). The mapping curve is then statistically determined. Usually, piece-wiselinear (PWL) mapping curve is used. Figure 3B shows an example of a piecewise linear (PWL) map with 3 segments, where two adjacent segments have different slopes. Dashed line 340 corresponds to a 1:1 mapping. If the samples ranging from 0 to 340 have a large spatial variance and a small number of occurrences, map the input range 0-340 to a smaller output range (i.e. 0-170) , shown as line segment 310 in Figure 3B. If the samples ranging from 340 to 680 have small spatial variation and a large number of occurrences, then the input range 340-680 is mapped to a larger output range (i.e., 170-850), as shown by line segment 320 in Figure 3B. If the samples ranging from 680 to 1023 have large spatial variation and a small number of occurrences, then the input range 680-1023 is mapped to a smaller output range (i.e., 850-1023), as shown in line segment 330 of Figure 3B . Figure 3B is intended to show a simple PWL mapping. In practice, a PWL map can have more or fewer segments.

Intra sub-block partition (ISP) and sub-block transform (SBT)

In order to generate better intra mode predictors, intra sub-block partitioning (ISP) can be applied. When ISP is applied, the luminance component is divided into multiple sub-TBs. Sub-TBs are rebuilt one by one. For each sub-TU, the reconstructed samples of adjacent sub-TBs can be used as adjacent reconstructed samples for intra prediction. For the chroma component TB, it is not divided into multiple sub-TBs like the luma.

Similar to ISP, sub-block transform (SBT) can be applied in inter mode. When applying SBT, only part of the CU data is converted. For example, the current block can be divided into two partitions by horizontal split or vertical split. Only one partition can be used for converting codecs. The residual of the other partition is set to zero. For example, a CU is divided into two TUs or four TUs. Only one of the TUs has non-zero coefficients.

Signaling of LMCS parameters

A syntax table for the LMCS parameters being considered by VVC is shown in Table 1.

Table 1.

In the above grammar table, the semantics of the grammar are defined as follows:

lmcs_min_bin_idx specifies the minimum data sub(bin) index of the PWL (piecewise linear) model for brightness mapping

lmcs_delta_max_bin_idx specifies a delta value between 15 and the maximum data subindex used in lmcs, LmcsMaxBinIdx. The value should be in the range 1 to 15 (inclusive).

lmcs_delta_cw_prec_minus1 plus 1 is the number of data used to represent the syntax lmcs_delta_abs_cw[i].

lmcs_delta_abs_cw[i] is the absolute deltacodeword value of the i-th data element.

lmcs_delta_sign_cw_flag[i] is the sign of variable lmcsDeltaCW[i].

The variable lmcsDeltaCW[i] is derived as follows:

lmcsDeltaCW[i]＝

(1-2*lmcs_delta_sign_cw_flag[i])*lmcs_delta_abs_cw[i].

The variable lmcsCW[i] specifies the number of codewords for each interval in the mapped domain, where i=0...15. It can be exported as follows:

OrgCW＝(1<<BitDepthY)/16

For i=0...lmcs_min_bin_idx-1, lmcsCW[i] is set equal to 0.

For i=lmcs_min_bin_idx...LmcsMaxBinIdx, the following applies:

lmcsCW[i]=OrgCW+lmcsDeltaCW[i]

The value of lmcsCW[i] should be in the range from (OrgCW>>3) to (OrgCW<<3-1) (including (OrgCW>>3) and (OrgCW<<3-1)).

For i=LmcsMaxBinIdx+1…15, lmcsCW[i] is set equal to 0.

In order to represent the PWL model of the reshaped curve, three variables LmcsPivot[i](i=0…16), ScaleCoeff[i](i=0…15), and InvScaleCoeff[i](i=0…15) are derived as follows :

In the above derivation, SCALE_FP_PREC is a constant value used to specify the precision.

In the LMCS process, the delay in scaling of the chroma residuals may have a negative impact on the processing speed due to the dependence on the corresponding luma data. Therefore, it is desirable to develop methods and apparatus that reduce the delay of chroma residual scaling.

[Content of the invention]

Video decoding methods and devices are disclosed. According to a method of the invention, a current chroma residual block is received. One or more chroma residual scaling factors are derived based on neighboring predicted or reconstructed luma samples of the collocated luma block, where Luminance samples correspond to samples among the M samples along the top boundary of the collocated luma block and the N samples along the left boundary of the collocated luma block, where M and N are positive integers. Chroma scaling is applied to the chroma residual samples of the current chroma residual block based on the derived one or more chroma residual scaling factors.

In one embodiment, adjacent predicted or reconstructed luma samples of a collocated luma block correspond to M samples along the top boundary of the collocated luma block. In another embodiment, adjacent predicted or reconstructed luma samples of a collocated luma block correspond to N samples along the left boundary of the collocated luma block. In yet another embodiment, adjacent predicted or reconstructed luma samples of a collocated luma block correspond to both M samples along the top boundary of the collocated luma block and N samples along the left boundary of the collocated luma block. sample.

In one embodiment, the boundary sample at the upper left position of the collocated luma block is used to derive the one or more chroma residual scaling factors, if the boundary sample at the upper left position of the collocated luma block is available. If the boundary sample at the upper left position of the collocated luma block is not available, the left boundary sample along the left boundary of the collocated luma block or the top boundary sample along the top boundary of the collocated luma block is used to derive the representation of one or more chroma residual scaling factor.

According to another method, chroma residual data associated with a current chroma processing data unit in a picture is received, wherein the picture is divided into a plurality of non-overlapping processing data units, and each processing data unit includes a luma processing data unit and one or more chroma processing data units. One or more chroma residual scaling factors are derived based on one or more reconstructed luma samples external to a collocated luma processing data unit associated with the current chroma processing data unit. Chroma scaling is then applied to the chroma residual samples of the current chroma processing data unit based on the derived one or more chroma residual scaling factors. According to a variant of the method, a chroma residual scaling factor is derived based on one or more reconstructed luma samples from a first codec unit (CU) covering the upper left position of the collocated luma processing data unit.

In one embodiment, the one or more reconstructed luma samples outside a first codec unit (CU) covering a collocated luma processing data unit correspond to one or more previously codec luma processing data units. One or more reconstructed brightness samples. In another embodiment, the one or more reconstructed luma samples of the one or more previously codec luma processing data units correspond to a first codec unit (CU) along the covering collocated luma one or more reconstructed luma samples along the top boundary of the CU, one or more reconstructed luma samples along the left boundary of the first codec unit (CU) covering the collocated luma, or both.

In one embodiment, the reconstructed luma samples outside the collocated luma processing data unit correspond to one or more reconstructed luma samples of one or more previously decoded luma processing data units. For example, the reconstructed luma samples of the one or more previously decoded luma processing data units correspond to one or more reconstructed luma samples along the top boundary of the collocated luma processing data unit, along the collocated luma processing data unit One or more reconstructed luminance samples, or both, of the cell's left boundary.

In yet another method, one or more chroma residual scaling factors are signaled in the APS (Adaptation Parameter Set) level of the video data stream on the encoder side, or from the decoder side The APS level resolves one or more chroma residual scaling factors.

[Picture description]

Figure 1 shows an exemplary block diagram of a video decoder incorporating a brightness reshaping process.

Figure 2 shows an exemplary block diagram of a video decoder that combines a luma reshaping process and chroma scaling.

Figure 3A shows an example of a 1:1 brightness mapping, where the output (i.e. the reshaped brightness) is the same as the input.

Figure 3B shows an example of piecewise linear (PWL) brightness mapping with 3 segments.

Figure 4 shows an example of deriving a chroma scaling factor based on reference reconstructed luma samples along a VPDU top boundary, a left boundary, or both, according to an embodiment of the present invention.

Figure 5 shows an example of deriving a chroma scaling factor based on a reference reconstructed luminance sample TL, A or L position according to an embodiment of the present invention.

6 illustrates a flowchart of an exemplary decoding system for deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of collocated luma blocks in accordance with an embodiment of the present invention.

7 illustrates a flowchart of another exemplary decoding system for deriving one or more colors based on one or more reconstructed luma samples external to a collocated luma processing data unit in accordance with one embodiment of the present invention. Residual scaling factor.

Figure 8 shows a flowchart of an exemplary codec system in which one or more chromas are signaled in the APS (Adaptive Parameter Set) level of the video data stream at the encoder side according to an embodiment of the invention. Residual scaling factors, or one or more chroma residual scaling factors parsed from the APS level of the video data stream on the decoder side.

【Detailed ways】

The following description is of the best mode contemplated for carrying out the invention. This description is made to illustrate the general principles of the invention and should not be considered limiting. The scope of the invention can best be determined by reference to the appended claims.

In chroma residual scaling, for a chroma TU, all corresponding luma predictors are used to derive a single scaling factor. Chroma sample reconstruction cannot be processed until scaling factors are exported. It introduces new data dependencies to the cross-component process, resulting in longer delays in chroma sample reconstruction. In VVC, some decoder auxiliary tools are introduced to improve the brightness predictor to improve encoding and decoding efficiency. These types of codec tools will also add to the critical path of the reconstruction loop. In inter and intra mode prediction, prediction samples of CU/PU/TU can be divided into multiple MxN blocks, and these blocks can be processed sequentially or in parallel.

In one embodiment, to reduce the latency of chroma sample reconstruction, for a CU/PU/TU, it only uses its upper left KxL luma samples (e.g., luma predictors or luma reconstruction samples or luma residuals) or its upper left corner M luma samples are used to derive one or more chroma residual scaling factors. K and L can be equal to 1, 2, 4, 8, 16, 32 or 64. One or more scaling factors are used for the entire chroma TU. For example, use the 16x15 luma sample at the top left. In another example, the top left 1x1 luma sample is used. In another example, the 256 luminance samples from the upper left are used. In another example, the 1 luma sample from the upper left is used. In another example, if the width and height of the luma CU/TU are greater than or equal to 16, then the top left 16x16 luma samples are used; otherwise, up to 256 top left luma samples are used. In one example, when ISP is applied, only the upper left KxL block or the upper left M samples of the first ISP sub-TB are used to derive the chroma residual scaling factor. In another example, when applying SBT, only the upper left KxL blocks or upper left M samples of the TU with non-zero coefficients are used to derive the scaling factor. In another embodiment, only a portion of the corresponding luma samples are used to derive the chroma residual scaling factor. For example, a portion of the top-row and a portion of the left column of the internally collocated luma CT/TU/PU boundary samples are used to export the color degree remaining scaling factor.

In another embodiment, to reduce the latency of chroma sample reconstruction for a CU/PU/TU, only adjacent boundary samples along the current TB are used (i.e., corresponding luma samples or called collocated luma samples) Returns one or more chroma residual scaling factors. The samples can be predicted samples or reconstructed samples of neighboring blocks. In one embodiment, M samples along the top boundary are used to derive one or more chroma residual scaling factors. In one embodiment, N samples along the left boundary are used to derive one or more chroma residual scaling factors. In one embodiment, M samples along the top boundary and N samples along the left boundary are used to derive one or more chroma residual scaling factors. Here, M and N can be 1, 2, 4, 8, 16, 32 or 64. In another embodiment, one or more chroma residual scaling factors are derived using samples located at the upper left corner of the L-shaped boundary. In another embodiment, the sample is used if the upper left neighbor sample is available. Otherwise, use one of the top adjacent samples or one of the left adjacent samples. In one example, if none of the above samples are available, the top left sample in the collocated luma block is used. One or more scaling factors are used for the entire chroma TU.

In another embodiment, to reduce the latency of chroma sample reconstruction when applying chroma residual scaling, it is proposed to partition the chroma TU into sub-blocks, such as KxL sub-blocks or sub-blocks with a block size equal to M. K and L can be 2, 4, 8, 16 or 32; M can be 4, 8, 16, 32, 64, 128, 256, 512 or 1024. For each KxL chroma residual sub-block, one or more scaling factors are derived. Different KxL chroma residual sub-blocks can have different scaling factors. For example, for an M×N block, where M is larger than K (i.e., the width threshold) and N is smaller than L (i.e., the height threshold), the M×N block is divided into M/K blocks of size (K×N) .

In another embodiment, when the size/area/width/height of the chroma residual TU is less than the first threshold or greater than the second threshold, no chroma residual scaling is applied. For example, chroma residual scaling is disabled when the TU size is less than or equal to 8 or 16 or 64. In another example, chroma residual scaling is disabled when the TU width or height is less than or equal to 2 or 4 or 8 or 16. In another example, chroma residual scaling is disabled when the TU size is greater than or equal to 16, 64, 256, or 1024. In another example, chroma residual scaling is disabled when the TU width or height is greater than or equal to 8 or 16 or 32. In another example, chroma residual scaling is disabled for certain prediction modes. For example, chroma residual scaling is disabled for blocks that have DMVR mode, BIO mode, LIC mode, diffusion mode, or a combination of these modes enabled.

DMVR (decoder side motion vector refinement) is a new encoding and decoding tool developed in recent years. DMVR exports MV refinement information on the decoder side to improve encoding and decoding performance. BIO is another new encoding and decoding tool developed in recent years. BIO derives sample-level motion refinement based on the assumptions of optical flow and stable motion, where the current pixel in the B slice (bidirectionally predicted slice) is predicted by a pixel in reference picture 0 and a pixel in reference picture 1. LIC (LocalIllumination Compensation) is a method that uses adjacent samples of the current block and the reference block to perform inter-frame prediction. It is based on a linear model using scaling factors and offsets.

In one embodiment, when ISP is applied, only a portion of the luma sub-TB is used to derive the chroma residual scaling factor. For example, only the first luminance TB is used to derive the chroma residual scaling factor. Using the first luma TB to generate scaling factors can reduce the delay in chroma sample reconstruction. In another example, only the last luminance TB is used to derive the chroma residual scaling factor.

In another embodiment, when applying ISP, each luma sub-TB can be treated as a separate TB. For each sub-TB, it can calculate its own chroma residual scaling factor. The method proposed above can also be applied, for example, dividing each luma sub-TB into several KxL sub-blocks and deriving a scaling factor for each sub-block. For chroma TB, even if it is not divided into multiple sub-TBs like luminance when performing transformation, chroma TB is also divided into multiple sub-regions when performing chroma residual scaling. Each sub-region corresponds to a brightness TB; each sub-region corresponds to one or more brightness sub-TBs; or one or more chroma sub-regions correspond to a brightness sub-TB. For each chroma sub-region, if the luma sub-TB is divided into sub-blocks to derive the scaling factor, it can be further divided into sub-blocks.

In another embodiment, when applying SBT, only luma partitions with non-zero coefficients are used to derive the chroma residual scaling factors. The brightness partition used can be divided into sub-blocks to derive the scaling factor. In another embodiment, when applying SBT, the luma samples of the entire CU can be used to derive one or more scaling factors.

In another embodiment, the luma samples of a CU (not a TU or TB) are used to derive the chroma residual scaling factor. When ISP is applied, the entire luma CU sample will be used to derive the chroma residual scaling factor. For example, luma CU samples can be divided into sub-blocks to derive different scaling factors for different sub-blocks. Sub-blocks can span ISP sub-TB boundaries.

In another embodiment, the chroma residual scaling factor derivation may be different for applying a transform or not (eg, transform skipping). For chroma residual scaling factor derivation, the values/factors/constants or equations can be different. In another embodiment, the chroma residual scaling factor derivation may be different for different prediction modes or different residual energy levels.

On the encoder side, scaling factor derivation typically involves deriving λ for the quantization parameter. In one embodiment, the entire TU prediction data is used to derive the lambda value. For chroma residual scaling, the TU is still divided into sub-blocks. Each sub-block can derive its own scaling factor.

The same types of process issues will be encountered during the BIO and DMVR processes. For example, for the BIO process, SAD (sum of absolute differences) calculation at the TU/PU/CU level is performed. If the calculated cost is small enough, the BIO process can be disabled. For the DMVR process, if the entire CU/PU/TU is used to derive an MV difference (MVD), this is not a friendly design. Therefore, it is proposed to align the BIO with DMVR, or even align the BIO and/or DMVR with the chroma residual scaling process, which divides the current block into KxL blocks. For example, for BIO and DMVR processes, the current block is divided into KxL blocks. For each KxL block, it can calculate the cost of its BIO early termination decision, or it can derive its own MVD using the DMVR process. In another example, for BIO or DMVR processes, the current block is divided into KxL blocks for performing BIO and DMVR processes, where the size of KxL (in luma sample precision) is the same as the base unit of the chroma residual scaling process same.

In another embodiment, different modes may use reference brightness samples at different locations.

In one embodiment, for blocks that can refer to adjacent reconstructed samples for the prediction process, the reference luma samples used for scaling value derivation are from adjacent reconstructed samples or reference boundaries used to generate predictors for the current CU or TU. sample. For example, if the current block is in intra prediction mode, the reference luma sample is the upper left, upper, or left reference boundary sample of the current CU. Therefore, for Intra Subpartition Prediction (ISP) mode, the chroma residual scaling values are derived using the upper left, upper, or left boundary reconstruction samples of the L-shaped boundary reconstruction samples of the current CU/TU (not the subpartition TU). In another example, if the current block is in intra prediction mode, the reference luma sample is the upper left reference boundary sample of the current TU. Therefore, for intra sub-partition prediction (ISP) mode, the chroma residual scaling value is derived using the upper, upper or left boundary reconstruction samples of the L-shaped boundary reconstruction sample of the current TU (sub-partition). The upper left L-shaped boundary reconstruction sample may be one sample.

In another example, if the current CU is in inter prediction mode but predicted by combined inter/intra mode (CIIP) or other prediction methods that require adjacent reconstructed samples, the reference luma samples can be reference boundary reconstructed samples Or the reference boundary sample used to generate the predictor for the current CU or TU (e.g., using the upper left neighbor reconstructed sample), as described above. As is known in the art, CIIP is another codec tool developed in recent years. CIIP uses a weighted average of inter- and intra-prediction signals to obtain CIIP predictions.

In another embodiment, if the current block is in inter prediction mode, the reference luma sample may be the upper left corner luma prediction sample of the current CU or TU.

In one embodiment, if it is CIIP mode, the reference luma sample is the upper left corner luma prediction sample of the inter predictor.

In another embodiment, if the current block is in inter prediction mode other than CIIP mode, the reference luma sample may be the upper left corner luma prediction sample of the current CU or TU. In this embodiment, blocks coded in CIIP mode are regarded as intra prediction modes, and any of the above methods related to intra prediction modes may be applied.

In another embodiment, if the current block is in IBC mode, the determination of reference luma samples is the same as in inter prediction mode. As known in the art, IBC (Intra Block Copy) is a new codec tool developed in recent years. IBC is similar to inter prediction mode. However, IBC uses reference pixels from the current frame instead of reference pixels from previously encoded and decoded frames.

In another embodiment, if the current block is IBC mode, the determination of the reference luma samples is the same as for intra prediction mode.

When one or more reference luma samples are prediction samples for the current CU or TU, a different number of samples may be used as described in the above embodiments.

In one embodiment, the above embodiments of intra and inter prediction modes may be combined.

In one embodiment, for intra prediction mode and CIIP mode, the reference luma sample is the upper left boundary reference sample used to generate the intra predictor, and for inter prediction mode except CIIP mode, the reference luma sample is the upper left boundary Angular brightness prediction sample.

In one embodiment, for intra prediction mode, the reference luma sample is the upper left boundary reference sample used to generate the intra predictor, and for inter prediction mode, except CIIP mode, the reference luma sample is the upper left luma prediction sample . For CIIP mode, the reference luma sample is the upper left corner luma prediction sample of the inter predictor. In other words, prediction samples are mixed with intra prediction samples before use.

In one embodiment, for intra prediction mode and CIIP mode, the reference luma sample is the top left, top, or left (first available) boundary reconstruction sample, and for inter prediction mode (except CIIP mode), the reference luma sample is the top left Corner brightness prediction sample. In other words, prediction samples are mixed with intra prediction samples before use.

In another example, only the upper left reconstructed sample is used.

If the reference sample is not available, the scaling factor is set to its default value. In one embodiment, the default value is equal to (1<<PREC), where PREC is the chroma-scaled prediction.

In one embodiment, for intra prediction mode, the reference luminance sample is the upper left, upper, or left (first available) boundary reconstruction sample, and for inter prediction mode (except CIIP mode), the reference luminance sample is the upper left corner luminance Prediction sample. For CIIP mode, the reference luma sample is the upper left corner luma prediction sample of the inter predictor. In other words, prediction samples are mixed with intra prediction samples before use.

Mode constrains and conditionally disables chroma splitting within the root block

In another embodiment, a root block is determined and the luminance components of the root block can be further divided into smaller blocks. According to this embodiment, whether the chrominance component of the root block can be further divided is determined based on the prediction mode of the luma block within the same root block.

In the previous approach, three cases of "same pattern" definition are listed below:

Case 1. The same mode means that all blocks within the root block must be in intra prediction mode, or all blocks within the root block must be in inter prediction mode, or all blocks within the root block must be in IBC mode.

Case 2. The same mode means that all blocks in the root block must be in intra prediction mode, or all blocks in the root block must be in one of inter prediction mode and IBC prediction mode (inter/IBC mode).

Case 3. The same mode means that all blocks in the root block must be in intra prediction mode and one of the IBC prediction modes (intra/IBC mode), or all blocks in the root block must be in inter prediction mode.

In one embodiment, if all blocks within the current root block are inter prediction mode, inter/IBC mode and inter prediction mode for case 1, case 2 and case 3 respectively, then the division of chroma components follows the luma block . If all blocks within the current root block are intra prediction mode, intra prediction mode and intra/IBC mode for case 1, case 2 and case 3 respectively, then the chroma components of this root block cannot be further split, thus resulting in Multiple luma blocks correspond to one chrominance block.

In another embodiment, a root block is determined and the luminance components of the root block can be further divided into smaller blocks. According to this embodiment, it is determined whether the chrominance component of the root block cannot be further divided. In this area, the luminance blocks may be of the same pattern or may be of different patterns.

In one embodiment, chroma residual scaling cannot be applied when the chroma components are not allowed to be further split. In another embodiment, chroma residual scaling can still be applied when the chroma components are not allowed to split further. The location of the reference brightness samples can vary. In one embodiment, the upper left NxM luma prediction samples of the collocated luma block are used. N and M can be 1, 2, 4, 8, 16, 32, 64 and 128. In another embodiment, the reconstructed top boundary K reference samples of the current root block are used. In another embodiment, the reconstructed left boundary K reference samples of the current root block are used. K can be 1, 2, 4, 8, 16, 32, 64 and 128. In another embodiment, the reconstructed upper left reference sample of the current root block is used.

In another embodiment, chroma residual scaling cannot be applied when the chroma components are not allowed to be further segmented and the chroma root block is coded in intra mode. In another example, when the chroma components are not allowed to be further split and the chroma root block is coded in IBC mode, then chroma residual scaling cannot be applied. In another embodiment, chroma residual scaling can still be applied when further splitting of the chroma components is not allowed and the chroma root block is coded in intra mode. In another example, when the chroma components are not allowed to be further split and the chroma root block is coded in IBC mode, chroma residual scaling can still be applied. The location of the reference brightness samples can vary. In one embodiment, the upper left NxM luma prediction samples of the collocated luma block are used. N and M can be 1, 2, 4, 8, 16, 32, 64 and 128. In another embodiment, the reconstructed top boundary K reference samples of the current root block are used. In another embodiment, the reconstructed left boundary K reference samples of the current root block are used. K can be 1, 2, 4, 8, 16, 32, 64 and 128. In another embodiment, the reconstructed upper left reference sample of the current root block is used.

In another embodiment, chroma residual scaling cannot be applied when the chroma block is in the chroma root block. In another embodiment, chroma residual scaling can still be applied when the chroma block is in the chroma root block. The location of the reference brightness samples can vary. In one embodiment, the upper left NxM luma prediction samples of the collocated luma block are used. N and M can be 1, 2, 4, 8, 16, 32, 64 and 128. In another embodiment, the reconstructed top boundary K reference samples of the current root block are used. In another embodiment, K reference samples of the reconstructed left boundary of the current root block are used. K can be 1, 2, 4, 8, 16, 32, 64 and 128. In another embodiment, the reconstructed upper left reference sample of the current root block is used.

LMCS maps samples in the original domain to the reshaped domain for better data estimation. The mapping curve is approximated by a piecewise linear (PWL) model. In order to convert the sample values from the original domain to the reshaped domain, a look-up-table (LUT) is used. The entry number of the LUT is the same as the input sample dynamic range. For example, if using 10 data inputs, use a LUT of 1024 entries. If 14 data inputs are used, an 8192 entry LUT is used. In hardware implementation, the cost of this LUT is high. Therefore, piecewise linear models can be used. The input can be compared to each of multiple fragments to find out which fragment the input belongs to. Within each segment, the corresponding output value can be calculated based on the characteristics of that segment.

Various methods of LMCS are disclosed according to embodiments of the present invention.

Method 1 - PCM mode with LMCS

LMCS maps samples in the original domain to the reshaped domain for better data estimation. The mapping curve is approximated by a piecewise linear model. Use a lookup table (LUT) to transform sample values from the original domain to the reshaped domain. The LUT has the same number of entries as the input sample dynamic range. For example, if using 10 data inputs, use a LUT of 1024 entries. If 14 data inputs are used, a LUT of 8192 entries is used.

In one embodiment, when using Pulse code Modulation (PCM) encoding and decoding, LMCS is disabled, which can achieve lossless encoding and decoding. This is because the mapping process may introduce some bit rounding, or it may not be accurately mapped back to the original value after performing the forward and backward mappings, resulting in lossy encoding and decoding. According to one embodiment of the present invention, one or more high-level syntaxes of PCM codec are sent at SPS/PPS/APS/slice/tile group/tile/picture level, and are sent before the LMCS syntax. When determining that a tile/tilegroup/picture/slice/sequence uses the PCM codec, LMCS-related syntax elements (reshape tools or reshape models) may be skipped, inferred as unused, or constrained to not be used ( For example, encoder constraints prohibit LMCS from being used for PCM codecs).

In another embodiment, reshaping can still be applied if PCM codec mode is applied in tile/tile group/slice/picture/sequence level regions. However, the mapping table for forward reshaping and reverse reshaping should be identity mapping, for example, the input is equal to the output, or the mapping function is a line with a slope equal to 1.

In one example, a mapping table can be signaled, but the mapping table should be an identity mapping table. In another example, the mapping table is not sent. Use the default identity mapping table. The default mapping is a simple identical mapping where input equals output.

In another embodiment, if CU/PU/TU level PCM codec and/or transform quantization bypass mode is applied, the residual or transformed residual should be coded in the original domain to achieve PCM codec . For example, predictors (eg, inter mode predictor, intra mode predictor, intra block copy mode predictor, palette mode predictor) should also be located in the original domain. For intra prediction or any other prediction mode that uses adjacent reconstructed samples to generate predictors (eg, combined inter/intra prediction), the adjacent reconstructed samples are converted to the original domain before generating the predictor. In another example, if a predictor is generated in the reshaped domain (eg, an intra-mode predictor), the generated predictor is converted to the original domain. In this example, for the inter mode predictor, it will not go through the forward reshaper and become the predictor of the reshaped domain when using PCM mode. The residual data is encoded and decoded in the original domain. Syntax for specifying the domain of reconstructed CU samples. Therefore, when applying CU/PU/TU level PCM codec and/or transform quantization bypass mode, only adjacent reconstructed samples in the reconstructed domain need to be inversely mapped to the original domain if intra prediction is used for prediction.

When encoding and decoding the current intra-frame CU in lossy encoding and decoding, forward mapping is required if the adjacent reconstructed samples are in the original domain. After mapping the neighboring reconstructed samples to the reshaped domain, the reshaped neighboring reconstructed samples are used to generate intra prediction samples.

In another embodiment, if the current intra-frame CU is coded in lossy coding, adjacent reconstructed samples are regarded as reconstructed samples regardless of which domain they belong to.

In another embodiment, if CU/PU/TU level PCM codec and/or transform quantization bypass mode is applied, predictors can still be generated in the reconstruction domain, but the reconstructed samples are not inversely mapped (to the original domain) reconstruction. However, the reconstructed samples in the reshaping domain should be the values obtained by performing PCM on the original samples. For example, for intra prediction or any other prediction mode that uses adjacent reconstructed samples to generate predictors, there is no need to convert adjacent samples back to the original domain. Reshape domain neighbor samples can be used to generate predictors. For inter prediction, the predictor may or may not be transformed via forward mapping as in lossy codecs. In another embodiment, backward mapping can still be applied. However, the mapping table for backward mapping is the same mapping, such as a one-to-one mapping with slope equal to 1 or output equal to input.

In another embodiment, if CU/PU/TU level PCM codec and/or transform quantization bypass mode is applied, forward and backward mapping are disabled or the same mapping is used (for all prediction modes). In another embodiment, the residual/predictor/reconstructed samples can still be encoded and decoded in the reconstruction domain. However, there are encoder constraints or data stream consistency requirements, that is, when applying PCM mode, the reconstructed samples can be converted to the original domain, and the original domain reconstructed samples should be the same as the input samples.

In one embodiment, if CU/PU/TU level PCM codec and/or transform quantization bypass mode is applied, chroma residual scaling is not applied, or the scaling factor is set to 1, or the scaling factor is limited to a range Inside. For example, the scaling factor should be no greater than 1 or no less than 1. In another embodiment, when transform skip mode is applied, no chroma residual scaling is applied. In another embodiment, when transform skip mode is applied to the chroma component, no chroma residual scaling is applied.

In another embodiment, if CU/PU/TU level PCM codec and/or transform quantization bypass mode is applied, the residual or transformed residual should be coded in the reconstruction domain, where the mapping table The output is the same as the input. Therefore, the mapping process does not introduce lossy codecs.

In another embodiment, if CU/PU/TU level PCM mode and/or transform quantization bypass mode is used, adjacent reconstructed samples are converted to the original domain. Prediction samples for the current block can still be transformed by reshaping. However, the mapping table for forward reshaping and reverse reshaping should be a one-to-one mapping, such that the output is equal to the input, or the mapping function corresponds to a line with a slope equal to 1.

Method 2 - Derivation of inverse scaling factors

In one embodiment, the inverse scaling factor can be derived as follows:

InvScaleCoeff[i]＝

OrgCW*((1<<SCALE_FP_PREC)/lmcsCW[i]).

Thus, since the number of possible values for the denominator (eg, lmcsCW[i]) is limited, a lookup table can be used to implement division for values that are not powers of two (eg, lmcsCW[i]). The lookup table contains the values of (1<<SCALE_FP_PREC)/lmcsCW[i].

Method 3 - LMCS with default number of codewords

In one embodiment, the number of codewords for each data bin in the mapping domain (eg, lmcsCW[i]) can be derived using a default number of codewords instead of using OrgCW (which only depends on the data depth of the input data).

In the proposed method, the variable lmcsCW[i] is derived according to the following formula, where i=lmcs_min_bin_idx to LmcsMaxBinIdx:

lmcsCW[i]=default_CW+lmcsDeltaCW[i],

Among them, default_CW is exported at the decoder or sent by the re-encoder.

In one embodiment, if default_CW is derived on the decoder side, it can be derived based on lmcs_min_bin_idx and LmcsMaxBinIdx. If the sum of the number of data children smaller than lmcs_min_bin_idx and the number of data children larger than LmcsMaxBinIdx is greater than lmcs_min_bin_idx, default_CW can be adjusted to a value greater than OrgCW.

For example, if the sum of the number of data sub-numbers less than lmcs_min_bin_idx and the number of data sub-numbers greater than LmcsMaxBinIdx is equal to 2, then default_CW is derived as default_CW=OrgCW+A, where A is a positive integer (such as 1, 2, 3...).

If the sum of the number of data sub-numbers less than lmcs_min_bin_idx and the number of data sub-numbers greater than LmcsMaxBinIdx is equal to 0, then default_CW is equal to OrgCW.

In one embodiment, if default_CW is signaled, the two syntaxes default_delta_abs_CW and default_delta_sign_CW_flag are signaled before lmcs_delta_cw_prec_minus1.

The variable default_delta_abs_CW represents the absolute difference between default_CW and OrgCW, and the variable default_delta_sign_CW_flag represents whether the incremental value is positive or negative. The default_delta_sign_CW_flag is sent only when default_delta_abs_CW is greater than 0.

In one embodiment, if default_CW is signaled, the syntax default_delta_CW is signaled before lmcs_delta_cw_prec_minus1.

The variable default_delta_CW represents the difference between default_CW and OrgCW.

Method 4 - Reshape Curve Update

In one embodiment, the reshaping curve is updated every frame or every other frame.

Chroma scaling with VPDU constraints

The picture can be divided into several non-overlapping MxN blocks. These MxN non-overlapping blocks as processing data units are called VPDUs. M and N can be 64, or any predefined or signaled value, or a value related to the maximum transform block size.

In one embodiment, for the chroma component, chroma residual scaling uses reference luma reconstruction samples external to the current VPDU, such as a previously encoded VPDU.

In one embodiment, the reference brightness sample may be one or more regions. For example, the reference sample is the KxL block outside the current VPDU. K and L can be 2, 4, 8, 16 or 32. In detail, according to this embodiment, the size of the current VPDU is equal to min(CtbSizeY, 64), and the number of reference brightness samples of the top boundary and the left boundary are respectively equal to min(CtbSizeY, 64). The variable CtbSizeY specifies the luminance width and luminance height of the luminance codec tree block.

In another embodiment, the reference reconstructed luma samples are along the VPDU top boundary or left boundary or both, as shown in Figure 4. The number of reference luminance samples is a power of 2.

In another embodiment, the reference reconstructed luminance sample is simply a sample value. In one embodiment, the position may be the upper left position of the L-shaped boundary of the current VPDU, such as the TL position in Figure 5. In another embodiment, the position of the reference sample may be the position above the current VPDU, such as position A in Figure 5. In another embodiment, the position of the reference sample may be the left position of the current VPDU, such as the L position in Figure 5.

In another embodiment, chroma scaling is derived only once per VPDU, and the scaling factor is derived by the first CU in each VPDU. In detail, the size of the VPDU is equal to Min(CtbSizeY, 64), and for all blocks in the Min(CtbSizeY, 64) times Min(CtbSizeY, 64) area (ie, in the same VPDU), according to this embodiment, use The reference luminance sample is the same as the one from which the chroma scaling factor is derived. The variable CtbSizeY specifies the luminance width and luminance height of the luminance codec tree block.

In another embodiment, the reference codec unit (CU) for chroma scaling is derived from the VPDU corresponding to the current block (e.g., the chroma scaling is always Derived from the first CU in the VPDU). In detail, according to this embodiment, the size of the VPDU is equal to Min(CtbSizeY, 64), and the reference CU covers the upper left position of the current VPDU. The reference luminance samples include Min(CtbSizeY, 64) reconstructed luminance samples along the top boundary of the reference CU and Min(CtbSizeY, 64) luminance reconstructed samples along the left boundary of the reference CU. In another embodiment, if chroma scaling is not applied to the first CU in the current VPDU, the scaling factor is set to a default value. In one embodiment, the default value is equal to (1<<PREC), where PREC is the chroma-scaled prediction.

In another embodiment, the chroma scaling factors are shared at the picture/slice level. In another embodiment, the chroma scaling factors are shared among APS levels. In other words, for each signaled mapping curve, a chroma scaling factor is derived. In one example, the derivation of the chromatic scaling factor for each reshaping curve is accomplished by averaging the scaling factors in all intervals (segments). In another embodiment, the scaling factor is derived by selecting the majority scaling factor in all intervals (segments). In another embodiment, the scaling factor is derived by directly dividing the difference between the maximum luminance sample and the minimum luminance sample by the difference between the maximum luminance sample in the reshaping domain and the minimum luminance sample in the reshaping domain.

Luminance residual with reduced latency

Instead of mapping the luma prediction samples, the mapping can be applied only to the luma residuals. In other words, the predicted samples of the luminance component are in the original domain, and the residuals of the luminance component will be scaled by the scaling factor. The scaling factor is derived by referencing the brightness prediction samples at different locations or in different ways. The above methods suggested for chroma scaling can also be applied to luma residual scaling.

In another embodiment, the scaling factor is the average of two scaling factors for two consecutive intervals.

In one embodiment, the scaling factors used for luma residual scaling and chroma residual scaling are the same.

Signaling chroma scaling factor

Instead of implicitly deriving the chroma scaling factor at the decoder side, embodiments of the present invention signal the chroma scaling factor at the TB, TU, CU, CTU, VPDU, slice level, block level or APS level.

In one embodiment, one or more chroma scaling factors are signaled in an APS.

In one embodiment, if the chroma scaling factor is signaled at the TU level, and if the Cbfs (coding block flag) of both Cb and Cr is equal to 0, then the chroma scaling factor is not signaled.

In another embodiment, if the chroma scaling factor is signaled at the TU level, and if the root Cbf is equal to 0, then the chroma scaling factor is not signaled.

In one embodiment, if the chroma scaling factor is signaled at the TB level for the chroma Cb component, and if Cbf of Cb is equal to 0, then the chroma scaling factor is not signaled; for the chroma Cr component, if Cbf of Cr is equal to 0, no chroma scaling factor is sent.

In some embodiments, the video encoder must follow the foregoing syntax design in order to generate a legal data stream, and the video decoder can correctly decode the data stream only if the parsing process complies with the foregoing syntax design. When syntax is skipped in the data stream, encoders and decoders should set the syntax value to an inferred value to ensure that the encoding and decoding results match.

6 illustrates a flowchart of an exemplary decoding system for deriving one or more chroma residual scaling factors based on neighboring predicted or reconstructed luma samples of collocated luma blocks in accordance with an embodiment of the present invention. The steps shown in the flowchart, as well as other subsequent flowcharts in this disclosure, may be implemented as executable on one or more processors (e.g., one or more CPUs) on the encoder side and/or the decoder side. Program code. The steps shown in the flowcharts may also be implemented on a hardware basis, such as one or more electronic devices or processors arranged to perform the steps in the flowcharts. According to the method, in step 610 the current chroma residual block is received. In step 620, one or more chroma residual scaling factors are derived based on neighboring predicted or reconstructed luma samples of the collocated luma block associated with the current chroma residual block, where the neighboring luma blocks of the collocated luma block are The predicted or reconstructed luma samples correspond to samples among the M samples that collocate the top boundary of the luma block and the N samples that collocate the left boundary of the luma block, where M and N are positive integers. Then in step 630, chroma scaling is applied to the chroma residual samples of the current chroma residual block according to the one or more chroma residual scaling factors.

7 illustrates a flow diagram of another exemplary decoding system for deriving one or more reconstructed luma samples based on one or more reconstructed luma samples external to a collocated luma processing data unit in accordance with an embodiment of the present invention. Chroma residual scaling factor. According to the method, chroma residual data associated with the current chroma processing data unit in the picture is received in step 710, wherein the picture is divided into a plurality of non-overlapping processing data units, and each processing data unit Includes one luminance processing data unit and one or more chrominance processing data units. In step 720, one or more chroma residual scaling factors are derived based on one or more reconstructed luma samples external to the collocated luma processing data unit associated with the current chroma processing data unit. In step 730, chroma scaling is applied to the chroma residual samples of the current chroma processing data unit according to the one or more chroma residual scaling factors.

Figure 8 shows a flow diagram of an exemplary codec system in which one or more chroma residuals are signaled in the APS (Adaptive Parameter Set) level of the video data stream on the encoder side according to an embodiment of the invention. Differential scaling factors, or one or more chroma residual scaling factors resolved from the APS level of the video data stream on the decoder side. According to the method, in step 810 the current chroma residual block is received. In step 820, one or more chroma residual scaling factors are signaled or parsed in the APS level of the video data stream. Chroma residual scaling factor. In step 830, chroma scaling is applied to the chroma residual samples of the current chroma residual block.

The flowchart shown is intended to illustrate an example of video encoding and decoding according to the present invention. Those skilled in the art may modify each step, rearrange steps, split steps or combine steps to practice the invention without departing from the spirit of the invention. In this disclosure, specific syntax and semantics have been used to illustrate examples for implementing embodiments of the invention. Those skilled in the art may practice the invention by replacing syntax and semantics with equivalent syntax and semantics without departing from the spirit of the invention.

The above description is presented to enable one skilled in the art to practice the invention in the context of a particular application and its requirements. Various modifications to the described embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Thus, the present invention is not intended to be limited to the specific embodiments shown and described but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details are set forth in order to provide a thorough understanding of the invention. However, those skilled in the art will understand that the invention may be practiced.

The embodiments of the present invention as described above can be implemented in various hardware, software codecs, or a combination of both. For example, an embodiment of the invention may be one or more electronic circuits integrated into a video compression chip or program code integrated into video compression software to perform the processes described herein. Embodiments of the invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The present invention may also be directed to any number of functions performed by a computer processor, digital signal processor, microprocessor or field programmable gate array (FPGA). These processors may be configured to perform specific tasks in accordance with the invention by executing machine-readable software code or firmware code that defines the specific methods embodied by the invention. Software code or firmware code can be developed in different programming languages and in different formats or styles. Software code can also be compiled for different target platforms. However, different codec formats, styles and languages of software code, and other means of configuring the code to perform tasks in accordance with the invention will not depart from the spirit and scope of the invention.

The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. The described examples should be considered in all respects as illustrative only and not restrictive. The scope of the invention is, therefore, indicated by the appended claims rather than the foregoing description. All changes that fall within the equivalent meaning and scope of the claims shall be included within their scope.

Claims (11)

1. A video decoding method, the method includes: Receive chroma residual data related to the current chroma processing data unit in the picture, wherein the picture is divided into multiple non-overlapping processing data units, each processing data unit includes a luminance processing data unit and one or more colorimetric processing data unit; deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples external to the collocated luma processing data unit associated with the current chroma processing data unit; and applying chroma scaling to the chroma residual samples of the current chroma processing data unit according to the derived one or more chroma residual scaling factors, The luminance size of one or more non-overlapping processing data units is equal to Min(CtbSizeY, 64) times Min(CtbSizeY, 64), and wherein CtbSizeY specifies the luminance width and luminance height of the luma codec tree block. 2. The method of claim 1, wherein the one or more reconstructed luma samples outside the collocated luma processing data unit correspond to one of one or more previously coded luma processing data units. or multiple reconstructed brightness samples. 3. The method of claim 2, wherein the one or more reconstructed luma samples of the one or more previously coded luma processing data units correspond to a top boundary along the collocated luma processing data unit one or more reconstructed luma samples of , one or more reconstructed luma samples along the left boundary of the collocated luma processing data unit, or both. 4. A video decoding device comprising one or more electronic circuits or processors arranged to: Receive chroma residual data related to the current chroma processing data unit in the picture, wherein the picture is divided into multiple non-overlapping processing data units, each processing data unit includes a luminance processing data unit and one or more colorimetric processing data unit; deriving one or more chroma residual scaling factors based on one or more reconstructed luma samples external to the collocated luma processing data unit associated with the current chroma processing data unit; and applying chroma scaling to the chroma residual samples of the current chroma processing data unit according to the derived one or more chroma residual scaling factors, The luminance size of one or more non-overlapping processing data units is equal to Min(CtbSizeY, 64) times Min(CtbSizeY, 64), and wherein CtbSizeY specifies the luminance width and luminance height of the luma codec tree block. 5. A video decoding method, the method comprising: Receive chroma residual data related to the current chroma processing data unit in the picture, wherein the picture is divided into a plurality of non-overlapping processing data units, each processing data unit includes a luminance processing data unit and one or more Colorimetric processing data unit; Derive one or more chroma residual scaling factors from a first codec unit that covers collocated luma associated with the current chroma processed data unit based on the one or more reconstructed luma samples Process the upper left position of the data unit; and applying chroma scaling to the chroma residual samples of the chroma residual data associated with the current chroma processing data unit based on the derived one or more chroma residual scaling factors, The luminance size of the one or more non-overlapping processing data units is equal to Min(CtbSizeY, 64) times Min(CtbSizeY, 64), and wherein CtbSizeY specifies the luminance width and luminance height of the luma codec tree block. 6. The method of claim 5, wherein the one or more reconstructed luma samples outside the first codec unit covering the collocated luma processing data unit correspond to one or more previous codecs. One or more reconstructed luma samples of the decoded luma processing data unit. 7. The method of claim 5, wherein the one or more reconstructed luma samples of the one or more previously coded luma processing data units correspond to images along the first codec unit (CU). ), one or more reconstructed luma samples along the left boundary of the first codec unit (CU), or both, where the first codec unit covers The concatenated brightness processing data unit. 8. The method of claim 7, wherein the number of reference reconstructed luma samples along the top boundary of the first codec unit covering the collocated luma processing data unit is equal to that of the luma processing data unit. width or height. 9. The method of claim 7, wherein the number of reference reconstructed luma samples along the left boundary of the first codec unit covering the collocated luma processing data unit is equal to the luma processing data unit. width or height. 10. A video encoding and decoding method, the method includes: Receive the current chroma residual block; signaling one or more chroma residual scaling factors in the adaptive parameter set level of the video data stream, or parsing the one or more chroma residual scaling factors in the adaptive parameter set level of the video data stream; as well as Applies chroma scaling to the chroma residual samples of this current chroma residual block. 11. A video encoding and decoding device, the device comprising one or more electronic circuits or processors, the electronic circuits or processors being arranged to: Receive the current chroma residual block; signaling one or more chroma residual scaling factors in the adaptive parameter set level of the video data stream, or parsing the one or more chroma residual scaling factors in the adaptive parameter set level of the video data stream; as well as Applies chroma scaling to the chroma residual samples of this current chroma residual block.