CN114630131A - Jump-out pixel codec method in index map codec - Google Patents

Abstract

A video codec method is disclosed that reduces implementation cost by reusing a transform coefficient buffer palette for palette codec. If the current prediction mode is the intra prediction mode or the inter prediction mode, information related to transform coefficients of the prediction residual of the current block generated by the intra prediction or the inter prediction is stored in the transform coefficient buffer. If the current prediction mode is the palette codec mode, information related to palette data associated with the current block is stored in the transform coefficient buffer. If the current block is encoded or decoded in the intra prediction mode or the inter prediction mode, the current block is encoded or decoded based on the information related to the transform coefficients, or if the current prediction mode is the palette codec mode, based on Information about the palette data stored in the transform coefficient buffer to encode or decode the current block.

Description

Jump-out pixel encoding and decoding method in index map encoding and decoding

【cross reference】

This application requires a U.S. Provisional Application with a filing date of November 12, 2014, a U.S. Provisional Application No. 62/078,595, a U.S. Provisional Application with a filing date of December 4, 2014, and a U.S. Provisional Application No. 62/087,454 , the filing date is February 24, 2015, the U.S. Provisional Application with U.S. Provisional Application No. 62/119,950, the filing date is April 10, 2015, and the U.S. Provisional Application and Application with U.S. Provisional Application No. 62/145,578 Priority of U.S. Provisional Application No. 62/162,313, dated May 15, 2015, and U.S. Provisional Application No. 62/170,828, filed June 4, 2015, The content of the above provisional application is incorporated into this application.

【Technical field】

The present invention relates to palette coding of video data. In particular, the present invention relates to reusing transform coefficient buffers or Group Escape Values, palette predictor initialization, palette predictor entry semantics, or Palette entry semantics to conserve system memory or various techniques to increase system throughput.

【Background technique】

High Efficiency Video Coding (HEVC for short) is a new codec standard that has been developed in recent years. In the High Efficiency Video Codec (HEVC) system, the fixed-size macro blocks of H.264/AVC are replaced by flexible blocks called coding units (CU for short). Pixels in a CU share the same codec parameters to improve codec efficiency. A CU may start with a Largest CU (LCU), which is also referred to as a coded tree unit (CTU) in HEVC. In addition to the concept of a codec unit, the concept of a prediction unit (PU for short) is also introduced in HEVC. Once the splitting of the CU-level tree is complete, each leaf CU is further split into one or more prediction units (PUs) according to prediction type and PU partitioning. Several codec tools have been developed for encoding and decoding screen content. These tools relevant to the present invention are briefly reviewed below.

palette codec

During the development of HEVC screen content coding (SCC for short), several proposals have been published to address palette-based codecs. For example, JCTVC-N0247 (Guo et al., "RCE3: Results of Test 3.1 on Palette Mode for Screen Content Coding", Joint Collaborative Group on ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Video Codecs (Joint Collaborative Team on VideoCoding) (JCT-VC), 14th Session: Vienna, AT, 2013.7.25-2013.8.2, Document Number: JCTVC-N0247) and JCTV C-O0218 (Guo et al., "Evaluation of Palette Mode Coding on HM-12.0+RExt-4.1”, ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Joint Collaborative Team on Video Coding (JCT-VC), Palette prediction and sharing techniques are disclosed in the 15th session: Geneva, CH, 2013.10.23–2013.11.1, Document No.: JCTVC-O0218). In JCTVC-N0247 and JCTVC-O0218, a palette for each color component is constructed and transmitted. The palette may be predicted (or shared) from its left neighbor CU to reduce bit rate. All pixels within a given block are then encoded and decoded using their palette indices. An example of the encoding process according to JCTVC-N0247 is shown below.

Palette transfer: The color index table (also called palette table) size is transferred first, followed by the palette elements (ie, color values).

Transmission of pixel palette index value (index points to a color in the palette): The index value of a pixel in the CU is encoded in raster scan order. For each location, a flag is first sent to indicate whether "run mode" or "copy above run mode" is being used.

a. "Run Mode": In "Run Mode", the palette index is emitted first, followed by "palette_run" (eg, M). Since there is the same palette index as the already marked palette index, no further information needs to be sent for the current position and the following M positions. The palette index (e.g., i) is shared by all three color components, which means the reconstructed pixel value is (Y, U, V) = (palette _Y [i], palette _U [i], palette _V [ i]) (assuming the color space is YUV).

b. "Copy Above Run Mode": In "Copy Above Run Mode", the value "copy_run" (eg, N) is sent to indicate that for the next N positions (including the current position), the palette index is equal to that in the top row Palette index at the same position.

Transfer of residuals: The palette indices sent in stage 2 are converted back to pixel values and used as predictions. Residual information is transmitted using the HEVC residual codec and added to the reconstruction prediction.

In JCTVC-N0247, a palette for each component is constructed and transmitted. The palette can be predicted (or shared) from its left neighbor CU to reduce bit rate. In JCTVC-O0218, each element in the palette is a triplet, representing a specific combination of three color components. Predictive codecs for across CU palettes are removed.

In JCTVC-O0182 (Guo et al., "AHG8: Major-color-based screen content coding", ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Joint Collaborative Team on Video Codecs Video Coding) (JCT-VC), the 15th meeting: Geneva, CH, 2013.10.23-2013.11.1, document number: JCTVC-O0182), another palette encoding and decoding method is disclosed. Instead of predicting the entire palette table from the left CU, individual palette color entries in the palette may be predicted from the exact corresponding palette color entry in the upper or left CU.

For the transmission of pixel palette index values, a predictive codec method is applied to the index according to JCTVC-O0182. Index rows can be predicted by different modes. In particular, the index row uses three row modes, namely horizontal mode, vertical mode and normal mode. In horizontal mode, all indices in the same row have the same value. If the value is the same as the first pixel of the row of pixels above, only line mode signaling bits are sent. Otherwise, the index value is also sent. In vertical mode, the current index row is the same as the upper index row. Therefore, only row mode signaling bits are transmitted. In normal mode, the exponents in a row are predicted individually. For each index position, the left or above neighbor is used as the predicted value and the predicted symbol is sent to the decoder.

Furthermore, according to JCTVC-00182, pixels are classified into major color pixels (with palette indices pointing to palette colors) and escape pixels. For primary color pixels, the decoder reconstructs the pixel value from the primary color index (ie, the palette index in JCTVC-N0247 and JCTVC-O0182) and the palette table. For bounced pixels, the encoder will send further pixel values.

Palette table signaling

In the reference software of Screen Content Coding (SCC for short) standard, SCM-2.0 (JCTVC-R1014: Joshi et al., Screen content coding test model 2 (SCM 2), ITU-TSG 16WP 3 and ISO/ IEC JTC 1/SC 29/WG 11 Joint Collaborative Team on Video Coding (JCT-VC), 18th meeting: Sapporo, JP, 2014.6.30-7.9, document number: JCTVC-R1014) The palette table of the last codec palette CU is used as the predicted value of the current palette table codec. In the palette table codec, the palette_share_flag is first marked. If palette_share_flag is 1, all palette colors in the last codec palette table will be reused for the current CU. In this case, the current palette size is equal to the palette size of the last codec palette CU. Otherwise (ie palette_share_flag is 0), the current palette table is signaled by indicating which palette color in the last codec palette table can be reused, or by sending a new palette color. The size of the current palette is set to the size of the predicted palette (ie, numPredPreviousPalette) plus the size of the passed palette (ie, num_signaled_palette_entries). The predicted palette is the palette obtained from the CU encoded and decoded from the previously reconstructed palette. When encoding the current CU into palette mode, the palette colors predicted without using the predicted palette are sent directly through the bitstream. For example, if the current CU is encoded in palette mode with palette size equal to 6. If three of the six dominant colors are predicted from palette predictions, three are transmitted directly through the bitstream. The following pseudocode illustrates an example of three palette colors transferred using the example syntax above.

Since the palette size is 6 in this example, a palette index from 0 to 5 is used to indicate each palette codec pixel, and each palette can be reconstructed into a palette color table main color in .

In SCM-2.0, if wavefront parallel processing (WPP for short) is not applied, the final coding is performed at the beginning of each slice or at the beginning of each tile. The decoded palette table is initialized (ie, reset). If WPP is applied, the palette table of the last codec is initialized (reset) not only at the beginning of each slice or at the beginning of each tile, but also at the beginning of each CTU row (i.e., reset).

Palette index map scan order

In SCM-3.0 (JCTVC-R1014: Joshi et al., Screen content coding test model 3 (SCM3), ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Video Codec Joint Collaborative Team on Video Coding) (JCT-VC), the 19th meeting: Strasbourg, FR, 2014.10.17-24, document number: JCTVC-S1014) palette mode codec, traverse scan (traverse scan) is used as shown in Figure 1 The index map codec shown. Figure 1 shows an example of a traversal scan of 8x8 blocks. In a traverse scan, even rows are scanned from left to right, and odd rows are scanned from right to left. Traversal scan works for all block sizes in palette mode.

Palette Index Map Codec in SCM-4.0

In SCM-4.0 (JCTVC-T1014: Joshi et al., Screen content coding test model 4 (SCM4), ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Video Codec Joint Collaborative Team on Video Coding) (JCT-VC), the 20th meeting: Geneva, CH, 2015.2.10-18, document number: JCTVC-T1014) In palette mode codec, the palette index is in the codec data of the corresponding block (ie, before palette_run_mode and palette_run codec) are aggregated and signaled. On the other hand, at the end of the codec data for the corresponding block, the codec escape pixel (escape pixel). The syntax elements palette_run_mode and palette_run are encoded and decoded between palette index and jump out pixels. Figure 2 shows an exemplary flow diagram of index mapping syntax signaling according to SCM 4.0. Indicates the number of indexes (210), last runtype (230) and clustered indexes (220). After the index information is marked, a pair of run types (240) and run times (250) are repeatedly marked. Finally, if necessary, a set of bounce values is indicated (260).

Palette prediction value initialization

In SCM-4.0, the global palette prediction value set is marked with PPS (picture parameter set, picture parameter set). Use values obtained from PPS instead of resetting all palette prediction states (including PredictorPaletteSize, PreviousPaletteSize, and PredictorPaletteEntries) to 0.

palette syntax

For the operation of the index in the index map, several elements that need to be marked include:

1) Type of run: It is a copy above run or a copy index run.

2) Palette Index: Used to indicate in a copy index run which index is used for this run.

3) Run Length: It represents the length of this run of replication above and replication index types.

4) Jumping out pixels: If there are N (N>=1) jumping out pixels in the operation, N pixel values need to be marked for these N jumping out pixels.

In JCTVC-T0064 (JCTVC-T0064: Joshi et al., Screen content coding test model 4 (SCM 4), ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Video Codec Joint Collaborative Group Team on Video Coding) (JCT-VC), Session 20: Geneva, CH, 2015.2.10-18, Document No.: JCTVC-T1014, all palette indices are clustered together. The palette is first marked The number of indices, followed by the palette indices.

According to the existing HEVC specification, when the palette indices for each color component are clustered together, most other palette codec related data for different color components is interleaved in the bitstream. In addition, a separate storage space is used to store Inter/Intra coded blocks and to store palette codec blocks. It is desirable to develop techniques to increase system throughput and/or reduce system implementation costs.

[Content of the invention]

A video codec method is disclosed that reduces implementation cost by reusing a transform coefficient buffer palette for palette codec. If the current prediction mode is the intra prediction mode or the inter prediction mode, information related to transform coefficients of the prediction residual of the current block generated by the intra prediction or the inter prediction is stored in the transform coefficient buffer. If the current prediction mode is the palette codec mode, information related to palette data associated with the current block is stored in the transform coefficient buffer. If the current block is encoded or decoded in the intra prediction mode or the inter prediction mode, the current block is encoded or decoded based on the information related to the transform coefficients, or if the current prediction mode is the palette codec mode, based on Information related to palette data stored in the transform coefficient buffer to encode or decode the current block.

If the current prediction mode is the palette codec mode, the palette data may correspond to the palette run type, palette index, palette run, jump value, jump flag, palette associated with the current block board table or any combination thereof. The information related to palette data may correspond to palette data, parsed palette data, or reconstructed palette data. For example, the parsed palette indices of the samples of the current block are reconstructed at the parsing stage, and the reconstructed palette indices and reconstructed skip values are stored in the transform coefficient buffer at the decoder side. In another example, the parsed palette indices of the samples of the current block are reconstructed, during the parsing phase, the reconstructed palette indices are further reconstructed into reconstructed pixel values using a palette table, and the reconstructed The pixel values and reconstructed skip values are stored in the transform coefficient buffer on the decoder side. Additionally, a storage area can be designated during the analysis phase to store the palette table, and the storage area can be released from use by the palette table during the reconstruction phase. The jump-out flag can also be stored in the transform coefficient buffer on the decoder side. In another example, the jump-out flag is stored in a portion of the transform coefficient buffer (eg, the most significant bit (MSB) portion of the transform coefficient buffer), and the reconstructed pixel value or jump-out value is stored in a in another part.

In another embodiment, if the current prediction mode is a palette codec mode, all the jump-out values for the same color component that are clustered together are parsed from the video bitstream at the decoder side, or for all the jump-out values for the same color component Gather together on the encoder side. The information containing the skip value is then used for encoding or decoding of the current block. Aggregate breakout values for the same color component may be indicated at the end of the codec palette data for the current block. Aggregated bounce values for different color components may be emitted separately for the current block. Aggregated dropout values for the same color component of the current block may be stored in a transform coefficient buffer. Aggregated skip values for different color components may share transform coefficient buffers by storing aggregated skip values for one color component in the transform coefficient buffer at a time.

In another embodiment, all initial palette predictions for the same color components grouped together in a sequence parameter set (SPS), picture parameter set (PPS), or slice header are parsed from the video bitstream at the decoder side value, or aggregate all initial palette predictions for the same color component together on the encoder side. At least one palette codec block within the corresponding sequence, picture or slice is encoded or decoded using the initial palette predictor.

In another embodiment, on the decoder side, all palette predictor entries or palette entries for the current block of the same color component grouped together are parsed from the video bitstream, or on the encoder side, will be used for All palette predictor entries or palette entries for the same color component are grouped together. The current block is then encoded or decoded using the palette predictor consisting of all palette predictor entries or the palette table consisting of all palette entries.

【Description of drawings】

Figure 1 shows an example of a traversal scan of 8x8 blocks.

Figure 2 illustrates exemplary palette index mapping syntax signaling according to Screen Content Codec Test Module Version 4 (SCM-4.0).

Figure 3A shows an example index map flip prior to index encoding and decoding.

FIG. 3B shows an example of a flipped index map corresponding to the index map in FIG. 3A.

Figure 4 shows an example of flipping the index map prior to index map encoding and decoding, where it is inefficient to predict the pixel in the last row in the physical location using the pixel constructed from the upper neighbor.

Figures 5A-B illustrate an example of prediction from an upper CU with flipped index mapping, where an index in a copy-over operating mode is always predicted from its physical upper position regardless of whether the index mapping is in accordance with an embodiment of the present invention is flipped. In Figure 5A, the line-filled blocks represent the flipped index map, while the transparent blocks in Figure 5B represent the original index map.

6A-B illustrate another example from an upper CU with a flipped index map, where a copy of the first row is predicted from its physically closest location sample, running codec pixels above, in accordance with an embodiment of the invention. In Figure 6A, the line-filled blocks represent the inverted index map, while the transparent blocks in Figure 6B represent the original index map.

7A-B illustrate another example of prediction from an upper CU with a flipped index map, where a copy of the last row of codec pixels above is predicted from the physical pixel locations of the upper neighboring CU, according to an embodiment of the invention. In Figure 7A, the line-filled blocks represent the inverted index map, while the transparent blocks in Figure 7B represent the original index map.

8A shows an example of an extended copy-over mode of operation, where two rows of pixels are copied from the previous row above the CU boundary (ie, L=2).

8B shows an example of cross-CU prediction indicating the prediction of the first M (ie, M=11) samples from reconstructed pixels by signaling syntax element pixel_num(M).

FIG. 9A shows an example of predicting the pixel values of the first two rows of samples from the reconstructed pixel values of the last row of the upper CU.

9B shows an example of predicting the pixel values of the first two columns of samples from the reconstructed pixel values of the rightmost column of the left CU.

10A-C illustrate three different scan modes for cross-CU prediction in accordance with embodiments of the present invention.

11A-C illustrate three different scan modes for cross-CU prediction in accordance with another embodiment of the present invention.

12A-B illustrate two different scan modes for inverse scan for cross-CU prediction in accordance with an embodiment of the present invention.

Figure 13 shows an example of extending row-based replicated pixels from adjacent CUs to 8x8 CUs coded in inter mode.

FIG. 14 shows an example of changing the positions of adjacent reference pixels, in which the upper right reference pixel is copied from the upper right CU, according to an embodiment of the present invention.

FIG. 15 shows another example of changing the position of adjacent reference pixels, wherein the upper right reference pixel is copied from the rightmost pixel of the third row, according to an embodiment of the present invention.

Figure 16 shows an example of decoded pop-out color according to an embodiment with N pop-out pixels in different positions within the current codec block (N=5), where the pixel value where each pop-out pixel occurs is still written to the bitstream , and scan using horizontal traversal.

Figure 17 shows an example of decoding a jump out color according to another embodiment, where there are N jump out pixels in different positions in the current codec block (N=5), where only non-repeating colors are decoded.

Figure 18 shows an example of using a special index denoted N adjacent construction pixels (NCPs) across CU boundaries.

FIG. 19 shows an example of using the special index of the adjacent construction pixel (NCP) in the case where the maximum index value is 0.

FIG. 20 shows an example of using a special index for a neighboring construction pixel (NCP) with a maximum index value of 1.

21 shows an exemplary flow diagram of signaling to support index prediction across CUs, where a new flag all_pixel_from_NCP_flag is added and used for index prediction across CUs according to the syntax of SCM3.0.

Figure 22 shows an exemplary flowchart for signaling to support index prediction across CUs, where a new flag all_pixel_from_NCP_flag is added, when all_pixel_from_NCP_flag is off, without cross-CU index prediction according to the syntax of SCM3.0 used.

FIG. 23 shows another exemplary flowchart similar to that of FIG. 22 . However, when the maximum index value is not 0, the syntax according to SCM3.0 is used for index prediction on the CU.

24 shows an exemplary flow diagram of signaling for supporting index prediction across CUs according to an embodiment of the present invention.

25 shows an exemplary flowchart of signaling for supporting index prediction across CUs according to another embodiment of the present invention.

26 shows an exemplary flowchart of signaling for supporting index prediction across CUs according to another embodiment of the present invention.

27 shows an exemplary flowchart of signaling for supporting index prediction across CUs according to another embodiment of the present invention.

28A shows an exemplary flowchart of signaling for supporting index prediction across CUs according to another embodiment of the present invention.

28B shows an exemplary flowchart of signaling for supporting index prediction across CUs according to another embodiment of the present invention.

Figure 29 shows an example of source pixels used for intra block copy prediction and compensation, where the dot padding regions correspond to unfiltered pixels in the current CTU (Codec Tree Unit) and the left CTU.

Figure 30 shows another example of source pixels for intra block copy prediction and compensation, where the dot padding area corresponds to the current CTU (Codec Tree Unit), the bottom four rows of the left and upper CTUs, and the bottom four rows of the left and upper CTUs Unfiltered pixels in the bottom four rows.

Figure 31 shows another example of source pixels for intra block copy prediction and compensation, where the dot padding area corresponds to the current CTU (codec tree unit), the N left CTUs, the bottom four rows of the upper CTU, and the N Unfiltered pixels in the bottom four rows of the top left CTU.

Figure 32 shows another example of source pixels for intra block copy prediction and compensation, where the dot padding area corresponds to the current CTU (Codec Tree Unit), the bottom four rows of the upper CTU, and the bottom four rows of the left CTU Unfiltered pixels in a row.

Figure 33 shows another example of source pixels for intra block copy prediction and compensation, where the dot padding area corresponds to the current CTU (codec tree unit), the N left CTUs, and the bottom four rows of the upper CTU, Unfiltered pixels in the bottom four rows of the N upper left CTUs and the right four columns of the N+1 th left CTU.

34 shows an exemplary flow diagram of a system for palette codec block sharing transform coefficient buffers incorporating embodiments of the present invention.

【Detailed ways】

The following description is of the best contemplated mode for carrying out the invention. This description is intended to illustrate the general principles of the invention and should not be considered limiting. The scope of the invention is best determined by the appended claims.

Reuse HEVC transform coefficient buffers for palette related information

In HEVC, in addition to palette codec, Inter prediction and Intra prediction are available codec modes. When using inter or intra prediction, transform coding is usually applied to the prediction residuals obtained by inter/intra prediction. The transform coefficients are then quantized and entropy coded for inclusion in the coded bitstream. On the decoder side, the reverse operation is applied to the received bitstream. In other words, entropy decoding is applied to the bitstream to recover the codec symbols corresponding to the quantized transform coefficients. The quantized transform coefficients are then dequantized and inverse transformed to reconstruct the inter/intra prediction residuals. Transform coefficient buffers are often used on both the encoder side and the decoder side to store transform coefficients as needed between entropy encoding and decoding and quantization/transform operations.

For color video data, multiple transform coefficient buffers may be required. However, it is also possible to configure the system to process one color component at a time, such that only one transform coefficient buffer is required. When a block is palette encoded, the block is not transformed, and no transform coefficient buffer is used. In order to save the system implementation cost, the embodiment of the present invention reuses the transform coefficient buffer to store the palette codec related data. Therefore, for inter/intra codec blocks, in the coefficient parsing stage, the decoded coefficients of the TU are stored in the coefficient buffer. However, for the SCC (Screen Content Codec) codec block, the palette codec block does not require residual codec. Therefore, according to an embodiment of the present invention, the transform coefficient buffer is used to store palette codec related information, which may include palette run type, palette index, palette run, jump value, jump flag, Swatches or any combination thereof.

For example, the parsing palette index of the samples of the reconstructed block during the parsing phase. Transform coefficient buffers are used to store reconstructed palette indices and skip values.

In another example, the parsed palette indices of the samples of the block are reconstructed at the parsing stage. The parsing index is used to reconstruct the pixel value of the corresponding color component using the palette table lookup during the parsing phase. Therefore, the coefficient buffer is used to store reconstructed pixel values and jump values, or for storing reconstructed pixel values, jump values and jump flags. In this case, the palette table only needs to be stored during the parsing phase. There is no need to store and maintain palette tables during the refactoring phase. The data depth required for the transform coefficients may be higher than the data depth of the palette codec-related data. For example, transform coefficients may require a 16-bit deep buffer. However, for palette encoding and decoding, if the maximum palette index is 63, the storage of the palette index only needs to be 6 bits deep. Furthermore, the maximum bit length of the jump value is equal to the bit depth which is usually 8 or 10 bits. Therefore, a portion of the 16 bits (ie, 8 or 6 bits) are free and can be used to store information for the jump flag. For example, the six MSBs (most significant bits) of the coefficient buffer can be used to store the pop-out flag. The remaining bits are used to store the reconstructed pixel value or jump out value. In the reconstruction stage, the MSB can be used to directly reconstruct the pop-out value or reconstruct the pixel value.

In yet another example, the parsed palette indices of the samples are reconstructed in the parsing stage, and palette table lookups are also used in the parsing stage to reconstruct the reconstructed values of the different color components. The pixel values of the bounced samples are also reconstructed from the bounced values in the parsing stage. Therefore, the coefficient buffer is used to store reconstructed pixel values. In this case, the palette table only needs to be stored during the parsing phase. The refactoring phase does not need to store and maintain the palette table.

Aggregate bounce values of the same color component

In JCTVC-T0064 (JCTVC-T0064: Joshi et al., Screen content coding test model 4 (SCM 4), ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Video Codec Joint Collaborative Group Team on Video Coding) (JCT-VC), 20th Session: Geneva, CH, 2015.2.10-18, Document Number: JCTVC-T1014, the bounce values of the three color components are clustered together. In other words, For each sample, the jump values of the three components are sequentially coded and decoded. The syntax of the jump value codec according to JCTVC-T1014 is shown in Table 1.

Table 1.

Since there is no residual codec for palette mode, coefficient buffers can be reused to store palette index mapping information. In HEVC coefficient parsing, only the TUs of one color component are parsed at a time. The parser can only have one coefficient buffer for a color component, and can only access one coefficient buffer for a color component. However, in palette encoding and decoding, a palette index represents the pixel values of multiple color components. Palette mode can decode multiple color component values at once. The palette index can be stored in a coefficient buffer for a color component. However, in the current SCM-4.0 parsing order, a jump-out value may require three coefficient buffers for three color components.

Therefore, according to another embodiment, the parsing order of the bounce values is changed to overcome the problem involving the need for three coefficient buffers to store the three color components. Bounce values for the same color component are grouped together and marked at the end of the block's palette syntax codec. The bounce values for different color components are signaled separately. An exemplary syntax of the palette codec in conjunction with this embodiment is shown in Table 2.

Table 2.

In Table 2, the text in the line-filled area indicates deletion. The do-loop statement for the different color components (i.e. "for(cIdx=0; cIdx<numComps; cIdx++)") moves up from the original position indicated by note (2-1) to the one indicated by note (2-2) new location. Therefore, it can resolve all bounce values for one color component of the CU. The palette index and dropout value for the first color component can be written out (ie, cIdx equals 0), and the parser can then reuse the same buffer to parse the information for the second color component. With the syntax changes in Table 2, one coefficient buffer to store palette codec-related data for one color component is sufficient for palette mode parsing. The implementation complexity and cost of the SCC parser do not increase.

Aggregate initialization of palette predictions for the same color components

In SCM 4.0, a set of PPS (Picture Parameter Set) syntax elements are indicated to specify palette predictor initializers. The existing syntax of the PPS extension is shown in Table 3.

table 3.

The initialization process of the palette prediction values according to the existing SCM 4.0 is as follows:

The output of this process is the initialized palette predictor variables PredictorPaletteSize and PredictorPaletteEntries.

PredictorPaletteSize is exported as follows:

– If palette_predictor_initializer_present_flag is equal to 1, PredictorPaletteSize is set equal to num_palette_predictor_initializer_minus1 plus 1.

- Otherwise (palette_predictor_initializer_present_flag is equal to 0), PredictorPaletteSize is set to 0.

The PredictorPaletteEntries array (array) is exported as follows:

– If palette_predictor_initializer_present_flag is equal to 1,

for(i=0; i<PredictorPaletteSize; i++)

for(comp=0;comp<3;comp++)

PredictorPaletteEntries[i][comp]=palette_predictor_initializers[i][comp]

- Otherwise (palette_predictor_initializer_present_flag is equal to 0), PredictorPaletteEntries is set to 0.

In one embodiment, the predictor initialization values may be combined together for the Y component, the Cb component, and the Cr component, respectively. The syntax changes are shown in Table 4. Compared to Table 3, the two do-loop statements are swapped as indicated in Notes (4-1) and (4-2) to combine the initialization of the palette predictions of the same color components.

Table 4.

An exemplary palette prediction value initialization process corresponding to the above-described embodiment is shown below.

The output of this process is the initialized palette predictor variables PredictorPaletteSize and PredictorPaletteEntries.

PredictorPaletteSize is exported as follows:

– If palette_predictor_initializer_present_flag is equal to 1, PredictorPaletteSize is set equal to num_palette_predictor_initializer_minus1 plus 1.

- Otherwise (palette_predictor_initializer_present_flag is equal to 0), PredictorPaletteSize is set equal to 0.

The PredictorPaletteEntries array is exported as follows:

– If palette_predictor_initializer_present_flag is equal to 1,

for(comp=0;comp<3;comp++)

for(i=0; i<PredictorPaletteSize; i++)

PredictorPaletteEntries[i][comp]=palette_predictor_initializers[i][comp]

- Otherwise (palette_predictor_initializer_present_flag is equal to 0), PredictorPaletteEntries is set to 0.

The same aggregation method can be applied if the palette predictor initializer is flagged at the SPS (sequence parameter set) or slice header. In other words, all initialization values of the Y component are marked together, all initialization values of the Cb component are marked together, and all initialization values of the Cr component are marked together.

Aggregate the update process of palette predictions for the same color components

The existing palette prediction value update process is as follows.

The variable numComps is derived as shown in equation (1):

numComps=(ChromaArrayType==0)? 1:3 (1)

The variables PredictorPaletteSize and Predictor Palette Entries are modified as shown in the pseudocode in Table 5.

table 5.

As shown in Table 5, the first part of the newPredictorPaletteEntries update process is performed in a double do-loop denoted by annotations (5-1) and (5-2), where the do-loop associated with the color components (i.e. cIdx) in the inner do-loop. Therefore, for each palette entry, the three color components are updated. The remaining two do-loop statements updated by newPredictorPaletteEntries are denoted by comments (5-3) and (5-4), where the do-loop statements for different color components correspond to the inner loop. Therefore, for each entry, the three color components are updated. The two do-loop statements updated by PredictorPaletteEntries are represented by annotations (5-5) and (5-6), where the do-loop statements for different color components correspond to the outer loop. Therefore, update the PredictorPaletteEntries value for each color component.

In one embodiment, the palette predictions for the three corresponding components are aggregated and then updated. Exemplary modifications to the existing update process are shown below.

The derivation of variable numComps remains the same as equation (1). The variables PredictorPaletteSize and PredictorPaletteEntries are modified as shown in the following exemplary pseudocode in Table 6.

Table 6.

In the above example, the two do-loop statements updated by the first part newPredictorPaletteEntries are represented by annotations (6-1) and (6-2), where the do-loop statements for different color components correspond to the outer loop. Therefore, the remaining newPredictorPaletteEntries values are updated for each color component. The update process of newPredictorPaletteEntries indicated by the double do-loop denoted by Note (6-3) and Note (6-4) is consistent with the existing update process in Table 5. The update process of PredictorPaletteEntries indicated by the double do-loop indicated by annotations (6-5) and (6-6) is consistent with the existing update process in Table 5.

In another embodiment, the variables PredictorPaletteSize and PredictorPaletteEntries are modified as shown in the following exemplary pseudocode in Table 7.

Table 7.

In the above example, the two do-loop statements of the first part of the newPredictorPaletteEntries update are represented by annotations (7-1) and (7-2), where the do-loop statements for different color components correspond to the outer loop. Therefore, the remaining newPredictorPaletteEntries values are updated for each color component. The newPredictorPaletteEntries update process indicated by a double do-loop is denoted by Notes (7-3) and (7-4), where the do-loop statements of different components correspond to the outer loop. Therefore, update the newPred ictorPaletteEntries value for each color component. The update process of Pre dictorPaletteEntries indicated by the double do-loops represented by Note(7-5) and Note(7-6) is consistent with the existing update process in Table 5.

In another embodiment, the variables PredictorPaletteSize and PredictorPaletteEntries are modified as shown in the following exemplary pseudocode in Table 8.

Table 8.

In the above example, the two-loop statement of the update process of newPredictorPaletteEntries and PredictorPaletteEntries is represented by annotation (8-1), annotation (8-2), annotation (8-3), and annotation (8-4), where The do-loop statements for the different color components correspond to the outer loop. Therefore, the newPredictorPaletteEntries value and PredictorPaletteEntries are updated for each color component.

Various examples for the update process to aggregate palette prediction values for the same color components have been shown in Tables 6 to 8, where HEVC syntax elements have been used to The component palette predictions are aggregated to demonstrate the update process. However, the invention is not limited to the specific syntax elements and specific pseudocode listed in these examples. Those skilled in the art can practice the present invention without departing from the spirit of the present invention.

Semantic clustering of palette entries of the same color component

In the current SCM4.0, the syntax element palette_entry is used to specify the value of the component in the palette entry of the current palette. The variable PredictorPaletteEntries[cIdx][i] specifies the ith element in the palette of predicted values for color component cIdx. The variable numComps is derived as shown in equation (1). The variable CurrentPaletteEntries[cIdx][i] specifies the ith element in the current palette of color components cIdx, and is derived as shown in the following exemplary pseudocode in Table 9.

Table 9.

As shown in Table 9, the CurrentPaletteEntries of the palette predictions are updated as shown in Notes (9-1) and (9-2), where do-loop statements for different color components correspond to inner loops. Therefore, for each entry, the three color components are updated. Additionally, the CurrentPaletteEntries from the new palette entry will be updated as shown in Notes (9-3) and (9-4), where the do-loop statements for the different color components correspond to the outer loop. Therefore, the CurrentPaletteEntries value is updated for each color component.

In one embodiment, the palette prediction values for each of the three components may be clustered together. Exemplary changes to existing procedures are shown below.

The variable numComps is derived as shown in equation (1). The variable CurrentPaletteEntries[cIdx][i] specifies the ith element in the current palette of color components cIdx, and is derived as shown in the following exemplary pseudocode in Table 10.

Table 10.

As shown in Table 10, the CurrentPaletteEntries from the palette predictions are updated as shown in annotations (10-1) and (10-2), where the do-loop statements for different color components correspond to the outer loop. Therefore, the CurrentPaletteEntries value is updated for each color component. The CurrentPaletteEn tries in the new palette entry are updated as indicated in Notes (10-3) and (10-4) and are consistent with Table 9.

In another embodiment, the three components may be combined together. The changes are as follows.

The variable numComps is derived as shown in equation (1):

The variable CurrentPaletteEntries[cIdx][i] specifies the ith element in the current palette for color component cIdx, and is derived as shown in the following exemplary pseudocode in Table 11.

Table 11.

As shown in Table 11, the CurrentPaletteEntries in the palette predictions and the CurrentPaletteEntries from the new palette entries are as Annotation(11-1), Annotation(11-2), Annotation(11-3) and Annotation(11-4 ), where the do-loop statements for the different color components correspond to the outer loop. Therefore, the CurrentPaletteEntries value is updated for each color component.

Various examples of aggregation of palette entry semantics for the same color component of update processing have been shown in Tables 10 and 11, where HEVC syntax elements have been used to pass the same color The palette entry semantics of the components are aggregated to demonstrate the update process. However, the invention is not limited to the specific syntax elements and specific pseudocode listed in these examples. Those skilled in the art can practice the present invention without departing from the spirit of the present invention.

Additional palette syntax aggregation

In SCM-4.0, palette indices are clustered first, and bounce values are clustered last. Palette indices and jump values are encoded and decoded using bypass bins. Aggregating bounced values using palette indices (ie, aggregating bypass-encoded binaries) can increase parsing throughput. In SCM-4.0, the number of bounce values to resolve depends on the palette run mode, palette index, and palette run. Therefore, bounce values can only be aggregated at the end. In order to aggregate the bounced values using the palette index, the following methods are exposed.

Method 1: Use the palette index to aggregate the previous bounce values:

If the bounces are clustered in front, the number of bounces to resolve should be independent of palette runs. For bounce value parsing, in order to remove the data dependency of the palette run, embodiments of the present invention change the copy-over-run mode behavior when copying bounce samples from the row above. If the predictor is a dropout sample, the dropout sample is considered a predefined color index in the predictor replication mode.

In Copy Above Run mode, the Palette Run value is transferred or exported to indicate the number of subsequent samples to be copied from the top row. The color index is equal to the color index in the upper row. In one embodiment, if the upper or left sample is a pop-out sample, the color index of the upper sample is treated as a predefined color index (eg, 0). The current index is set to a predefined index. These predictor copy modes do not require jump values. In this method, palette run can be flagged for index run mode even if flagged index is equal to jump index. If the run of the skip index is greater than 0 (eg N, N>0), then reconstruct the first sample with the codec's skip value. The index of the first sample can be set to a jump index or a predefined color index. The index of the remaining N samples is set to a predetermined index (eg, index 0), and the remaining N samples are reconstructed using the value of the predetermined index (eg, index 0). In one embodiment, the maximum codeword index in a mode of operation (eg, adjustIndexMax) is fixed (eg, fixed at indexMax-1) except for the first sample of the index mode of operation. For the first sample in the CU, adjustIndexMax equals indexMax. Redundant index removal can still be applied. In this approach, the number of bounce values that need to be resolved depends on the number of resolved/refactored indexes equal to the bounce index. For example, if the palette index is encoded with a truncated binary code, and the encoded and decoded binaries are both 1, then comparing the parsed index is the jump index. The number of bounce values that need to be resolved is independent of palette runs. Therefore, using the palette index, the syntax of the jump value can be brought to the front (ie, before the palette runs).

Syntax sequence example 1: number of copy_above_run or number_run → last_run_mode → run type aggregation (context codec) → palette index aggregation (bypass codec) → jump value aggregation (bypass codec) → run length aggregation.

In this case, last_run_mode indicates that the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, and last_run_mode is index_run, the run type aggregation will be terminated. If the number of codec/decode index_runs is equal to the number of index_runs, and the last_run_mode is copy_above_run, the run type aggregation will also be terminated, and copy_above_run is inserted at the end.

Syntax sequence example 2: number of copy_above_run or index_run → run type aggregation (context codec) → last_run_mode → palette index aggregation (bypass codec) → jump value aggregation (bypass codec) → run length aggregation.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated and last_run_mode is marked. If last_run_mode is copy_above_run, insert copy_above_run at the end.

Syntax sequence example 3: number of copy_above_run or index_run → run type aggregation (context codec) → palette index aggregation (bypass codec) → skip value aggregation (bypass codec) → last_run_mode → run length aggregation.

In this case, last_run_mode indicates that the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated. Mark last_run_mode. If last_run_mode is copy_above_run, insert a copy_above_run at the end.

Syntax sequence example 4: number of copy_above_runs or number of index_runs → run type aggregation (context codec) → palette index aggregation (bypass codec) → last_run_mode → skip value (bypass codec) aggregation → run length aggregation.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated. Mark last_run_mode. If last_run_mode is copy_above_run, insert a copy_above_run at the end.

Syntax sequence example 5: number of copy_above_runs or number of index_runs → palette index aggregation → jump value aggregation → last_run_mode → run type aggregation → run length aggregation.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated. If last_run_mode is copy_above_run, insert an unmarked copy_above_run at the end. For the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

Syntax sequence example 6: number of copy_above_runs or number of index_runs → palette index aggregation → last_run_mode → jump value aggregation → run type aggregation → run length aggregation.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated. If last_run_mode is copy_above_run, copy_above_run is inserted at the end without marking. For the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

Syntax sequence example 7: number of copy_above_runs or number of index_runs → palette index aggregation → skip value aggregation → last_run_mode → interleave {palette run type, palette run length}.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated. If last_run_mode is copy_above_run, copy_above_run is inserted at the end without marking. For the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

Syntax sequence example 8: number of copy_above_runs or number of index_runs → palette index aggregation → last_run_mode → skip value aggregation → interleave {palette run type, palette run length}.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, in encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated. If last_run_mode is copy_above_run, copy_above_run is inserted at the end without being marked. For the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

In the above example, copy_above_run corresponds to the above-described exemplary syntax element of "copy above run mode". Furthermore, index_run corresponds to an exemplary syntax element of "run mode".

In Examples 1, 2, 3, and 5 of the above syntax sequence, "palette index aggregation (bypass codec) → skip (bypass codec) value aggregation" can be "interleave { palette index, skip value }". Palette indices and jump values can be encoded and decoded in an interleaved fashion. If the parsed index is equal to the bounce index, the bounce value can be parsed immediately.

In Examples 2 to 7 of the above syntax sequence, "last_run_mode" may be indicated after "number of copy_above_runs or number of index_runs".

In Examples 1 to 8 of the above syntax sequence, "last_run_mode" may be indicated anywhere before the last palette run is indicated.

In Examples 1 to 4 of the above syntactic sequence, the palette-run-gather is decoded before the palette-run-gather. Thus, for palette run signaling, the maximum possible run may be further subtracted by the number of index run modes remaining, the number of copy-over run modes remaining, or the number of index runs remaining pattern + the number of remaining patterns to run above the copy. For example, maxPaletteRun=nCbS*nCbS–scanPos–1–number of COPY_INDEX_MODEs remaining, or maxPaletteRun=nCbS*nCbS–scanPos–1–number of COPY_ABOVE_MODEs remaining, or maxPaletteRun=nCbS*nCbS–scanPos–1–number of COPY_ABOVE_MODEs remaining Quantity – The number of COPY_INDEX_MODE remaining. In the above example, maxPaletteRun corresponds to an exemplary syntax element for maximum palette run, nCbS corresponds to an exemplary syntax element for the size of the current luma codec block, and scanPos corresponds to the scan position of the current pixel.

In Examples 1 to 7 of the above syntax sequence, for the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

Method 2: Cluster the palette index to the end using a bounce value.

In SCM-4.0, the context in which the palette operates depends on the palette index. Therefore, the palette index can only be encoded and decoded before the palette is run. In order to cluster the palette index to the end, the context in which the palette runs should be formed independent of the palette index.

Therefore, the context formation of the palette run needs to be changed so that the context formation of the palette run will depend only on the current palette run mode, previous palette run mode, previous palette run, previous sample's palette palette run mode, palette run of the previous sample, or a combination of the above. Alternatively, palette runs can be encoded and decoded with bypass binaries.

Various examples of syntax orders for palette index mapping codecs are shown below.

Syntax sequence example 1: number of copy_above_run or number of index_run → last_run_mode → run type aggregation (context codec) → run length aggregation (does not depend on palette index) → palette index aggregation (bypass codec) → jump out Value aggregation (codec bypass).

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, when encoding/decoding run type aggregation, if the number of index_runs for encoding/decoding is equal to the number of index_runs, and last_run_mode is index_run, the run type aggregation will be terminated. If the number of codec/decode index_runs is equal to the number of index_runs, and the last_run_mode is copy_above_run, the run type aggregation will also be terminated, and copy_above_run is inserted at the end. For the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

Syntax sequence example 2: number of copy_above_runs or number of index_runs → run type aggregation → last_run_mode → run length aggregation → palette index aggregation → jump value aggregation.

In this case, last_run_mode indicates whether the last run mode was copy_above_run or index_run. For example, when encoding/decoding run type grouping, if the number of index_runs for encoding/decoding is equal to the number of index_runs, the run type aggregation will be terminated and last_run_mode will be indicated. If last_run_mode is copy_above_run, insert copy_above_run at the end. For the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

Syntax Sequence Example 3: Interleave {Palette Run Type, Palette Run Length}, Palette Index Aggregation → Jump Value Aggregation.

In Examples 1 to 3 of the above syntactic order, "palette index aggregation → jump value aggregation" may be "interleaved {palette index, jump value}". Palette indices and jump values can be encoded and decoded in an interleaved fashion. If the parsed index is equal to the bounce index, the bounce value can be parsed immediately.

In Examples 1 to 3 of the above syntax sequence, "last_run_mode" may be indicated anywhere before the last palette run is indicated.

In examples 1 and 2 of the above syntactic sequence, the palette run aggregation can be decoded before the palette index aggregation. Thus, for palette run signaling, the maximum possible run can be further subtracted from the number of remaining index run modes, the number of remaining copy-over run modes, or the number of remaining index run modes + the number of remaining copy-over run modes. For example, maxPaletteRun=nCbS*nCbS–scanPos–1–number of COPY_INDEX_MODE remaining, or maxPaletteRun=nCbS*nCbS–scanPos–1–number of COPY_ABOVE_MODE remaining, or maxPaletteRun=nCbS*nCbS–scanPos–1–number of COPY_ABOVE_MODE remaining–remaining Number of COPY_INDEX_MODEs.

In Examples 1 and 2 of the above syntax sequence, for the last palette run mode, the palette run is inferred to be at the end of the PU/CU.

The present invention also relates to various aspects of the palette codec disclosed below.

Remove line buffer in palette index map parsing

In SCM-3.0, four palette index mapping syntax elements (ie, palette run mode, palette index, palette run, and skip value) are encoded and decoded in an interleaved fashion. Although the contextual form of the palette run mode is modified to be independent of the palette run mode of the upper sample in SCM-4.0, index map resolution requires information from the upper row. For example, when using the copy above run mode, the number of bounce values to resolve depends on the number of bounce pixels to be copied from the row above. When the previous codec mode is the copy-above run mode, the index reconstruction also depends on the palette index of the upper sample. Several methods for removing data dependencies from the above example are disclosed in order to save the line buffer in palette index map parsing.

Method-1: If the predicted value is a jump-out sample, copy the jump-out value directly in the predicted value copy mode.

In order to remove dependencies during the calculation of the number of bounced pixels to resolve in the run-over-copy mode, embodiments in accordance with the present invention modify the run-over-copy mode behavior for copy-removing samples from the upper row.

In Copy Above Run mode, the Palette Run value is transferred or exported to indicate the number of the following samples to be copied from the top row. The color index is equal to the color index in the upper row. According to one embodiment, if the predicted value (ie, the upper position) is a jump sample, the current sample not only copies the index (the jump index), but also copies the jump value from the upper row. There is no need to parse the bounce value in these samples. In this method, even if the marked index is equal to the jump index, the run can be marked for the index run mode. If the run of the skip index is greater than 0 (eg N, N>0), the decoder will fill in the reconstructed value (or skip value) of N samples starting from the first sample. Escape values can be marked after running the syntax.

In order to remove the data dependency of index resolution and index run mode refactoring when the previous mode was a run-over-copy mode, redundant index deletion is disabled when the previous mode is run-above-copy mode, as described in JCTVC-T0078 ( JCTVC-T00784: Kim et al., CE1-related: simplification for index map coding inpalette mode, ITU-T SG 16WP 3 and ISO/IEC JTC 1/SC 29/WG 11 Joint Collaborative Team on Video Coding ) (JCT-VC), 20th meeting: Geneva, CH, 2015.2.10-18, Document No.: JCTVC-T0078).

Based on method-1, index map resolution does not rely on information from the upper row. The entropy decoder can parse all palette index mapping syntaxes by using only the dependencies of the previous codec palette operating mode and palette index.

In one embodiment, in the copy index run mode, a "palette run" value is sent or derived to indicate the number of following samples to be encoded and decoded in the bitstream. The color index of the current position is encoded and decoded. However, if the sample at the current position is a jump-out sample, not only the index of the current sample (the jump-out index) but also the jump-out value is encoded and decoded. In this method, the run can be flagged for the index run mode even if the flagged index is equal to the jump index. If the run of the skip index is greater than 0 (eg, N, N>0), the decoder will fill in the reconstructed values (or skip values) for N samples, starting from the first sample. Escape values can be marked after running the syntax.

In order to remove the data dependency of index resolution and reconstruction of the replicated index operational mode when the previous mode is the replicated index operational mode, redundant index deletion is disabled when the previous mode is the replicated index operational mode.

With the above approach, index map resolution does not depend on information from previous indexes. The entropy decoder can parse all palette index mapping syntaxes by using dependencies on the palette operating mode and palette index of the previous codec.

Method-2: If the predicted value is a skip sample, the skip sample is processed as a predetermined color index in the predicted value copy mode.

In Copy Above Run mode, the Palette Run value is transferred or exported to indicate the number of the following samples to be copied from the top row. The color index is equal to the color index in the row above. According to one embodiment, if the upper or left sample is a pop-out sample, the color index of the upper sample is treated as a predefined color index (eg, 0). The current index is set to a predefined index. These predictor copy modes do not require jump values. According to one embodiment, a palette run may be marked for an indexed run mode even if the marked index is equal to the jump index. Escape values can be marked after running the syntax. If the run of the skip index is greater than 0 (eg N, N>0), then reconstruct the first sample with the codec's skip value. The index of the first sample can be set to a jump index or a predefined color index. The index of the remaining N samples is set to a predetermined index (eg, index 0). The remaining N samples are reconstructed with the value of a predefined index (eg index 0). In another example, if the run of the jump index is greater than 0 (eg, N, N>0), then the first sample is reconstructed with the jump value, and the jump value for the following N samples also needs to be indicated. The remaining samples are respectively marked with bounce value reconstruction. The indices of the first sample and the remaining N samples may be set to predetermined indices (eg, index 0).

According to method-2, the maximum codeword index in the run mode (ie, adjustedIndexMax) is fixed (eg, fixed at indexMax-1), except for the first sample of the index run mode. For the first sample in the CU, adjustIndexMax is equal to indexMax. Redundant index removal can continue to be applied.

According to method-2, although the index values of the above samples are still required for index map reconstruction, the entropy decoder can resolve all palettes by using only dependencies on previously encoded palette operating modes and palette indices Index mapping syntax.

According to JCTVC-T0078, in order to remove the data dependency of index rebuilds for index run mode when the previous mode was run above copy mode, redundant index deletion will be disabled when the previous mode is run above copy mode.

Copy-above mode under index map rollover

The present invention also relates to problems related to index map flipping before index map encoding and decoding. After the decoder flips the index map, the prediction source in the copy above mode of operation is the physical above pixels at different physical locations than before. Figure 3A shows an example index map flip prior to index encoding and decoding. FIG. 3B shows an example of a flip index map.

FIG. 3A shows an example of an original codec unit. After the index map is flipped, the pixels in the last row (ie, pixels 0 to 7) will be flipped to the first row, as shown in Figure 3B. If the prediction can span CUs, the current first row of pixels is predicted from the neighboring constructed pixels (NCP for short) above. As shown in Figure 3B, the line-filled blocks indicate the inverted index map, while the clear blocks in Figure 3A represent the original index map. For pixels in other rows after flipping, the predictions in the upper operating mode are copied as predictions from the pixel positions of the underlying physical locations before flipping. In this approach, index reconstruction does not require a second pass.

However, flipping the index map before encoding and decoding the index map means using the above NCP to predict the pixel in the last row in the physical location, as shown in Figure 4. This prediction process is ineffective because of the large distance between the predicted value and the potential index to be predicted.

Therefore, the following discloses a method for improving codec efficiency related to index map flipping.

Method 1: As shown in Figure 5A and Figure 5B, regardless of whether the index map is flipped, the index in the copy-up mode of operation is from its physical upper position (left if the transpose flag is on). location) predicted. As shown in Figure 5A, the line-filled blocks represent the inverted index map, while the transparent blocks in Figure 5B represent the original index map.

Method 2: Signal different starting positions of the running scan. This method is similar to method 1, where the index in the copy-over mode of operation is predicted from the pixel in the physical over-position. Additional information can be marked to indicate the "running scan start position" or "scan mode". The Run Scan Start Position can be top left, top right, bottom left, or bottom right. The "scan pattern" can be horizontal scan, vertical scan, horizontal traversal scan, or vertical traverse scan.

Approach 3: If the index map is flipped, run the codec pixel over a copy of the first row of sample predictions from its physically closest position, as shown in Figures 6A and 6B. Figure 6A shows the flipped samples as indicated by the line-filled blocks, and Figure 6B shows the samples at the original physical location. If the transpose flag is off, the codec running above the copy of the last row in the physical location pixel is predicted from the sample at its physically closest location.

Method 4: If the index map is flipped, run the codec pixel above the copy of the last row of predictions from the physical pixel positions of the adjacent CUs described above, as shown in Figures 7A and 7B. Figure 7A shows flipped pixels indicated by line fill blocks, and Figure 7B shows a sample of the original physical location. If the transpose flag is turned off, the first row (or first M rows) of the physical location can be predicted from the physical pixel locations of the aforementioned adjacent CUs. M can be 1, 2, or 3. M may also depend on the CU size. M can be marked so that the decoder can decode accordingly.

Cross-CU prediction

In order to further improve the efficiency of encoding and decoding, a special operation is disclosed. This particular run extends the copy above run from the first sample of the palette codec CU. The special run can be signaled once. Samples in the operating mode above extended replication are predicted from reconstructed pixels in adjacent CUs. The remaining samples in this CU are encoded and decoded using the palette syntax specified in SCM-4.0 or SCM-3.0, with the difference that the total palette encoded/decoded samples in the PU/CU are reduced.

Method 1: First denote a syntax element (eg, line_num denoted by L) to indicate that the first L lines of samples are predicted from reconstructed pixels in neighboring CUs, where L is a positive integer. The remaining samples are encoded and decoded using the palette syntax in SCM-4.0 or SCM-3.0, with the difference that the total palette codec samples in the PU/CU are reduced. For the first L rows of samples, their pixel values are predicted from reconstructed pixels in their neighboring CUs. For example, if palette_transpose_flag is 0, the reconstructed pixels in the CU above are used. The pixel values of the first L rows of samples are the reconstructed pixel values of the last row of the upper CU. Similar to applying Intra vertical prediction to the first L rows of samples, while the remaining rows are encoded and decoded with the normal palette mode.

FIG. 8A shows an example of an extended copy-over mode of operation, where two rows of pixels are copied from the previous row above the CU boundary 810 (ie, L=2).

Method 2: First denote a syntax element (eg, pixel_num denoted by M) to indicate that the first M samples are predicted from reconstructed pixels in neighboring CUs, where M is a positive integer. The remaining samples are encoded and decoded using the palette syntax in SCM-4.0 or SCM-3.0, with the difference that the total palette codec samples in the PU/CU are reduced. For example, if palette_transpose_flag is 0, the reconstructed pixels in the CU above are used. The pixel values of the first M samples are the reconstructed pixel values of the last row of the CU above. Similar to applying vertical intra-prediction to the first M rows of samples. If the width of the CU is CU_width, according to the syntax in SCM-4.0, starting from the first sample at (M+1) to the first CU_width sample of (M+CU_width) samples in the CU cannot run above the copy codec in mode. In other words, according to the syntax in SCM-4.0, samples with scan positions equal to M to (M+CU_width-1) cannot be encoded and decoded in the run-up-copy mode. 8B illustrates an example of cross-CU prediction indicating the prediction of the first M (ie, M=11) samples from reconstructed pixels by denoting the syntax element pixel_num(M).

For example, the syntax table of palette_coding in SCM-3.0 can be modified, as shown in Table 12.

Table 12.

As shown in Table 12, the syntax element pixel_num is incorporated as shown in Note (12-1), where pixel_num is indicated before palette index mapping encoding and decoding. The rest of the palette samples are encoded and decoded from the scan position starting with pixel_num as shown in Note (12-2). The previous palette sample positions were derived from annotations (12-3). The first sample line after the first pixel_num samples does not allow the above pattern to be copied, as indicated in comment (12-4).

The variable adjustIndexMax representing the adjusted maximum index is derived as follows:

adjustedIndexMax=indexMax

if(scanPos>pixel_num)

adjustedIndexMax-=1

The variable adjustRefIndexMax representing the adjusted maximum reference index is derived as follows:

In the above method 1 and method 2, the syntax element copy_from_neighboring_CU_flag may be marked first. If copy_from_neighboring_CU_flag is 0, line_num and pixel_num will not be marked and inferred to be 0. If copy_from_neighboring_CU_flag is 1, line_num and pixel_num will be marked. The actual line_num and pixel_num equal to the parsed line_num and pixel_num are incremented by 1.

Method 3: In this method, neighboring pixels are used to predict the current pixel encoded and decoded in palette mode. Num_copy_pixel_line is first labeled, indicating that the first num_copy_pixel_line sample lines are predicted from reconstructed pixels in neighboring CUs. Except that the starting point is changed and the total palette codec samples in the PU/CU are reduced, the remaining samples are encoded and decoded by the normal palette index map codec.

For the first num_copy_pixel_line lines of samples, their pixel values are predicted from reconstructed pixels in neighboring CUs, where the syntax element num_copy_pixel_line corresponds to the number of pixel lines to copy. For example, if palette_transpose_flag is 0, the reconstructed pixels in the upper CU are used, as shown in Figure 9A. The pixel values of the first num_copy_pixel_line (eg, K) lines of samples are predicted by the reconstructed pixel values of the last line of the upper CU. It is similar to applying Intra vertical prediction to the first K rows of samples, while the remaining rows are encoded and decoded with the normal palette mode. If palette_transpose_flag is 1, the reconstructed pixels in the left CU are used, as shown in Figure 9B.

The syntax of num_copy_pixel_line can be indicated after the palette mode indication and before the palette table codec. If num_copy_pixel_line is equal to CU width or CU height, then flag palette_transpose_flag to indicate that the entire CU is predicted from upper or left pixels. Since all samples in the current CU are predicted, the syntax for palette table codec and index map codec is skipped.

If num_copy_pixel_line is less than CU width or CU height, normal palette table codec and palette index map codec are applied. In SCM-4.0, if MaxPaletteIndex is equal to 0, palette_transpose_flag is inferred to be 0. However, according to the current method, if MaxPaletteIndex is equal to 0 and num_copy_pixel_line is not equal to 0, palette_transpose_flag still needs to be flagged to indicate the first num_copy_pixel_line row or column samples to be predicted from reconstructed pixels in the upper CU or left CU. For index map encoding and decoding, the starting sample position is set to num_copy_pixel_line*CU_width. For samples with sample positions between num_copy_pixel_line*CU_width and (num_copy_pixel_line+1)*CU_width – 1, copy-above run mode cannot be selected. In other words, samples whose sample positions are smaller than CU_width*(num_copy_pixel_line+1) cannot be marked for copy-up operation mode.

Table 13 shows an exemplary syntax table for palette encoding and decoding according to embodiments of the methods disclosed above.

Table 13.

In Table 13, the syntax element num_copy_pixel_line is incorporated in front of all syntaxes, as shown in Note (13-1). If num_copy_pixel_line is equal to nCbS (ie CU width or CU height), the syntax palette_transpose_flag is merged as shown in Note (13-2). The palette index for the entire CU is assigned -1, as shown in note (13-3), which indicates that pixel values are copied from neighboring CUs. If num_copy_pixel_line is not equal to 0 and MaxPaletteIndex is not greater than 0 (eg MaxPaletteIndex is equal to 0), the syntax element palette_transpose_flag is merged as indicated in note (13-4). The palette index for the samples in the num_copy_pixel_line line is assigned -1, as indicated in comment (13-5). The first sample line after the first nCbS*num_copy_pixel_line line does not allow copying of the pattern above, as indicated in note (13-6).

The variable AdjustedMaxPaletteIndex is exported as follows:

AdjustedMaxPaletteIndex=MaxPaletteIndex

if(PaletteScanPos>num_copy_pixel_line*nCbS)

AdjustedMaxPaletteIndex-=1

The variable adjustRefPaletteIndex is derived as follows:

If PaletteIndexMap[xC][yC] is equal to -1, then the corresponding pixel value is the same as its neighbors. According to this method, if the upper or left pixels are not available (eg, samples at frame boundaries or inter-coded samples when restrictive intra prediction is applied), then the ones with a palette index equal to 0 are used. color. If the palette table of the current CU is not encoded (eg, num_copy_pixel_line is equal to CU_width), the first palette in the palette predictor table is used.

In another example, if upper or left pixels are not available, HEVC intra-prediction boundary pixel padding methods may be used to generate replacement adjacent pixels.

The number of lines using the Copy Above Pixel Line mode can be derived from the syntax element num_copy_pixel_line_indication instead of being directly indicated as num_copy_pixel_line. If num_copy_pixel_line_indication is 0, export num_copy_pixel_line as 0. If num_copy_pixel_line_indication is 1, export num_copy_pixel_line as N, which is a predefined number. If num_copy_pixel_line_indication is k, then export num_copy_pixel_line as k*N.

Method 4: In this method, the num_copy_pixel_line syntax is marked before the palette_transpose_flag syntax. Table 14 shows an exemplary palette codec syntax table according to an embodiment of the method.

Table 14.

In Table 14, after the palette_escape_val_present_flag signaling, the syntax element num_copy_pixel_line is incorporated as shown in Note (14-1). If num_copy_pixel_line is not equal to 0 and MaxPaletteIndex is not greater than 0 (eg MaxPaletteIndex is equal to 0), the syntax palette_transpose_flag is incorporated as indicated in Note (14-2). The palette index of the samples in the num_copy_pixel_line line is assigned -1 as indicated in comment (14-3). The first sample line after the first nCbS*num_copy_pixel_line does not allow copying of the upper pattern, as indicated in note (14-4).

The syntax element num_copy_pixel_line may be indicated before the syntax element palette_escape_val_present_flag. Table 15 shows an exemplary syntax table for palette codec according to this method.

Table 15.

In Table 15, before the palette_escape_val_present_flag syntax signaling, the syntax element num_copy_pixel_line is incorporated as shown in Note (15-1). If num_copy_pixel_line is not equal to 0 and MaxPaletteIndex is not greater than 0 (eg MaxPaletteIndex is equal to 0), the syntax palette_transpose_flag is merged as shown in Note (15-2). The palette index of the samples in the num_copy_pixel_line line is assigned -1 as indicated in comment (15-3). The first sample line after the previous nCbS*num_copy_pixel_line line does not allow copying of the pattern above, as indicated in note (15-4).

In this embodiment, the syntax elements NumPredictedPaletteEntries and num_signaled_palette_entries should both be zero if the syntax element num_copy_pixel_line is equal to CU_width. The first codec syntax element palette_predictor_run shall be 1.

In the syntax design of signaling (signaling) num_copy_pixel_line before palette_escape_val_present_flag (eg Table 15), NumPredictedPaletteEntries and num_signaled_palette_entries should both be 0, palette_predictor_run of the first codec should be 1, and palette_escape_val_present_flag is inferred to be 0.

Method 5: According to this method, num_copy_pixel_line is marked after palette_transpose_flag. In this syntax design, even if MaxPaletteIndex is equal to 0, the syntax element palette_transpose_flag needs to be marked. Table 16 shows an exemplary syntax table for palette encoding and decoding according to an embodiment of the method.

Table 16.

Since the syntax element palette_transpose_flag needs to be flagged even when MaxPaletteIndex is equal to 0, syntax is merged outside of the test "if(MaxPaletteIndex>0)" (as shown in Note (16-1)). At the same time, the syntax element palette_transpose_flag in the test "if(MaxPaletteIndex>0)" is removed as indicated by the line padding text in comment (16-2). The syntax element num_copy_pixel_line is incorporated as indicated in note (16-3). The palette index of the samples in the num_copy_pixel_line line is assigned -1 as indicated in comment (16-4).

If num_copy_pixel_line is even, it is natural to use a left-to-right scan for the first normal line, as shown in Figure 10A. However, if num_copy_pixel_line is odd, there are two types of scans to choose from. One is a left-to-right scan of the first normal as shown in FIG. 10B , and the other is a right-to-left scan of the first normal as shown in FIG. 10C .

As shown in Figure 10B, it moves a traverse scan downward or uses a traverse from the first normal. In Figure 10C, it is using a traversal scan from the first sample in the current CU, and skipping the scan for the first num_copy_pixel_line lines. In Tables 13 to 16, the scan in Figure 1OC was used. The syntax table can be modified accordingly for the scan in Figure 10B.

Method 6: Table 17 shows an exemplary syntax table for the palette codec scanned in Figure 10B.

Table 17.

In Table 17, the syntax element num_copy_pixel_line is incorporated in front of all syntaxes, as shown in Note (17-1). If num_copy_pixel_line is equal to nCbS (ie CU width or CU height), the syntax palette_transpose_flag is incorporated as shown in Note (17-2). The palette index for the entire CU is assigned -1, as indicated by note (17-3), which indicates that pixel values are copied from neighboring CUs. If num_copy_pixel_line is not equal to 0, and MaxPaletteIndex is not greater than 0 (eg MaxPaletteIndex is equal to 0), the syntax palette_transpose_flag will be merged as in Note (17-4). The palette indices of the samples in the num_copy_pixel_line line are assigned -1 as indicated in note (17-5). PaletteScanPos will be reset to 0 as indicated in note (17-6). Since PaletteScanPos is reset, the actual sample index needs to be increased by num_copy_pixel_line*nCbS, as shown in Notes (17-7 and 17-10 to 17-12). The vertical position needs to be increased by num_copy_pixel_line as indicated in comments (17-8, 17-9 and 17-13).

The variable AdjustedMaxPaletteIndex is exported as follows:

AdjustedMaxPaletteIndex=MaxPaletteIndex

if(PaletteScanPos>0)

AdjustedMaxPaletteIndex-=1

The variable adjustRefPaletteIndex is derived as follows:

Note that in the above methods 3 to 6, the signaling of num_copy_pixel_line can also be constructed by using the palette_run syntax element and context in the existing HEVC SCC specification. For example, it can be labeled as shown in Table 18.

Table 18.

In Table 18, using the above syntax, the decoded copy_pixel_line_length can semantically be the number of copied pixel lines. The maximum value of this codec element is block height-1.

The decoded copy_pixel_line length can also be the actual number of samples using the copy pixel mode, and the number of copied pixel lines is derived as copy_pixel_line length/block_width. Note that in this method, a conformance constraint is imposed on the copy_pixel_line length such that it must be a multiple of block_width.

Method 7: In this method, the current syntax structure with the modified decoding process is used to indicate whether the block starts from a copied pixel pattern outside the current CU, and the number of rows (samples) of the copied pixel pattern is used. For example, this can be achieved by:

· Allows duplicating the run pattern above for the first sample in the block. This is indicated by using the syntax element palette_run_type_flag[0][0]. If palette_run_type_flag[0][0] is 1, the following lines are filled with samples of the indicated line number with replicated pixels. If palette_run_type_flag[0][0] is 0, the remaining syntax signaling (syntaxsignaling) will remain the same as the current syntax structure.

• When palette_run_type_flag[0][0] is 1, use the same syntax element used to signal the palette run length to indicate the number of lines using duplicated pixels. There are two ways to use the palette run-length syntax to notify the number of lines.

o The decoded palette run-length R semantics for replicating pixel patterns means the number of rows (not the number of samples). Therefore, the actual operation of copying pixels is the decoded value R*block_width for horizontal scanning, or the decoded value R*block_height for vertical scanning. The maximum value of this decoded R shall be the block height (or width).

o Another way is that the decoded palette run length R is the actual replicated pixel run. This method does not require changing the semantics and decoding process of the copied pixel operation.

Note that for this method, a consistency constraint must be imposed on the decoded value R such that it must be a multiple of block_width.

· When palette_run_type_flag[0][0] is 1, after copying the pixel row, the next row of samples cannot use the copy above run mode. Modify the parsing criteria based on this condition to not parse the run_type flag for this line.

· When palette_run_type_flag[0][0] is 0, the rest of the samples in the first row cannot use the copy above run mode. The parsing standard is modified according to this condition to not parse the run_type flag of these samples.

Table 19 shows an exemplary syntax table for palette codec according to this method. The codec semantics for the "palette run length R" of the copied pixel pattern means the number of rows (not the number of samples).

Table 19.

In Table 19, since the syntax element palette_transpose_flag needs to be marked even if MaxPaletteIndex is equal to 0, the syntax (as shown in Note (19-1)) is combined in addition to the test "if(MaxPaletteIndex>0)". Meanwhile, the line padding text in comment (19-2) indicates the deletion of the syntax element palette_transpose_flag within the test "if(MaxPaletteIndex>0)". The palette index for the entire CU is first reset to -1, as shown in note (19-3), which indicates that pixel values are copied from neighboring CUs. Palette_run_type_flag and paletteRun of the first sample (PaletteScanPos==0) are used to indicate num_copy_pixel_line. As noted in Note (19-5), for the first sample, if palette_run_type_flag is equal to COPY_ABOVE_MODE, the maxPaletteRun is set equal to nCbS-1, as noted in Note (19-4), and num_copy_pixel_line is equal to the decoded paletteRun. Otherwise, num_copy_pixel_line is set to 0.

Table 20 shows another exemplary syntax table for palette codec according to this method. The codec for the "palette run length R" of the replicated pixel mode is the actual run length and imposes consistency constraints such that R must be a multiple of CU_width.

Table 20.

In Table 20, since the syntax element palette_transpose_flag needs to be marked even if MaxPaletteIndex is equal to 0, the syntax (as shown in Note (20-1)) is combined outside the test "if (MaxPaletteIndex>0"). At the same time, the note ( The line padding text in 20-2) indicates the deletion of the syntax element palette_transpose_flag within the test "if(MaxPaletteIndex>0)". The palette index for the entire CU is first reset to -1 as shown in note (20-3) , represents that pixel value is copied from adjacent CU. The palette_run_type_flag and paletteRun of the first sample (PaletteScanPos==0) are used to indicate num_copy_pixel_line.As shown in note (20-4), for the first sample, if palette_run_type_flag is equal to COPY_ABOVE_MODE , then num_copy_pixel_line is equal to decoded paletteRun/nCbs. Note that in this case, a consistency constraint is imposed on paletteRun so that it must be a multiple of nCbs. Otherwise (ie palette_run_type_flag is not equal to COPY_ABOVE_MODE), num_copy_pixel_line is set to 0.

Method 8: According to this method, only num_copy_pixel_line=0 or CU_width is tested. Provides a shortcut to palette mode by introducing a new syntax element pred_from_neighboring_pixels. If pred_from_neighboring_pixels is 1, display palette_transpose_flag.

If pred_from_neighboring_pixels is 1 and palette_transpose_flag is 0, all samples are predicted from the pixels of the CU above. If pred_from_neighboring_pixels is 1 and palette_transpose_flag is 1, all samples are predicted from the pixels of the left CU. If adjacent pixels are not available, there are two ways to generate replacement pixels. According to the first method, an intra prediction boundary pixel generation method can be used. It is similar to horizontal or vertical intra prediction without residual coding. According to the second method, a color with a palette index equal to 0 is used. If the palette table of the current CU is not encoded (eg, num_copy_pixel_line is equal to CU_width), the first palette in the palette predictor table is used.

Table 21 shows an exemplary syntax table for palette codec according to this method.

Table 21.

In Table 21, the syntax element pred_from_neighboring_pixels syntax is merged as shown in Note (21-1). If pred_from_neighboring_pixels is true (true), the palette_transpose_flag is merged, and the palette index for the entire CU is set to -1, as indicated in note (21-2).

In method 3 to method 8, only one palette_transpose_flag is marked. If palette_transpose_flag is equal to 0, the vertical copy pixel mode is used first, then the indexed horizontal scan is used. Otherwise, copy pixels horizontally first, then scan vertically using indexing.

Alternatively, two transpose flags can be flagged, eg palette_copy_pixel_transpose_flag and palette_scan_transpose_flag. Palette_copy_pixel_transpose_flag indicates the direction of copying pixel patterns from neighboring CUs. Palette_scan_transpose_flag indicates the direction of palette scan. For example, palette_copy_pixel_transpose_flag equal to 0 indicates the copied pixel mode from the upper CU, while palette_copy_pixel_transpose_flag equal to 1 indicates the copied pixel mode from the left CU. Palette_scan_transpose_flag equal to 0 means to use horizontal scan, and Palette_scan_transpose_flag equal to 1 means to use vertical scan.

For example, in FIG. 11A, palette_copy_pixel_transpose_flag is 0, and palette_scan_transpose_flag is 0. In FIG. 11B, palette_copy_pixel_transpose_flag is 0, and palette_scan_transpose_flag is 1. In FIG. 11C, palette_copy_pixel_transpose_flag is 1 and palette_scan_transpose_flag is 0.

Prediction on CU with reverse traversal scan and rotation traversal scan

According to one embodiment, prediction across CUs may be implemented with reverse traversal scans and rotational traversal scans. FIG. 12A shows an example of cross-CU prediction by reverse traversal scanning, and FIG. 12B shows an example of cross-CU prediction by rotated traversal scanning. The normal index mapping codec for both scans starts at 0 and ends at nCbS*nCbS-num_copy_pixel_line*nCbS–1. For the remaining samples with sample positions equal to or greater than nCbS*nCbS-num_copy_pixel_line*nCbS, PaletteIndexMap[xC][yC] is set to -1, which means that their pixel values are the same as adjacent pixels.

Context formation and binarization of num_copy_pixel_line

The syntax element num_copy_pixel_line can use an Exponential-Golomb code of order K (EG-K code), a truncated Exponential-Golomb code of order K (truncated EG-K code), an N-bit truncated unary code (Unary code) + EG-K code, or the same binarization method used in the palette run (binarized as palette_run_msb_id_plus1 and palette_run_refinement_bits).

Context bins can be used to encode and decode num_copy_pixel_line. For example, we can use the same binarization method used in the palette run of num_copy_pixel_line. The first N bits of palette_run_msb_id_plus1 can use context codec bins. For example, N can be 3. The remaining bins are encoded and decoded in bypass bins. Codec shared contexts can be run through the palette. For example, a context can be shared with an extended copy-over-run mode.

Since the probability that num_copy_pixel_line is equal to CU_width and 0 is greater than the probability that num_copy_pixel_line is other numbers, the code binarization is modified to reduce the codeword of num_copy_pixel_line equal to CU_width. For example, the value of CU_width may be inserted before the number P (eg, 1) in the codeword table for binarization. The value of num_copy_pixel_line equal to or greater than P will be incremented by 1 on the encoder side. On the decoder side, if the parsed codeword is P, it means that num_copy_pixel_line is equal to CU_width. If the parsed codeword is less than P, then num_copy_pixel_line is equal to the parsed codeword. If the parsed codeword is greater than P, then num_copy_pixel_line is equal to the parsed codeword minus 1.

In another embodiment, two additional bits are used to indicate whether num_copy_pixel_line is equal to 0, CU_width, or other numbers. For example, "0" means that num_copy_pixel_line is equal to 0, "10" means that num_copy_pixel_line is equal to CU_width, and "11+codeord-L" means that num_copy_pixel_line is equal to L+1.

In another embodiment, two additional bits are used to indicate whether num_copy_pixel_line is equal to 0, CU_width, or other number. For example, "0" means that num_copy_pixel_line is equal to CU_width, "10" means that num_copy_pixel_line is equal to 0, and "11+codeord-L" means that num_copy_pixel_line is equal to L+1.

For methods 4 to 7 mentioned in cross-CU region prediction, num_copy_pixel_line is marked after palette table encoding and decoding. The binarization of num_copy_pixel_line can be modified according to the decoding information of the palette table codec. For example, if NumPredictedPaletteEntries and num_signaled_palette_entries are both 0, this means that at least one sample row/column is encoded and decoded via normal palette mode. Therefore, num_copy_pixel_line should not be equal to CU_width. Therefore, the codeword range of num_copy_pixel_line is limited from 0 to (CU_width-1). For example, if you use a truncated exponential-Golomb code of order K (truncated EG-K code), an N-bit truncated unary code + EG-K code, or the same binarization method used in the palette run with For encoding and decoding num_copy_pixel_line, cMax, MaxPaletteRun, or the maximum possible value is set to CU_width-1. The binarization method for palette run will binarize the palette run into palette_run_msb_id_plus1 and palette_run_refinement_bits.

Determine the search method for num_copy_pixel_line

In another embodiment, a search method for determining the number of copied pixel lines (ie, num_copy_pixel_line) is disclosed.

Method 1: On the encoder side, determine the value of num_copy_pixel_line. Predict the first num_copy_pixel_line columns/rows from neighboring pixels. The remaining samples are used to derive a palette of remaining samples and index mappings.

Method 2: On the encoder side, use the samples of the entire CU to first derive the palette. From this palette table, an optimal value for num_copy_pixel_line can be determined using a rate-distortion-optimization (RDO) process. Interpolation can be used to estimate the bit cost of different num_copy_pixel_lines. For example, if the bit cost of num_copy_pixel_line=0 is R0 and CU_width is 16, then the bit cost of num_copy_pixel_line=3 is equal to R0*(CU_width-3)/CU_width (ie 13/16*R0).

After num_copy_pixel_line is determined, the first num_copy_pixel_line columns/rows are predicted from neighboring pixels. The remaining samples are used to re-export the remaining samples and index-mapped palettes.

Row-based replicated pixels from adjacent CUs

In the previous section, row-based replicated pixels from adjacent CUs were disclosed for palette mode codec. The syntax element num_copy_pixel_row representing the number of copied pixel rows is first indicated to indicate the first num_copy_pixel_row row samples to be predicted from reconstructed pixels in adjacent CUs. Similar concepts can be applied to other modes such as Inter mode, Intra BC mode and Intra BC mode.

In one embodiment, row-based replicated pixels from adjacent CUs are applied to inter-mode and/or intra-mode PUs or CUs. The syntax element num_copy_pixel_row is first marked. If num_copy_pixel_row is not equal to 0, the syntax element copy_pixel_row_direction_flag is flagged to indicate the direction of copying the pixel row. If copy_pixel_row_direction_flag is 0, then num_copy_pixel_row represents the first num_copy_pixel_row row samples predicted from the reconstructed pixels in the CU above. If copy_pixel_row_direction_flag is 1, then num_copy_pixel_row represents the first num_copy_pixel_row column samples predicted from the reconstructed pixels in the left CU. For example, FIG. 13 shows an example of an 8x8 CU encoded and decoded in inter mode. The value of num_copy_pixel_row of the CU is 3, and the copy_pixel_row_direction_flag of the CU is 0. The predicted values in the top three rows are replaced by the reconstructed pixel values of the last row of the CU above. The remaining pixels are predicted by the original inter mode. It is similar to performing inter-mode prediction on the entire CU/PU and then replacing the first num_copy_pixel_row rows or columns with adjacent pixels.

Intra-predicted neighbor construction can be used to generate neighbor reference pixels. If adjacent pixels are not available (eg, outside the image boundaries), the reference pixel padding method in intra prediction can be applied to generate adjacent reference pixels. The smoothing filter in intra prediction can be applied or turned off.

The syntax elements num_copy_pixel_row and copy_pixel_row_direction_flag may indicate signals at the CU level or the PU level. Num_copy_pixel_row and copy_pixel_row_direction_flag may be indicated before the CU or PU, or may be indicated in the middle of the CU or PU, or may be indicated at the end of the CU or PU. For example, if num_copy_pixel_row and copy_pixel_row_direction_flag are indicated at the CU level and are indicated before the CU, then num_copy_pixel_row and copy_pixel_row_direction_flag may be indicated before part_mode. The codeword of part_mode may be adaptively changed according to the values of num_copy_pixel_row and copy_pixel_row_direction_flag. For example, if copy_pixel_row_direction_flag is 0 and num_copy_pixel_row is equal to or greater than CU_height/2, then PART_2NxN and PART_2NxnU are invalid. Modify the codeword binarization of part_mode. For example, if PART_2NxN and PART_2NxnU are removed, the binarization of part_mode is shown in Table 22.

Table 22.

In Table 22, text with a line-filled background corresponds to deleted text. In another example, if num_copy_pixel_row and copy_pixel_row_direction_flag are indicated at the CU level, and num_copy_pixel_row and copy_pixel_row_direction_flag are indicated at the end of the CU or in the middle of the CU, then num_opy_pixel_row and copy_pixel_row_direction_flag may be indicated after part_mode. After receiving part_mode, the values of num_copy_pixel_row and copy_pixel_row_direction_flag can be limited. For example, if part_mode is PART_2NxN and copy_pixel_row_direction_flag is 0, the value of num_copy_pixel_row will be restricted to the range from 0 to CU_height/2.

Intra Boundary reference pixel

In intra mode, if num_copy_pixel_row is not 0, the adjacent reference pixel may be the same as the pixel from HEVC origin. It is the same as performing intra prediction of the PU and then replacing the first few rows or columns with pixels from neighboring CUs.

In another embodiment, if num_copy_pixel_row is not 0, the position of the adjacent reference pixel is changed. 14 and 15 illustrate an example of changing the positions of adjacent reference pixels according to the present embodiment. In Figures 14 and 15, num_copy_pixel_row is 3, and copy_pixel_row_direction_flag is 0. The upper reference pixel and the upper left reference pixel are moved to the third row. As shown in Figure 14, the upper right reference pixel is copied from the upper right CU. As shown in Figure 15, the upper right reference pixel is copied from the rightmost pixel in the third row.

Residual coding for regions predicted from neighboring CUs

According to one embodiment, if num_copy_pixel_row is N and copy_pixel_row_direction_flag is 0, the upper N rows are predicted from neighboring CUs. The residuals in the upper N rows can be constrained to 0. According to another embodiment, the residuals of the upper N rows may be marked. For inter mode, HEVC residual quadtree is applied.

Context formation and binarization of num_copy_pixel_row

num_copy_pixel_row can use K-order exponential Golomb code (EG-K code), K-order truncated exponential Golomb code (truncated EG-K code), N-bit truncated unary code + EG-K code or used in palette operation The same binarization method of (ie, is binarized as palette_run_msb_id_plus1 and palette_run_refinement_bits).

The context binary can be used to encode and decode num_copy_pixel_row. For example, the same binarization method as num_copy_pixel_row in the palette run codec can be used. The first N bits of palette_run_msb_id_plus1 can use the context codec binary. For example, N can be 3. The rest of the binaries are encoded and decoded in bypass binaries. Codec shared contexts can be run through the palette. For example, the context can be shared with the run mode above the extended replication.

Since the probability of num_copy_pixel_row being equal to CU_width and 0 is higher than the probability of num_copy_pixel_row being other numbers, the code binarization can be modified to reduce the codewords of num_copy_pixel_row equal to CU_width. For example, the value of CU_width may be inserted before the number M (eg, 1) in the codeword table. The value of num_copy_pixel_row equal to or greater than M will be incremented by 1 on the encoder side. On the decoder side, if the parsed codeword is M, it means that num_copy_pixel_row is equal to CU_width. If the parsed codeword is less than M, then num_copy_pixel_row is equal to the parsed codeword. If the parsed codeword is greater than M, then num_copy_pixel_row is equal to the parsed codeword minus 1.

According to another embodiment, two additional bits are used to indicate whether num_copy_pixel_row is equal to 0, CU_width, or other number. For example, "0" means that num_copy_pixel_row is equal to 0, "10" means that num_copy_pixel_row is equal to CU_width, and "11+codeord-L" means that num_copy_pixel_row is equal to L+1. In another example, "0" means that num_copy_pixel_row is equal to CU_width, "10" means that num_copy_pixel_row is equal to 0, and "11+codeord-L" means that num_copy_pixel_row is equal to L+1.

For methods 4 to 7 mentioned in the prediction of the cross-CU part, num_copy_pixel_row can be indicated after the palette table encoding and decoding. The binarization of num_copy_pixel_row can be modified according to the decoding information of the palette table codec. For example, if both NumPredictedPaletteEntries and num_signaled_palette_entries are 0, it means that at least one row/column of samples is encoded and decoded in normal palette mode. num_copy_pixel_row should not be equal to CU_width. Therefore, the codeword range of num_copy_pixel_row is limited from 0 to (CU_width-1). For example, if K-order truncated exponential – Golomb code (truncated EG-K code), N-bit truncated unary code + EG-K code, the same binarization method used in the palette run (i.e. Change to palette_run_msb_id_plus1 and palette_run_refinement_bits) for encoding and decoding num_copy_pixel_row, set cMax, MaxPaletteRun, or the maximum possible value to CU_width-1. num_copy_pixel_row should not be equal to CU_width

In this section, the above-mentioned CU_width may be replaced by CU_height, PU_width, or PU_height of a row-based copy pixel from a CU-level or PU-level of an adjacent method.

Number of index codecs

In the SCM-4.0 palette index mapping codec, the index number is first marked. To indicate the number of indices, the variable "number of indices - palette size" is first derived. A mapping process is then performed to map "number of indices-palette size" to "mapped value". The mapped values are binarized using the same binarization method as "coeff_abs_level_remaining" and marked. The prefix part is represented by a truncated Rice code. The suffix part is represented by exponential Golomb code. This binarization process can be thought of as having an input cParam, and cParam is set to (2+indexMax/6). However, indexMax/6 requires division or lookup table operations. Therefore, according to an embodiment of the present invention, cParam is set to (2+indexMax/M), and M is a power-of-2 (eg, 2 ⁿ , where n is an integer). Therefore, indexMax/M can be achieved by right-shifting indexMax by n bits. For example, setting cParam to (2+indexMax/4) or (2+indexMax/8) can be achieved by shifting indexMax to the right by 2 or 3 bits, respectively.

Gather all bounce colors before encoding/parsing the palette index map

In the current HEVC SCC specification or previous versions, during encoding and decoding of the index map, the values of the jumping out pixels in the modulation board codec are marked with other regular indexes in an interleaved manner, or after the encoding and decoding of the index map is completed, the pixel values come together. According to one embodiment of the present invention, all jumping out pixel values are aggregated before index map encoding and decoding.

It is assumed that there are N jump-out pixels at different positions within the current codec block, where N is a positive integer. In one embodiment, all color values of these skip pixels are encoded/decoded together before encoding/decoding the palette index map for the codec block. In this way, when an index is decoded as a jump index, its corresponding pixel value no longer needs to be decoded. Note that some bounce pixels may have the same color value. In one embodiment, the pixel value of each pop-out pixel occurrence is still written to the bitstream.

Figure 16 is an example of decoding a pop-out color with N=5, where the pixel value at which each pop-out pixel occurs is still written to the bitstream, according to an embodiment of the method. In this example, a horizontal traversal scan is used. According to the decoding order, each jumping out pixel can find the corresponding color in the decoding table.

In another embodiment, only non-repeating color values are written to the bitstream, and an index identifying these written colors occurs for each bounce pixel.

Figure 17 shows an example of decoding a pop-out color according to an embodiment of the N=5 method, where only non-repeating color values are written to the bitstream. In this example, a horizontal traversal scan is used. Only non-repeating colors are decoded (eg, M=3), and the index in the color table for each pop-out pixel occurrence is signaled (eg, N=5). According to the decoding order, each jumping out pixel can find the corresponding color in the decoding table.

Jump out color signaling termination

In one embodiment, the number of bounced colors encoded/decoded is indicated. For example, after each escape color is encoded or decoded, a 1-bit flag "end_of_escape_color_flag" is used to signal if this is the last color to be encoded or decoded. When the decoded end_of_escape_color_flag is 1, no more escape colors need to be decoded. Assuming that there are N jump colors in the current codec block, the last M pixels have the same color value, where M and N are integers, and M<=N. In another embodiment, only one color value of the M pixels needs to be sent, and end_of_escape_color_flag is set to 1. The last (M-1) bounce pixels are inferred to share the last decoded bounce color value. An example of this method is shown in Table 23 for N=5 and M=3.

Table 23.

In another embodiment, the total number of bounced colors is explicitly indicated before encoding/decoding the color values. Also, end_of_escape_color_flag can be bypass codec or context codec.

Limit on the total number of bounce colors allowed in a codec block

The total number of decoded bounce colors can be limited by setting the maximum allowed number at an advanced header such as sequence level, picture level or slice level.

If the maximum number is reached, according to one embodiment, the remaining bounce pixels are inferred to share the value of the last decoded bounce color. In another embodiment, the remaining bounce pixels are inferred to share the value of a particular decoded bounce color, such as the most commonly used color.

Copy pixels from outside the CU

In SCM 3.0, the range of color index values depends on the palette size and the bounce flag. If the jump flag is off, the maximum index value is equal to the palette size. If the jump flag is on, the maximum index value is equal to (palette size + 1). If the maximum index value is equal to N, the range of possible index values is 0 to (N-1). In SCM 3.0, a maximum index value equal to 0 is prohibited. If the maximum index value is equal to 1, then all color indices in the CU will be presumed to be 0. If there is only one possible index value, it is assumed that all color indices should be 0.

According to the above disclosure, pixels can be marked by COPY_ABOVE. In this case it will not only copy the pixel index of the pixel above, but also the pixel value of the pixel above. The decoder can reconstruct the pixels in COPY_ABOVE from the copied pixel values without having to reference the palette. If the above-mentioned pixel crosses a CU boundary, according to the above disclosure, a special index (denoted as N) of the adjacent structuring pixel (NCP) of the adjacent CU is assigned. When a pixel is denoted by COPY_ABOVE, it copies not only the pixel index (N) of the pixel above, but also the pixel value of the pixel above indicated by the dot-filled area in Figure 18, which shows the CU boundary (1810).

Based on the method of copying the pixel values of the NCP, the assumption that the CU of the palette codec is processed with zero/one index value is not true. For the case where the maximum index value is equal to 0, all pixels in the CU can be predicted from the NCP, as shown in Figure 19.

If the maximum index value is equal to 1, not all color indices can be 0. A portion of the pixels can be 0, and a portion of the pixels can be predicted from the NCP, as shown in Figure 20.

In the examples shown in Figure 19 and Figure 20, there is no corresponding syntax available in SCM3.0. Therefore, a new syntax for denoting these cases is disclosed below.

Syntax elements for index prediction across CUs

In SCM3.0, the palette codec CU contains the following syntax:

Palette_share_flag equal to 1 specifies that the palette size is equal to the previous previousPetteBoxEntries, and the entire palette entry is the same as the previous palette entry.

Palette_transpose_flag equal to 1 specifies that the transpose process is applied to the current CU's associated palette index. Palette_transpose_flag equal to 0 specifies that the transpose process should not be applied to the current CU's associated palette index.

· Palette_escape_val_present_flag specifies the sample value of the skip codec.

Palette_prediction_run[i] specifies the difference between the index of the current reused palette entry in the previous palette previousPaletteEntries and the index of the next recalled palette entry, with the following exceptions: palette_prediction_run equal to 0 means the current and next reused entry The difference between the indices is 1, and palette_prediction_run equal to 1 means no more entries from the previous palette previousPaletteEntries are reused.

Num_signaled_palette_entries specifies the number of entries in the palette that are explicitly marked for the current codec unit.

Palette_entries specifies the ith element in the palette of the color component cIdx

· Palette_run_coding() specifies the run codec mode of the index map.

To provide syntax for index prediction across CUs, the following syntax examples are disclosed according to embodiments of the present invention:

Syntax Example 1: Add a new flag all_pixel_from_NCP_flag. If all_pixel_from_NCP_flag is off, other syntax is the same as SCM3.0. In the first row, a copy run mode may be indicated to allow cross-CU prediction. If all_pixel_from_NCP_flag is on, it means (imply) that all pixels will be predicted from NCP. Palette_transpose_flag may indicate prediction from left NCP or above. Other prediction directions may also be indicated. If all_pixel_from_NCP_flag is on, signaling palette_share_flag, palette_escape_val_present_flag, palette_prediction_run, num_signaled_palette_entries, palette_entries, or palette_run_coding() can be skipped.

21 illustrates an exemplary flow diagram of signaling to support index prediction across CUs according to the above-described examples. In step 2110, it is tested whether all_pixel_from_NCP_flag is equal to 1. If the result is "yes", step 2130 is performed. If the result is "no", step 2120 is performed. In step 2130, palette_transpose_flag is flagged to indicate prediction from either the left NCP or the upper NCP. In step 2120, SCM3.0 based syntax is used for index prediction across CUs.

Syntax example 2: Add new flag any_pixel_from_NCP_flag. If any_pixel_from_NCP_flag is off, other syntax is the same as SCM3.0. In the first row, the copy run mode is not indicated (no cross-CU prediction). If any_pixel_from_NCP_flag is on, it is implied that some pixels are to be predicted from NCP. The encoder can flag palette_share_flag, palette_prediction_run, num_signaled_palette_entries, palette_escape_val_present_flag, and the decoder can calculate the maximum index value based on this information. If the maximum index value is equal to 0, all pixels are predicted from NCP and palette_run_coding() can be skipped. If the maximum index value is greater than 0, partial pixels are predicted from NCP, and palette_run_coding() can be flagged.

22 illustrates an exemplary flow diagram of signaling to support index prediction across CUs according to the above-described examples. As shown in step 2210, test whether any_pixel_from_NCP_flag is equal to 1. If the result is "yes", step 2230 is performed. If the result is "no", step 2220 is performed. In step 2220, SCM3.0 based syntax is used for index prediction and no cross-CU prediction is required. In step 2230, various syntax elements including palette_share_flag, palette_prediction_run, num_signaled_palette_entries, and palette_escape_val_present_flag are marked. The decoder calculates the maximum index value based on this information and checks in step 2240 whether the maximum index value is equal to zero. If the result is "yes", step 2260 is performed. If the result is "no", step 2250 is performed. In step 2250, flag palette_transpose_flag and palette_run_coding(). In step 2260, the palette_transpose_flag is flagged, and palette_run_coding() is skipped (ie, all pixels are predicted from NCP).

Syntax Example 3: Add new flag any_pixel_from_NCP_flag. If any_pixel_from_NCP_flag is off, other syntax is the same as SCM3.0. In the first row, the copy run mode is not indicated (no cross-CU prediction). If any_pixel_from_NCP_flag is on, it implies that partial pixels are to be predicted from NCP. The encoder can flag palette_share_flag, palette_prediction_run, num_signaled_palette_entries, palette_escape_val_present_flag, and the decoder can calculate the maximum index value based on this information. If the maximum index value is equal to 0, all pixels are predicted from NCP and palette_run_coding() can be skipped, as shown in Figure 2. Otherwise, other syntax is the same as SCM3.0. In the first row, the copy operation mode is indicated (cross-CU prediction). Note that palette_run_coding() and palette_transpose_flag can be skipped if the maximum index value is equal to 1.

23 shows an exemplary flow diagram of signaling to support index prediction across CUs according to the above-described examples. The flowchart is essentially the same as in Figure 22, except for the case where the maximum index is not 0 (ie, the "no" path from step 2240). In this case, as shown in step 2310, the syntax according to SCM3.0 is used for prediction across CUs.

Syntax Example 4: "all_pixel_from_NCP_flag" in Syntax Example 1 and "any_pixel_from_NCP_flag" in Syntax Example 2 or 3 can be combined into palette_prediction_run. In SCM3.0, palette_prediction_run is the run-length codec. All_pixel_from_NCP_flag or any_pixel_from_NCP_flag is inferred to be on (on) if the first run (ie palette_prediction_run[0]) is equal to a fixed or derived value. The value can be 0 or 1.

Syntax Example 5: As shown in step 2410 of Figure 24, the encoder may flag the palette_share_flag, palette_prediction_run, and num_signaled_palette_entries signals. The palette size can then be derived based on the information.

The palette size is checked to determine if it is equal to 0, as shown in step 2420. If the palette size is greater than 0 (ie, the "NO" path from step 2420), the other syntax is the same as for SCM3.0, as shown in step 2430. In the first row, the copy operation mode may be indicated according to whether it is a cross-CU prediction.

If the palette size is equal to 0 (ie, the "yes" path from step 2420), then signal_from_NCP_flag is flagged. In step 2440 it is checked whether any_pixel_from_NCP_flag is on. If any_pixel_from_NCP_flag is off (ie, "NO" path from step 2440), then palette_escape_val_present_flag is inferred to be on, as shown in step 2450, and the SCM3.0-based syntax is used for index prediction, and cross-CU prediction is not required. If any_pixel_from_NCP_flag is on, flag palette_escape_val_present_flag. If any_pixel_from_NCP_flag is on (ie, "yes" path from step 2440 ) and palette_escape_val_present_flag is off (ie, "no" path from step 2460 ), then all pixels can be predicted from NCP and palette_run_coding( ) can be skipped as in step 2470 shown. If any_pixel_from_NCP_flag is on (ie, the "yes" path from step 2440 ), and the palette_escape_val_present_flag is on, then some pixels are predicted from NCP, and palette_run_coding( ) may be flagged, as shown in step 2480 .

Syntax Example 6: This example is essentially the same as Syntax Example 5, except that any_pixel_from_NCP_flag is on (ie, the "yes" path from step 2440) and palette_escape_val_present_flag is on (ie, the "yes" path from step 2460). In this case, as shown in step 2510 of Figure 25, all pixels are jump indices.

Syntax Example 7: The encoder may flag palette_share_flag, palette_prediction_run, num_signaled_palette_entries, as shown in step 2610 of FIG. 26 . The palette size can then be derived based on the information.

The palette size is checked in step 2620 to determine if it is equal to zero. If the palette size is equal to 0 (ie, "Yes" from step 2620), then flag all_pixel_from_NCP_flag and check in step 2640 whether all_pixel_from_NCP_flag is on. If all_pixel_from_NCP_flag is on (ie, "yes" path from step 2640 ), then all pixels are implied to be predicted from NCP, as shown in step 2660 . In this case, the palette_transpose_flag can be flagged to imply that the prediction is from the left NCP or the upper NCP. Other prediction directions may also be indicated. Otherwise, the syntax is the same as SCM3.0 shown in step 2650. In the first row, the replication mode of operation (ie, prediction across CUs) may be indicated.

Syntax Example 8: In Syntax Example 8, the codec may be marked to run to indicate the situation in FIG. 27 . The flowchart in FIG. 27 is similar to the flowchart in FIG. 26 . However, steps 2630 and 2650 in FIG. 26 are replaced by steps 2710 and 2720 (ie, the flags palette_escape_val_Present_flag, palette_transcope_flag, and palette_run_coding( )).

Syntax Example 9: As shown in step 2810 of Figure 28A, the encoder may send palette_share_flag, palette_reuse_flag(), num_signaled_palette_entries, palette_escape_val_present_flag, and the decoder may calculate the maximum indicator value based on this information. If the maximum index value is equal to 0 or 1 (ie, the "no" path from step 2820), then palette_run_coding() may be skipped. If the maximum index value is equal to 0 (ie, the "NO" path from step 2830), then as shown in step 2850, all pixels are predicted from the NCP. As shown in step 2840, if the maximum index value is greater than 1 (ie, the "yes" path from step 2820), the palette_transpose_flag may be flagged to imply prediction from either the left NCP or the upper NCP. Other prediction directions may also be indicated. If the maximum index value is equal to 1 (ie, the "yes" path from step 2830 ), then all color indices in the CU will be inferred to 0 or jumped out, as shown in step 2860 .

Syntax Example 10: As shown in step 2812 of FIG. 28B, the encoder can flag palette_share_flag, palette_reuse_flag(), num_signaled_palette_entries, palette_escape_val_present_flag, and the decoder can calculate the maximum index value based on this information. If the maximum index value is equal to 0 (ie, the "NO" path from step 2822), then as shown in step 2832, all pixels are predicted from the NCP. If the maximum index value is greater than 0 (ie, the "yes" path from step 2822 ), the palette_transpose_flag may be flagged to imply prediction from either the left NCP or the upper NCP, as shown in step 2842 . Other prediction directions may also be indicated. Palette_run_coding() can be skipped. If the maximum index value is greater than 0, palette_transpose_flag and palette_run_coding() can be marked, as shown in Figure 20.

In the above syntax example, the NCP can be the closest upper row or the closest left column. If the number of NCP rows is greater than 1 (eg, the two closest upper rows or the two closest left columns), additional signaling may be required to indicate which NCP row is used for prediction. In the case of "all pixels predicted from NCP" in the syntax example, the NCP may be restricted to the nearest upper row or the nearest left column.

Although specific syntax elements are used to illustrate syntax examples for supporting index prediction across CUs according to embodiments of the present invention, these specific syntax elements should not be construed as limitations of the present invention. Those skilled in the art may use other syntax elements and semantics to perform index prediction across CUs without departing from the spirit of this disclosure.

Syntax element for enabling index prediction across CUs

For index prediction across CUs, an enable flag may be indicated in PPS (Picture Parameter Set) or SPS (Sequence Parameter Set). Also, flags can be flagged only when palette_mode_enabled_flag in SPS is true. Otherwise it will be inferred to be false. When the enable flag for index prediction across CUs is false, index prediction across CUs is disabled. In another embodiment, when the enable flag for index prediction across CUs is false, adjacent indices or values may be inferred to be predetermined values. For example, the adjacent index of each color component can be set to 0, and the adjacent value can be set to 128. In the present invention, the method of predicting an index or value in a block is not limited to information of adjacent pixels (or blocks).

Context selection of operating mode: In SCM 3.0, the syntax element palette_mode used for index map encoding and decoding is context encoded. palette_mode has two contexts. Context is selected according to palette_mode indexed above. However, for the index of the first row, there is no upper index.

Several methods for handling context selection are exposed in this case:

1. The context of the first line uses a fixed context. This context can be the first context used when the upper index is encoded and decoded by INDEX-RUN, or the second context used when the upper index is encoded and decoded by COPY-run mode.

2. The index of the first row can be the third context.

3. All indices in the CU use the same context to encode and decode palette_mode. The context can be selected according to the CU size, and all indexes in the CU use the same context to encode and decode palette_mode. There can be two contexts in this case. If the CU size is larger than a fixed or derived threshold, the first context is used. Otherwise, use another context. In another example, the number of contexts can be reduced to one. In this case, the context is the same for all CU sizes.

Modified intra prediction scheme

In another embodiment, to achieve the same effect as the prediction schemes disclosed in Figures 19 and 20, based on HEVC, HEVC range extension, or conventional intra prediction in HEVC SCC, the syntax element rqt_root_cbf is marked to indicate the Whether the TU (transform unit) rooted from the current CU has residual. The signaling of rqt_root_cbf may be the same as the signaling of rqt_root_cbf for inter-CU redundancy codec in HEVC.

Based on intra prediction with rqt_root_cbf according to an embodiment of the present invention, rqt_root_cbf may be selectively applied to a subset of intra prediction modes including luma and chroma intra prediction modes. rqt_root_cbf is only used for those intra prediction modes in the subset. In one example, the modification scheme applies only to the luma intra prediction mode equal to the horizontal or vertical prediction mode, while the chroma intra prediction mode is not modified.

In another embodiment, a CU-level flag is used for an intra CU (intra CU) to indicate whether there is a residual signal for encoding and decoding the intra CU. Similarly, this flag can be selectively applied to a subset of intra prediction modes.

Intra Block Copy (IntraBC) search

One embodiment of the present invention changes the source pixels of IntraBC. The pixels used for IntraBC prediction and compensation may be unfiltered pixels (ie, before deblocking) or filtered pixels (ie, after deblocking and SAO (Sampling Adaptive Offset)) depending on the position of the pixel.

For example, as shown in Figure 29, pixels used for IntraBC prediction and compensation may be based on unfiltered pixels of pixels in the current CTU (2910) and the left CTU (2920). Other pixels still use filtered pixels. Figure 29 shows an example of a source pixel according to an embodiment of the present invention, where the dot fill pixels are from unfiltered pixels and the transparent pixels are from filtered pixels for IntraBC.

In another example, as shown in Figure 30, the pixels used for IntraBC prediction and compensation are the current CTU (3010), the left CTU (3020), and the bottom four rows of the upper CTU (3030) and the upper left CTU (3040). Unfiltered pixels for the pixels in the bottom four rows. Other pixels use filtered pixels. In Figure 30, the dot fill pixels are from unfiltered pixels, and the transparent pixels are from filtered pixels for IntraBC.

In another example, as shown in FIG. 31, the pixels used for IntraBC prediction and compensation are the pixels in the current CTU, the N CTUs to the left, and the bottom four rows of the upper CTU and the bottom four rows of the N upper left CTUs. Unfiltered pixels. N is a positive integer. Other pixels use filtered pixels. In Figure 31, the dot-fill pixels are from unfiltered pixels, while the transparent pixels are from IntraBC's filtered pixels.

In another example, the pixels used for IntraBC prediction and compensation are the unfiltered pixels of the pixels in the current CTU, the bottom four rows of the upper CTU, and the right four columns of the left CTU. Other pixels are filtered using as shown in Figure 32. In Figure 32, the dot fill pixels are from unfiltered pixels, and the transparent pixels are from IntraBC's filtered pixels.

In another example, the pixels used for IntraBC prediction and compensation are the current CTU, the N left CTUs, the bottom four rows in the upper CTU and the bottom four rows of the N upper left CTUs, and the (N+1)th left CTU The unfiltered pixels of the pixels in the four columns to the right of the square CTU. N is a positive integer value. Other pixels use filtered pixels. In Figure 33, the dot-fill pixels are from unfiltered pixels, while the transparent pixels are from IntraBC's filtered pixels.

34 shows an exemplary flow diagram of a system for palette codec block sharing transform coefficient buffers incorporating embodiments of the present invention. The system determines the current prediction mode for the current block in step 3410 and designates the storage area as a transform coefficient buffer in step 3420. In step 3430, if the current prediction mode is the intra prediction mode or the inter prediction, store the first information related to the transform coefficient of the prediction residual of the current block generated by the intra prediction or the inter prediction in the transform coefficient buffer . In step 3440, if the current prediction mode is the palette codec mode, information related to palette data associated with the current block is stored in the transform coefficient buffer. In step 3450, the current block is encoded or decoded based on the information related to the transform coefficients if the current block is encoded or decoded in the intra prediction mode or the inter prediction mode, or if the current block is in the modulation plate codec mode For encoding and decoding, the current block is encoded, decoded or decoded based on the information stored in the transform coefficient buffer related to the palette data.

The above description is presented to enable one of ordinary skill in the art to implement the invention in the context of a particular application and its requirements. Various modifications to the described embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments. Therefore, the present invention is not intended to be limited to the specific embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In the foregoing detailed description, various specific details are set forth in order to provide a thorough understanding of the present invention. However, those skilled in the art will understand that the present invention may be practiced.

Embodiments of the present invention as described above may be implemented in various hardware, software codes, or a combination of both. For example, embodiments of the present invention may be one or more electronic circuits integrated into a video compression chip or program code integrated into video compression software to perform the processes described herein. Embodiments of the present invention may also be program code to be executed on a digital signal processor (DSP) to perform the processes described herein. The invention may also relate to many functions performed by a computer processor, digital signal processor, microprocessor, or field programmable gate array (FPGA). These processors may be configured to perform certain tasks in accordance with the present invention by executing machine-readable software code or firmware code that defines the particular methods embodied by the present invention. Software code or firmware code can be developed in different programming languages and in different formats or styles. Software code can also be compiled for different target platforms. However, different code formats, styles and languages of the software code according to the present invention and other ways of configuring the code will not depart from the spirit and scope of the present invention.

The present invention may be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. The described examples are to be considered in all respects only as illustrative and not restrictive. Accordingly, the scope of the invention is to be indicated by the appended claims rather than the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims (2)

1. A video coding method using a plurality of prediction modes including a palette coding mode, the method comprising:

parsing all initial palette predictors of a same color component grouped together in a sequence parameter set, a picture parameter set, or a slice header from a video bitstream at a decoder side or grouping all initial palette predictors of the same color component together at an encoder side; and

decoding or encoding at least one palette coded block within a respective sequence, picture or slice using the initial palette predictor value.

2. A video coding method using a plurality of prediction modes including a palette coding mode, the method comprising:

at the decoder side, parsing all palette predictor entries or palette entries of a current block of the same color component aggregated together from the video bitstream, or aggregating all palette predictor entries or palette entries of the same color component at the encoder side; and

decoding or encoding the current block using a palette predictor consisting of all palette predictor entries or a palette table consisting of all palette entries.

Images