RU2821429C1 - Prevention of redundant signaling in multilayer bit video streams - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to video encoding techniques. Result is achieved by receiving a bit sub-stream having a sequence parameter set (SPS) with a decoded picture buffer (DPB) syntax for an output layer set (OLS) with one single layer; setting the number of layers in the OLS to a value equal to one, based on the fact that the OLS has one single layer; obtaining a DPB syntax structure for OLS with one layer from the SPS, when the number of layers in the OLS is equal to one; and the single layer image is decoded using the DPB syntax to obtain a decoded image.

EFFECT: high efficiency of video encoding.

18 cl, 11 dwg, 1 tbl

Description

Field of technology to which the invention relates

In general, the present disclosure describes techniques for multilayer video bitstreams in video encoding. More specifically, the present disclosure describes techniques for eliminating redundancy at signaling parameters corresponding to layers in a multilayer bitstream in video encoding.

State of the art

The amount of video data required to display even a relatively short video can be significant, which can lead to difficulties in cases where the data must be streamed or otherwise transmitted over a communications network with limited bandwidth. Thus, video data is typically compressed before being transmitted over modern telecommunications networks. Video size can also be an issue when the video is stored in a storage device, as memory resources may be limited. Video compression devices often use software and/or hardware at the source to encode video data before transmission or storage, thereby reducing the amount of data required to represent digital video images. The compressed data is then received at its destination by a video decompressor, which decodes the video data. With limited network resources and ever-increasing demands for higher video quality, improved compression and decompression technologies are desired that improve the compression ratio with little or no sacrifice in image quality.

Disclosure of the invention

The first aspect relates to a decoding method implemented by a video decoder, comprising: receiving, by the video decoder, a bit substream having a sequence parameter set (SPS) with a decoded picture buffer (DPB) syntactic structure for an output layer set (OLS) with one single layer; obtaining, by the video decoder, a DPB syntax structure for the OLS with one layer from the SPS; and decoding, by the video decoder, the image of one single layer using a DPB syntax structure to obtain a decoded image.

The method provides techniques that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the DPB syntactic structure is denoted dpb_parameters().

As an alternative to any of the previous aspects, another implementation of the aspect is provided, wherein the single layer is the i-th layer, and wherein the OLS is the i-th OLS.

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the DPB syntax structure is present in the i-th dpb_parameters() syntax structure in the SPS.

As an alternative to any of the previous aspects, another implementation of the aspect is provided wherein the bitstream does not include a video parameter set (VPS).

As an alternative to any of the previous aspects, another implementation of the aspect involves storing the decoded image in the DPB before display.

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the SPS is referenced by a single layer in the OLS.

As an alternative to any of the previous aspects, another implementation of the aspect involves displaying the decoded image on a display of an electronic device.

The second aspect relates to an encoding method implemented by a video encoder, the method comprising: generating, by the video encoder, a video parameter set (VPS) and a sequence parameter set (SPS), wherein the VPS includes a list of decoded picture buffer (DPB) syntax structures for all sets of layers output (OLS), and where the SPS includes a DPB syntax structure for each OLS having one single layer; encoding, video encoder, VPS and SPS into a bit video stream; and storing, by the video encoder, the video bit stream for transmission to the video decoder.

The method provides techniques that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs.

As an alternative to any of the previous aspects, another implementation of the aspect is provided where the VPS includes an index for a list of DPB syntax structures, and where the index is denoted vps_ols_dpb_params_idx[i].

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the value of vps_ols_dpb_params_idx[ i ] ranges from zero to the total number of DPB syntax structures in the VPS.

As an alternative to any of the previous aspects, another implementation of the aspect is provided wherein the VPS includes vps_num_dpb_params_minus1, and where vps_num_dpb_params_minus1 plus 1 indicates the total number of DPB syntax structures in the VPS.

As an option in any of the previous aspects, another implementation of the aspect involves extracting a subbitstream from the bitstream, wherein the subbitstream includes an SPS from the bitstream but does not include a VPS from the bitstream.

As an alternative to any of the previous aspects, another implementation of the aspect provides for transmitting the bit substream to a video decoder.

The third aspect relates to a decoding apparatus comprising: a receiver configured to receive a bit substream having a sequence parameter set (SPS) with a decoded picture buffer (DPB) syntax structure for an output layer set (OLS) with one single layer; and a processor coupled to the memory, wherein the processor is configured to execute instructions that instruct the decoding device to: set the number of layers in the OLS to a value equal to one based on the fact that the OLS has one single layer; obtain the syntactic structure of DPB for OLS with one layer from SPS when the number of layers in OLS is one; and decode the image of one single layer using a DPB syntax structure to obtain a decoded image.

The decoder provides technologies to enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

As an alternative to any of the previous aspects, another implementation of the aspect is provided wherein the syntactic structure of the DPB is denoted dpb_parameters(), where the single layer is the i-th layer, and where the OLS is the i-th OLS.

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the DPB syntax structure is present in the i-th dpb_parameters() syntax structure in the SPS.

As an alternative to any of the previous aspects, another implementation of the aspect is provided wherein the subbitstream does not include a video parameter set (VPS).

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the SPS is referenced by a single layer in the OLS.

As an option in any of the previous aspects, another implementation of the aspect provides a display configured to display a decoded image.

The fourth aspect relates to an encoding device comprising: a memory containing instructions; a processor coupled to the memory, wherein the processor is configured to implement instructions that instruct the encoding device to: generate a video parameter set (VPS) and a sequence parameter set (SPS), where the VPS includes a list of decoded picture buffers (DPBs). ) syntax structures for all output layer sets (OLS), where the SPS includes a DPB syntax structure for each OLS having one single layer; and encode VPS and SPS into a video bitstream; and a transmitter coupled to the processor, wherein the transmitter is configured to transmit the video bit stream to the video decoder.

The encoding apparatus provides techniques that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

As an alternative to any of the previous aspects, another implementation of the aspect is provided where the VPS includes an index for a list of DPB syntax structures, and where the index is denoted vps_ols_dpb_params_idx[i].

As an alternative to any of the previous aspects, another implementation of the aspect is provided in which the value of vps_ols_dpb_params_idx[ i ] ranges from zero to the total number of DPB syntax structures in the VPS.

As an alternative to any of the previous aspects, another implementation of the aspect is provided wherein the VPS includes vps_num_dpb_params_minus1, and where vps_num_dpb_params_minus1 plus 1 indicates the total number of DPB syntax structures in the VPS.

The fifth aspect relates to the encoding device. The encoding device includes a receiver configured to receive an image for encoding or receive a bitstream for decoding; a transmitter connected to the receiver, wherein the transmitter is configured to transmit a bit stream to a decoder or transmit a decoded image to a display; a memory connected to at least one of: a receiver or a transmitter, the memory being configured to store instructions; and a processor coupled to the memory, the processor configured to execute instructions stored in the memory to perform any of the methods disclosed herein.

The encoding apparatus provides techniques that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

As an option in any of the previous aspects, another implementation of the aspect provides a display configured to display a decoded image.

The sixth aspect relates to the system. The system includes an encoder; and a decoder operative with the encoder, wherein the encoder or decoder includes a decoding apparatus, an encoding apparatus, or an encoding apparatus disclosed herein.

The system provides technologies that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

The seventh aspect relates to the encoding medium. The encoding means includes receiving means configured to receive an image for encoding or receive a bit stream for decoding; transmission means connected to the reception means, wherein the transmission means is configured to transmit a bit stream to the decoding means or transmit a decoded image to the display means; storage means connected to at least one of: receiving means or transmitting means, wherein the storage means is configured to store instructions; and processing means connected to the storage means, wherein the processing means is configured to execute instructions stored in the storage means to perform any of the methods disclosed herein.

The encoding engine provides techniques that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

For purposes of clarity, any of the above embodiments may be combined with any one or more other embodiments above to create a new embodiment within the scope of this disclosure.

These and other features will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings and claims.

Brief description of drawings

For a more complete understanding of the present disclosure, reference is now made to the following brief description, taken in conjunction with the accompanying drawings and detailed description, in which like reference numerals denote like parts.

Fig. 1 is a flowchart of an exemplary method for encoding a video signal.

Fig. 2 is a schematic diagram of an exemplary encoding and decoding system (codec) for video encoding.

Fig. 3 is a schematic diagram illustrating an exemplary video encoder.

Fig. 4 is a schematic diagram illustrating an exemplary video decoder.

Fig. 5 is an example of multilayer coding for spatial scalability.

Fig. 6 is an example of multi-layer encoding using output layer sets (OLS).

Fig. 7 is an embodiment of a video bit stream.

Fig. 8 illustrates an embodiment of a method for decoding an encoded video bit stream.

Fig. 9 is an embodiment of a method for encoding a coded video bit stream.

Fig. 10 is a schematic diagram of a video encoding apparatus.

Fig. 11 is a schematic diagram of an embodiment of an encoding means.

Carrying out the invention

First, it should be noted that while an illustrative implementation of one or more embodiments is presented below, the disclosed systems and/or methods may be implemented using any number of technologies currently known or existing. The present disclosure is not to be limited in any way to the illustrative implementations, drawings and techniques illustrated below, including the illustrative designs and implementations illustrated and described herein, but may be varied within the scope of the appended claims along with their full scope of equivalents.

The following terms are defined as follows unless otherwise used herein. In particular, the following definitions are intended to provide further clarity of this disclosure. However, the terms may be described differently in different contexts. Accordingly, the definitions below should be considered in addition to, and should not be construed as limiting, any other definitions of the descriptions provided for such terms herein.

A bitstream is a sequence of bits comprising video data that is compressed for transmission between an encoder and a decoder. An encoder is a device that is configured to use encoding processes to compress video data into a bitstream. A decoder is a device that is configured to use decoding processes to recover video data from a bitstream for display. An image is an array of luma samples and/or an array of chrominance samples that create a frame or region of it. The image that is being encoded or decoded may be referred to as the current image for clarity of discussion. A reference picture is an image that contains reference samples that can be used when encoding other pictures by reference in accordance with inter-picture prediction and/or inter-layer prediction. The reference picture list is a list of reference pictures used for inter-picture prediction and/or inter-layer prediction. Some video encoding systems use two reference picture lists, which can be referred to as reference picture list 1 (one) and reference picture list 0 (zero). The reference picture list structure is an addressable syntactic structure that contains multiple reference picture lists.

Inter-frame prediction is a mechanism for encoding samples of a current image by referring to said samples in a reference image that is different from the current image, where the reference image and the current image are in the same layer. A reference picture list structure entry is an addressable cell in the reference picture list structure that specifies a reference picture associated with the reference picture list.

The slice header is part of an encoded slice containing data elements related to all video data in the tile represented in the slice. A picture parameter set (PPS) is a set of parameters that contains data related to the entire image. More specifically, a PPS is a syntax structure containing syntax elements that apply to zero or more complete encoded pictures, as determined by a syntax element found in each picture header. A sequence parameter set (SPS) is a set of parameters that contains data related to a sequence of images. An access unit (AU) is a set of one or more encoded pictures associated with the same display time (e.g., the same picture queue counter) for output from a decoded picture buffer (DPB) (e.g., for display to a user) . An Access Unit Delimiter (AUD) is an indicator or data structure used to indicate the start of an AU or the boundary between AUs. The decoded video sequence is a sequence of images reconstructed by the decoder in preparation for display to the user.

A Network Abstraction Layer (NAL) unit is a syntactic structure containing raw byte sequence payload (RBSP) data, a data type indicator, and optionally interleaved emulation prevention bytes. A video coding layer (VCL) NAL unit is a NAL unit encoded to contain video data, such as a coded image slice. A non-VCL NAL unit is a NAL unit that contains non-video data, such as syntax and/or parameters that support decoding video data, performing conformance checking, or other operations. A layer is a collection of VCL NAL units that share a given characteristic (eg, overall resolution, frame rate, image size, etc.) and associated non-VCL NAL units. Layer VCL NAL units can share a specific NAL unit header layer identifier value (nuh_layer_id). A coded picture is a coded representation of the picture containing VCL NAL units with a specific NAL unit header layer identifier value (nuh_layer_id) in the access unit (AU) and containing all the coding tree units (CTUs) of the picture. A decoded image is an image obtained by applying a decoding process to an encoded image.

An output layer set (OLS) is a layer set for which one or more layers are defined as one or more output layers. The output layer is a layer intended for output (for example, to a display). A null (0th) OLS is an OLS that contains only the bottom layer (the layer with the bottom layer ID) and therefore only contains the output layer. The number of layers in an OLS can be indicated by a syntax element denoted NumLayersInOLS. A video parameter set (VPS) is a block of data that contains parameters relevant to the entire video. Inter-layer prediction is a mechanism for encoding the current image in the current layer by referring to a reference image in the reference layer, where the current image and the reference image are included in the same AU, and the reference layer includes a lower nuh_layer_id than the current layer.

A profile is a specific set of encoding tools used to create a compatible or conforming bitstream. Each profile defines a subset of algorithmic features and constraints that must be supported by all decoders conforming to that profile. Ranks and levels are constraints that define a bitstream in terms of maximum bit rate, maximum luminance sampling rate, maximum luminance image size, minimum compression ratio, maximum number of slices allowed, and maximum number of tiles allowed. Lower layers are constrained to a greater extent than higher layers, and lower layers are constrained to a greater extent than higher layers. Profile, Rank, and Level (PTL) parameters are a syntactic structure that provides profile, rank, and/or level information.

A time-scaled bitstream is a bitstream encoded in multiple layers providing different temporal resolution/frame rates (eg, each layer is encoded to support a different frame rate). The sublayer is a time-scaled time-scaled bitstream layer including VCL NAL units with a specific temporal identifier value and associated non-VCL NAL units. For example, a temporal sublayer is a layer that contains video data associated with a given frame rate. A sublayer representation is a subset of a bitstream containing the NAL units of a particular sublayer and lower sublayers. Thus, one or more temporal sublayers can be combined to achieve a sublayer representation that can be decoded to obtain a video sequence at a given frame rate. An OLS index is an index that uniquely identifies the corresponding OLS. A temporal identifier (ID) is a data element indicating that the data corresponds to a temporal location in a video sequence. The substream bitstream extraction process is a process that removes NAL units from the bitstream that do not belong to the target set, as determined by the target OLS index and the target highest temporary identifier. The result of the bitstream extraction process is an output bitstream containing the NAL units from the bitstream that are part of the target set.

A hypothetical reference decoder (HRD) is a decoder model running within an encoder that examines the variability of the bitstreams produced by the encoding process to test against specified constraints. A bitstream conformance test is a test to determine whether an encoded bitstream conforms to a standard such as Versatile Video Coding (VVC). HRD parameters are syntactic elements that initiate and/or define HRD operating conditions. Sequence-level HRD parameters are HRD parameters that apply to the entire encoded video sequence. The maximum HRD temporal identifier (ID) (hrd_max_tid[ i ]) specifies the temporal ID of the top layer view for which the HRD parameters are contained in the i-th OLS HRD parameter set. The syntactic structure of the general HRD parameters (general_hrd_parameters) is a syntactic structure that contains the HRD parameters at the sequence level. An operating point (OP) is a temporary subset of the OLS that is identified by the OLS index and the largest temporary ID. The OP under test (targetOp) is the OP selected for compliance testing in the HRD. The target OLS is the OLS selected to be extracted from the bitstream. The decoding unit HRD parameters present flag (decoding_unit_hrd_params_present_flag) is a flag that indicates whether the corresponding HRD parameters operate at the decoding unit (DU) level or at the AU level. The Coded Picture Buffer (CPB) is a reverse store buffer in HRD that holds coded pictures in decoding order for use during bitstream consistency checking. A decoded picture buffer (DPB) is a buffer for storing decoded pictures for reference, output reordering, and/or output delay.

Decoded picture buffer (DPB) parameters are a syntactic structure that provides the size of the DPB and optionally information about the maximum number of picture reorders and maximum delay. Information about the maximum number of image reorders and maximum delay can be collectively abbreviated as MRML.

The following abbreviations are used in this document: Coding Tree Block (CTB), Coding Tree Unit (CTU), Coding Unit (CU), Coded Video Sequence (CVS), Joint Video Expert Team (JVET), Motion Constrained Tile Set (MCTS) ), Maximum Transmission Unit (MTU), Network Abstraction Layer (NAL), Picture Queue Counter (POC), Picture Parameter Set (PPS), Raw Byte Sequence Payload (RBSP), Sequence Parameter Set (SPS), Universal Video Coding ( VVC) and working draft (WD).

In fig. 1 is a flowchart of an exemplary video encoding method 100. In particular, the video signal is encoded in the encoder. The encoding process compresses the video signal using various mechanisms to reduce the size of the video file. The smaller file size allows the compressed video file to be transmitted to the user while reducing the associated bandwidth overhead. The decoder then decodes the compressed video file to reconstruct the original video signal for display to the end user. The decoding process typically repeats the encoding process, allowing the decoder to reconstruct the video signal sequentially.

At step 101, the video signal is input to the encoder. For example, the video signal may be an uncompressed video file stored in memory. As another example, a video file may be captured by a video capture device, such as a video camera, and encoded to support live video streaming. A video file may include both an audio component and a video component. The video component contains a series of image frames that, when viewed in sequence, create the visual impression of movement. Frames contain pixels that are expressed in terms of light, referred to herein as luminance components (or luminance samples), and color, which is referred to as chrominance components (or color samples). In some examples, frames may also contain depth values to support 3D viewing.

At step 103, the video is divided into blocks. Division involves dividing the pixels in each frame into square and/or rectangular blocks for compression. For example, in High Efficiency Video Coding (HEVC) (also known as H.265 and MPEG-H part 2), a frame may first be divided into coding tree units (CTUs), which are blocks of a given size (for example, sixty-four pixels by sixty four pixels). CTUs contain both luma and chrominance samples. Encoding trees can be used to divide a CTU into blocks, and then recursively subdivide the blocks until configurations that support further encoding are reached. For example, the luminance components of a frame may be separated until the individual blocks contain relatively uniform light values. Additionally, the chrominance components of a frame can be separated until the individual blocks contain relatively uniform color values. Accordingly, the separation mechanisms differ depending on the content of the video frames.

At step 105, various compression mechanisms are used to compress the image blocks divided at step 103. For example, inter-frame prediction and/or intra-frame prediction may be used. Inter-frame prediction is designed to take advantage of the fact that objects in the overall scene tend to appear in consecutive frames. Accordingly, a block representing an object in a reference frame does not need to be re-described in adjacent frames. In particular, an object such as a table may remain in a constant position for several frames. Therefore, the table is described once, and neighboring frames can refer to the reference frame. Pattern matching engines can be used to match objects across multiple frames. In addition, moving objects can be represented in multiple frames, for example due to object movement or camera movement. As a specific example, a video might show a car moving on screen for several frames. Motion vectors can be used to describe such motion. A motion vector is a two-dimensional vector that provides a displacement of the coordinates of an object in the frame to the coordinates of the object in the reference frame. Thus, inter-frame prediction allows an image block in the current frame to be encoded as a set of motion vectors indicating an offset from the corresponding block in the reference frame.

In intra-frame prediction, blocks are encoded in a common frame. Intra-frame prediction takes advantage of the fact that luma and chrominance components tend to cluster together within a frame. For example, a patch of greenery in a part of a tree tends to be located next to similar patches of greenery. Intra-frame prediction uses several directional prediction modes (e.g., thirty-three in HEVC), planar mode, and direct current (DC) mode. Directional modes indicate that the current block is similar/same as the samples of the neighboring block in the corresponding direction. Planar mode indicates that a sequence of blocks along rows/columns (eg, in a plane) can be interpolated based on adjacent blocks at the edges of the row. Planar mode essentially refers to a smooth transition of light/color across a row/column by using a relatively constant slope as values change. The DC mode is used to smooth edges and indicates that a block is the same as the average value associated with samples of all neighboring blocks associated with the angular directions of the directional prediction modes. Accordingly, the intra-frame prediction blocks may represent the image blocks as different relative prediction mode values instead of actual values. In addition, inter-frame prediction blocks may represent image blocks as motion vector values instead of actual values. However, in some cases, prediction blocks may not accurately represent image blocks. Any differences are stored in residual blocks. Transformations can be applied to the residual blocks to further compress the file.

At step 107, various filtering technologies may be used. In HEVC, filters are applied according to an in-loop filtering scheme. The block-based prediction discussed above can result in blocky images being produced in the decoder. In addition, the block-based prediction scheme allows a block to be encoded, and then the encoded block to be recovered for later use as a reference block. The in-loop filtering scheme iteratively applies noise reduction filters, deblocking filters, adaptive loop filters, and sample adaptive offset (SAO) filters to blocks/frames. These filters reduce such blocking artifacts so that the encoded file can be accurately reconstructed. In addition, these filters reduce artifacts in the reconstructed reference blocks, so that artifacts are less likely to create additional artifacts in subsequent blocks that are encoded based on the reconstructed reference blocks.

After the video signal has been separated, compressed, and filtered, the resulting data is encoded into a bitstream at step 109. The bitstream includes the data discussed above as well as any signaling data necessary to support proper reconstruction of the video signal at the decoder. For example, such data may include partition data, prediction data, residual blocks, and various flags providing encoding instructions to the decoder. The bit stream can be stored in memory for transmission to the decoder on demand. The bitstream may also be broadcast and/or multicast to a plurality of decoders. Creating a bitstream is an iterative process. Accordingly, steps 101, 103, 105, 107 and 109 may be performed continuously and/or simultaneously across multiple frames and blocks. The order shown in FIG. 1 is presented for clarity and ease of discussion and is not intended to limit the video encoding process to a particular order.

The decoder receives the bit stream and begins the decoding process at step 111. Specifically, the decoder uses an entropy decoding scheme to convert the bit stream into appropriate syntax and video data. The decoder uses syntax data from the bitstream to determine partitions for frames at step 111. The partitioning should match the results of block partitioning at step 103. The entropy encoding/decoding used at step 111 will now be described. During the compression process, the encoder makes a selection from various options, for example, selects block partitioning schemes from several possible options based on the spatial positioning of values in the input image(s). Precise selection signaling can use a large number of bins. As used herein, a bean is a binary value that is treated as a variable (eg, a bit value that can change depending on the context). Entropy coding allows the encoder to discard any options that are clearly not suitable for a particular case, leaving a set of valid options. Each valid option is then assigned a code word. The length of the codewords depends on the number of valid options (for example, one bin for two options, two bins for three or four options, etc.). The encoder then encodes the codeword for the selected option. This scheme reduces the size of codewords because codewords are as large as necessary to unambiguously indicate a choice from a small subset of valid options, as opposed to unambiguously indicating a choice from a potentially large set of all possible options. The decoder then decodes the selected option, determining the set of valid options in a similar manner to an encoder. Having determined the set of valid options, the decoder can read the codeword and determine the choice made by the encoder.

At step 113, the decoder performs block decoding. In particular, the decoder uses inverse transforms to generate residual blocks. The decoder then uses the residual blocks and corresponding prediction blocks to reconstruct the image blocks according to the division. The prediction blocks may include both intra-prediction blocks and inter-prediction blocks generated by the encoder at step 105. The reconstructed image blocks are then placed into frames of the reconstructed video signal in accordance with the partitioning data determined at step 111. The syntax at step 113 may also be denoted in the bitstream using entropy encoding as discussed above.

At step 115, frames of the reconstructed video signal are filtered in a manner similar to step 107 in the encoder. For example, noise reduction filters, deblocking filters, adaptive edge filters, and SAO filters can be applied to frames to remove blocking artifacts. Once the frames are filtered, the video signal can be displayed at step 117 for viewing by the end user.

In fig. 2 shows a schematic diagram of an exemplary encoding and decoding (codec) system 200 for video encoding. In particular, the encoding/decoding system 200 provides functionality to support the implementation of the method 100 of operation. The encoding/decoding system 200 is generalized to display components used in both the encoder and the decoder. The encoding/decoding system 200 receives and splits the video signal as described with respect to steps 101 and 103 in the operating method 100, resulting in a divided video signal 201. The encoding/decoding system 200 then compresses the divided video signal 201 into an encoded bit stream, acting as an encoder , as described with respect to steps 105, 107 and 109 in the method 100. When operating as a decoder, the encoding/decoding system 200 produces a video output signal from the bitstream, as described with respect to steps 111, 113, 115 and 117 in the method 100 of operation. The encoding/decoding system 200 includes a general encoder control component 211, a upscaling and quantization component 213, an intra-frame estimation component 215, an intra-frame prediction component 217, a motion compensation component 219, a motion estimation component 221, a scaling and inverse transform component 229, a 227 filter control analysis, in-loop filters component 225, decoded image buffer component 223, and header formatting and context adaptive binary arithmetic coding (CABAC) component 231. Such components are connected as shown in the drawing. In fig. 2, black lines indicate the movement of data to be encoded/decoded, and dotted lines indicate the movement of control data that controls the operation of other components. All components of the encoding/decoding system 200 may be present in the encoder. The decoder may include a subset of components of the encoding/decoding system 200. For example, the decoder may include an intra-frame prediction component 217, a motion compensation component 219, a scaling and demapping component 229, an in-loop filter component 225, and a decoded image buffer component 223. These components will now be described.

The divided video signal 201 is a captured video sequence that has been divided into blocks of pixels using a coding tree. The coding tree uses different partitioning modes to divide a block of pixels into smaller blocks of pixels. These blocks can then be divided into even smaller blocks. The blocks may be called nodes of the coding tree. Larger parent nodes are split into smaller child nodes. The number of times a node is subdivided is called the depth of the node/encoding tree. In some cases, the divided blocks may be included in coding units (CUs). For example, a CU may be part of a CTU that contains a luminance block, a red chrominance (Cr) block(s), and a blue chrominance (Cb) block(s), along with corresponding syntax instructions for the RK. Partitioning modes may include binary tree (BT), ternary tree (TT), and quadtree (QT), used to split a node into two, three, or four child nodes, respectively, of varying shapes depending on the partitioning modes used. The divided video signal 201 is sent to a general encoder control component 211, a scaling and quantization transform component 213, an intra-frame estimation component 215, a filter control analysis component 227, and a motion estimation component 221 for compression.

The general encoder control component 211 is configured to make decisions associated with encoding images of a video sequence into a bitstream in accordance with application constraints. For example, the general encoder control component 211 controls the optimization of the bit rate/bit stream size depending on the reconstruction quality. Such decisions can be made based on disk space/bandwidth availability and image resolution requests. The general encoder control component 211 also controls buffer usage in light of the bit rate to mitigate buffer flushing and overflow problems. To cope with these problems, the general encoder control component 211 controls the division, prediction, and filtering with the help of other components. For example, the general encoder control component 211 may dynamically increase compression complexity to increase resolution and increase bandwidth utilization, or decrease compression complexity to decrease resolution and bandwidth utilization. Therefore, the general encoder control component 211 controls the other components of the encoding/decoding system 200 to balance the quality of the video signal reconstruction with bit rate issues. The encoder general control component 211 creates control data that controls the operation of other components. The control data is also sent to header and CABAC formatting component 231 for encoding into a bitstream to signal parameters for decoding at the decoder.

The divided video signal 201 is also sent to the motion estimation component 221 and the motion compensation component 219 for inter-frame prediction. A frame or portion of the divided video signal 201 may be divided into multiple video blocks. The motion estimation component 221 and the motion compensation component 219 perform interframe predictive coding of a received video block with respect to one or more blocks in one or more reference frames to provide temporal prediction. The encoding/decoding system 200 may perform multiple encoding passes, for example, to select an appropriate encoding mode for each block of video data.

The motion estimation component 221 and the motion compensation component 219 may be highly integrated, but are illustrated separately for conceptual purposes. The motion estimation performed by the motion estimation component 221 is a motion vector generation process in which motion is estimated for video blocks. For example, the motion vector may indicate the offset of the encoded object relative to the prediction block. A prediction block is a block that exactly matches the block being coded in terms of pixel difference. The prediction block may also be referred to as a reference block. Such pixel difference may be determined by sum of absolute difference (SAD), sum of squared differences (SSD), or other difference measures. HEVC uses several coded entities, including CTUs, coding tree blocks (CTBs), and CUs. For example, CTUs can be divided into CTBs, which can then be divided into CBs to be included in a CU. The CU may be encoded as a prediction unit (PU) containing prediction data and/or a transformation unit (TU) containing the transformed residual data for the CU. Motion estimation component 221 generates motion vectors, PUs and TUs, using velocity-distortion analysis as part of the velocity-distortion optimization process. For example, motion estimation component 221 may determine multiple reference blocks, multiple motion vectors, etc. for the current block/frame and can select reference blocks, motion vectors, etc. that have the best speed-distortion characteristics. The best rate-distortion performance balances video restoration quality (eg, amount of data lost during compression) with encoding efficiency (eg, final encoding size).

In some examples, encoding/decoding system 200 may calculate values for sub-integer pixel positions of reference images stored in decoded image buffer component 223. For example, video encoding/decoding system 200 may interpolate quarter pixel position values, eighth pixel positions, or other fractional pixel positions of a reference image. Thus, the motion estimation component 221 can search for motion with respect to whole pixel positions and fractional pixel positions and output a motion vector with fractional pixel precision. Motion estimation component 221 calculates a motion vector for a video block PU in an inter-frame coded slice by comparing the position of the PU with the position of a reference picture prediction block. The motion estimation component 221 outputs the calculated motion vector as motion data for header formatting, and the CABAC component 231 for encoding and motion to the motion compensation component 219.

The motion compensation performed by the motion compensation component 219 may include sampling or generating a prediction block based on the motion vector determined by the motion estimation component 221 . Again, in some examples, motion estimation component 221 and motion compensation component 219 may be functionally integrated. After receiving the motion vector for the PU of the current video block, motion compensation component 219 can locate the prediction block pointed to by the motion vector. A residual video block is then generated by subtracting the pixel values of the prediction block from the pixel values of the current video block being encoded, generating pixel difference values. In general, motion estimation component 221 performs motion estimation with respect to luminance components, and motion compensation component 219 uses motion vectors calculated based on luminance components for both chrominance and luminance components. The prediction block and the residual block are sent to the scaling and quantization transform component 213.

The divided video signal 201 is also sent to the intra-frame estimation component 215 and the intra-frame prediction component 217. As with motion estimation component 221 and motion compensation component 219, intra-frame estimation component 215 and intra-frame prediction component 217 may be highly integrated, but they are illustrated separately for conceptual purposes. The intra-frame estimation component 215 and the intra-frame prediction component 217 intra-frame predict the current block relative to blocks in the current frame as an alternative to the inter-frame prediction performed by the motion estimation component 221 and the inter-frame motion compensation component 219 as described above. In particular, intra-frame estimation component 215 determines the intra-frame prediction mode that is used to encode the current block. In some examples, intra-frame estimation component 215 selects an appropriate intra-prediction mode for encoding the current block from several tested intra-prediction modes. The selected intra-prediction modes are then sent to the header formatting component and CABAC 231 for encoding.

For example, intra-frame estimation component 215 calculates rate-distortion values using rate-distortion analysis for various tested intra-prediction modes and selects the intra-prediction mode having the best rate-distortion characteristics among the tested modes. Typically, rate-distortion analysis determines the degree of distortion (or error) between a coded block and the original uncoded block that was coded to produce the coded block, as well as the bit rate (eg, number of bits) used to produce the coded block. Intra-frame estimation component 215 calculates distortion-rate ratios for various coded blocks to determine which intra-prediction mode exhibits the best rate-distortion value for the block. In addition, the intra-frame estimation component 215 may be configured to encode the depth blocks of the depth map using a depth modeling mode (DMM) based on rate-distortion optimization (RDO).

The intra-frame prediction component 217 may produce a residual block from the prediction block based on selected intra-prediction modes determined by the intra-frame estimation component 215 when implemented in an encoder, or read the residual block from a bitstream when implemented in a decoder. The residual block includes the difference in values between the prediction block and the original block, represented as a matrix. The residual block is then sent to the scaling and quantization component 213 . The intra-frame estimation component 215 and the intra-frame prediction component 217 may operate on both luma and chrominance components.

The scaling and quantization component 213 is configured to further compress the residual block. The scaling and quantization transform component 213 applies a transform, such as a discrete cosine transform (DCT), a discrete sine transform (DST), or a conceptually similar transform, to the residual block, creating a video block containing the residual values of the transform coefficients. In addition, wavelet transforms, integer transforms, subband transforms, or other types of transforms can also be used. The transform allows the residual information to be converted from the pixel value domain to a transform domain such as the frequency domain. The scaling and quantization component 213 is also configured to scale the transformed residual information, for example, based on frequency. Such scaling involves applying a scaling factor to the residual information such that information at different frequencies is quantized to different degrees of detail, which can affect the final visual quality of the reconstructed video. The scaling and quantization component 213 is also configured to quantize the transform coefficients to further reduce the bit rate. The quantization process allows the bit depth associated with some or all of the coefficients to be reduced. The degree of quantization can be changed by adjusting the quantization parameter. In some examples, the scaling and quantization component 213 may then scan a matrix including the quantized transform coefficients. The quantized transform coefficients are sent to the header formatting component and CABAC 231 for encoding into a bitstream.

The scaling and inverse transform component 229 applies the inverse operation of the scaling and quantizing transform component 213 to support motion estimation. The scaling and detransforming component 229 applies descaling, transforming, and/or quantization to reconstruct a residual block in the pixel domain, for example, for later use as a reference block that can become a prediction block for another current block. Motion estimation component 221 and/or motion compensation component 219 may calculate a reference block by adding a residual block back to the corresponding prediction block for use in motion estimation of a later block/frame. Filters are applied to the reconstructed reference blocks to remove artifacts that occur during scaling, quantization, and transformation. Otherwise, such artifacts may cause inaccurate prediction (and create additional artifacts) when predicting subsequent blocks.

Filter control analysis component 227 and in-loop filter component 225 apply filters to residual blocks and/or reconstructed image blocks. For example, the transformed residual block from the upscaling and demapping component 229 may be combined with the corresponding prediction block from the intra-frame prediction component 217 and/or the motion compensation component 219 to restore the original image block. Filters can then be applied to the reconstructed image block. In some examples, filters may be applied to residual blocks instead. As with the other components shown in FIGS. 2, the filter control analysis component 227 and the in-loop filter component 225 are highly integrated and may be implemented together, but are depicted separately for conceptual purposes. Filters applied to reconstructed reference blocks apply to specific areas of space and include several parameters to control how those filters are applied. The filter control analysis component 227 analyzes the reconstructed reference blocks to determine where such filters should be applied and sets the appropriate parameters. Such data is sent to header formatting component 231 and CABAC as filter control data for encoding. The in-loop filter component 225 applies such filters based on filter control data. The filters may include a deblocking filter, a noise reduction filter, a SAO filter, and an adaptive loop filter. Depending on the example, such filters may be applied in the spatial/pixel domain (eg, to a reconstructed block of pixels) or in the frequency domain.

When operating as an encoder, the filtered reconstructed image block, residual block, and/or prediction block are stored in decoded image buffer component 223 for later use in motion estimation, as discussed above. When operating as a decoder, the decoded image buffer component 223 stores and forwards the reconstructed and filtered blocks to the display as part of the output video signal. The decoded image buffer component 223 may be any memory device capable of storing prediction blocks, residual blocks, and/or reconstructed image blocks.

The header formatting and CABAC component 231 receives data from various components of the encoding/decoding system 200 and encodes such data into a coded bitstream for transmission to the decoder. In particular, the header and CABAC formatting component 231 produces various headers for encoding control data such as general control data and filter control data. In addition, prediction data, including intra-frame prediction and motion data, as well as residual data in the form of quantized transform coefficient data, are encoded entirely into a bitstream. The final bitstream includes all information necessary for the decoder to reconstruct the original split video signal 201. Such information may also include intra-frame prediction mode index tables (also called codeword mapping tables), definitions of encoding contexts for various blocks, indications of the most likely modes intra-frame prediction, indication of partition information, etc. Such data can be encoded using entropy coding. For example, the information may be encoded using context adaptive variable length coding (CAVLC), CABAC based on context adaptive binary arithmetic coding (SBAC) syntax, probabilistic interframe entropy encoding (PIPE), or other entropy encoding technique. After entropy encoding, the encoded bitstream can be transferred to another device (such as a video decoder) or archived for later transmission or retrieval.

In fig. 3 is a block diagram illustrating an exemplary video encoder 300. The video encoder 300 may be used to implement encoding functions of the encoding/decoding system 200 and/or implement steps 101, 103, 105, 107 and/or 109 of the method 100 of operation. Encoder 300 splits the input video signal, resulting in split video 301, which is substantially the same as split video 201. The split video 301 is then compressed and encoded into a bitstream by components of encoder 300.

Specifically, the divided video signal 301 is sent to the intra prediction component 317 for intra prediction. The intra-frame prediction component 317 may be substantially similar to the intra-frame estimation component 215 and the intra-frame prediction component 217 . The divided video signal 301 is also sent to motion compensation component 321 for inter-frame prediction based on reference blocks in decoded image buffer component 323. The motion compensation component 321 may be substantially similar to the motion estimation component 221 and the motion compensation component 219 . The prediction blocks and residual blocks from the intra-frame prediction component 317 and the motion compensation component 321 are sent to the transform and quantize component 313 to transform and quantize the residual blocks. The transform and quantization component 313 may be substantially similar to the scaling and quantization component 213 . The transformed and quantized residual blocks and corresponding prediction blocks (along with corresponding control data) are sent to entropy encoding component 331 for encoding into a bitstream. The entropy encoding component 331 may be substantially similar to the header formatting component 231 and CABAC.

The transformed and quantized residual blocks and/or corresponding prediction blocks are also sent from the transform and quantization component 313 to the inverse transform and quantization component 329 for reconstruction into reference blocks for use by the motion compensation component 321 . The demapping and quantization component 329 may be substantially similar to the scaling and demapping component 229 . The in-loop filters in the in-loop filter component 325 are also applied to the residual blocks and/or recovered reference blocks, depending on the example. The in-loop filter component 325 may be substantially similar to the filter control analysis component 227 and the in-loop filter component 225 . The in-loop filter component 325 may include a plurality of filters, as described with respect to the in-loop filter component 225. The filtered blocks are then stored in the decoded image buffer component 323 for use as reference blocks by the motion compensation component 321 . The decoded image buffer component 323 may be substantially similar to the decoded image buffer component 223.

In fig. 4 is a block diagram illustrating an exemplary video decoder 400. The video decoder 400 may be used to implement decoding functions of the encoding/decoding system 200 and/or implement steps 111, 113, 115 and/or 117 of the method 100 of operation. Decoder 400 receives a bit stream, for example, from encoder 300, and produces a reconstructed video output based on the bit stream for display to an end user.

The bitstream is received by entropy decoding component 433. The entropy decoding component 433 is configured to implement an entropy decoding scheme such as CAVLC, CABAC, SBAC, PIPE, or other entropy encoding technologies. For example, entropy decoding component 433 may use header information to provide context for interpreting additional data encoded as codewords into the bitstream. The decoded information includes any required information for decoding a video signal, such as general control data, filter control data, division information, motion data, prediction data, and quantized transform coefficients from residual blocks. The quantized transform coefficients are sent to the inverse transform and quantization component 429 for reconstruction into residual blocks. The inverse transform and quantization component 429 may be similar to the inverse transform and quantization component 329 .

The reconstructed residual blocks and/or prediction blocks are sent to intra-prediction component 417 for restoration into image blocks based on intra-prediction operations. The intra-prediction component 417 may be similar to the intra-frame estimation component 215 and the intra-prediction component 217 . Specifically, intra-prediction component 417 uses prediction modes to detect a reference block in a frame and applies a residual block to the result to reconstruct intra-predicted image blocks. The reconstructed intra-frame predicted image blocks and/or residual blocks and corresponding inter-prediction data are sent to the decoded image buffer component 423 through the in-loop filter component 425, which may be substantially similar to the decoded image buffer component 223 and to the in-loop filter component 225, respectively. The in-loop filter component 425 filters the reconstructed image blocks, residual blocks, and/or prediction blocks, and such information is stored in the decoded image buffer component 423. The reconstructed image blocks from the decoded image buffer component 423 are sent to the motion compensation component 421 for inter-frame prediction. Motion compensation component 421 may be substantially similar to motion estimation component 221 and/or motion compensation component 219 . Specifically, motion compensation component 421 uses motion vectors from the reference block to form a prediction block and applies the residual block to the result to reconstruct the image block. The resulting reconstructed blocks may also be forwarded through the in-loop filter component 425 to the decoded image buffer component 423. The decoded image buffer component 423 continues to store additional reconstructed image blocks that can be reconstructed into frames using the division information. Such frames can also be placed in a sequence. The sequence is displayed as a reconstructed video output.

Considering the above, video compression technologies perform spatial (intra-frame) prediction and/or temporal (inter-frame) prediction to reduce or eliminate the redundancy inherent in video sequences. For block-based video encoding, a video slice (i.e., a video image or part of a video image) can be divided into video blocks, which may also be called tree blocks, coding tree blocks (CTB), coding tree units (CTU), coding units (CU), and/or coding nodes. Video blocks in an intra-frame (I) coded image slice are encoded using spatial prediction with respect to reference samples in adjacent blocks of the same image. Video blocks in an interframe (P or B) coded slice of an image may use spatial prediction with respect to reference samples in adjacent blocks in the same picture, or temporal prediction with respect to reference samples in other reference pictures. The pictures may be referred to as frames, and the reference pictures may be referred to as reference frames. POC is a variable associated with each picture that uniquely identifies the associated picture among all pictures in the coded layer video sequence (CLVS), indicates when the corresponding picture should be output from the DPB, and indicates the position of the associated picture in the output order relative to the output order positions other images in the same CLVS that should be output from the DPB. A flag is a variable or one-bit syntax element that can take one of two possible values: 0 and 1.

Spatial or temporal prediction results in a block prediction for the block to be encoded. The residual data is the difference in pixels between the original encoded block and the predicted block. The inter-frame predicted block is encoded in accordance with a motion vector that indicates a block of reference samples forming a prediction block, and residual data indicating the difference between the coded block and the predicted block. The intra-predicted block is encoded in accordance with the intra-prediction mode and the residual data. For further compression, the residual data can be converted from a pixel domain to a transform domain, resulting in residual transform coefficients that can then be quantized. The quantized transform coefficients, initially placed in a two-dimensional array, can be scanned to obtain a one-dimensional vector of transform coefficients, and entropy coding can be applied to achieve even greater compression.

Image and video compression has evolved at a rapid pace, leading to the emergence of various encoding standards. Such video coding standards include ITU-T H.261, International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group (MPEG)-1 Part 2, ITU-T H.262, or ISO/IEC MPEG-2 Part 2, ITU-T H.263, ISO/IEC MPEG-4 Part 2, Advanced Video Coding (AVC), also known as ITU-T H.264 or ISO/IEC MPEG-4 Part 10, and High Efficiency video coding (HEVC), also known as ITU-T H.265 or MPEG-H part 2. AVC includes extensions such as scalable video coding (SVC), multi-view video coding (MVC), and multi-view plus depth video coding ( MVC+D), and three-dimensional (3D) AVC (3D-AVC). HEVC includes extensions such as scalable HEVC (SHVC), multi-view HEVC (MV-HEVC), and 3D HEVC (3D-HEVC).

There is also a new video coding standard called Versatile Video Coding (VVC), which is being developed by the Joint Video Experts Team (JVET) of ITU-T and ISO/IEC. Although the VVC standard has several working drafts, one VVC working draft (WD) in particular is mentioned in this document, namely B. Bross, J. Chen, and S. Liu, “Versatile Video Coding (Draft 5),” JVET- N1001-v3, 13th JVET Meeting, March 27, 2019 (VVC Draft 5).

Scalability in video encoding is usually supported by the use of multi-layer encoding technologies. The multilayer bitstream contains a base layer (BL) and one or more enhancement layers (EL). Examples of scalability include spatial scalability, quality/signal-to-noise ratio (SNR) scalability, multi-view scalability, etc. When multi-layer encoding technology is used, an image or part thereof can be encoded (1) without using a reference image, that is, using intra-frame prediction; (2) by referring to reference images that are in the same layer, that is, using inter-frame prediction; or (3) by accessing reference images that are in other layer(s), that is, using inter-layer prediction. The reference image used for inter-layer prediction of the current image is called inter-layer reference picture (ILRP).

In fig. 5 is a schematic diagram illustrating an example of layer-based prediction 500, such as how it is performed to determine motion vectors (MVs) in block compression step 105, block decoding step 113, motion estimation component 221, motion compensation component 219, motion compensation component 321 and/or motion compensation component 421. Layer-based prediction 500 is compatible with unidirectional inter-prediction and/or bi-directional inter-prediction, but is also performed between images in different layers.

Layer-based prediction 500 is applied between images 511, 512, 513 and 514 and images 515, 516, 517 and 518 in different layers. In the example shown, images 511, 512, 513 and 514 are part of layer N+1 532, and images 515, 516, 517 and 518 are part of layer N 531. A layer such as layer N 531 and/or layer N+1 532, is a group of images associated with a similar characteristic value, such as similar size, quality, resolution, signal-to-noise ratio, capabilities, etc. In the example shown, layer N+1,532 is associated with an image size larger than layer N 531. Accordingly, in this example, images 511, 512, 513, and 514 in layer N+1,532 have a larger image size (e.g., greater height and width and therefore more samples) than images 515, 516, 517 and 518 in layer N 531. However, such images may be separated between layer N+1 532 and layer N 531 based on other characteristics. Although only two layers are shown, layer N+1 532 and layer N 531, the set of images can be divided into any number of layers based on associated characteristics. Layer N+1 532 and layer N 531 may also be designated by layer ID. A layer ID is a data element that is associated with an image and indicates that the image is part of the specified layer. Accordingly, each image 511-518 may be associated with a corresponding layer ID to indicate which layer N+1 532 or layer N 531 includes the corresponding image.

Images 511-518 in different layers 531-532 are configured to be displayed in an alternative embodiment. Thus, images 511-518 in different layers 531-532 can share the same temporary identifier (ID) and can be included in the same AU. As used herein, the term AU is a set of one or more encoded pictures associated with the same display time for output from the DPB. For example, the decoder may decode and display image 515 at the current display time if a smaller image size is required, or the decoder may decode and display image 511 at the current display time if a larger image size is required. Thus, images 511-514 in the top layer N+1 532 contain substantially the same image data as the corresponding images 515-518 in the bottom layer N 531 (despite the difference in image size). Specifically, image 511 contains substantially the same image data as image 515, image 512 contains substantially the same image data as image 516, and so on.

Images 511-518 may be encoded by reference to other images 511-518 in the same N 531 or N+1 layer 532. Encoding an image by reference to another image in the same layer results in inter-frame prediction 523 that is compatible with unidirectional interframe prediction and/or bidirectional interframe prediction. Inter-frame prediction 523 is shown by solid arrows. For example, picture 513 may be encoded using inter-picture prediction 523 using one or two pictures 511, 512 and/or 514 in the N+1 layer 532 as a reference, where one picture is used as a reference for unidirectional inter-picture prediction, and/or the two images are used as a reference for bidirectional inter-frame prediction. In addition, picture 517 may be encoded using inter-picture prediction 523 using one or two pictures 515, 516 and/or 518 in layer N 531 as a reference, where one picture is used as a reference for unidirectional inter-picture prediction, and/or two the images are used as a reference for bidirectional inter-frame prediction. When performing inter-picture prediction 523, an image is used as a reference image for another image in the same layer, that image may be referred to as a reference image. For example, picture 512 may be a reference picture used to encode picture 513 in accordance with inter-picture prediction 523. Inter-picture prediction 523 may also be referred to as intra-layer prediction in a multi-layer context. Thus, inter-picture prediction 523 is a mechanism for encoding samples of a current picture by referring to said samples in a reference picture that is different from the current picture, where the reference picture and the current picture are in the same layer.

Images 511-518 can also be encoded by referencing other images 511-518 in other layers. This process is known as interlayer prediction 521 and is shown by dotted arrows. Inter-layer prediction 521 is a mechanism for encoding samples of a current image by reference to said samples in a reference image, where the current image and the reference image are in different layers and therefore have different layer IDs. For example, an image in the lower layer N+1 531 may be used as a reference image to encode a corresponding image in the upper layer N+1 532. As a specific example, the image 511 may be encoded by reference to the image 515 in accordance with the inter-layer prediction 521. In such a case, image 515 is used as an interlayer reference image. An inter-layer reference picture is a reference picture used for inter-layer prediction 521. In most cases, inter-layer prediction 521 is limited such that the current picture, such as picture 511, can only use inter-layer reference pictures that are included in the same AU and are in a lower layer, such as image 515. When multiple layers are available (eg, more than two), inter-layer prediction 521 may encode/decode the current image based on multiple inter-layer reference images on layers lower than the current image.

A video encoder may use multi-layer video sequence 500 to encode images 511-518 with a variety of different combinations and/or permutations of inter-picture prediction 523 and inter-layer prediction 521. For example, image 515 may be encoded in accordance with inter-picture prediction. Images 516-518 may then be encoded according to inter-frame prediction 523 using image 515 as a reference image. In addition, image 511 may be encoded in accordance with inter-layer prediction 521 using image 515 as an inter-layer reference image. The pictures 512-514 may then be encoded according to inter-frame prediction 523 using the picture 511 as a reference picture. Thus, the reference image can serve as either a single-layer reference image or an inter-layer reference image for various encoding mechanisms. By encoding N+1532 top layer images based on N531 bottom layer images, the N+1532 top layer can avoid using intra-frame prediction, which has much lower coding efficiency than inter-frame prediction 523 and inter-layer prediction 521. Thus, low efficiency Intra prediction coding may be limited to the smallest/lowest quality images and therefore limited to encoding the least amount of video data. Pictures used as reference pictures and/or inter-layer reference pictures may be specified in reference picture list(s) entries contained in the reference picture list structure.

Each AU 506 shown in FIG. 5 may contain multiple images. For example, one AU 506 may contain images 511 and 515. Another AU 506 may contain images 512 and 516. In fact, each AU 506 is a set of one or more encoded images associated with the same display time (e.g., one and same temporary ID) for output from the decoded picture buffer (DPB) (for example, for display to the user). Each AUD 508 is an indicator or data structure used to indicate the start of an AU (eg, AU 508) or the boundary between AUs.

Previous H.26x video encoding families provided scalability support in a separate profile(s) from the profile(s) for single-layer encoding. Scalable Video Coding (SVC) is a scalable extension to AVC/H.264 that provides support for spatio-temporal scalability and quality scalability. For SVC, a flag is signaled in each macroblock (MB) in the EL pictures to indicate whether the EL MB is predicted using a co-located block from the lower layer. The prediction from the combined block may include texture, motion vectors, and/or encoding modes. SVC implementations cannot directly reuse unmodified AVC implementations in their design. The EL SVC macroblock syntax and decoding process are different from the H.264/AVC syntax and decoding process.

Scalable HEVC (SHVC) is an extension of the HEVC/H.265 standard that provides support for spatial and quality scalability, multi-view HEVC (MV-HEVC) is an extension of HEVC/H.265 that provides support for multi-view scalability, and 3D HEVC (3D -HEVC) is an extension to HEVC/H.264 that provides support for three-dimensional (3D) video encoding that is more advanced and efficient than MV-HEVC. It should be noted that temporal scalability is included as an integral part of the single-layer HEVC codec. The HEVC multi-layer extension design uses the idea that decoded pictures used for inter-layer prediction come only from the same access unit (AU) and are treated as long-term reference pictures (LTRP) and are assigned reference indices in the list(s) reference images along with other temporary reference images in the current layer. Inter-layer prediction (ILP) is achieved at the prediction unit (PU) level by setting a reference index value to reference the inter-layer reference picture(s) in the reference picture list(s).

Notably, the reference image resampling and spatial scaling functions require resampling the reference image or part of it. Reference picture resampling (RPR) can be implemented either at the picture level or at the coding block level. However, when RPR is referred to as a coding function, it represents single-layer coding functions. Even so, it is possible or even preferable from a codec design point of view to use the same oversampling filter for both the single-layer encoding RPR function and the multi-layer encoding spatial scalability function.

In fig. 6 shows an example of layer-based prediction 600 using output layer sets (OLS), for example, as performed to determine the MV at block compression step 105, block decoding step 113, motion estimation component 221, motion compensation component 219, motion compensation component 321 and/or motion compensation component 421. Layer-based prediction 600 is compatible with unidirectional inter-frame prediction and/or bi-directional inter-frame prediction, but is also performed between images in different layers. Prediction based on the layers shown in FIG. 6, similar to that shown in FIG. 5. Therefore, for the sake of brevity, the full description of layer-based prediction is not repeated.

Some of the layers in the coded video sequence (CVS) 690 shown in FIG. 6 are included in OLS. An OLS is a set of layers for which one or more layers are specified as output layers. The output layer is an OLS output layer. In fig. 6 shows three different OLSs, namely OLS 1, OLS 2 and OLS 3. As shown, OLS 1 includes layer N 631 and layer N+1 632. OLS 2 includes layer N 631, layer N+1 632, layer N+2. 633 and layer N+3 634. OLS 3 includes layer N 631. That is, OLS 3 includes one single layer. In practical applications, other OLS may also contain a single layer. Although three OLS are shown, different numbers of OLS may be used in practical applications. Other layers, such as the N+4 635 layer, are not included in the OLSs illustrated (e.g., OLS 1, 2, and 3) but are available for inclusion in other OLSs.

Each of the different OLSs can contain any number of layers. Different OLSs are developed to accommodate the encoding capabilities of different devices having different encoding capabilities. For example, OLS 3, which contains one single layer, could be designed to work with a mobile phone with relatively limited encoding capabilities. On the other hand, OLS 2, which contains four layers, can be designed to work with a large screen TV that is capable of decoding higher layers than a mobile phone. OLS 1, which contains two layers, can be designed to work with a personal computer, laptop or tablet computer that can decode higher layers than a mobile phone, but cannot decode the highest layers such as a large screen TV.

All layers shown in Fig. 6, can be independent of each other. That is, each layer can be encoded without using inter-layer prediction (ILP). In this case, the layers are called simulcast layers. One or more layers shown in FIG. 6 may also be encoded using ILP. Whether the layers are simulcast layers, or whether some of the layers are encoded using ILP, is signaled by a flag in the Video Parameter Set (VPS), which will be discussed in more detail below. When some layers use ILP, layer dependency relationships between layers are also signaled to the VPS.

In the embodiment, when the layers are simulcast layers, one single layer is selected for decoding and output. In an embodiment, when some layers use ILP, all layers (eg, the entire bitstream) are set for decoding, and certain layers among the layers are set as output layers. The output layer or layers may be, for example, 1) only the highest layer, 2) all layers, or 3) the highest layer plus a set of specified lower layers. For example, when the highest layer plus a set of specified lower layers are designated for output by flag in the VPS, layer N+3634 (which is the highest layer) and layers N631 and N+1632 (which are lower layers) from OLS 2 are output.

As shown in FIG. 6, each layer consists of any number of images. For example, layer N 631 includes images 615-618, layer N+1 632 includes images 611-614, layer N+2 633 includes images 641-644, layer N+3 634 includes images 651- 654, and layer N+4 635 includes images 661-664.

Video coding standards such as HEVC, SHVC, MV-HEVC and VVC define and signal profile, rank and level. Profiles, ranks, and levels define restrictions on bitstreams and therefore limit the capabilities required to decode the bitstreams. Profiles, ranks, and levels can also be used to specify points of interoperability between individual decoder implementations. A profile is a specific set of encoding tools used to create a compatible or conforming bitstream. Each profile defines a subset of algorithmic features and constraints that must be supported by all decoders conforming to that profile.

A level is a set of constraints on a bit stream (eg maximum luminance sampling rate, maximum bit rate for resolution, etc.). That is, a level is a set of constraints indicating the required decoder performance to reproduce a bit stream of a given profile.

The levels are divided into two ranks: basic and high. Major Rank is a lower rank than High Rank. Ranks were created to work with applications that differ in their maximum data transfer rates. Basic rank was designed for most applications, while high rank was designed for very demanding applications. A set of restrictions and/or ranks corresponding to levels may be referred to herein as level information.

Each rank level defines a set of restrictions on the values that syntactic elements of a particular standard (e.g., HEVC, VVC) can take. The same set of rank and level definitions is used with all profiles, but individual implementations may support a different rank, and within a rank, a different level for each supported profile. For any given profile, the rank level typically corresponds to the specific decoder computational load and memory capabilities.

When using the HEVC standard, an illustrative example of levels and ranks for various profiles is presented in Table 1 below.

Table 1

Level Maximum bit rate for main profile 12 4:4:4
(1000 bps) Maximum data transfer rate for high throughput
4:4:4 16 Internal profile
(1000 bps) Example of image resolution at maximum frame rate
(MaxDPBsize) Main rank High rank Main rank High rank 5.2 180000 720000 5760000 23040000 4096×2160 at 120 Hz 6 180000 720000 5760000 23040000 8192x4320 at 30 Hz 6.1 180000 720000 11520000 46080000 8192x4320 at 60 Hz 6.2 720000 2400000 23040000 76800000 8192x4320 at 120 Hz

The profile, rank, and level (PTL) parameters are a syntactic structure (e.g., the i-th syntax structure profile_tier_level()) that provides a profile, rank, and/or level, as specified in the table above. The decoded picture buffer (DPB) parameters are a syntax structure (eg, the i-th syntax structure dpb_parameters()) that provides the size of the DPB, optionally the maximum number of picture reorders, and maximum delay information. The maximum number of image reorders and maximum delay information may be collectively abbreviated MRML. HRD parameters are a syntactic structure (eg, the i-th syntax structure ols_timing_hrd_parameters()) that initializes and/or defines the operating conditions of the HRD. Additional details and specifics regarding one or more of the PTL parameters, DPB parameters and HRD parameters are presented below.

In an embodiment, the maximum number of picture reorders is denoted by dpb_max_num_reorder_pics[ i ], which specifies the maximum allowed number of OLS pictures that can precede any picture in the OLS in decoding order and follow that picture in output order when the Htid value is i. The value of dpb_max_num_reorder_pics[ i ] must be in the range from 0 to dpb_max_dec_pic_buffering_minus1[ i ], inclusive. When i is greater than 0, dpb_max_num_reorder_pics[ i ] must be greater than or equal to dpb_max_num_reorder_pics[i - 1]. When dpb_max_num_reorder_pics[ i ] is missing for i in the range 0 to MaxSubLayersMinus1 - 1 inclusive, because the subLayerInfoFlag is 0, it is assumed to be equal to dpb_max_num_reorder_pics[ MaxSubLayersMinus1 ]. Htid is a variable that identifies the highest temporal sublayer to be decoded.

dpb_max_dec_pic_buffering_minus1[ i ] plus 1 specifies the maximum required DPB size in units of image storage buffers when Htid is equal to i. The value of dpb_max_dec_pic_buffering_minus1[ i ] shall be in the range from 0 to MaxDpbSize - 1 inclusive, where MaxDpbSize corresponds to subclause A.4.2 of the VVC standard. When i is greater than 0, dpb_max_dec_pic_buffering_minus1[ i ] must be greater than or equal to dpb_max_dec_pic_buffering_minus1[i - 1]. When dpb_max_dec_pic_buffering_minus1[ i ] is missing for i in the range 0 to MaxSubLayersMinus1 - 1 inclusive, because the subLayerInfoFlag is 0, it is assumed to be equal to dpb_max_dec_pic_buffering_minus1[ MaxSubLayersMinus1 ].

In an embodiment, the maximum latency information is denoted as dpb_max_latency_increase_plus1[ i ]. The parameter dpb_max_latency_increase_plus1[ i ], not equal to 0, is used to calculate the MaxLatencyPictures[ i ] value, which specifies the maximum number of pictures in OLS that can precede any picture in OLS in output order and follow that picture in decoding order when the Htid value is i.

MaxLatencyPictures[ i ] = dpb_max_num_reorder_pics[ i ] + dpb_max_latency_increase_plus1[ i ] - 1.

The DPB syntax structures (also known as DPB parameters) for the OLS described above are included in the bitstream VPS. Unfortunately, signaling all of these DPB syntactic structures in the VPS is ineffective when the OLS contains one single layer (eg, OLS 3 shown in Fig. 6).

Disclosed herein are technologies that enable decoded picture buffer (DPB) parameters to be included in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

In fig. 7 shows an embodiment of a video bitstream 700. As used herein, the video bitstream 700 may also be referred to as a coded video bitstream, a bitstream, or variants thereof. As shown in FIG. 7, bitstream 700 includes a video parameter set (VPS) 704, a sequence parameter set (SPS) 706, a picture parameter set (PPS) 708, a slice header 712, and image data 720. Each of the VPS 704, SPS 706 and PPS 708 may be collectively referred to as a parameter set. In an embodiment, other parameter sets and/or headers not shown in FIG. 7 may also be included in the bit stream 700.

VPS 704 includes decoding dependency or information for constructing a set of enhancement layer reference images. VPS 704 provides an overall perspective or view of the scalable sequence, including the types of operating points provided, the profile, rank and level of operating points, and certain other high-level properties of the bitstream 700 that can be used as the basis for session negotiation and content selection, etc.

In an embodiment, the VPS 704 includes a list of DPB syntax structures 780 for all OLSs. In an embodiment, each of the DPB syntax structures in the list of DPB syntax structures 780 is designated dpb_parameters(). The VPS 704 also includes a vps_ols_dpb_params_idx[ i ] syntax structure 782. The vps_ols_dpb_params_idx[ i ] syntax structure 782 specifies the DPB syntax structure index that applies to the i-th OLS. That is, each index value corresponds to one of the DPB syntax structures in the list. For example, index value 1 in the list corresponds to dpb_parameters() for OLS 1, index value 2 corresponds to dpb_parameters() for OLS 2, and index value 3 corresponds to dpb_parameters() for OLS 3.

The value of vps_ols_dpb_params_idx[ i ], if present, is in the range from 0 to VpsNumDpbParams - 1 inclusive.

The syntax element vps_num_dpb_params_minus1 781 specifies the number of dpb_parameters() syntax structures in the VPS 704. The value of vps_num_dpb_params_minus1 781 ranges from zero to less than the total number of multilayer OLSs, which can be denoted as NumMultiLayerOlss.

The variable VpsNumDpbParams - 1 is obtained by the video decoder (that is, the variable is not signaled in the bitstream) based on the syntax element vps_num_dpb_params_minus1 781, which is signaled in the bitstream.

In an embodiment, the variable VpsNumDpbParams - 1 is obtained as follows.

if( vps_each_layer_is_an_ols_flag )

VpsNumDpbParams = 0 … (34)

else

VpsNumDpbParams = vps_num_dpb_params_minus1 + 1

The variable VpsNumDpbParams - 1 plus 1 indicates the number of DPB syntax structures in the VPS 704. For example, when the value of VpsNumDpbParams - 1 is two, the number of DPB syntax structures in the VPS 704 is three (2+1=3).

In an embodiment, the VPS 704 includes a DPB presence flag 783, which may be designated vps_sublayer_dpb_params_present_flag, used to control the presence of dpb_max_dec_pic_buffering_minus1[ j ], dpb_max_num_reorder_pics[ j ] and dpb_max_latency_increase_plus1[ j ] syntax elements in dpb syntax structures _parameters() in VPS for j in the range from 0 to vps_dpb_max_tid[ i ] - 1 inclusive, when vps_dpb_max_tid[ i ] is greater than 0. If the value of vps_sub_dpb_params_info_present_flag is missing, it is assumed to be 0.

SPS 706 contains data that is common to all images in a Sequence of Pictures (SOP). SPS 706 is a syntax structure containing syntax elements that apply to zero or more complete CLVS, as determined by the content of the syntax element found in the PPS referenced by the syntax element found in the header of each image.

In an embodiment, the dpb_parameters() parameters that apply to the i-th OLS 784 are present in the SPS 706 referenced by the layer in the i-th OLS when the i-th OLS is a single-layer OLS. That is, for a single-layer OLS (eg, OLS 2 shown in FIG. 6), the applicable dpb_parameters() syntax structure is present in the SPS referenced by the layer in the OLS.

Based on the above, it should be understood that when the OLS is a single-layer OLS (for example, the OLS contains one single layer), the dpb_parameters() that apply to the i-th OLS 784 are present in both the SPS 706 and the VPS 704 (i.e., syntactic The DPB structure that applies to the i-th OLS is one of the DPB syntactic structures included in the DPB structures 780). Moreover, the dpb_parameters() parameters that apply to the i-th OLS 784 in the SPS 706 are identical to the dpb_parameters() parameters that apply to the i-th OLS 784 in the VPS 704.

Unlike SPS 706, PPS 708 contains data that is common to the entire image. PPS 708 is a syntax structure containing syntax elements that apply to zero or more complete encoded pictures. The slice header 712 is part of an encoded slice containing data elements related to all video data in the tile represented in the slice. The slice header 712 contains parameters that are specific to each slice in the image. Thus, there may be one slice header 712 per slice in a video sequence. The slice header 712 may contain slice type information, picture order counters (POCs), reference picture lists (RPLs), prediction weights, tile insertion points, or deblocking parameters. The slice header 712 may also be referred to as a tile group header and/or a picture header (PH).

In an embodiment, VPS 704 and/or SPS 706 comprise the following SPS syntax and semantics for implementing one or more embodiments disclosed herein.

VPS syntax includes the following.

video_parameter_set_rbsp() { Descriptor vps_video_parameter_set_id u(4) vps_max_layers_minus1 u(6) vps_max_sub_layers_minus1 u(3) if( vps_max_layers_minus1 > 0 && vps_max_sub_layers_minus1 > 0 ) vps_all_layers_same_num_sub_layers_flag u(1) if( vps_max_layers_minus1 > 0 ) vps_all_independent_layers_flag u(1) ... vps_num_ptls u(8) for( i = 0; i < vps_num_ptls; i++ ) { if( i > 0 ) pt_present_flag[i] u(1) if( vps_max_sub_layers_minus1 > 0 && !vps_all_layers_same_num_sub_layers_flag ) ptl_max_temporal_id[ i ] u(3) } while( !byte_aligned() ) vps_ptl_byte_alignment_zero_bit /* equal to 0 */ u(1) for( i = 0; i < vps_num_ptls; i++ ) profile_tier_level( pt_present_flag[ i ], ptl_max_temporal_id[ i ] ) for( i = 0; i < TotalNumOlss; i++ ) if( NumLayersInOls[ i ] > 1 && vps_num_ptls > 1 ) ols_ptl_idx[ i ] u(8) if( !vps_all_independent_layers_flag ) vps_num_dpb_params ue(v) if( vps_num_dpb_params > 0 ) { same_dpb_size_output_or_nonoutput_flag u(1) if( vps_max_sub_layers_minus1 > 0 ) vps_sub_layer_dpb_params_present_flag u(1) } for( i = 0; i < vps_num_dpb_params; i++ ) { dpb_size_only_flag[ i ] u(1) if( vps_max_sub_layers_minus1 > 0 && !vps_all_layers_same_num_sub_layers_flag ) dpb_max_temporal_id[ i ] u(3) dpb_parameters( dpb_size_only_flag[ i ], dpb_max_temporal_id[ i ],
vps_sub_layer_dpb_params_present_flag ) } for( i = 0; i < vps_max_layers_minus1 && vps_num_dpb_params > 1; i++ ) { if( !vps_independent_layer_flag[ i ] ) layer_output_dpb_params_idx[ i ] ue(v) if( LayerUsedAsRefLayerFlag[ i ] && !same_dpb_size_output_or_nonoutput_flag ) layer_nonoutput_dpb_params_idx[ i ] ue(v) } general_hrd_params_present_flag u(1) if( general_hrd_params_present_flag ) { num_units_in_tick u(32) time_scale u(32) general_hrd_parameters( ) } vps_extension_flag u(1) if( vps_extension_flag ) while( more_rbsp_data() ) vps_extension_data_flag u(1) rbsp_trailing_bits( ) }

The SPS raw byte sequence payload (RBSP) syntax includes the following.

seq_parameter_set_rbsp( ) { Descriptor sps_decoding_parameter_set_id u(4) sps_video_parameter_set_id u(4) sps_max_sub_layers_minus1 u(3) sps_reserved_zero_4bits u(4) sps_ptl_dpb_present_flag u(1) if( sps_ptl_dpb_present_flag ) profile_tier_level( 1, sps_max_sub_layers_minus1 ) gdr_enabled_flag u(1) sps_seq_parameter_set_id ue(v) chroma_format_idc ue(v) ... log2_max_pic_order_cnt_lsb_minus4 ue(v) poc_msb_in_rap_pics_flag u(1) if( poc_msb_in_rap_pics_flag > 0 ) poc_msb_len_minus1 ue(v) if( sps_max_sub_layers_minus1 > 0 ) sps_sub_layer_dpb_params_flag u(1) if( sps_ptl_dpb_present_flag ) dpb_parameters( 0, sps_max_sub_layers_minus1, sps_sub_layer_dpb_params_flag ) long_term_ref_pics_flag u(1) ... sps_scaling_list_enabled_flag u(1) vui_parameters_present_flag u(1) if( vui_parameters_present_flag ) vui_parameters( ) sps_extension_flag u(1) if( sps_extension_flag ) while( more_rbsp_data() ) sps_extension_data_flag u(1) rbsp_trailing_bits( ) }

DPB syntax includes the following.

dpb_parameters( dpbSizeOnlyFlag, maxSubLayersMinus1, subLayerInfoFlag ) { Descriptor for( i = ( subLayerInfoFlag ? 0 : maxSubLayersMinus1 );
i <= maxSubLayersMinus1; i++ ) { max_dec_pic_buffering_minus1[ i ] ue(v) if( !dpbSizeOnlyFlag ) { max_num_reorder_pics[ i ] ue(v) max_latency_increase_plus1[ i ] ue(v) } } }

HRD parameter syntax includes general HRD parameter syntax, OLS HRD parameter syntax, and sublayer HRD parameter syntax.

The syntax for common HRD options includes the following.

general_hrd_parameters() { Descriptor general_nal_hrd_params_present_flag u(1) general_vcl_hrd_params_present_flag u(1) if( general_nal_hrd_params_present_flag | | general_vcl_hrd_params_present_flag ) { decoding_unit_hrd_params_present_flag u(1) if( decoding_unit_hrd_params_present_flag ) { tick_divisor_minus2 u(8) decoding_unit_cpb_params_in_pic_timing_sei_flag u(1) } bit_rate_scale u(4) cpb_size_scale u(4) if(decoding_unit_hrd_params_present_flag) cpb_size_du_scale u(4) } if( vps_max_sub_layers_minus1 > 0 ) sub_layer_cpb_params_present_flag u(1) if( TotalNumOlss > 1 ) num_ols_hrd_params_minus1 ue(v) hrd_cpb_cnt_minus1 ue(v) for( i = 0; i <= num_ols_hrd_params_minus1; i++ ) { if( vps_max_sub_layers_minus1 > 0 && !vps_all_layers_same_num_sub_layers_flag ) hrd_max_temporal_id[ i ] u(3) ols_hrd_parameters( hrd_max_temporal_id[ i ]) } if( num_ols_hrd_params_minus1 > 0 ) for( i = 1; i < TotalNumOlss; i++ ) ols_hrd_idx[ i ] ue(v) }

HRD OLS parameter syntax includes the following.

ols_hrd_parameters(hrdMaxTid) { Descriptor firstSubLayer = sub_layer_cpb_params_present_flag ? 0:hrdMaxTid for( i = firstSubLayer; i <= hrdMaxTid; i++ ) { fixed_pic_rate_general_flag[ i ] u(1) if( !fixed_pic_rate_general_flag[ i ] ) fixed_pic_rate_within_cvs_flag[ i ] u(1) if( fixed_pic_rate_within_cvs_flag[ i ] ) elemental_duration_in_tc_minus1[ i ] ue(v) else if( hrd_cpb_cnt_minus1 = = 0 ) low_delay_hrd_flag[ i ] u(1) if( general_nal_hrd_params_present_flag ) sub_layer_hrd_parameters( i ) if( general_vcl_hrd_params_present_flag ) sub_layer_hrd_parameters( i ) } }

The HRD sublayer parameter syntax includes the following.

sub_layer_hrd_parameters( subLayerId ) { Descriptor for( j = 0; j <= hrd_cpb_cnt_minus1; j++ ) { bit_rate_value_minus1[ subLayerId ][ j ] ue(v) cpb_size_value_minus1[ subLayerId ][ j ] ue(v) if( decoding_unit_hrd_params_present_flag ) { cpb_size_du_value_minus1[ subLayerId ][ j ] ue(v) bit_rate_du_value_minus1[ subLayerId ][ j ] ue(v) } cbr_flag[ subLayerId ][ j ] u(1) } }

The semantics of VPS are presented as follows.

vps_max_layers_minus1 plus 1 specifies the maximum number of layers allowed in each CVS referencing the VPS.

vps_max_sub_layers_minus1 plus 1 specifies the maximum number of temporary sublayers that can be present in each CVS referencing the VPS. The value of vps_max_sub_layers_minus1 must be in the range from 0 to 7 inclusive.

The vps_all_layers_same_num_sub_layers_flag equal to 1 indicates that the number of temporary sublayers is the same for all layers in each CVS referencing the VPS. The vps_all_layers_same_num_sub_layers_flag equal to 0 indicates that the layers in each CVS referencing the VPS may or may not have the same number of temporary sublayers. If it is missing, vps_all_layers_same_num_sub_layers_flag is assumed to be 1.

The vps_all_independent_layers_flag equal to 1 indicates that all layers in CVS are encoded independently without using inter-layer prediction. A vps_all_independent_layers_flag of 0 indicates that one or more layers in CVS can use inter-layer prediction. If it is missing, the value of vps_all_independent_layers_flag is assumed to be 1. When the value of vps_all_independent_layers_flag is 1, the value of vps_independent_layer_flag[ i ] is assumed to be 1. When the value of vps_all_independent_layers_flag is 0, the value of vps_independent_layer_flag[ 0 ] is assumed to be 1.

The vps_direct_dependency_flag[ i ][ j ] flag equal to 0 indicates that the layer with index j is not a direct link layer for the layer with index i. The vps_direct_dependency_flag [ i ][ j ] flag equal to 1 indicates that layer index j is the direct link layer for layer index i. When vps_direct_dependency_flag[ i ][ j ] is missing for i and j in the range 0 to vps_max_layers_minus1 inclusive, it is assumed to be 0.

The DirectDependentLayerIdx[ i ][ j ] variable indicating the jth directly dependent layer of the i th layer, and the LayerUsedAsRefLayerFlag[ j ] variable indicating whether the layer at layer index j is used as a reference layer flag by any other layer is obtained as follows way:

for( i = 0; i <= vps_max_layers_minus1; i++ )

LayerUsedAsRefLayerFlag[ j ] = 0

for( i = 1; i < vps_max_layers_minus1; i++ )

if( !vps_independent_layer_flag[ i ] )

for( j = i - 1, k = 0; j >= 0; j- - )

if( vps_direct_dependency_flag[ i ][ j ] ) {

DirectDependentLayerIdx[ i ][ k++ ] = j

LayerUsedAsRefLayerFlag[ j ] = 1

}

The variable GeneralLayerIdx[ i ], indicating the index of the layer with nuh_layer_id equal to vps_layer_id[ i ], is obtained as follows:

for( i = 0; i <= vps_max_layers_minus1; i++ )

GeneralLayerIdx[ vps_layer_id[ i ] ] = i

Each_layer_is_an_ols_flag equal to 1 indicates that each set of output layers contains only one layer, and each layer in the bitstream is itself a set of output layers, with the only layer included being the only output layer. Each_layer_is_an_ols_flag equal to 0 indicates that an output layer set can contain more than one layer. If the value of vps_max_layers_minus1 is 0, the value of each_layer_is_an_ols_flag is assumed to be 1. Otherwise, when the value of vps_all_independent_layers_flag is 0, the value of each_layer_is_an_ols_flag is assumed to be 0.

An ols_mode_idc equal to 0 indicates that the total number of OLSs specified by the VPS is vps_max_layers_minus1 + 1, the i-th OLS includes layers with layer indices from 0 to i inclusive, and for each OLS only the top layer in the OLS is output. An ols_mode_idc equal to 1 indicates that the total number of OLSs specified by the VPS is vps_max_layers_minus1 + 1, the i-th OLS includes layers with layer indices from 0 to i inclusive, and for each OLS all layers in the OLS are output. An ols_mode_idc equal to 2 indicates that the total number of OLS specified by the VPS is explicitly signaled, and for each OLS the top layer and the explicitly signaled set of bottom layers in the OLS are output. The ols_mode_idc value must be between 0 and 2, inclusive. Value 3 ols_mode_idc reserved for future use by ITU-T | ISO/IEC. When vps_all_independent_layers_flag is 1 and each_layer_is_an_ols_flag is 0, the value of ols_mode_idc is assumed to be 2.

num_output_layer_sets_minus1 plus 1 indicates the total number of OLS set by the VPS when ols_mode_idc is two.

The TotalNumOlss variable indicating the total number of OLSs specified by the VPS is obtained as follows.

if( vps_max_layers_minus1 = = 0 )

TotalNumOlss = 1

else if( each_layer_is_an_ols_flag | | ols_mode_idc = = 0 | | ols_mode_idc = = 1 )

TotalNumOlss = vps_max_layers_minus1 + 1

else if( ols_mode_idc == 2 )

TotalNumOlss = num_output_layer_sets_minus1 + 1

layer_included_flag[ i ]][ j ] indicates whether the j-th layer (that is, the layer with nuh_layer_id equal to vps_layer_id[ j ]) is included in the i-th OLS when ols_mode_idc is equal to 2. layer_included_flag[ i ][ j ] equal 1 indicates that the j-th layer is included in the i-th OLS. layer_included_flag[ i ][ j ] equal to 0 indicates that the j-th layer is not included in the i-th OLS.

] The NumLayersInOls[ i ] variable, which specifies the number of layers in the i-th OLS, and the LayerIdInOls[ i ][ j ] variable, which specifies the nuh_layer_id value of the j-th layer in the i-th OLS, are obtained as follows.

NumLayersInOls[ 0 ] = 1

LayerIdInOls[ 0 ][ 0 ] = vps_layer_id[ 0 ]

for( i = 1, i < TotalNumOlss; i++ ) {

if( each_layer_is_an_ols_flag ) {

NumLayersInOls[ i ] = 1

LayerIdInOls[ i ][ 0 ] = vps_layer_id[ i ]

} else if( ols_mode_idc = = 0 | | ols_mode_idc = = 1 ) {

NumLayersInOls[ i ] = i + 1

for( j = 0; j < NumLayersInOls[ i ]; j++ )

LayerIdInOls[ i ][ j ] = vps_layer_id[ j ]

} else if( ols_mode_idc == 2 ) {

for( k = 0, j = 0; k <= vps_max_layers_minus1; k++ )

if( layer_included_flag[ i ][ k ] )

LayerIdInOls[ i ][ j++ ] = vps_layer_id[ k ]

NumLayersInOls[ i ] = j

}

}

The OlsLayeIdx[ i ][ j ] variable, which specifies the OLS layer index of the layer with nuh_layer_id equal to LayerIdInOls[ i ][ j ], is obtained as follows.

for( i = 0, i < TotalNumOlss; i++ )

for j = 0; j < NumLayersInOls[ i ]; j++ )

OlsLayeIdx[ i ][ LayerIdInOls[ i ][ j ] ] = j

The bottom layer in each OLS must be an independent layer. In other words, for every i in the range 0 to TotalNumOlss - 1 inclusive, the value of vps_independent_layer_flag[GeneralLayerIdx[LayerIdInOls[ i ][ 0 ]]] must be equal to 1.

Each layer must be included in at least one OLS defined by the VPS. In other words, for each layer with a particular value nuh_layer_id nuhLayerId equal to one from vps_layer_id[ k ] for k ranging from zero to vps_max_layers_minus1 inclusive, there must be at least one pair of values i and j, where i is in the range zero to TotalNumOlss - 1 inclusive, and j is in the range NumLayersInOls[ i ] - 1 inclusive, so the value of LayerIdInOls[ i ][ j ] is nuhLayerId.

Any layer in OLS must be an OLS output layer or a reference layer (direct or indirect) to an OLS output layer.

vps_output_layer_flag[ i ][ j ] indicates whether the j-th layer is output in the i-th OLS when ols_mode_idc is equal to 2. vps_output_layer_flag[ i ] equal to one indicates that the j-th layer is output in the i-th OLS . vps_output_layer_flag[ i ] equal to 0 indicates that the j-th layer in the i-th OLS is not output. When vps_all_independent_layers_flag is 1 and ach_layer_is_an_ols_flag is 0, vps_output_layer_flag[ i ] is assumed to be 1.

The OutputLayerFlag[ i ][ j ] variable, for which a value of 1 indicates that the j-th layer in the i-th OLS is output, and a value of 0 indicates that the j-th layer in the i-th OLS is not output, is obtained as follows :

for( i = 0, i < TotalNumOlss; i++ ) {

OutputLayerFlag[ i ][ NumLayersInOls[ i ] - 1 ] = 1

for( j = 0; j < NumLayersInOls[ i ] - 1; j++ )

if( ols_mode_idc[ i ] = = 0 )

OutputLayerFlag[ i ][ j ] = 0

else if( ols_mode_idc[ i ] = = 1 )

OutputLayerFlag[ i ][ j ] = 1

else if( ols_mode_idc[ i ] = = 2 )

OutputLayerFlag[ i ][ j ] = vps_output_layer_flag[ i ][ j ]

}

Note. The 0th OLS contains only the bottom layer (the layer with nuh_layer_id equal to vps_layer_id[ 0 ]), and for the 0th OLS only the included layer is output.

vps_num_ptls indicates the number of profile_tier_level() syntax structures in the VPS.

The pt_present_flag[ i ] flag equal to 1 indicates that information about the profile, tier and general restrictions is present in the i-th profile_tier_level() syntactic structure in the VPS. The pt_present_flag[ i ] flag equal to 0 indicates that information about the profile, rank and general restrictions is not present in the i-th profile_tier_level() syntactic structure in the VPS. The value of pt_present_flag[ 0 ] is assumed to be 0. When the flag pt_present_flag[ i ] is 0, the profile, rank and general constraint information for the ith syntax structure profile_tier_level() in VPS is assumed to be the same as for ( i - 1 )-th syntactic structure profile_tier_level() in VPS.

ptl_max_temporal_id[ i ] specifies the TemporalId of the highest sublayer representation for which tier information is present in the i-th profile_tier_level() syntax structure in the VPS. The value of ptl_max_temporal_id[ i ] must be in the range from 0 to vps_max_sub_layers_minus1 inclusive. When VPS_Max_Sub_Layers_Minus1 is equal to 0, it is assumed that PTL_MAX_TEMPOral_id [i] is 0. When vPs_max_sub_layers_minus1 is more than 0, and vps_all_layers_sab_Layers_flag is 1 Temporal_id [i] is supposed to be equal to VPS_MAX_USSUBLAYERS1_MAX_USSUBLAYERS1_MIN.

The value of vps_ptl_byte_alignment_zero_bit must be 0.

The index ols_ptl_idx[ i ] specifies the index in the profile_tier_level() syntax structure list in the VPS of the profile_tier_level() syntax structure that applies to the i-th OLS. When present, the value of ols_ptl_idx[ i ] must be in the range 0 to vps_num_ptls - 1, inclusive.

When the value of NumLayersInOls[ i ] is 1, the profile_tier_level() syntax structure that applies to the i-th OLS is present in the SPS referenced by the layer in the i-th OLS.

vps_num_dpb_params indicates the number of dpb_parameters() syntax structures in the VPS. The value of vps_num_dpb_params must be in the range from 0 to 16 inclusive. If this value is missing, vps_num_dpb_params is assumed to be 0.

A value of same_dpb_size_output_or_nonoutput_flag equal to 1 indicates that the VPS does not have the layer_nonoutput_dpb_params_idx[ i ] syntax element. A same_dpb_size_output_or_nonoutput_flag value of 0 indicates that the layer_nonoutput_dpb_params_idx[ i ] syntax elements may or may not be present in the VPS.

vps_sub_layer_dpb_params_present_flag is used to control the presence of the max_dec_pic_buffering_minus1[ ], max_num_reorder_pics[ ] and max_latency_increase_plus1[ ] syntax elements in the dpb_parameters() syntax structures in the VPS. If vps_sub_dpb_params_info_present_flag is not present, it is assumed to be 0.

A dpb_size_only_flag[ i ] value of 1 indicates that the syntax elements max_num_reorder_pics[] and max_latency_increase_plus1[] are not present in the i-th dpb_parameters() syntax structure in the VPS. A dpb_size_only_flag[ i ] value of 1 indicates that the max_num_reorder_pics[] and max_latency_increase_plus1[ ] syntax elements may be present in the i-th dpb_parameters() syntax structure in the VPS.

dpb_max_temporal_id[ i ] specifies the TemporalId of the highest sublayer representation for which DPB parameters can be present in the i-th dpb_parameters() syntax structure in the VPS. The value of dpb_max_temporal_id[ i ] must be in the range from 0 to vps_max_sub_layers_minus1 inclusive. When the value of vps_max_sub_layers_minus1 is 0, the value of dpb_max_temporal_id[ i ] is assumed to be 0. When the value of vps_max_sub_layers_minus1 is greater than 0, and the value of vps_all_layers_same_num_sub_layers_flag is 1, the value of dpb_max_temporal_id[ i ] is assumed to be vpss_minus_sub.

The index layer_output_dpb_params_idx[ i ] specifies the index in the list of dpb_parameters() syntax structures in the VPS of the dpb_parameters() syntax structure that applies to the i-th layer when it is an output layer in OLS. When present, the value of layer_output_dpb_params_idx[ i ] must be in the range 0 to vps_num_dpb_params - 1, inclusive.

If the value of vps_independent_layer_flag[ i ] is 1, the dpb_parameters() syntax structure that applies to the ith layer when it is an output layer is the dpb_parameters() syntax structure present in the SPS referenced by the layer.

Otherwise (when vps_independent_layer_flag[ i ] is 0), the following applies:

When vps_num_dpb_params is 1, layer_output_dpb_params_idx[ i ] is assumed to be 0.

A requirement for bitstream matching is that the value of layer_output_dpb_params_idx[ i ] must be such that the value of dpb_size_only_flag[layer_output_dpb_params_idx[ i ]] is 0.

The index layer_nonoutput_dpb_params_idx[ i ] specifies the index in the dpb_parameters() syntax structure list in the VPS of the dpb_parameters() syntax structure that applies to the i-th layer when it is not an output layer in OLS. When present, the value of layer_nonoutput_dpb_params_idx[ i ] must be in the range 0 to vps_num_dpb_params - 1, inclusive.

If same_dpb_size_output_or_nonoutput_flag is 1, the following applies:

If the value of vps_independent_layer_flag[ i ] is 1, the dpb_parameters() syntax structure that applies to the ith layer when it is not an output layer is the dpb_parameters() syntax structure present in the SPS referenced by the layer.

Otherwise (when vps_independent_layer_flag[ i ] is 0), layer_nonoutput_dpb_params_idx[ i ] is assumed to be layer_output_dpb_params_idx[ i ].

Otherwise (when same_dpb_size_output_or_nonoutput_flag is 0), when vps_num_dpb_params is 1, layer_output_dpb_params_idx[ i ] is assumed to be 0.

A vps_extension_flag value of 0 indicates that there are no vps_extension_data_flag syntax elements in the VPS RBSP syntax structure. A vps_extension_flag value of 1 indicates that vps_extension_data_flag syntax elements are present in the VPS RBSP syntax structure.

The vps_extension_data_flag flag can have any value. Its presence and meaning do not affect the decoder's compliance with the profiles specified in this version of this specification. Decoders conforming to this version of this specification shall ignore all vps_extension_data_flag syntax elements.

The semantics of RBSP SPS is presented as follows.

The RBSP SPS must be available to the decoding process before being referenced, it must be included in at least one access unit with a TemporalId equal to 0 or provided through an external facility, and the NAL SPS unit containing the RBSP SPS must have a nuh_layer_id equal to the smallest nuh_layer_id value of the NAL PPS units that reference it.

All SPS NAL units with a particular sps_seq_parameter_set_id value in CVS must have the same content.

The ID sps_decoding_parameter_set_id, when greater than 0, specifies the dps_decoding_parameter_set_id value for the DPS referenced by the SPS. When the ID sps_decoding_parameter_set_id is 0, the SPS does not reference the DPS, and the DPS is not referenced when decoding each CLVS that references the SPS. The value of sps_decoding_parameter_set_id must be the same in all SPS referenced by encoded images in the bitstream.

The ID sps_video_parameter_set_id, when greater than 0, specifies the vps_video_parameter_set_id value for the VPS referenced by the SPS. When the sps_video_parameter_set_id ID is 0, the SPS does not reference the VPS, and the VPS is not referenced when decoding each CLVS that references the SPS, and the value of GeneralLayerIdx[nuh_layer_id] is assumed to be 0, and the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is assumed to be 1.

When vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is equal to 1, the SPS referenced by CLVS with a particular nuhLayerId value nuh_layer_id must have a nuh_layer_id equal to nuhLayerId.

sps_max_sub_layers_minus1 plus 1 specifies the maximum number of temporary sublayers that can be present in each CLVS referencing the SPS. The value of sps_max_sub_layers_minus1 must be in the range from 0 to vps_max_sub_layers_minus1 inclusive.

The value of sps_reserved_zero_4bits shall be 0 in bitstreams conforming to this version of this specification. Other values for sps_reserved_zero_4bits are reserved for future use by ITU-T | ISO/IEC.

The sps_ptl_dpb_present_flag flag equal to 1 indicates that the profile_tier_level() syntax structure and the dpb_parameters() syntax structure are present in SPS. The sps_ptl_dpb_present_flag flag equal to 0 indicates that SPS does not have the profile_tier_level() syntax structure and the dpb_parameters() syntax structure. The value of sps_ptl_dpb_present_flag should be equal to vps_independent_layer_flag[nuh_layer_id].

If the value of vps_independent_layer_flag[GeneralLayerIdx[nuh_layer_id]] is 1, the MaxDecPicBuffMinus1 variable is set to max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1] in the dpb_parameters( ) syntax structure in SPS. Otherwise, MaxDecPicBuffMinus1 is set equal to max_dec_pic_buffering_minus1[sps_max_sub_layers_minus1] in the layer_nonoutput_dpb_params_idx[GeneralLayerIdx[nuh_layer_id]]-th dpb_parameters() syntax structure in the VPS.

A gdr_enabled_flag equal to 1 indicates that progressive decoding update (GDR) pictures may be present in the CLVS referencing the SPS. A gdr_enabled_flag of 0 indicates that GDR images are not present in the CLVS referencing the SPS.

The sps_sub_layer_dpb_params_flag is used to control the presence of the max_dec_pic_buffering_minus1[ i ], max_num_reorder_pics[ i ] and max_latency_increase_plus1[ i ] syntax elements in the dpb_parameters() syntax structure in SPS. If it is missing, sps_sub_dpb_params_info_present_flag is assumed to be 0.

A long_term_ref_pics_flag of 0 indicates that LTRP is not used for inter-picture prediction of any coded picture in CLVS. A long_term_ref_pics_flag equal to 1 indicates that LTRPs can be used for inter-picture prediction of one or more coded pictures in CLVS.

The overall profile, rank and semantics of the layer are as follows.

The profile_tier_level() syntactic structure provides tier information and, optionally, profile, rank, subprofile, and general constraint information (denoted PT information).

When the profile_tier_level() syntax structure is included in the DPS, OlsInScope is an OLS that includes all layers in the entire bitstream that references the DPS. When the profile_tier_level() syntax structure is enabled in a VPS, OlsInScope is one or more OLSs specified by the VPS. When the profile_tier_level() syntax structure is included in the SPS, OlsInScope is an OLS that includes only the bottom layer among the layers that reference the SPS, which must be an independent layer.

general_profile_idc indicates the profile to which OlsInScope corresponds, as specified in Annex A of the VVC standard. Bitstreams shall not contain general_profile_idc values other than those specified in Annex A. Other general_profile_idc values are reserved for future use by ITU-T | ISO/IEC.

general_tier_flag specifies the rank context for interpreting the general_level_idc, as specified in Appendix A.

num_sub_profiles specifies the number of general_sub_profile_idc[ i ] syntax elements.

general_sub_profile_idc[ i ] indicates the i-th cross-compatible metadata registered according to ITU-T Recommendation T.35, the contents of which are not specified in the VVC standard.

general_level_idc indicates the level to which OlsInScope corresponds, as specified in Annex A. Bitstreams shall not contain general_level_idc values other than those specified in Annex A. Other general_level_idc values are reserved for future use by ITU-T | ISO/IEC.

Note 1: A larger general_level_idc value indicates a higher level. The maximum level signaled in the DPS for the OlsInScope may be higher than the level signaled in the SPS for the CVS contained in the OlsInScope.

Note 2: When OlsInScope matches multiple profiles, the general_profile_idc shall indicate the profile that provides the preferred decoding result or preferred bitstream identification as determined by the encoder (in a manner not specified herein).

Note 3: When the syntax structure profile_tier_level() included in DPS and CVS OlsInScope corresponds to different profiles, general_profile_idc and level_idc must indicate the profile and level for a decoder capable of decoding OlsInScope.

sub_layer_level_present_flag[ i ] equal to 1 indicates that layer information is present in the syntactic structure of profile_tier_level() to represent the sublayer with TemporalId equal to i. sub_layer_level_present_flag[ i ] equal to 0 indicates that there is no level information in the syntactic structure of profile_tier_level() to represent the sublayer with TemporalId equal to i.

The value of ptl_alignment_zero_bits must be 0.

The semantics of the sub_layer_level_idc[ i ] syntax element are, except for the description of the output of missing values, the same as the general_level_idc syntax element, but apply to the representation of a sublayer with a TemporalId equal to i.

The semantics of DPB is presented as follows.

The dpb_parameters(maxSubLayersMinus1, subLayerInfoFlag) syntax structure provides information about the DPB size, the maximum number of image reorders, and the maximum latency for each CVS CLVS.

When the dpb_parameters() syntax structure is included in a VPS, the OLSs to which the dpb_parameters() syntax structure applies are specified by the VPS. When the dpb_parameters() syntax structure is included in the SPS, it applies to the OLS, which includes only the bottom layer among the layers referencing the SPS, which must be an independent layer.

max_dec_pic_buffering_minus1[ i ] plus 1 specifies, for each CVS CLVS, the maximum required decoded picture buffer size in units of picture storage buffers when the value of Htid is i. The value of max_dec_pic_buffering_minus1[ i ] must be in the range 0 to MaxDpbSize - 1, inclusive, where MaxDpbSize is specified elsewhere. When i is greater than 0, max_dec_pic_buffering_minus1[ i ] must be greater than or equal to max_dec_pic_buffering_minus1[i - 1]. When max_dec_pic_buffering_minus1[ i ] is missing for i in the range 0 to maxSubLayersMinus1 - 1 inclusive, because subLayerInfoFlag is 0, its value is assumed to be max_dec_pic_buffering_minus1[ maxSubLayersMinus1 ].

max_num_reorder_pics[ i ] specifies, for each CLVS CVS, the maximum allowed number of CLVS pictures that can precede any picture in the CLVS in decoding order and follow that picture in output order when the value of Htid is i. The max_num_reorder_pics[ i ] value must be between 0 and max_dec_pic_buffering_minus1[ i ], inclusive. When i is greater than 0, max_num_reorder_pics[ i ] must be greater than or equal to max_num_reorder_pics[i - 1]. When max_num_reorder_pics[ i ] is missing for i in the range 0 to maxSubLayersMinus1 - 1 inclusive, because the subLayerInfoFlag is 0, it is assumed to be equal to max_num_reorder_pics[ maxSubLayersMinus1 ].

max_latency_increase_plus1[ i ], not equal to 0, is used to calculate the value MaxLatencyPictures[ i ], which specifies, for each CLVS in CVS, the maximum number of images in the CLVS that can precede any image in the CLVS in output order and follow that image in output order decoding when Htid is equal to i.

When max_latency_increase_plus1[ i ] is not equal to 0, the value of MaxLatencyPictures[ i ] is specified as follows:

MaxLatencyPictures[ i ] =

max_num_reorder_pics[ i ] + max_latency_increase_ plus1[ i ] - 1

When max_latency_increase_plus1[ i ] is 0, the corresponding limit is not expressed.

The value of max_latency_increase_plus1[ i ] must be in the range 0 to 232 - 2 inclusive. When max_latency_increase_plus1[ i ] is missing for i in the range 0 to maxSubLayersMinus1 - 1 inclusive, because the subLayerInfoFlag is 0, it is assumed to be equal to max_latency_increase_plus1[ maxSubLayersMinus1 ].

The semantics of HRD parameters includes the semantics of general HRD parameters.

The semantics of the general HRD parameters is presented as follows.

The general_hrd_parameters() syntax structure provides the HRD parameters used in HRD operations.

num_ols_hrd_params_minus1 plus 1 indicates the number of ols_hrd_parameters() syntax structures present in the general_hrd_parameters() syntax structure. The value of num_ols_hrd_params_minus1 must be in the range from 0 to 73 inclusive. When TotalNumOlss is greater than 1, the value of num_ols_hrd_params_minus1 is assumed to be 0.

The parameter hrd_cpb_cnt_minus1 plus one specifies the number of alternative CPB specifications in the CVS bitstream. The value of hrd_cpb_cnt_minus1 must be in the range from 0 to 31 inclusive.

hrd_max_temporal_id[ i ] specifies the TemporalId of the top layer view for which the HRD parameters are contained in the i-th syntactic structure layer_level_hrd_parameters( ). The value of hrd_max_temporal_id[ i ] must be in the range from 0 to vps_max_sub_layers_minus1 inclusive. When vps_max_sub_layers_minus1 is zero, the value of hrd_max_temporal_id[ i ] is assumed to be 0.

ols_hrd_idx[ i ] specifies the index of the ols_hrd_parameters() syntax structure that applies to the i-th OLS. The value of ols_hrd_idx[[ i ] must be in the range from 0 to num_ols_hrd_params_minus1 inclusive. If this value is missing, the value of ols_hrd_idx[[ i ] is assumed to be 0.

The semantics of the reference image list structure is presented as follows.

The syntax structure ref_pic_list_struct(listIdx, rplsIdx) can be present in the SPS or in the slice header. Depending on whether the syntax structure is included in the slice header or in the SPS, the following applies:

- The syntactic structure ref_pic_list_struct(listIdx, rplsIdx), if present in the slice header, specifies the reference image list listIdx of the current image (the image containing the slice).

- Otherwise, the syntactic structure ref_pic_list_struct( listIdx, rplsIdx ) (present in SPS) specifies a candidate for the listIdx of reference pictures, and the term "current picture" in the semantics specified in the remainder of this clause refers to each picture that 1) has one or more slices containing ref_pic_list_idx[ listIdx ] equal to the index in the list of syntax structures ref_pic_list_struct( listIdx, rplsIdx ) included in the SPS, and 2) located in the CVS referencing the SPS.

num_ref_entries[ listIdx ][ rplsIdx ] specifies the number of entries in the syntax structure ref_pic_list_struct( listIdx, rplsIdx ). The value of num_ref_entries[listIdx][rplsIdx] must be in the range 0 to MaxDecPicBuffMinus1 + 14, inclusive.

VPS 704, SPS 706 and PPS 708 are contained in different types of Network Abstraction Layer (NAL) units. A NAL unit is a syntactic structure containing an indication of the data type to be followed (eg, encoded video data). NAL units are divided into video coding layer (VCL) units and non-VCL NAL units. VCL NAL units contain data that represents sample values in video images, and non-VCL NAL units contain any associated additional information, such as parameter sets (important data that can be applied to a number of VCL NAL units) and additional extension information (time and other additional data that may enhance the usability of the decoded video signal, but is not necessary for decoding sample values in video images).

In an embodiment, the VPS 704 is contained in a non-VCL NAL unit, designated a VPS NAL unit. Thus, a NAL VPS unit has a NUT VPS. In an embodiment, SPS 706 is a non-VCL NAL unit, designated an SPS NAL unit. Thus, the SPS NAL block has an SPS NUT. In an embodiment, PPS 708 is contained in a non-VCL NAL unit, designated a PPS NAL unit. Thus, a unit of NAL PPS has a NUT PPS.

Slice header 712 is a syntax structure containing syntax elements that apply to all slices of an encoded image (eg, image 725). In an embodiment, the slice header 712 is in the NAL VCL.

The image data 720 includes video data encoded according to inter-layer prediction, intra-frame prediction, or inter-layer prediction, as well as corresponding transformed and quantized residual data. As shown in FIG. 7, image data 720 includes one or more OLS 721. The one or more OLS 721 are similar to OLS 1, OLS 2 and OLS 3 shown in FIG. 6. Each OLS 721 includes one or more layers 723. Layers 723 are similar to layers 531-532 shown in FIG. 5. Each of the layers 723 includes one or more images 725. The images 725 are similar to the images 615-618, 611-614, 641-644, 651-654 and 661-664 shown in FIG. 6.

Each image 725 is an array of luminance samples in monochrome format or an array of luminance samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2 and 4:4:4 color formats. Images 725 can be either a frame or a field. However, in one CVS (eg, CVS 790), either all images 725 are frames or all images 725 are fields. CVS 790 is a coded video sequence for each coded layer video sequence (CLVS) in video bitstream 700. Notably, CVS 790 and CLVS are the same when video bitstream 700 includes a single layer. CVS 790 and CLVS are different only when the video bitstream 700 includes multiple layers (eg, as shown in FIGS. 5 and 6).

Each image 725 contains one or more slices 727. A slice 727 represents an integer number of full tiles or an integer number of consecutive full CTU lines in an image tile (eg, image 725). Each slice 727 is contained exclusively in one NAL unit (eg, a VCL NAL unit). A tile (not shown) is a rectangular CTU area in a particular column of tiles and a particular row of tiles in an image (eg, image 725). A CTU (not shown) represents a CTB of luma samples, two corresponding CTBs of chroma samples of an image that has three arrays of samples, or a CTB of samples of a monochrome image or an image that is encoded using three separate color planes and the syntax structures used to encode the samples. A CTB (not shown) is a block of N×N samples for some value of N, so that dividing a component into a CTB is a partition. A block (not shown) is an MxN array (M columns by N rows) of samples (eg, pixels) or an MxN array of transform coefficients.

The images 725 and their slices 727 contain data associated with the images or videos to be encoded or decoded. Thus, the images 725 and their slices 727 may simply be referred to as the payload or data carried in the bitstream 700.

Those skilled in the art will appreciate that in practical use, bit stream 700 may contain other parameters and information.

The subbitstream 701 may be extracted from the bitstream 700 in accordance with the subbitstream extraction process 703. The subbitstream extraction process 703 is a predetermined mechanism that removes NAL units from the bitstream 700 that are not part of the target set, resulting in an output subbitstream 701 that includes NAL units that are included in the target set. The sub-bitstream extraction process 703 may be performed by an encoder or associated slicer configured to dynamically change the bitstream 700 based on user behavior/requests.

Since SPS 706 contains dpb_parameters() for the i-th OLS 784, when the i-th OLS contains one single layer (eg, OLS 3 in FIG. 6), the VPS 704 can be removed during the extraction process 703. That is, the bit substream 701 does not need to be included by the VPS 704 in the bit substream 701 since identical dpb_parameters() for the i-th OLS 784 are also present in the SPS 706. Thus, the decoder can obtain the dpb_parameters() for the i-th OLS 784 from SPS 706 when the i-th OLS contains one single layer.

In fig. 8 illustrates an embodiment of a decoding method 800 implemented by a video decoder (eg, video decoder 400). Method 800 may be performed after the bit stream has been directly or indirectly received from a video encoder (eg, video encoder 300). Method 800 improves the decoding process by including decoded picture buffer (DPB) parameters in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

At step 802, the video decoder receives a bit substream (eg, bit substream 701) having a set of sequence parameters (eg, SPS 706) with a decoded picture buffer (DPB) syntax structure (eg, dpb_parameters()) for the i-th OLS 784) for the set output layers (OLS) with one single layer (for example, OLS 3).

In an embodiment, the DPB syntax structure is denoted dpb_parameters(). In an embodiment, the single layer is the i-th layer, and where OLS is the i-th OLS. In an embodiment, the DPB syntax structure is present in the i-th profile_tier_level() syntax structure in the SPS. In an embodiment, the subbit stream does not include a video parameter set (VPS 704). That is, the VPS was removed as part of the extraction process 703.

At step 804, the video decoder obtains the DPB syntax structure for the OLS with one layer from the SPS. In an embodiment, the SPS is referenced by a single layer in the OLS.

At step 806, the video decoder decodes an image (eg, image 725) of one single layer using a DPB syntax structure to obtain a decoded image. In an embodiment, the decoded image is stored in a decoded picture buffer (DPB) before or before display.

Once the image has been decoded, the image can be used to generate or obtain an image or video sequence for display to a user on the display or screen of an electronic device (eg, smartphone, tablet computer, laptop, personal computer, etc.).

In fig. 9 illustrates an embodiment of a method 900 for encoding a video bitstream implemented by a video encoder (eg, video encoder 300). Method 900 may be performed when an image (eg, from a video) needs to be encoded into a video bit stream and then transmitted to a video decoder (eg, video decoder 400). Method 900 improves the encoding process by including decoded picture buffer (DPB) parameters in a sequence parameter set (SPS) when the output layer set (OLS) contains a single layer. Since SPS includes DPB parameters for single-layer OLS, the video parameter set (VPS) can be removed from the bitstream, reducing redundancy and improving coding efficiency. Thus, the encoder/decoder (aka "codec") for video encoding is improved compared to modern codecs. From a practical standpoint, the improved video encoding process provides the user with a better user experience when sending, receiving and/or viewing video.

At step 902, the video encoder generates a video parameter set (eg, VPS 704) and a sequence parameter set (eg, SPS 706). The VPS includes a list of decoded picture buffer (DPB) syntax structures for all sets of output layers (eg, PTL syntax structures 780). The SPS includes a DPB syntax structure for each OLS having one single layer (eg, dpb_parameters() for the i-th OLS 784). For example, the SPS includes the DPB syntax structure for OLS 3 in FIG. 6, which is denoted dpb_parameters() for OLS3 in FIG. 7. SPS does not include the DPB syntax structure for any OLS that has more than one layer. For example, the SPS does not include the DPB syntax structure for OLS 1 and OLS 2 in FIG. 6, which is denoted dpb_parameters() for OLS2 and dpb_parameters() for OLS3 in FIG. 7.

In an embodiment, the VPS includes an index for a list of PTL syntax structures, and where the index is denoted as vps_ols_dpb_params_idx[i]. In an embodiment, the value of vps_ols_dpb_params_idx[ i ] ranges from zero to the total number of DPB syntax structures in the VPS. In an embodiment, the VPS includes vps_num_dpb_params_minus1, and where vps_num_dpb_params_minus1 plus 1 indicates the total number of DPB syntax structures.

At step 904, the video encoder encodes the VPS and SPS into a video bit stream. At step 906, the video encoder stores the video bit stream for transmission to the video decoder. In an embodiment, method 900 further includes extracting a bit stream (eg, bit substream 701) from a bit stream (eg, bit stream 700). As described above, extraction process 703 retains the SPS but removes the VPS. That is, the sub-bitstream includes the SPS from the bitstream, but does not include the VPS from the bitstream.

Similar to a video bitstream, a subbitstream can be stored in memory for transmission to a video decoder. In an embodiment, the video bit stream and/or sub-bit stream is transmitted to a video decoder. The encoded video bit stream and/or the encoded sub-bit stream, when received by a video decoder, may be decoded (e.g., as described above) to generate or obtain an image or video sequence for display to the user on the display or screen of an electronic device (e.g., smartphone, tablet computer, laptop, personal computer, etc.).

Additional and/or added details are provided below.

The general decoding process is as follows.

The input to this process is the bitstream BitstreamToDecode. The output of this process is a list of decoded images.

The decoding process is specified such that all decoders that match a given profile and level produce numerically identical framed decoded output pictures when invoking the decoding process associated with that profile for a bitstream corresponding to that profile and level. Any decoding process that produces cropped decoded output images identical to those produced by the process described in this document (with the correct output order or output timing as specified) meets the VVC standard decoding process requirements.

For each IRAP AU in the bitstream, the following applies.

- If the AU is the first AU in the bitstream in decoding order, each picture is an instantaneous decoding update (IDR) picture, or each picture is the first layer picture that follows the end of the sequence of NAL units in decoding order, the NoIncorrectPicOutputFlag variable is set to 1.

- Otherwise, if any external facility not specified in this description is available to set the HandleCraAsCvsStartFlag variable to a value for the AU, HandleCraAsCvsStartFlag is set to the value provided by the external facility, and NoIncorrectPicOutputFlag is set to HandleCraAsCvsStartFlag.

- Otherwise, HandleCraAsCvsStartFlag and NoIncorrectPicOutputFlag are set to 0.

For each GDR AU in the bitstream, the following applies.

- If the AU is the first AU in the bitstream in decoding order, or each picture is the first layer image that follows the end-of-sequence NAL unit in decoding order, the NoIncorrectPicOutputFlag variable is set to 1.

- Otherwise, if any external facility not specified herein is available to set the HandleGdrAsCvsStartFlag variable to a value for the AU, HandleGdrAsCvsStartFlag is set to the value provided by the external facility, and NoIncorrectPicOutputFlag is set to HandleGdrAsCvsStartFlag.

- Otherwise, HandleGdrAsCvsStartFlag and NoIncorrectPicOutputFlag are set to 0.

Note: The above operations for both IRAP and GDR images are required to identify the CVS in the bitstream.

Clause 8.1.2 is called repeatedly for each encoded image in BitstreamToDecode in decoding order.

The decoding process for generating reference picture lists is as follows.

This process is called at the beginning of the decoding process for each layer of the non-IDR image.

The reference images are addressed by reference indices. The reference index is an index in the list of reference images. In the case of I-slice decoding, the reference picture list is not used when decoding the slice data. In the case of P-slice decoding, only the reference picture list 0 (ie, RefPicList[ 0 ]) is used when decoding the slice data. In the case of B-slice decoding, both reference picture list 0 and reference picture list 1 (ie, RefPicList[1]) are used when decoding the slice data.

The following restrictions apply to the bitstream compliance requirement.

- For each i equal to 0 or 1, num_ref_entries[ i ][RplsIdx[ i ]] must not be less than NumRefIdxActive[ i ].

- The image referenced by each active entry in RefPicList[ 0 ] or RefPicList[1] must be present in the DPB and must have a TemporalId less than or equal to the TemporalId value of the current image.

- The picture referenced by each entry in RefPicList[ 0 ] or RefPicList[1] must not be the current picture and must have a non_reference_picture_flag equal to 0.

- A short-term reference picture (STRP) entry in the RefPicList[ 0 ] or RefPicList[1] of an image slice and an LTRP entry in the RefPicList[ 0 ] or RefPicList[1] of the same slice or another slice of the same image shall not refer to the same image.

- RefPicList[ 0 ] or RefPicList[1] must not contain LTRP entries for which the difference between the PicOrderCntVal of the current image and the PicOrderCntVal of the image referenced by the entry is greater than or equal to 224.

- Let setOfRefPics be a set of unique images referenced by all entries in RefPicList[ 0 ] that have the same nuh_layer_id as the current image, and all entries in RefPicList[1] that have the same nuh_layer_id as the current image. The number of images in setOfRefPics must be less than or equal to MaxDecPicBuffMinus1, and setOfRefPics must be the same for all image slices.

- When the current image is a step-by-step temporal sublayer access (STSA) image, there must be no active entry in RefPicList[ 0 ] or RefPicList[1] that has a TemporalId equal to the ID of the current image.

When the current picture is a picture that follows in decoding order an STSA picture that has a TemporalId equal to the ID of the current picture, there shall not be a picture that has a TemporalId equal to the value of the current picture included as the active entry in RefPicList[ 0 ] or RefPicList [1], which precedes the STSA image in decoding order.

- The image referenced by each ILRP entry in the RefPicList[ 0 ] or RefPicList[1] of the current image slice must be in the same access unit as the current image.

- The image referenced by each ILRP entry in the RefPicList[ 0 ] or RefPicList[1] of the current image slice must be present in the DPB and must have a nuh_layer_id less than that of the current image.

- Each ILRP entry in a slice's RefPicList[ 0 ] or RefPicList[1] must be an active entry.

General aspects of HRD in clause C.1 of the VVC standard include the following.

This appendix specifies a hypothetical reference decoder (HRD) and its use for checking bitstream and decoder compliance.

A set of bitstream consistency tests is required to test the consistency of a bitstream referred to as the entire bitstream, designated fullBitstream. A set of bitstream conformance tests are designed to check the conformance of each OP of each OLS specified by the VPS.

For each test, the following ordered steps apply in the order listed, followed by the processes described after those steps in this clause:

1. The test operating point, designated targetOp, is selected by selecting the target OLS with the OLS index opOlsIdx and the highest OpTid TemporalId value. The opOlsIdx value ranges from 0 to TotalNumOlss - 1, inclusive. The opTid value ranges from 0 to vps_max_sub_layers_minus1 inclusive. Each pair of selected values of opOlsIdx and opTid shall be such that the subbitstream output of a call to the subbitstream extraction process as specified in clause C.6, with the entire bitstreams, opOlsIdx and opTid as input, satisfies the following conditions:

- There is at least one VCL NAL unit with nuh_layer_id equal to each of the nuh_layer_id values in LayerIdInOls[opOlsIdx] in BitstreamToDecode.

- There is at least one VCL NAL unit with TemporalId equal to opTid in BitstreamToDecode.

2. If the layers in targetOp include all layers in fullBitstream, and opTid is greater than the highest TemporalId among all NAL units in fullBitstream, BitstreamToDecode is set to totalBitstream. Otherwise, BitstreamToDecode is set as the output by calling the bitstream extraction process as specified in clause C.6 with the entireBitstream, opOlsIdx, and opTid as input.

3. The TargetOlsIdx and Htid values are set to opOlsIdx and opTid, respectively, for targetOp.

4. ScIdx value is selected. The selected ScIdx value must be in the range from 0 to hrd_cpb_cnt_minus1 inclusive.

5. The access unit in the BitstreamToDecode parameter associated with the buffer period SEI messages (either present in the TargetLayerBitstream or accessible through an external means not specified herein) applicable to the TargetOlsIdx is selected as the HRD initialization point and is referred to as access unit 0 for each layer in the target OLS.

6. The ols_hrd_parameters() syntax structure and the sub_layer_hrd_parameters() syntax structure applied to BitstreamToDecode are selected as follows:

- The syntax structure ols_hrd_idx[TargetOlsIdx]-th ols_hrd_parameters() is selected in the VPS (or provided by an external tool not specified in this description).

- In the selected syntax structure ols_hrd_parameters( ), if BitstreamToDecode is a type I bitstream, the syntax structure sub_layer_hrd_parameters( Htid ) is selected, which immediately follows the condition "if( general_vcl_hrd_params_present_flag )", and the variable NalHrdModeFlag is set to 0; otherwise (when BitstreamToDecode is a Type II bitstream), the syntax structure is sub_layer_hrd_parameters( Htid ), which immediately follows either the "if( general_vcl_hrd_params_present_flag )" condition (in which case the variable NalHrdModeFlag is set to 0) or the "if( general_nal_hrd_params_present_flag )" condition. " (in this case, the NalHrdModeFlag variable is set to 1). When BitstreamToDecode is a Type II bitstream and the NalHrdModeFlag flag is 0, all non-VCL NAL units except filler data NAL units, and all syntax elements leading_zero_8bits, zero_byte, start_code_prefix_one_3bytes, and trailing_zero_8bits that form a byte stream from the stream of NAL units (as specified in Appendix B), if present, are discarded from BitstreamToDecode and the remaining bitstream is assigned to the BitstreamToDecode parameter.

7. When the decoding_unit_hrd_params_present_flag flag is 1, the CPB is scheduled to operate either at the access unit level (in which case the DecodingUnitHrdFlag variable is set to 0) or at the decoding unit level (in which case the DecodingUnitHrdFlag variable is set to 1). Otherwise, the DecodingUnitHrdFlag flag is set to 0 and the CPB is scheduled to operate at the access unit level.

8. For each access unit in BitstreamToDecode, starting with access unit 0, a buffer period SEI message (either present in BitstreamToDecode or accessible through an external means not specified) is selected that is associated with the access unit and applied to TargetOlsIdx, a synchronization SEI message is selected image (present in BitstreamToDecode or accessible through an external means not specified herein) that is associated with the access unit and applied to the TargetOlsIdx, and when the DecodingUnitHrdFlag flag is 1 and the decoding_unit_cpb_params_in_pic_timing_sei_flag is 0, the decoding unit information SEI messages ( present in BitstreamToDecode or accessible through external means not specified herein) that are associated with decoding units in an access unit and applied to TargetOlsIdx.

Each matching test consists of a combination of one option in each of the above steps. When more than one option exists for a step, only one option is selected for any given compliance test. All possible combinations of all stages form a complete set of compliance tests. For each operating point tested, the number of bitstream compliance tests performed is n0 * n1 * n2 * n3, where the values of n0, n1, n2 and n3 are given as follows:

- n1 is equal to hrd_cpb_cnt_minus1 + 1.

- n1 represents the number of access units in BitstreamToDecode that are associated with buffer period SEI messages.

- n2 is obtained as follows:

- If BitstreamToDecode is a Type I bitstream, n0 is 1.

- Otherwise (when BitstreamToDecode is a Type II bitstream) n0 is 2.

- n3 is obtained as follows:

- If the decoding_unit_hrd_params_present_flag is 0, n3 is 1.

- Otherwise n3 is equal to 2.

The HRD contains a bitstream extractor (optional), a coded picture buffer (CPB), a flash decoding process, a decoded picture buffer (DPB), which conceptually contains a sub-DPB for each layer, and an output crop.

For each bitstream matching test, the CPB size (number of bits) is equal to CpbSize[ Htid ][ ScIdx ] as specified in clause 7.4.5.2, where ScIdx and HRD parameters are specified above in this clause, and DPB parameters max_dec_pic_buffering_minus1 [ Htid ], max_num_reorder_pics The [ Htid ] and MaxLatencyPictures[ Htid ] for each layer are found in or derived from the dpb_parameters() syntax structure, which is applied to the layer depending on whether the layer is an independent layer and whether the layer is the output layer of the target OLS.

HRD works as follows.

- The HRD is initialized at decode block 0, with both the CPB and each sub-DPB in the DPB set to empty (the sub-DPB padding for each sub-DPB is set to 0).

Note: Once initialized, the HRD is not reinitialized by subsequent buffer period SEI messages.

- Data associated with decoding units that arrive at each CPB according to a given arrival schedule are delivered by a hypothetical stream scheduler (HSS).

- The data associated with each decoding unit is removed and decoded instantly in the instantaneous decoding process at the time the CPB of the decoding unit is deleted.

- Each decoded image is placed in DPB.

- The decoded picture is removed from the DPB when it is no longer required for inter-prediction reference and is no longer required for output.

The decoded image buffer is running.

The specifications in this section apply independently to each set of decoded picture buffer (DPB) parameters selected as specified in clause C.1.

The decoded image buffer is conceptually composed of sub-DPBs, and each sub-DPB contains image storage buffers for storing decoded images of one layer. Each of the image storage buffers may contain a decoded image that is marked as "used for reference" or stored for future output. The processes specified in clauses C3.2, C3.3 and C3.4 are applied sequentially as follows and are applied independently for each layer, starting with the lowest layer in the OLS, in ascending order of the nuh_layer_id values of the layers in the OLS. When these processes are applied to a specific layer, only the sub-DPB for the specific layer is affected. In descriptions of these processes, DPB refers to the sub-DPB for a particular layer, and the particular layer is referred to as the current layer.

Note: In the DPB output synchronization operation, decoded pictures with PicOutputFlag equal to 1 in the same access unit are output sequentially in ascending order of the nuh_layer_id values of the decoded pictures.

Let image n and the current image be the encoded image or decoded image of access unit n for a particular nuh_layer_id value, where n is a non-negative integer.

The following describes how to remove images from the DPB before decoding the current image.

Removing pictures from the DPB before decoding the current picture (but after parsing the slice header of the first slice of the current picture) occurs instantly during CPB removal of the first decoding unit of access unit n (containing the current picture) and is done as follows:

- The decoding process is started to list the reference pictures as specified in clause 8.3.2, and the decoding process is started to mark the reference pictures as specified in clause 8.3.3.

- When the current AU is a CVSS AU that is not AU 0, the following ordered steps apply:

1. The NoOutputOfPriorPicsFlag variable is obtained for the decoder under test as follows:

- If the pic_width_max_in_luma_samples, pic_height_max_in_luma_samples, chroma_format_idc, separate_colour_plane_flag, bit_depth_luma_minus8, bit_depth_chroma_minus8 or max_dec_pic__minus1 [Htid] value obtained for any image in the current AU differs from the pic_width_in_luma_samples values, height_in_luma_samples, chroma_format_idc, separate_colour_plane_, bit_depth_luma_minus8, bit_depth_chrom a_minus8 or max_dec_pic_buffering_minus1[ Htid ], , obtained respectively for the previous image in the same CLVS, the NoOutputOfPriorPicsFlag flag may (but should not) be set by the decoder under test to 1, regardless of the value of no_output_of_prior_pics_flag.

Note: Although setting NoOutputOfPriorPicsFlag equal to no_output_of_prior_pics_flag is preferred under these conditions, the decoder under test is allowed to set NoOutputOfPriorPicsFlag to 1 in this case.

- Otherwise, NoOutputOfPriorPicsFlag is set to no_output_of_prior_pics_flag.

2. The NoOutputOfPriorPicsFlag value obtained for the decoder under test is applied to the HRD, so when the NoOutputOfPriorPicsFlag value is 1, all image storage buffers in the DPB are cleared without outputting the images they contain, and the DPB occupancy is set to 0.

- When both of the following conditions are met for any k images in the DPB, all such k images in the DPB are removed from the DPB:

- image k is marked as "not used for reference".

- picture k has a PictureOutputFlag equal to 0, or its DPB output time is less than or equal to the CPB removal time of the first decoding unit (denoted as decoding unit m) of the current picture n; that is, DpbOutputTime[ k ] is less than or equal to DuCpbRemovalTime[ m ].

- For each image that is removed from the DPB, the DPB padding is reduced by one.

Below is a description of the operation of DPB in order of output.

The specifications in this clause apply independently to each set of decoded picture buffer (DPB) parameters selected as specified in clause C.1.

The decoded image buffer is conceptually composed of DPB sub-blocks, and each DPB sub-block contains image storage buffers for storing decoded images of one layer. Each of the image storage buffers contains a decoded image, marked as "used for reference" or saved for future output.

The process of outputting and removing pictures from the DPB is initiated before decoding the current picture as specified in clause C.5.2.2, followed by the process of marking and storing the current decoded picture as specified in clause C.3.4, and finally by calling the process additional ejection as specified in paragraph C.5.2.3. The "push" process is specified in clause C.5.2.4 and is initiated as specified in clauses C.5.2.2 and C.5.2.3.

These processes are applied independently for each layer, starting with the bottom layer in OLS, in ascending order of the nuh_layer_id values of the layers in OLS. When these processes are applied to a specific layer, only the sub-DPB for the specific layer is affected.

Note: In the output queuing DPB operation, as well as in the output synchronization DPB operation, decoded pictures with PicOutputFlag equal to 1 in the same access unit are also output sequentially in ascending order of the nuh_layer_id values of the decoded pictures.

Let image n and the current image be the encoded image or decoded image of access unit n for a particular nuh_layer_id value, where n is a non-negative integer.

The following describes how to output and delete images from DPB.

The output and removal of pictures from the DPB before decoding the current picture (but after parsing the slice header of the first slice of the current picture) occurs instantly when the first decoding unit of the access unit containing the current picture is removed from the DPB and acts as follows:

- The decoding process for listing the reference pictures as specified in clause 8.3.2 and the decoding process for marking the reference pictures as specified in clause 8.3.3 are initiated.

- If the current AU is a CVSS AU that is not AU0, the following ordered steps apply:

1. The NoOutputOfPriorPicsFlag variable is obtained for the decoder under test as follows:

- If the pic_width_max_in_luma_samples, pic_height_max_in_luma_samples, chroma_format_idc, separate_colour_plane_flag, bit_depth_luma_minus8, bit_depth_chroma_minus8 or max_dec_pic__minus1 [Htid] value obtained for any image of the current AU differs from the pic_width_in_luma_samples value, ight_in_luma_samples, chroma_format_idc, separate_colour_plane, bit_depth_luma_minus8, bit_depth_chroma_minus8 or max_dec_pic_buffering_[ Htid ] obtained, respectively , for a previous picture in the same CLVS, NoOutputOfPriorPicsFlag may (but should not) be set by the decoder under test to 1, regardless of the value of no_output_of_prior_pics_flag.

Note: Although setting the NoOutputOfPriorPicsFlag to no_output_of_prior_pics_flag is preferable under these conditions, the decoder under test is allowed to set NoOutputOfPriorPicsFlag to 1 in this case.

- Otherwise, NoOutputOfPriorPicsFlag is set to no_output_of_prior_pics_flag.

2. The NoOutputOfPriorPicsFlag value obtained for the decoder under test is applied to the HRD as follows:

- If the NoOutputOfPriorPicsFlag flag is 1, all image storage buffers in the DPB are cleared without outputting the images they contain, and the DPB padding is set to 0.

- Otherwise (when the value of NoOutputOfPriorPicsFlag is 0), all image storage buffers containing an image that is marked as "not needed for output" and "not used for reference" are flushed (no output), and all image storage buffers in the DPB are flushed are cleared by repeatedly calling the "push" process specified in clause C.5.2.4, and the DPB padding is set to 0.

- Otherwise (when the current image is not a CLVSS image), all image storage buffers containing an image that is marked as "not needed for output" and "not used for reference" are cleared (without output). For each image storage buffer cleared, the DPB fill value is decreased by one. When one or more of the following conditions are true, the "push" process specified in clause C.5.2.4 is re-invoked, further reducing the DPB fill by one for each additional image storage buffer that is flushed, until the one of the following conditions:

The number of images in the DPB that are marked as "required for output" is greater than max_num_reorder_pics[ Htid ].

The value of max_latency_increase_plus1[ Htid ] is not equal to 0, and there is at least one image in the DPB that is marked as "required for output" for which the associated PicLatencyCount variable is greater than or equal to MaxLatencyPictures[ Htid ].

The number of images in DPB is greater than or equal to max_dec_pic_buffering_minus1[Htid] + 1.

Below is a further description of the ejection process.

The processes specified in this section occur instantly when the last decoding unit from access unit n containing the current picture is removed from the CPB.

When the current image has a PictureOutputFlag equal to 1, for each image in the DPB that is marked as "needed for output" and follows the current image in output order, the associated PicLatencyCount variable is set to PicLatencyCount + 1.

The following applies:

- If the currently decoded picture has a PictureOutputFlag of 1, it is marked as "needed for output" and its associated PicLatencyCount variable is set to 0.

- Otherwise (when the currently decoded picture has a PictureOutputFlag value of 0) it is marked as “not needed for output”.

When one or more of the following conditions are true, the "push" process specified in clause C.5.2.4 is called repeatedly until one of the following conditions is met:

- The number of images in DPB marked as "required for output" is greater than max_num_reorder_pics[ Htid ].

- max_latency_increase_plus1[ Htid ] is not equal to 0 and the DPB has at least one image marked as "needed for output" for which its associated PicLatencyCount variable is greater than or equal to MaxLatencyPictures[ Htid ].

Below is a description of the ejection process.

The “pushing” process consists of the following ordered steps:

The image or images that are output first are selected as having the lowest PicOrderCntVal value of all the images in the DPB that are marked as "required for output".

Each of these images, in ascending order by nuh_layer_id, is cropped using a matching crop window for the image, the cropped image is output, and the image is marked as “not needed for output.”

Each image storage buffer that contains an image marked as "not used for reference" and which was one of the cropped and output images is cleared and the occupancy of the associated sub-DPB is reduced by one.

Note: For any two images picA and picB that belong to the same CVS and are output in a "push process" when picA is produced before picB, the PicOrderCntVal value of picA is less than the PicOrderCntVal value of picB.

The process of extracting a bit stream is as follows.

The inputs to this process are the bitstream inBitstream, the target OLS index targetOlsIdx, and the target largest TemporalId tIdTarget.

The output of this process is the outBitstream.

The bitstream matching requirement for an input bitstream is that any output bitstream resulting from the process specified in this clause with a bitstream targetOlsIdx equal to the index of the OLS list specified by the VPS and tIdTarget equal to any value in the range of 0 up to and including 7 as input, and which satisfies the following conditions must be a corresponding bitstream.

- The output bitstream contains at least one VCL NAL unit with a nuh_layer_id equal to each of the nuh_layer_id values in LayerIdInOls[ targetOlsIdx ].

- The output bitstream contains at least one VCL NAL unit with a TemporalId equal to tIdTarget.

Note: The corresponding bitstream contains one or more coded slice NAL units with TemporalId equal to 0, but need not contain coded slice NAL units with nuh_layer_id equal to 0.

The output bit stream OutBitstream is obtained as follows.

- The outBitstream bitstream is set to be identical to the inBitstream bitstream.

- Remove from outBitstream all NAL units with a TemporalId greater than tIdTarget.

- Remove from outBitstream all NAL units with nuh_layer_id not included in the LayerIdInOls[ targetOlsIdx ] list.

- Remove from outBitstream all NAL SEI units that contain a scalable nesting SEI message that has a nesting_ols_flag equal to 1 and that does not have an i value in the range 0 to nesting_num_olss_minus1 inclusive, so that NestingOlsIdx[ i ] equals targetOlsIdx.

- When targetOlsIdx is greater than 0, remove from outBitstream all NAL SEI units that contain an unscaled embedded SEI message with a payloadType of 0 (buffering period), 1 (image timing), or 130 (decoding unit information).

The scalable nesting SEI message syntax is provided as follows.

scalable_nesting( payloadSize ) { Descriptor nesting_ols_flag u(1) if( nesting_ols_flag ) { nesting_num_olss_minus1 ue(v) for( i = 0; i <= nesting_num_olss_minus1; i++ ) nesting_ols_idx_delta_minus1[ i ] ue(v) } else { nesting_all_layers_flag u(1) if( !nesting_all_layers_flag ) { nesting_num_layers_minus1 ue(v) for( i = 1; i <= nesting_num_layers_minus1; i++ ) nesting_layer_id[ i ] u(6) } } nesting_num_seis_minus1 ue(v) while( !byte_aligned() ) nesting_zero_bit /* equal to 0 */ u(1) for( i = 0; i <= nesting_num_seis_minus1; i++ ) sei_message( ) }

The approximate general semantics of an SEI payload is as follows.

The following applies to applied layers or OLS of unscaled nested SEI messages.

- For a non-scalable embedded SEI message, when payloadType is zero (buffering period), one (picture timing) or 130 (decoding unit information), the non-scaled embedded SEI message applies only to the 0th OLS.

- For a non-scaled nested SEI message, when payloadType is equal to any value among the VclAssociatedSeiList, the non-scaled nested SEI message applies only to the layer for which the VCL NAL units have a nuh_layer_id equal to the nuh_layer_id of the NAL SEI unit containing the SEI message.

The bitstream compliance requirement is that the following restrictions apply to the nuh_layer_id value of SEI NAL blocks:

- When a non-scaleable nested SEI message has a payloadType value of 0 (buffering period), 1 (picture timing) or 130 (decoding unit information), the NAL SEI unit containing the non-scaleable nested SEI message nuh_layer_id shall be equal to vps_layer_id[ 0 ] .

- When a non-scalable nested SEI message has a payloadType equal to any value in the VclAssociatedSeiList, the NAL SEI unit containing the non-scalable nested SEI message shall have a nuh_layer_id equal to the nuh_layer_id value of the VCL NAL unit associated with the NAL SEI unit.

- A NAL SEI unit containing a scalable nesting SEI message shall have a nuh_layer_id equal to the smallest nuh_layer_id value of all layers to which the scalable nesting SEI message applies (when the nesting_ols_flag of the scalable nesting SEI message is 0), or the smallest nuh_layer_id value of all layers in the OLS to which the scalable nested SEI message is applied (when the nesting_ols_flag of the scalable nested SEI message is 1).

Scalable nesting SEI message semantics are provided.

The scalable embedding SEI message provides a mechanism for associating SEI messages with specific OLSs or with specific layers.

A scalable embedding SEI message contains one or more SEI messages. SEI messages contained in a scalable nested SEI message are also called scalable nested SEI messages.

The bitstream compliance requirement is that the following restrictions apply to the content of SEI messages in a scalable nesting SEI message:

- An SEI message with a payload type of 132 (decoded image hash) or 133 (scalable attachment) shall not be contained in a scalable embedding SEI message.

- When a scalable embedding SEI message contains an SEI message about buffering period, image timing, or decoding unit information, the scalable enclosing SEI message shall not contain any other SEI message with payloadType not equal to 0 (buffering period), 1 (image timing ) or 130 (decoding unit information).

The bitstream compliance requirement is that the following restrictions apply to the nal_unit_type value of the NAL SEI unit containing the scalable nesting SEI message:

- When the scalable embedding SEI message contains an SEI message that has a payloadType value of 0 (buffering period), 1 (picture timing), 130 (decoding unit information), 145 (RAP dependent pointer), or 168 (frame information) field), a NAL SEI unit containing a scalable nesting SEI message shall have a nal_unit_type equal to PREFIX_SEI_NUT.

A nesting_ols_flag value of 1 indicates that scalable nested SEI messages are applied to specific OLSs. A nesting_ols_flag value of 0 indicates that scalable nested SEI messages are applied to specific layers.

The bitstream compliance requirement is that the following restrictions apply to the nesting_ols_flag value:

- When a scalable nesting SEI message contains an SEI message that has a payloadType value of 0 (buffering period), 1 (image timing), or 130 (decoding unit information), the value of nesting_ols_flag shall be equal to 1.

- If a scalable nesting SEI message contains an SEI message that has a payloadType equal to the value in VclAssociatedSeiList, the value of nesting_ols_flag must be 0.

nesting_num_olss_minus1 plus 1 specifies the number of OLSs to which scalable nested SEI messages apply. The value of nesting_num_olss_minus1 must be in the range from 0 to TotalNumOlss - 1, inclusive.

nesting_ols_idx_delta_minus1[ i ] is used to obtain the variable NestingOlsIdx[ i ], which specifies the OLS index of the i-th OLS to which scaled nested SEI messages are applied when nesting_ols_flag is 1. The value of nesting_ols_idx_delta_minus1[i ] must be in the range from 0 to TotalNumOlss - 2 inclusive.

The NestingOlsIdx[ i ] variable is obtained as follows.

if( i == 0 )

NestingOlsIdx[ i ] = nesting_ols_idx_delta_minus1[ i ]

else

NestingOlsIdx[ i ] = NestingOlsIdx[ i - 1 ] + nesting_ols_idx_delta_minus1[ i ] + 1

A nesting_all_layers_flag value of 1 indicates that scalable nested SEI messages are applied to all layers that have a nuh_layer_id greater than or equal to the nuh_layer_id of the current NAL SEI unit. A nesting_all_layers_flag value of 0 indicates that scalable nested SEI messages may or may not apply to all layers whose nuh_layer_id is greater than or equal to the nuh_layer_id of the current NAL SEI unit.

nesting_num_layers_minus1 plus 1 specifies the number of layers to which scalable nested SEI messages are applied. The nesting_num_layers_minus1 value must be in the range 0 to vps_max_layers_minus1 - GeneralLayerIdx[nuh_layer_id] inclusive, where nuh_layer_id is the nuh_layer_id of the current NAL SEI unit.

nesting_layer_id[ i ] specifies the nuh_layer_id value of the ith layer to which scaled nested SEI messages are applied when nesting_all_layers_flag is 0. The nesting_layer_id[ i ] value must be greater than nuh_layer_id, where nuh_layer_id represents the nuh_layer_id of the current NAL SEI unit.

When nesting_ols_flag is 0, a NestingNumLayers variable indicating the layer number to which scalable nested SEI messages are applied, and a list of NestingLayerId[ i ] for i ranging from 0 to NestingNumLayers - 1 inclusive, specifying the list of nuh_layer_id values of layers to which scalable nesting is applied SEI messages are obtained as follows, where nuh_layer_id is the nuh_layer_id of the current NAL SEI unit.

if( nesting_all_layers_flag ) {

NestingNumLayers =

ps_max_layers_minus1 + 1 - GeneralLayerIdx[ nuh_layer_id ]

for( i = 0; i < NestingNumLayers; i++)

NestingLayerId[ i ] = vps_layer_id[ GeneralLayerIdx[ nuh_layer_id ] + i ]

} else {

NestingNumLayers = nesting_num_layers_minus1 + 1

for( i = 0; i < NestingNumLayers; i++)

NestingLayerId[ i ] = ( i = = 0 ) ? nuh_layer_id : nesting_layer_id[ i ]

}

}

nesting_num_seis_minus1 plus 1 specifies the number of nested SEI messages to scale. The nesting_num_seis_minus1 value must be in the range 0 to 73, inclusive.

The value of nesting_zero_bit must be 0.

In fig. 10 is a schematic diagram of a video encoding apparatus 1000 (eg, video encoder 300 or video decoder 400) in accordance with an embodiment of the present disclosure. The video encoding apparatus 1000 is suitable for implementing the disclosed embodiments that are described herein. The video encoding device 1000 includes input ports 1010 and receiver units (Rx) 1020 for receiving data; a processor, logical unit, or central processing unit (CPU) 1030 for processing data; transmitter (Tx) blocks 1040 and data output ports 1050; and 1060 memory for data storage. The video encoding apparatus 1000 may also include optical-electrical (OE) components and electro-optical (EO) components connected to input ports 1010, receiving units 1020, transmitting units 1040, and output ports 1050 for outputting or inputting optical or electrical signals.

The 1030 processor is implemented using hardware and software. The processor 1030 may be implemented as one or more CPU chips, cores (eg, a multi-core processor), field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), and digital signal processors (DSPs). Processor 1030 interfaces with input ports 1010, receiver blocks 1020, transmitter blocks 1040, output ports 1050, and memory 1060. Processor 1030 includes an encoding module 1070. Encoding module 1070 implements the disclosed embodiments described above. For example, encoding module 1070 implements, processes, prepares, or provides various codec functions. Thus, turning on the encoding module 1070 provides a significant improvement in the functionality of the video encoding apparatus 1000 and causes the video encoding apparatus 1000 to be placed in a different state. Alternatively, encoding module 1070 is implemented as instructions stored in memory 1060 and executed by processor 1030.

The video encoding device 1000 may also include input and/or output (I/O) devices 1080 for transmitting data to and from the user. The input/output devices 1080 may include output devices such as a display for displaying video data, speakers for outputting audio data, etc. I/O devices 1080 may also include input devices such as a keyboard, mouse, trackball, etc. and/or appropriate interfaces for interacting with such output devices.

Memory 1060 includes one or more disks, tape drives, and solid-state drives and can be used as an overflow storage device for storing programs when such programs are selected for execution, and for storing instructions and data that are read during program execution. Memory 1060 may be volatile and/or nonvolatile and may be read only memory (RAM), random access memory (ROM), ternary content addressable memory (TCAM), and/or static random access memory (SRAM).

In fig. 11 shows a schematic diagram of an embodiment of an encoding means 1100. In an embodiment, the encoding engine 1100 is implemented in a video encoding device 1102 (eg, video encoder 300 or video decoder 400). The video encoding device 1102 includes a receiving means 1101. The receiving means 1101 is configured to receive an image for encoding or receive a bit stream for decoding. The video encoding device 1102 includes a transmission means 1107 connected to a reception means 1101. The transmission means 1107 is configured to transmit a bit stream to a decoder or transmit a decoded image to a display device (eg, one of the input/output devices 1080).

The video encoding device 1102 includes storage means 1103. The storage means 1103 is connected to at least one of the receiving means 1101 or the transmitting means 1107. The storage means 1103 is configured to store instructions. The video encoding device 1102 also includes processing means 1105. The processing means 1105 is connected to the storage means 1103. The processing means 1105 is configured to execute instructions stored in the storage means 1103 to perform the methods disclosed herein.

In addition, it should also be understood that the steps of the exemplary methods set forth herein need not be performed in the order described, and the order of the steps of such methods should be considered as exemplary only. Likewise, additional steps may be included in such methods, and some steps may be omitted or combined in methods consistent with various embodiments of the present disclosure.

Although several embodiments have been presented in the present disclosure, it should be understood that the disclosed systems and methods may be embodied in many other specific forms without departing from the spirit or scope of the present disclosure. The present examples are to be considered illustrative and non-limiting, and the examples are not intended to limit the invention to the details set forth herein. For example, various elements or components may be combined or integrated into another system, or some functionality may be omitted or not implemented.

In addition, the technologies, systems, subsystems, and methods described and illustrated in various embodiments as discrete or separate may be combined or integrated with other systems, components, technologies, or methods without departing from the scope of the present disclosure. Other elements shown or viewed as being directly connected or coupled to each other may be indirectly connected or coupled through some interface, device, or intermediate component, whether electrically, mechanically, or otherwise. Other examples of changes, substitutions and modifications may be ascertained by one skilled in the art and may be made without departing from the spirit and scope disclosed herein.

Claims (35)

1. A decoding method comprising the steps of: receive a bit stream having a decoded picture buffer (DPB) syntax structure, wherein when the DPB syntax structure is contained in a video parameter set (VPS), output layer sets (OLSs) to which the DPB syntax structure applies are defined by the VPS, and when the syntax structure The DPB is contained in a Sequence Parameter Set (SPS), the syntactic structure of the DPB is applied to the OLS, which has only one layer; obtaining a DPB syntax structure for OLS with one layer from SPS or a DPB syntax structure for OLS from VPS; And decoding an image from an OLS having only one layer or from OLSs using a DPB syntax structure to obtain a decoded image. 2. The method according to claim 1, in which the syntactic structure of DPB is designated as dpb_parameters(). 3. The method of claim 1 or 2, wherein said only one layer is the i-th layer and the OLS is the i-th OLS. 4. Method according to any one of paragraphs. 1-3, in which the DPB syntax structure is present in the i-th syntax structure dpb_parameters() in the SPS. 5. Method according to any one of paragraphs. 1-4, wherein the bit stream does not include a video parameter set (VPS) when the DPB syntax structure is contained in the SPS. 6. Method according to any one of paragraphs. 1-5, further comprising the step of storing the decoded image in the DPB before display. 7. Method according to any one of paragraphs. 1-6, in which the SPS is referenced by the specified only one layer in the OLS. 8. Method according to any one of paragraphs. 1-7, further comprising the step of displaying the decoded image on a display of the electronic device. 9. A decoding device comprising: a receiver configured to receive a bit stream having a decoded picture buffer (DPB) syntax structure, wherein when the DPB syntax structure is contained in a video parameter set (VPS), output layer sets (OLSs) to which the DPB syntax structure applies are defined by the VPS , and when the DPB syntax structure is contained in a Sequence Parameter Set (SPS), the DPB syntax structure is applied to the OLS, which has only one layer; And a processor coupled to the memory, wherein the processor is configured to execute instructions causing execution by a decoding device: obtaining a DPB syntax structure for OLSs with only one layer from the SPS or a DPB syntax structure for OLSs from the VPS; And decoding an image from OLSs with only one layer or OLSs using the DPB syntax structure to obtain the decoded image. 10. The decoding apparatus of claim 9, wherein the DPB syntax structure is designated dpb_parameters(), wherein said only one layer is the i-th layer, wherein the OLS is the i-th OLS. 11. The decoding device according to claim 9 or 10, wherein the DPB syntax structure is present in the i-th dpb_parameters() syntax structure in the SPS. 12. Decoding device according to any one of claims. 9-11, wherein the bit stream does not include a video parameter set (VPS) when the DPB syntax structure is contained in the SPS. 13. Decoding device according to any one of paragraphs. 9-12, in which the SPS is referenced by only one specified layer in the OLS. 14. Decoding device according to any one of paragraphs. 9-13, further comprising a display configured to display the decoded image. 15. An encoding device containing: a receiver configured to receive an image for encoding or receive a bitstream for decoding; a transmitter coupled to the receiver, wherein the transmitter is configured to transmit a bit stream to a decoder or transmit a decoded image to a display; a memory connected to at least one of a receiver and a transmitter, the memory being configured to store instructions; And a processor coupled to the memory, wherein the processor is configured to execute instructions stored in the memory to implement the method of any one of claims. 1-8. 16. The encoding device according to claim 15, further comprising a display configured to display the decoded image. 17. Video encoding/decoding system containing: encoder; And a decoder communicatively coupled to the encoder, wherein the encoder or decoder includes a decoding device or an encoding device according to any one of claims. 9-16. 18. Encoding tool containing: receiving means configured to receive an image for encoding or receive a bit stream for decoding; transmission means connected to the reception means, the transmission means being configured to transmit a bit stream to the decoding means or transmit a decoded image to the display means; storage means connected to at least one of: receiving means and transmitting means, wherein the storage means is configured to store instructions; And processing means connected to the storage means, wherein the processing means is configured to execute instructions stored in the storage means to implement the method according to any one of claims. 1-8.