RU2775390C1 - Adaptive image size signaling in video bitstream - Google Patents

Abstract

FIELD: video processing and transmission.

SUBSTANCE: invention relates to the field of video processing and transmission, and more specifically to the transmission of image signals or image parts, the size of which can vary from image to image or from image part to image part. The proposed method for decoding an encoded image from an encoded video sequence is performed by at least one processor, and the method includes decoding, from a set of parameters, a plurality of possible decoded resolutions; selecting, by means of an index encoded in the temporal header structure applicable to the sample group, a possible decoded resolution among the plurality of possible decoded resolutions; resampling a sample from the group of samples based on the output resolution and the selected candidate decoded resolution; and resampling the prediction using the resampled sample.

EFFECT: improving encoding efficiency.

19 cl, 10 dwg

Description

Cross-reference to related application

[0001] The present application claims priority from U.S. Provisional Application No. 62/816667, filed March 11, 2019, with the U.S. Patent and Trademark Office, which is incorporated herein by reference in its entirety.

Prerequisites for the creation of the invention

1. Technical field

[0002] The disclosed subject matter relates to video encoding and decoding, and more specifically, to the transmission of image signals or parts of an image, the size of which may vary from image to image or from image part to image part.

2. Description of the prior art

[0003] Video encoding and decoding using motion-compensated inter-picture prediction has been known for decades. An uncompressed digital video signal may consist of a sequence of images, each image having a spatial size of, for example, 1920x1080 luminance samples and their associated chrominance samples. An image sequence can have a fixed or variable image rate (informally also known as frame rate), such as 60 images per second or 60 Hz. Uncompressed video has significant bitrate requirements. For example, 1080p60 4:2:0 video with 8 bits per sample (resolution 1920×1080 luma samples at 60Hz) requires a bandwidth close to 1.5 Gbps. An hour of such video requires more than 600 GB.

[0004] One of the goals of video encoding and decoding may be to reduce the redundancy of the input video through compression. Compression can help reduce the aforementioned bandwidth or storage requirements, in some cases by two orders of magnitude or more. Both lossless and lossy compression, as well as a combination of both, can be used. Lossless compression refers to technologies where an exact copy of the original signal can be reconstructed from the compressed original signal. When lossy compression is used, the reconstructed signal may not be identical to the original signal, but the distortion between the original and reconstructed signals is small enough to make the reconstructed signal useful for the intended application. In the case of video, lossy compression is widely used. The amount of distortion allowed depends on the application; for example, users of certain consumer streaming applications may tolerate higher distortion than users of television signaling applications. The achievable compression ratio may reflect the following: higher acceptable/acceptable distortion may result in a higher compression ratio.

[0005] The video encoder and decoder may use technologies from several broad categories, including, for example, motion compensation, transformation, quantization, and entropy coding, some of which will be presented below.

[0006] Historically, video encoders and decoders tended to work with a given image size, which in most cases was determined and remained constant for a coded video sequence (CVS, Coded Video Sequence), a group of pictures (GOP, Group of Pictures) or a similar time interval from multiple images. For example, the MPEG-2 Moving Picture Experts Group (MPEG) standard knows system structures for changing horizontal resolution (and hence picture size) depending on factors such as scene activity, but only for I- images (Intra-coded picture image is encoded using only the information contained in itself), therefore, usually for a group of GOP images. Reference picture resampling to use different resolutions in CVS is known, for example, from Annex P of Recommendation H.263 of the International Telecommunications Union Telecommunication Standardization Sector (ITU-T). However, here the image size is not changed, only the reference images are resampled so that only parts of the image canvas (in the case of downsampling) or only parts of the captured scene (in the case of upsampling) can be used. In addition, Annex Q of Recommendation H.263 allows an individual macroblock to be resampled by a factor of two (in each dimension), up or down. Again, the image size remains the same. The macroblock size is fixed in H.263 and therefore does not need to be signaled.

[0007] Image resizing in predictive images has become an increasingly common trend in modern video coding. For example, the VP9 standard (Video Processor 9 Video Processor 9) allows you to resample the reference image and change the resolution for the entire image. Similarly, some of the proposals made regarding Versatile Video Coding (VVC) (including, for example, Hendry, et. al, "On adaptive resolution change (ARC) for VVC", document JVET-M0135-vl, 9 -January 19, 2019, JVET (Joint Video Experts Team) within the joint program (JVT, Joint Video Team, fully included in this document) allows resampling of entire reference pictures with other, higher or lower, permissions. This document proposes to encode the various possible resolutions in a sequence parameter set and refer to them using the syntax elements for each image in the image parameter set.

The essence of the invention

[0008] According to embodiments of the invention, a method for decoding an encoded image from an encoded video sequence is performed by at least one processor and includes decoding, from a set of parameters, a set of possible decoded resolutions, selecting, by means of an index encoded in a temporary header structure applicable to a group of samples, a possible decoded resolution among the plurality of possible decoded resolutions, resampling samples from the group of samples based on the output resolution and the selected candidate decoded resolution, and predictive resolution using the resampled sample.

[0009] According to embodiments of the invention, an apparatus for decoding an encoded image from an encoded video sequence comprises at least one memory configured to store computer program code and at least one processor configured to access at least one memory and operate in accordance with a computer program code, the computer program code comprising a decoding code configured to decode from a set of parameters of a plurality of possible decoded resolutions; a selection code configured to select, by means of an index encoded in the temporary header structure applicable to the sample group, a possible decoded grant among the plurality of possible decoded grants; a resampling code configured to resample a sample from the group of samples based on the output resolution and the selected candidate decoded resolution; and a prediction enable code configured to enable prediction using the resampled sample.

[0010] According to embodiments of the invention, a computer-readable storage medium storing a program for decoding an encoded image from an encoded video sequence includes instructions that cause the processor to decode from a set of parameters a plurality of possible decoded resolutions, select by an index encoded in a temporary header structure applicable to a sample group , a possible decoded resolution among the plurality of possible decoded resolutions, resampling a sample from the group of samples based on the output resolution and the selected candidate decoded resolution, and resolve prediction using the resampled sample.

Brief description of the drawings

[0011] Additional features, essence and various advantages of the disclosed subject matter will be more apparent from the following detailed description and the accompanying drawings, in which:

Fig. 1 is a schematic diagram of a simplified block diagram of a communication system in accordance with an embodiment of the invention.

Fig. 2 is a schematic diagram of a simplified block diagram of a communication system in accordance with an embodiment of the invention.

Fig. 3 is a schematic diagram of a simplified block diagram of a decoder in accordance with an embodiment of the invention.

Fig. 4 is a schematic diagram of a simplified block diagram of an encoder in accordance with an embodiment of the invention.

Fig. 5A is a schematic representation of options for signaling Adaptive Resolution Change (ARC) parameters as indicated.

Fig. 5B is a schematic representation of possible ARC parameter signaling options in accordance with an embodiment of the invention as indicated.

Fig. 6A is a schematic representation in accordance with an embodiment of the invention.

Fig. 6B is a flowchart illustrating a method for decoding an encoded picture from an encoded video sequence according to an embodiment of the invention.

Fig. 6C is a simplified block diagram of an apparatus for controlling decoding of a video sequence according to an embodiment of the invention.

Fig. 7 is a schematic representation of a computer system in accordance with an embodiment of the invention.

Detailed description

[0022] Recently, some attention has been drawn to the aggregation of a compressed area, or the extraction of multiple semantically independent parts of an image into a single video image. In particular, in the context of e.g. 360-degree encoding or certain surveillance applications, multiple semantically independent source images (e.g., six cube surfaces of a cube-projected 360-degree scene, or individual camera inputs in the case of a multi-camera surveillance setup) may require separate adaptive settings. resolution to deal with the different activity of each scene at a given point in time. In other words, the encoder at a given time may choose to use different resampling factors for different semantically independent images that make up the entire 360 degree or scene. When combined into a single picture, this in turn requires that the reference picture be resampled and that there be adaptive resolution coding signaling for portions of the encoded picture.

[0023] FIG. 1 is a simplified block diagram of a communication system (100) according to one embodiment of the present invention. System (100) may include at least two terminals (110-120) connected to each other via network (150). For unidirectional data transmission, the first terminal (110) may encode video data at the local location for transmission to another terminal (120) via the network (150). The second terminal (120) may receive the encoded video data of another terminal from the network (150), decode the encoded data, and display the recovered video data. Unidirectional data transmission may be common in multimedia service applications and the like.

[0024] In FIG. 1 shows a second pair of terminals (130, 140) providing support for bidirectional encoded video transmission, such as may occur during a video conference. For bi-directional data transmission, each terminal (130, 140) may encode video data captured at a local location for transmission to another terminal via a network (150). Each terminal (130, 140) may also receive the encoded video data transmitted by the other terminal, may decode the encoded data, and may display the recovered video data on a local display device.

[0025] In FIG. 1, terminals (110-140) may be shown as servers, personal computers, and smartphones, but the principles of the present invention may not be limited to this. Embodiments of the present invention find application in laptop computers, tablet computers, media players and/or specialized video conferencing equipment. Network (150) represents any number of networks that transmit encoded video data between terminals (110-140), including, for example, wired and/or wireless communication networks. The communications network (150) may communicate over circuit-switched and/or packet-switched channels. Representative networks include telecommunications networks, local area networks, wide area networks and/or the Internet. For the purposes of this discussion, the architecture and topology of the network (150) may not be essential to the operation of the present invention unless explained herein below.

[0026] FIG. 2 shows, as an example of application for the disclosed subject matter, the placement of a video encoder and decoder in a streaming environment. The disclosed subject matter may be equally applicable to other video-enabled applications, including, for example, video conferencing, digital television, compressed video storage on digital media, including CD, Compact Disk, DVD, Digital Versatile Disk), memory card, etc.

[0027] The streaming system may include a video capture subsystem (213), which may include a source (201) of video signals, such as a digital camera, creating, for example, a stream (202) of uncompressed video samples. This stream (202) of samples, depicted with a thick line to emphasize the large amount of data compared to the encoded video bitstreams, can be processed by an encoder (203) connected to the camera (201). The encoder (203) may include hardware, software, or a combination thereof to enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video bitstream (204), drawn with a thin line to emphasize the smaller amount of data compared to the sample stream, may be stored on the streaming server (205) for future use. One or more streaming clients (206, 208) can access the streaming server (205) to obtain copies (207, 209) of the encoded video bitstream (204). The client (206) may include a video decoder (210) that decodes an incoming copy (207) of the encoded video bitstream and produces an outgoing stream (211) of video samples that can be displayed on a display (212) or other visualization device (not shown) . In some streaming systems, video bitstreams (204, 207, 209) may be encoded according to certain video coding/compression standards. Examples of these standards include ITU-T Recommendation H.265. A video coding standard known informally as Versatile Video Coding (VVC) is under development. The disclosed subject matter may be used in the context of the VVC standard.

[0028] FIG. 3 may represent a functional block diagram of a video decoder (210) according to one embodiment of the present invention.

[0029] The receiver (310) may receive one or more encoded video sequences to be decoded by the decoder (210); in the same or another embodiment of the invention, one encoded video sequence at a time, where the decoding of each encoded video sequence is independent of other encoded video sequences. An encoded video sequence may be received from a channel (312), which may be a hardware/software connection to a storage device that stores the encoded video data. The receiver (310) may receive encoded video data along with other data, such as encoded audio data and/or ancillary data streams, which may be forwarded to their respective uses (not shown). The receiver (310) may separate the encoded video sequence from other data. To combat network jitter, a buffer memory (315) can be included between the receiver (310) and the entropy decoder/parser (320) (hereinafter referred to as "parser"). When the receiver (310) receives data from a storage/transfer device with sufficient bandwidth and controllability, or from an isosynchronous network, the buffer (315) may not be needed or may be small. For use in packet networks with maximum efficiency, such as the Internet, a buffer (315) may be required, it may be relatively large, and preferably may have an adaptive size.

[0030] The video decoder (210) may include a parser (320) for recovering symbols (321) from the entropy coded video sequence. The categories of these symbols include information used to control the operation of the decoder (210) and potentially information to control a visualization device such as a display (212) that is not an integral part of the decoder but can be attached to it as shown. in Fig.2. The control information for the rendering device(s) may be in the form of Supplementary Enhancement Information (SEI) or Video Usability Information (VUI) parameter set fragments (not shown). A parser (320) may parse/entropy decode the received encoded video sequence. Encoding of the encoded video sequence may be in accordance with a video coding technology or standard, including variable length coding, Huffman coding, arithmetic coding with or without context, etc. The parser (320) can extract from the encoded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. Subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, code packs (CU, Coding Unit), blocks, transformation packs (TU, Transform Unit), prediction packs (PU, Prediction Unit,) and so on. .d. The entropy decoder/parser may also extract information from the encoded video sequence, such as transform coefficients, quantizer parameter values, motion vectors, and so on.

[0031] The parser (320) may perform an entropy decoding/parsing operation on the video sequence received from the buffer (315) so as to create symbols (321).

[0032] Symbol recovery (321) may include several different blocks depending on the type of encoded video image or parts thereof (e.g., intra-inter prediction encoded image, intra-inter prediction encoded block) and other factors. Which blocks to use and how can be controlled by the subgroup control information that has been parsed from the encoded video sequence by the parser (320). The flow of such subgroup control information between the parser (320) and the plurality of blocks is not shown for clarity.

[0033] In addition to the functional blocks already mentioned, the decoder 210 can be conceptually subdivided into a number of functional blocks, as described below. In a practical implementation operating under commercial constraints, many of these blocks interact closely with each other and can, at least in part, be integrated into each other. However, for purposes of describing the disclosed subject matter, the conceptual division into functional blocks below is appropriate.

[0034] The first block is the scaling/inverse transform block (351). The scaling/inverse transform block (351) receives a quantized transform coefficient as well as control information including which transform to use, block size, quantization factor, quantization scaling matrices, and so on. in the form of character(s) (321) from the parser (320). It can output blocks containing sample values that can be entered into the aggregator (355).

[0035] In some cases, the output samples of the scaling/inverse transform block (351) may refer to a block encoded with intra prediction, that is: a block that does not use prediction information from previously restored images, but can use prediction information from previously restored parts current image. Such prediction information may be provided by intra prediction block 352 . In some cases, the block (352) intra-prediction generates a block of the same size and shape as the restored block, using the surrounding already restored information extracted from the current (partially restored) image (356). The aggregator (355) in some cases adds, for each sample, the prediction information generated by the intra prediction block (352) to the output sample information provided by the scaling/inverse transform block (351).

[0036] In other cases, the output samples of the scaling/inverse transform block (351) may refer to a block encoded with inter prediction and possibly with motion compensation. In such a case, the motion-compensated predictor (353) may access the reference picture memory (357) to fetch samples used for prediction. After compensating for the movement of the selected samples according to the symbols (321) associated with the block, these samples can be added by the aggregator (355) to the output signal of the scaler/detransform block (in this case called residual samples or residual signal) to form information about output counts. Reference picture memory addresses from where the motion compensator selects prediction samples may be controlled by motion vectors available to the motion compensator in symbol form (321), which may have, for example, X, Y, and reference picture components. Motion compensation may also include interpolation of sample values retrieved from reference picture memory when accurate subsample motion vectors, motion vector prediction mechanisms, and so on are used.

[0037] The output samples of the aggregator (355) may be subjected to various loop filtering techniques in the loop filter block (356). The video compression technologies may include in-loop filtering technologies that are controlled by parameters included in the encoded video bitstream and provided to the loop filter block (356) in the form of symbols (321) from the parser (320), but may also respond to the metainformation received during decoding of the previous (in decoding order) parts of the encoded image or encoded video sequence, as well as in response to previously restored and filtered by the loop filter sample values.

[0038] The output of the loop filter block (354) may be a stream of samples that may be output to the renderer (212) as well as stored in reference picture memory for use in future inter-picture prediction.

[0039] Some encoded pictures after full recovery can be used as reference pictures for future prediction. Once the encoded picture has been completely reconstructed and the encoded picture has been identified as a reference picture (eg, by parser 320), the current reference picture 356 may become part of the reference picture buffer 357 and fresh current picture memory may be reallocated. before starting the reconstruction of the next encoded image.

[0040] Video decoder 210 may perform decoding operations according to a predetermined video compression technology that may be documented in a standard such as ITU-T Rec. H.265. The encoded video sequence may conform to the syntax specified by the video compression technology or standard used, in the sense that it adheres to the syntax of the video compression technology or standard as specified in the video compression technology document or standard and, in particular, in the profiles documented therein. Compliance also requires that the complexity of the encoded video sequence be within the limits defined by the level of technology or video compression standard. In some cases, levels limit the maximum image size, the maximum frame rate, the maximum recovery sample rate (measured in Ms/s, for example), the maximum reference image size, and so on. The limits set by the layers may in some cases be further limited by Hypothetical Reference Decoder (HRD) specifications and metadata for managing the HRD buffer signaled in the encoded video sequence.

[0041] In one form of the invention, the receiver (310) may receive additional (redundant) data with the encoded video. Additional data may be included as part of the encoded video sequence(s). The additional data may be used by the video decoder (210) to correctly decode the data and/or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement levels, redundant slices, redundant images, forward error correction codes, and so on.

[0042] FIG. 4 may be a functional block diagram of a video encoder (203) according to one embodiment of the present invention.

[0043] The encoder (203) may receive video samples from a video source (201) (not part of the encoder) that may capture video(s) for encoding by the encoder (203).

[0044] The video source (201) may provide the original video sequence to be encoded by the encoder (203) in the form of a digital video sample stream that may have some suitable bit depth (eg: 8 bit, 10 bit, 12 bit , ...), some color space (for example, Y CrCb space as recommended by ITU-R BT.601, RGB, ...), some suitable sampling structure (for example, Y CrCb 4:2:0, Y CrCb 4: 4:4). In a media serving system, the video source (201) may be a storage device storing a pre-prepared video. In a videoconferencing system of video signals, the source (203) may be a camera that captures local image information as a video sequence.

The video data may be provided as a plurality of individual images which, when viewed sequentially, convey motion. The images themselves can be organized as a spatial array of pixels, with each pixel containing one or more samples depending on the sample structure used, color space, and so on. Those skilled in the art can easily understand the relationship between pixels and samples. The description below is based on readings.

[0045] According to one embodiment of the invention, the encoder (203) can encode and compress images of the original video sequence into the encoded video sequence (443) in real time or with any other time constraints as required by the application. Providing an appropriate coding rate is one of the functions of the controller (450). The controller controls other functional blocks as described below and is operatively connected to these blocks. The connection is not shown for clarity. The parameters set by the controller may include parameters related to speed control (picture skip, quantizer, lambda value for speed-distortion optimization technologies, ...), picture size, group of pictures (GOP) structure, maximum motion vector search range, etc. .d. Those skilled in the art can easily identify other functions of the controller (450) as they may relate to a video encoder (203) optimized for a particular system design.

[0046] Some video encoders operate in what those skilled in the art readily recognize as a "coding loop". As a simplified description, an encoding loop may consist of an encoding part of an encoder (430) (hereinafter "source encoder") (responsible for creating symbols based on the input image to be encoded and the reference image(s)), and a (local) decoder (433) built into the encoder (203) that decompresses the symbols to create sample data that the (remote) decoder can also produce (because any compression between the symbols and the encoded video bitstream is lossless in the video compression technologies discussed in the disclosed subject matter inventions). This reconstructed sample stream is entered into the reference picture memory (434). Since the decoding of a symbol stream results in bit-accurate results regardless of the location of the decoder (local or remote), the contents of the reference picture buffer also have bit-accuracy between the local encoder and the remote encoder. In other words, the prediction portion of the encoder "sees" as reference picture samples exactly the same sample values that the decoder would "see" using prediction during decoding. This is the principle of reference picture synchronism (and the resulting drift if synchronism cannot be maintained, for example due to channel errors).

[0047] The operation of the "local" decoder (433) may be the same as that of the "remote" decoder (210) already described in detail above in connection with FIG. 3. Referring also briefly to FIG. 3-4, however, since the symbols are available and encoding/decoding of the symbols into the encoded video sequence by the entropy encoder (445) and the parser (320) can be lossless, the entropy decoding part of the decoder (210), including the channel (312), receiver ( 310), buffer (315), and parser (320) may not be fully implemented in the local decoder (433).

[0048] At this point, it can be observed that any decoding technology other than parsing/entropy decoding that is present in a decoder must also necessarily be present in a substantially identical functional form in a corresponding encoder. For this reason, the disclosed subject matter focuses on the operation of the decoder. The description of encoding technologies can be abbreviated as they are the inverse of fully described decoding technologies. Only in some areas a more detailed description is required, which is given below.

[0049] As part of its operation, the source encoder (430) may perform motion-compensated predictive encoding that predictively encodes the input frame with reference to one or more previously encoded frames from the video sequence, which have been designated "reference frames". Thus, the coding engine (432) encodes the differences between the pixel blocks of the input frame and the pixel blocks of the reference frame(s) that can be selected as prediction references for the input frame.

[0050] The local video decoder (433) may decode the encoded video data of the frames, which may be designated as reference frames, based on the symbols generated by the source encoder (430). The operations of the encoding engine (432) may be predominantly lossy processes. When the encoded video data can be decoded in a video decoder (not shown in FIG. 4), the reconstructed video sequence can typically be a copy of the original video sequence with some errors. The local video decoder (433) replicates the decoding processes that may be performed by the video decoder on the reference frames and may cause the recovered reference frames to be stored in the reference picture cache (434). Thus, the encoder (203) can locally store copies of the reconstructed key frames that share the same content as the reconstructed key frames to be received by the far end video decoder (in the absence of transmission errors).

[0051] The predictor (435) may perform predictive searches for the encoding engine (432). That is, for a new frame to be encoded, the predictor (435) may search the reference picture memory (434) for sample data (as candidate pixel reference blocks) or certain metadata such as reference picture motion vectors, block shapes, etc. .d., this can serve as a suitable reference for prediction for new images. The predictor (435) may operate on a block of samples per block of pixels basis to find appropriate references for prediction. In some cases, as determined by the search results obtained by the predictor (435), the input picture may have prediction references taken from a plurality of reference pictures stored in the reference picture memory (434).

[0052] The controller (450) may control the encoding operations of the video encoder (430), including, for example, setting parameters and subgroup parameters used to encode video data.

[0053] The output of all of the above functional blocks may be entropy encoded in an entropy encoder (445). The entropy encoder translates symbols generated by various functional blocks into an encoded video sequence by lossless symbol compression according to various techniques such as Huffman coding, variable length coding, arithmetic coding, and so on.

[0054] The transmitter (440) may buffer the encoded video sequence(s) generated by the entropy encoder (445) to prepare it for transmission over a communication channel (460), which may be a hardware/software communication channel to a data storage device that will store encoded video data. The transmitter (440) may combine the encoded video data from the video encoder (430) with other data to be transmitted, such as encoded audio data and/or ancillary data streams (sources not shown).

[0055] The controller (450) may control the operation of the encoder (203). During encoding, the controller (450) may assign to each encoded picture a certain type of encoded picture, which may affect the encoding technologies that can be applied to the corresponding picture. For example, images can often be assigned as one of the following frame types:

[0056] An intra picture (I picture, Intra Picture) may be one that can be encoded and decoded without using any other frame in the sequence as a prediction source. Some video codecs allow the use of various intra-encoded types, including, for example, updated images, using an independent decoder. Variants of I-images and their respective uses and features may be used.

[0057] A predictively encoded picture (P picture) (P picture, Predictive picture) may be one that can be encoded and decoded using intra prediction or inter prediction using at most one motion vector and a reference index for prediction sample values of each block.

[0058] A picture encoded with bi-directional prediction (B-picture) (In Picture, Bi-directionally predictive picture) may be one that can be encoded and decoded using intra prediction or inter prediction using no more than two motion vectors and reference indexes for predicting the values of the samples of each block. Similarly, multi-prediction encoded pictures may use more than two reference pictures and their associated metadata to reconstruct a single block.

[0059] Source images can typically be subdivided spatially into multiple blocks of samples (eg, blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and encoded on a block-by-block basis. The blocks may be predictively encoded with reference to other (already encoded) blocks, as determined by the coding assignment applied to the corresponding block images. For example, I-picture blocks may be predictively encoded or they may be predictively encoded with reference to already encoded blocks of the same picture (spatial prediction or intra prediction). The pixel blocks of P-pictures may be encoded without prediction, by spatial prediction, or by temporal prediction with reference to one previously encoded reference picture. Blocks of B-pictures may be encoded without prediction, by spatial prediction, or by temporal prediction with reference to one or two previously encoded reference pictures.

[0060] The video encoder (203) may perform encoding operations according to a predetermined video coding technology or standard, such as ITU-T Rec. H.265. During operation, the video encoder (203) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The encoded video data may therefore follow the syntax given by the video coding technology or standard used.

[0061] In one form of the invention, the transmitter (440) may transmit additional data with encoded video. Video encoder (430) may include such data as part of an encoded video sequence. The additional data may comprise temporal/spatial/SNR enhancement levels, other forms of redundant data such as redundant images and slices, extended supplementary information (SEI) messages, video property information (VUI) parameter set fragments, and so on.

[0062] Before a more detailed description of certain aspects of the disclosed subject matter, it is necessary to introduce several terms that will be referred to in the remainder of this description.

[0063] Hereinafter, a sub-image refers in some cases to a rectangular arrangement of samples, blocks, macroblocks, code packets, or the like, which are semantically grouped and which can be independently encoded with a changed resolution. One or more sub-images may be for an image. One or more encoded image fragments may form an encoded image. One or more sub-images may be assembled into an image, and one or more sub-images may be extracted from the image. In certain environments, one or more encoded sub-images may be collected in the compressed region without being re-coded to the sample level in the encoded image, and in the same or some other cases, one or more encoded sub-images may be extracted from the encoded image in the compressed region.

[0064] Adaptive resolution change (ARC) now refers to mechanisms that allow the resolution of an image or sub-image in an encoded video sequence to be changed, for example, by resampling a reference image. The ARC parameters may refer to the types of control information needed to perform adaptive resizing, which may include, for example, filter parameters, scaling factors, output and/or reference picture resolutions, various control flags, and so on. Thus, ARC information may refer to data and values corresponding to ARC parameters.

[0065] The above description focuses on encoding and decoding a single semantically independent encoded video image. Before describing the consequences of encoding/decoding multiple sub-images with independent ARC parameters and its expected additional complexity, there should be a description of the option for ARC parameter signaling.

[0066] FIG. 5A shows several new options for ARC parameter signaling. As noted for each of the options, they have certain advantages and certain disadvantages in terms of coding efficiency, complexity and architecture. The video coding standard or technology may choose one or more of these options or other options for signaling the ARC parameters. The options may not be mutually exclusive and may possibly be interchanged depending on the needs of the application, the standards used, or the choice of encoder.

[0067] ARC parameter classes may include:

[0068] - upsampling/downsampling factors, either alone or combined across the X and Y dimensions;

[0069] - upsampling/downsampling factors, with the addition of a temporal dimension, indicative of upscaling/downscaling at a constant rate for a given number of images.

[0070] Any of the above two may include encoding one or more supposedly short syntax elements that may point to a table containing the coefficient(s).

[0070] is the resolution for measuring X or Y in units of samples, blocks, macroblocks, CU bursts, or any other suitable granularity of the input picture, output picture, reference picture, coded picture, combined or alone. If there is more than one resolution (eg, one for the input image, one for the reference image), then in some cases one set of values may be derived from another set of values. This can be controlled, for example, using flags. See below for a more detailed example.

[0072] - "warp" coordinates like those used in H.263 Annex P, again with the appropriate granularity as described above. Annex P H.263 defines one efficient way to encode such coordinate warp coordinates, but other, potentially more efficient ways may also be developed. For example, the reversible "Huffman" style variable length encoding for warp coordinates according to Annex P can be replaced by a suitable length binary encoding, where the length of the binary codeword can be, for example, obtained from the maximum image size, perhaps multiplied by a certain factor and shifted by a certain value to allow for "warping" beyond the bounds of the maximum image size.

[0073] - filter parameters with upsampling or downsampling. In the simplest case, there can only be one filter for upsampling and/or downsampling. However, in some cases it may be beneficial to provide more flexibility in the filter structure, and this may require filter parameter signaling. Such parameters may be selected by an index into a list of possible filter structures, a filter may be fully specified (e.g., by a list of filter coefficients using suitable entropy coding techniques), a filter may be implicitly selected by upsampling/downsampling relationships, according to which, in turn, are signaled according to any of the mechanisms mentioned above, and so on.

[0074] The following description assumes encoding a finite set of upsampling/downsampling coefficients (the same coefficient to be used in both the X and Y dimensions) indicated by the codeword. This codeword may advantageously be encoded with a variable length, for example, using the exponential Golomb code (Exp-Golomb) common to certain syntax elements in video coding specifications such as H.264 and H.265. One suitable mapping of values to upsampling/downsampling factors may, for example, correspond to the following table.

[0075] Many similar displays can be designed in accordance with the needs of the application and the capabilities of the zoom in and out mechanisms available in the video compression technology or standard. The table can be expanded to more values. Values may also be represented by entropy encoding mechanisms other than Exponential Golomb codes, such as using binary encoding. This may have certain advantages when the oversampling factors were of interest outside the video processing mechanisms themselves (primarily the encoder and decoder), for example, through network elements that respond to media data (MANE, Media Aware Network Element). It should be noted that for the (presumably) most common case where no resolution change is required, a short exponential Golomb code can be chosen; in the table above, only one bit. This may have an advantage in coding efficiency over using binary codes for the most common case.

[0076] The number of entries in the table, as well as their semantics, can be fully or partially configurable. For example, the underlying table structure may be conveyed in a "high" parameter set such as a sequence or a decoder parameter set. Alternatively, or in addition, one or more such tables may be defined in a video coding technology or standard and may be selected, for example, by a decoder or sequence parameter set.

[0077] Next, we describe how the upsampling/downsampling ratio (ARC information) encoded as described above can be included in the syntax of a video coding technology or standard. Similar considerations may apply to one or more codewords driving the upsampling/downsampling filters. See the discussion below for when a filter or other data structure requires relatively large amounts of data.

[0078] Appendix P H.263 includes ARC information 502 in the form of four warp coordinates in the image header 501, in particular in the H.263 PLUSPTYPE (503) header extension. This form of ARC information 502 may be used when a) there is an available picture header and b) frequent changes in the ARC information are expected. However, the amount of overhead when using H.263-style signaling can be quite high, and the scaling factors may not apply to picture boundaries because the picture header may be temporary.

[0079] The JVCET-M135-vl document cited above includes ARC reference information (505) (index) located in an image parameter set (504) indexing a table (506) including target resolutions, which in turn is located within a set (507) of sequence parameters. Placing a candidate resolution in a table (506) in a sequence parameter set (507) may, in accordance with the verbal statements made by the authors, be justified by using the SPS (Sequence Parameter Set) as a compatibility negotiation point during capability exchange. The resolution can be changed within the limits set by the values in the table (506) from image to image by referring to the corresponding set (504) of image parameters.

[0080] For the transmission of ARC information in a video bitstream, the following additional options may exist. Each of these possible options has certain advantages, as described above. The options may be present in the same video coding technology or standard at the same time.

[0081] In the embodiment shown in FIG. 5B, ARC information (509) such as a resampling (scaling) factor may be present in a slice header, GOB header, tile header, or tile group header (508) (hereinafter referred to as tile group header). This may be sufficient for small size ARC information such as a single variable length codeword ue(v) or a fixed length codeword of several bits, such as shown above. Having the ARC information in the tile group header directly has the additional advantage that the ARC information can be applied to a sub-image represented by that tile group, for example, rather than to the entire image. See also below. In addition, even if a video compression technology or standard only provides for adaptive resizing of the entire image (as opposed to, for example, adaptive resizing based on a group of tiles), placing it in an H.263-style image header has certain advantages in terms of providing error tolerance.

[0082] In the same or another embodiment of the invention, the ARC information (512) itself may be present in the corresponding parameter set (511), such as, for example, a picture parameter set, a header parameter set, a tile parameter set, an adaptation parameter set, etc. d. (a set of adaptation parameters is shown). The scope of this set of parameters may advantageously be no larger than an image, eg represent a group of tiles. The use of ARC information is implicitly done by activating the appropriate set of options. For example, when a video coding technology or standard considers only image-based ARC, then a set of image parameters or its equivalent may be appropriate.

[0083] In the same or another embodiment of the invention, the ARC reference information (513) may be present in a tile group header (514) or a similar data structure. This reference information (513) may refer to a subset of the ARC information (515) available in a parameter set (516) with a scope larger than a single picture, such as a sequence parameter set or a decoder parameter set.

[0084] An additional level of indirection involving the activation of a Picture Parameter Set (PPS) from a Tile Element Group Header, PPS, Sequence Parameter Set (SPS), as used in JVET-M0135-vl, seems unnecessary because Picture Parameter Sets, just like sequence parameter sets can be used (and are used in some standards such as RFC3984) to negotiate capabilities or declarations. A PPS set generally refers to a syntax structure containing syntax elements that apply to zero or more integer coded pictures as defined by the syntax element found in each slice header. An SPS set generally refers to a syntax structure containing syntax elements that apply to zero or more complete Coded Layer Video Sequences (CLVS) as defined by the content of the syntax element found in the PPS set referenced by the syntax element found in each image header. If, however, the ARC information is to be applicable to a sub-image represented, for example, also by tile groups, a parameter set with an activation area limited by the tile group, such as an adaptation parameter set or a header parameter set, may be a better choice. Also, if the ARC information is of more than negligible size, e.g., contains filter control information such as multiple filter coefficients, then the parameter may be a better choice than using the (508) header directly in terms of coding gain, since these settings may be reusable by future images or image fragments by reference to the same set of parameters.

[0085] When using a sequence parameter set or other higher-level parameter set with a scope spanning multiple images, certain considerations may apply:

[0086] 1. The parameter set for storing the ARC information table (516) may in some cases be a sequence parameter set, but in other cases preferably a decoder parameter set. The decoder parameter set may have an activation area of several CVS coded video sequences, namely the coded video stream, i. e. all bits of the encoded video from session start to session break. Such scope may be more appropriate because possible ARC factors could be a decoder function, perhaps implemented in hardware, and hardware functions generally do not change with any CVS (which, at least in some entertainment systems, is a group images that are one second or less in length). However, the inclusion of a table in the sequence parameter set is explicitly included in the placement options described here, in particular in connection with point 2 below.

[0087] 2. The ARC reference information (513) can advantageously be placed directly in the image/tile slice/group of blocks (GOB, Group Of Blocks)/rpynnbi header (514) of tiles (hereinafter referred to as the tile group header) rather than in the set image parameters, as in JVCET-M0135-vl. The reason is that when an encoder wants to change one value in a picture parameter set, such as ARC help information, for example, it must create a new PPS set and refer to this new PPS set. Assume that only the ARC reference information is changed, but other information, such as quantization matrix information in the PPS set, for example, remains. Such information may be of considerable size and will need to be retransmitted in order to complete a new PPS set. Since the ARC reference information may be a single codeword, such as an index in a table (513), and this will be the only value that changes, it would be cumbersome and wasteful to retransmit all, for example, quantization matrix information. In this regard, it can be significantly better in terms of coding efficiency to avoid indirection through the PPS set as proposed in JVET-M0135-vl. Similarly, placing the ARC help information in the PPS has the additional disadvantage that the ARC information referenced by the ARC help information (513) must necessarily apply to the entire image, and not to a sub-image, since the task of activating the parameter set image is an image.

[0088] In the same or another embodiment of the invention, ARC parameter signaling may follow a detailed example as shown in FIG. 6A. Fig. 6A is a diagram showing syntax examples, in the representation used in video coding standards since at least 1993. The presentation of such diagrams showing examples of syntax is roughly equivalent to C-style programming. Lines in bold indicate syntax elements present in the bit stream, lines without bold type often indicate control flow or variable settings.

[0089] A tile group header (601) as an exemplary header syntax structure applicable to a (possibly rectangular) part of an image, may conditionally contain a variable length syntax element encoded with exponential Golomb code, dec_pic_size_idx (602) (in bold) . The presence of this syntax element in a tile group header can be triggered by using adaptive resolution (603) - here the flag value is not in bold, meaning that the flag is present in the bitstream at the point where it occurs in the diagram showing the syntax example. Whether adaptive resolution is used for this image or parts of it can be signaled in any high-level syntactic structure inside or outside the bitstream. In the example shown, this is signaled in the sequence parameter set as shown below.

[0090] Still referring to FIG. 6A also shows a fragment of a set (610) of sequence parameters. The first syntax element shown is adaptive_pic_resolution_change_flag (611). When true, this flag may indicate the use of adaptive resolution, which in turn may require certain control information. In the example, such control information is conditionally present based on the value of the flag based on the if() statement in the parameter set (612) and the tile group header (601).

[0091] When adaptive resolution is used, this example encodes the output resolution in units of samples (613). Numeral 613 refers to both the output_pic_width_in_luma_samples element and the output_pic_height_in_luma_samples element, which together can determine the resolution of the output image. Elsewhere in a video coding technology or standard, certain restrictions may be defined for each value. For example, the level definition may limit the number of total output samples that can be the product of the values of these two syntax elements. In addition, certain video coding technologies or standards, or external technologies or standards such as, for example, system standards, may restrict the range of a numerical value (for example, one or both measurements must be divided by a power of 2) or the aspect ratio of an image (for example, width and height should be in a ratio such as 4:3 or 16:9). Such restrictions may be introduced to facilitate hardware implementations or for other reasons.

[0092] In some applications, it may be desirable for the encoder to instruct the decoder to use a certain reference picture size rather than implicitly assume that this size is the size of the output picture. In this example, the reference_pic_size_present_flag (614) syntax element triggers the conditional presence of the reference picture sizes (615) (again, the number refers to both width and height).

[0093] Finally, a table of possible width and height of the decoded image is shown. Such a table may be expressed, for example, by specifying a table (num_dec_pic_size_in_luma_samples_minusl) (616). The "minusl" element may refer to the interpretation of the meaning of this syntax element. For example, if the encoded value is zero, there is one entry in the table. If the value is five, there are six table entries. Then, for each "row" in the table, the width and height of the decoded picture are included in the syntax (617).

[0094] Represented (617) table elements can be indexed using the dec_pic_size_idx (602) syntax element in the tile group header, thereby allowing different decoded sizes - essentially scaling factors - for each tile group.

[0095] Some video coding technologies or standards, such as VP9, support spatial scalability by implementing certain forms of reference picture resampling (signaled quite differently than in the disclosed subject matter) in combination with temporal scalability to provide spatial scalability, in particular certain the reference pictures may be upsampled using ARC-style techniques to higher resolution to form the basis of the spatial enhancement layer. These upsampled images can be enhanced using conventional high resolution prediction engines to add detail.

[0096] The disclosed subject matter may be used in such an environment. In some cases, in the same or another embodiment of the invention, a value in the Network Abstraction Level (NAL) packet header, such as the Temporal ID field, can be used to indicate not only the temporal but also the spatial level. This provides certain advantages for certain system structures; for example, Selected Forwarding Units (SFU) servers built and optimized for temporal selective forwarding, selected based on the Temporal ID value of the NAL packet header, can be used without modification for scalable environments. To make this possible, there may be a requirement to map between the encoded picture size and the temporal level indicated by the Temporal ID field in the NAL packet header.

[0097] FIG. 6B is a flowchart illustrating a method (620) for decoding an encoded picture from an encoded video sequence according to an embodiment of the invention. In some implementations, one or more processing units in FIG. 6B may be performed by the decoder (210). In some implementations, one or more processing units in FIG. 6B may be performed by another device or group of devices separate from or including the decoder (210), such as an encoder (203).

[0098] As shown in FIG. 6B, the method (620) includes determining whether ARC information is available (621), and if it is determined that ARC information is not available, then the method ends (650). If it is determined that ARC information is available, then the method includes decoding, from the parameter set, a set of possible decoded grants (625).

[0099] The method (620) includes selecting, by means of an index encoded in a temporary header structure applicable to a group of samples, a possible decoded resolution among a plurality of possible decoded resolutions (630).

[0100] The method (620) includes resampling a sample from a group of samples based on an output resolution and a selected candidate decoded resolution (635).

[0101] Method (620) includes enabling prediction using a resampled sample (640).

[0102] The method (620) may further include that the temporary header structure is any one or any combination of an image header, a tile group header, a tile header, a slice header, and a block group header.

[0103] The method (620) may further include that the index is encoded with an Exponential Golomb code.

[0104] The method (620) may further include that the temporary header structure is a tile group header, and the tile group header contains a resampling factor.

[0105] The method (620) may further include that the transition header structure is a tile group header, and the tile group header contains adaptive resolution change (ARC) help information.

[0106] The method (620) may further include that the ARC reference information refers to a subset of the ARC information available in the parameter set.

[0107] The method (620) may further include that the number of possible decoded permissions is encoded in the exponential Golomb encoded syntax element preceding the set of possible decoded permissions.

[0108] Although in FIG. 6B shows exemplary blocks of method (620), in some implementations, method (620) may include additional blocks, fewer blocks, different blocks, or blocks arranged differently from those depicted in FIG. 6B. Additionally or alternatively, two or more blocks of the method (620) may be executed in parallel.

[0109] In addition, the proposed methods can be implemented by processing circuits (eg, one or more processors or one or more integrated circuits). In one example, one or more processors execute a program that is stored on a non-volatile computer-readable medium to perform one or more of the proposed methods.

[IT] FIG. 6C is a simplified block diagram of an apparatus (660) for decoding an encoded picture of a video sequence according to an embodiment of the invention.

[0111] As shown in FIG. 6C, the device (660) contains a decoding code (670), a selection code (675), a resampling code (680), and a permission code (685).

[0112] The decoding code (670) is configured to decode a set of possible decoded grants from the parameter set.

[0113] The selection code (675) is configured to select, by means of an index encoded in the temporary header structure applicable to the sample group, a possible decoded grant among a plurality of possible decoded grants.

[0114] The resampling code (680) is configured to resample a sample from a group of samples based on the output resolution and the selected candidate decoded resolution.

[0115] The enable code (685) is configured to enable prediction using the oversampled sample.

[0116] The technologies for signaling adaptive resolution parameters described above may be implemented in computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, in FIG. 7 shows a computer system 700 suitable for implementing some embodiments of the disclosed subject matter.

[0117] Computer software may be coded using any suitable machine code or computer language that may be the subject of translation, compilation, linking, or similar mechanisms to produce code containing instructions that can be executed directly or through interpretation, execution of microcode, and etc. computer central processing units (CPU, Central Processing Unit), graphic processors (GPU, Graphics Processing Unit), etc.

[0118] The commands may be executed on various types of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, IoT devices, and the like.

[0119] The components shown in FIG. 7 for computer system 700 are illustrative in nature and are not intended to suggest any limitation as to the scope of use or functionality of the computer software implementing the embodiments of the present invention. The component configuration is also not to be interpreted as having any dependency or requirement related to any one or combination of components shown in the exemplary computer system 700.

[0120] Computer system 700 may include certain input devices with a user interface. Such a user interface input device may respond to input by one or more human users through, for example, tactile input (e.g., keystrokes, swipe, movement of an information glove), audio input (e.g., voice, hand clapping), visual input (for example: gestures), olfactory input (not shown). User interface devices may also be used to capture certain multimedia data not necessarily directly related to conscious human input, such as sound (e.g. speech, music, ambient sound), images (e.g. scanned images, photographic images taken from a camera for still images), video (eg 2D video, 3D video including stereoscopic video).

[0121] User interface input devices may include one or more of (only one of each depicted): a keyboard 701, a mouse 702, a trackpad 703, a touch screen 710, an information glove 704, a joystick 705, a microphone 706, a scanner 707, cameras 708.

[0122] Computer system 700 may also include certain user interface output devices. Such user interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell/taste. Such user interface output devices may include tactile output devices (e.g., haptic feedback from a touch screen 710, information glove 704, or joystick 705, but may also be tactile feedback devices that do not serve as input devices), audio output devices (such as speakers 709, headphones (not shown)), visual output devices (for example, screens 710 including cathode ray tube screens), liquid crystal display (LCD, Liquid-Crystal Display) screens ), plasma screens, Organic Light-Emitting Diode (OLED) screens, each with or without touch capability, each with or without tactile feedback; - some of these may be capable of 2D visual output, or more than three-dimensional output by means such as stereographic output; virtual reality glasses (not pictured), holographic displays, smoke tanks (not pictured), and printers (not pictured).

[0123] The computer system 700 may also include user-accessible storage devices and associated media, such as optical media including read-only (ROM, Read Only Memory)/read and rewrite (RW, Read and Write) 720 with CDs (CD, Compact 018k)/Digital Versatile Discs (DVD, Digital Versatile Disc) or the like 721, flash drive 722, removable hard disk or solid state drive 723, traditional magnetic media such as tape and floppy disk ( not shown), Read Only Memory ROM/Application-Specific Integrated Circuit (ASIC)/Programmable Logic Device (PLD)-based dedicated devices such as security keys (not shown) etc.

[0124] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the currently disclosed subject matter does not encompass transmission media, carrier waves, or other transient signals.

[0125] Computer system 700 may also include an interface to one or more communication networks (755). Networks (755) may be, for example, wireless, wired, optical. Networks (755) may further be local, wide area, metropolitan, automotive, industrial, real-time, latency tolerant, and so on. Example networks (755) include local area networks such as Ethernet, wireless local area networks (LAN, Local Area Network), cellular networks, including global system for mobile communications (GSM, Global System for Mobile communications), third generation communications networks (3G , 3rd Generation), fourth generation communication networks (4G, 4th Generation), fifth generation communication networks (5G, 5th Generation), long-term evolution technology communication network (LTE, Long-Term Evolution), etc. etc., television (TV, Television) wired or wireless global digital networks, including cable television (TV), satellite television (TV) and terrestrial television broadcasting, automotive and industrial, including a local area network of controllers (CANBus, Controller Area Network Bus), etc. .d. Some networks (755) typically require external network interface adapters (754) that are connected to certain general purpose data ports or peripheral buses (749) (such as, for example, the Universal Serial Bus (USB) ports of the computer system 700; others are typically integrated into the core of the computer system 700 by connecting to a system bus as described below (for example, an Ethernet interface to a Personal Computer system or a cellular network interface to a smartphone computer system).Using any of these networks (755 ), computer system 700 can communicate with other entities. Such communication can be unidirectional, receive-only (e.g., TV broadcast), unidirectional, transmit-only (e.g., CANbus to certain CANbus devices), or bidirectional, e.g., to other computer systems, using local or wide digital networks Certain protocols and protocol stacks may be used on each of these networks (755) and network interfaces (754) as described above.

[0126] The aforementioned user interface devices, user accessible storage devices, and network interfaces (754) may be attached to the core 740 of the computer system 700.

[0127] The core 740 may contain one or more central processing units (CPU, Central Processing Unit) 741, graphics processors (GPU, Graphics Processing Unit) 742, specialized programmable processing units in the form of user-programmable matrices 743 logical elements (FPGA, Field Programmable Gate Area), hardware accelerators 744 for certain tasks, and so on. These devices, along with Readonly Memory (ROM) 745, Random-Access Memory (RAM) 746, graphics adapter 750, high-capacity internal storage 747, such as internal hard drives that are not user accessible, solid state drives (SSD, Solid State Drive), etc., can be connected via the 748 system bus. In some computer systems, the 748 system bus may be available in the form of one or more physical connectors to allow expansion with additional processors, graphics processors, and the like. Peripherals can be connected either directly to the core system bus 748 or via a peripheral bus 749. Peripheral bus architectures include an I/O bus for connecting peripherals PCI (Peripheral Component Interconnect), USB, and the like.

[0128] The CPUs 741, GPUs 742, FPGAs 743, and accelerators 744 may execute certain instructions, which in combination may constitute the aforementioned machine code. This machine code may be stored in ROM 745 or RAM 746. Temporary data may also be stored in RAM 746, while permanent data may be stored in internal mass storage 747, for example. Fast storage and retrieval capability for any of the storage devices may be provided through the use of a cache that may be tightly coupled to one or more of the CPU 741, GPU 742, mass storage 747, ROM 745, RAM 746, and etc.

[0129] The computer-readable medium may contain computer code for performing various computer-implemented operations. The media and computer code may be specially designed and constructed for the purposes of the present invention, or they may be well known and available to those skilled in the art of computer software.

[0130] By way of example, and not limitation, a computer system having architecture 700, and in particular core 740, may provide functionality as a result of execution by processor(s) (including CPUs, GPUs, FPGAs, accelerators, etc.) software embodied on one or more tangible, machine-readable media. Such computer-readable media can be media associated with user-accessible mass storage as described above, as well as certain core storage devices 740 that are non-volatile in nature, such as internal mass storage 747 or ROM 745. various embodiments of the present invention may be stored in such devices and executed by the core 740. The computer-readable medium may include one or more memories or integrated circuits, in accordance with particular needs. The software may cause core 740, and in particular the processors therein (including CPUs, GPUs, FPGAs, and the like), to execute certain processes or certain portions of specific processes described herein, including determining data structures stored in RAM 746, and changing such data structures in accordance with processes defined by the software. In addition or alternatively, a computer system may provide functionality as a result of logic embedded in hardware circuitry or otherwise embodied in circuitry (e.g., an accelerator 744) that can operate in place of or in conjunction with software to perform certain processes or certain portions of particular processes. described here. The link to the software may include logic and vice versa where appropriate. Reference to a computer-readable medium may encompass a circuit (for example, an integrated circuit (IC, Integrated Circuit) storing software to execute, a circuit embodying logic to execute, or both, where appropriate. The present invention encompasses any suitable combination of hardware and software.

[0131] While several illustrative embodiments of the invention are described in this disclosure, there are variations, permutations, and various replacement equivalents that are within the scope of the invention. Thus, it should be appreciated that those skilled in the art will be able to develop numerous systems and methods that, although not expressly shown or described herein, embody the principles of the invention and thus fall within its spirit and scope.

Claims (33)

1. A method for decoding an encoded image from an encoded video sequence, wherein the method is performed by at least one processor, and the method includes: decoding, from a set of parameters, a plurality of possible decoded resolutions; selecting, by means of an index encoded in the temporary header structure applicable to the sample group, a possible decoded resolution among the plurality of possible decoded resolutions; resampling the sample from the group of samples based on the output resolution and the selected candidate decoded resolution; and resolution of the prediction using the resampled sample based on the meta-information obtained during the decoding of the previous parts of the encoded image or the encoded video sequence, as well as the values of the samples previously restored and filtered by the loop filter. 2. The method of claim 1, wherein the temporary header structure is any one or any combination of an image header, a tile group header, a tile header, a slice header, and a block group header. 3. The method of claim 2, wherein the index is encoded with an exponential Golomb code. 4. The method of claim 2, wherein the temporary header structure is a tile group header and the tile group header contains a resampling factor. 5. The method of claim 2, wherein the temporary header structure is a tile group header and the tile group header contains adaptive resolution change (ARC) help information. 6. The method of claim 5, wherein the ARC reference information refers to a subset of the ARC information available in the parameter set. 7. The method of claim 1, wherein the number of possible decoded permissions is encoded in a syntax element encoded by the Exponential Golomb code and preceding the set of possible decoded permissions. 8. A device for decoding an encoded image from an encoded video sequence, comprising: at least one memory configured to store computer program code; as well as at least one processor configured to access at least one memory and operate in accordance with the computer program code, the computer program code comprising: a decoding code configured to cause at least one processor to decode, from the parameter set, a plurality of possible decoded resolutions; a selection code configured to cause at least one processor to select, by means of an index encoded in the temporary header structure applicable to the sample group, a possible decoded grant among a plurality of possible decoded grants; a resampling code configured to cause at least one processor to resample a sample from the group of samples based on the output resolution and the selected candidate decoded resolution; as well as a prediction resolution code configured to cause at least one processor to resolve prediction using a resampled sample based on meta-information obtained during decoding of previous parts of the encoded image or encoded video sequence, as well as previously restored and loop filtered sample values. 9. The apparatus of claim 8, wherein the temporary header structure is any one or any combination of an image header, a tile group header, a tile header, a slice header, and a block group header. 10. The apparatus of claim 9, wherein the index is encoded with an exponential Golomb code. 11. The apparatus of claim 9, wherein the temporary header structure is a tile group header and the tile group header contains a resampling factor. 12. The apparatus of claim 9, wherein the temporary header structure is a tile group header and the tile group header contains adaptive resolution change (ARC) help information. 13. The apparatus of claim 12, wherein the ARC reference information refers to a subset of the ARC information available in the parameter set. 14. The apparatus of claim 8, wherein the number of decoded candidate permissions is encoded in a syntax element encoded by the exponential Golomb code and preceding the set of possible decoded permissions. 15. A computer-readable storage medium that stores a program for decoding an encoded image from an encoded video sequence, the program containing instructions that cause the processor to: decode, from the parameter set, a plurality of possible decoded resolutions; select, by means of an index encoded in the temporary header structure applicable to the sample group, of a possible decoded resolution among the plurality of possible decoded resolutions; resampling a sample from the group of samples based on the output resolution and the selected candidate decoded resolution; and allow prediction using the resampled sample based on the meta-information obtained during the decoding of the previous parts of the encoded image or the encoded video sequence, as well as the values of the samples previously restored and filtered by the loop filter. 16. The computer-readable storage medium of claim 15, wherein the temporary header structure is any one or any combination of an image header, a tile group header, a tile header, a slice header, and a block group header. 17. The computer-readable storage medium of claim 16, wherein the index is encoded with an exponential Golomb code. 18. The computer-readable storage medium of claim 16, wherein the temporary header structure is a tile group header and the tile group header contains a resampling factor. 19. The computer-readable storage medium of claim 15, wherein the number of possible decoded permissions is encoded in the Exponential Golomb encoded syntax element preceding the set of possible decoded permissions.

Images