CN105191320A - Determining palettes in palette-based video coding - Google Patents
Determining palettes in palette-based video coding Download PDFInfo
- Publication number
- CN105191320A CN105191320A CN201480019413.2A CN201480019413A CN105191320A CN 105191320 A CN105191320 A CN 105191320A CN 201480019413 A CN201480019413 A CN 201480019413A CN 105191320 A CN105191320 A CN 105191320A
- Authority
- CN
- China
- Prior art keywords
- palette
- entry
- decoding
- block
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 claims abstract description 103
- 241001269238 Data Species 0.000 claims 2
- 238000005516 engineering process Methods 0.000 description 91
- 239000000523 sample Substances 0.000 description 79
- 230000008569 process Effects 0.000 description 44
- 238000013507 mapping Methods 0.000 description 36
- 238000006243 chemical reaction Methods 0.000 description 32
- 238000012545 processing Methods 0.000 description 27
- 238000013139 quantization Methods 0.000 description 24
- 230000000875 corresponding effect Effects 0.000 description 22
- 208000037170 Delayed Emergence from Anesthesia Diseases 0.000 description 21
- 238000004891 communication Methods 0.000 description 16
- 238000003860 storage Methods 0.000 description 11
- 230000009466 transformation Effects 0.000 description 11
- 230000006835 compression Effects 0.000 description 10
- 238000007906 compression Methods 0.000 description 10
- 230000005540 biological transmission Effects 0.000 description 8
- 239000003086 colorant Substances 0.000 description 7
- 238000013500 data storage Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 238000005192 partition Methods 0.000 description 6
- 230000000576 supplementary effect Effects 0.000 description 6
- 230000008859 change Effects 0.000 description 5
- 230000009471 action Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 4
- 239000013074 reference sample Substances 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000006073 displacement reaction Methods 0.000 description 3
- 238000011002 quantification Methods 0.000 description 3
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 238000012432 intermediate storage Methods 0.000 description 2
- 210000003127 knee Anatomy 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000009877 rendering Methods 0.000 description 2
- 230000011218 segmentation Effects 0.000 description 2
- 230000011664 signaling Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 235000009470 Theobroma cacao Nutrition 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000008901 benefit Effects 0.000 description 1
- 230000002457 bidirectional effect Effects 0.000 description 1
- 244000240602 cacao Species 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 238000009795 derivation Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000002035 prolonged effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000004513 sizing Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/93—Run-length coding
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
In an example, a method of coding video data includes determining a first palette having first entries indicating first pixel values, determining, based on the first entries of the first palette, one or more second entries indicating second pixel values of a second palette, and coding pixels of a block of video data using the second palette.
Description
Subject application advocates the apply on April 5th, 2013 the 61/809th, apply in No. 236 U.S. Provisional Application cases and on April 10th, 2013 the 61/810th, the rights and interests of No. 649 U.S. Provisional Application cases, the whole content of above application case is incorporated herein by reference separately.
Technical field
The present invention relates to Video coding and decoding.
Background technology
Digital video capabilities can be incorporated in diversified device, described device comprises Digital Television, digital direct broadcast system, wireless broadcast system, personal digital assistant (PDA), on knee or desktop PC, flat computer, electronic book reader, digital camera, digital recorder, digital media player, video game apparatus, video game console, honeycomb fashion or satellite radiotelephone (so-called " smart phone "), video conference call device, video streaming device and fellow thereof.Digital video apparatus implements video compression technology, such as, the video compression technology described in expansion of the standard defined by MPEG-2, MPEG-4, ITU-TH.263, ITU-TH.264/MPEG-4 the 10th part, advanced video decoding (AVC), high efficiency video coding (HEVC) standard developed at present and this class standard.Video-unit efficiently can be launched by implementing this type of video compression technology, receiving, encode, decoding and/or storing digital video information.
Video compression technology performs space (in picture) prediction and/or time (between picture) prediction reduces or removes redundancy intrinsic in video sequence.For block-based video coding, video segment (that is, a part for frame of video or frame of video) can be divided into video block.Video block in intra-encoded (I) section of picture uses to encode relative to the spatial prediction of the reference sample in the adjacent block in same picture.The video block in interframe decoding (P or B) section of picture can use spatial prediction relative to the reference sample in the adjacent block in same picture, or relative to the time prediction of the reference sample in other reference picture.Picture can be described as frame, and reference picture can be described as reference frame.
Space or time prediction cause the predictability block for block to be decoded.Residual data represents that the pixel between original block to be decoded and predictability block is poor.Encode through the residual data of difference between decode block and prediction block according to the motion vector and instruction that point to the reference sample block forming prediction block through interframe decode block.Encode according to Intra coding modes and residual data through intra-coding block.In order to realize further compression, residual data can be transformed to transform domain from pixel domain, thus produce residual coefficients, then can quantize residual coefficients.The coefficient through quantizing that initial placement is two-dimensional array can being scanned, to produce the one-dimensional vector of coefficient, and entropy decoding can be applied to realize more multiple pressure contracting.
Such as can produce multi views decoding bit stream from multiple visual angles coded views.Develop some three-dimensionals (3D) video standard of multi views decoding aspect.For example, different views can transmit left eye and right-eye view to support 3D video.Or some 3D video coding processes can apply the decoding of so-called multi views plus depth.In the decoding of multi views plus depth, 3D video bit stream can not only containing texture view component but also containing depth views component.For example, each view can comprise a texture view component and a depth views component.
Summary of the invention
Technology of the present invention relates to the video coding based on palette.For example, based in the decoding of palette, video decoder (video encoder or Video Decoder) can form so-called " palette " as representing the table of the color of the video data of specific region (such as, given piece).Decoding based on palette can be particularly useful for carries out decoding to the video data area of the color with relatively small amount.Not carry out decoding to actual pixel value (or it is remaining), video decoder can carry out decoding to the index value of one or many person in described pixel, and described index value makes described pixel relevant to the entry of the color representing described pixel in palette.Palette can explicitly encoded and be sent to decoder, to predict from previous palette entry, or its combination.The technology described in the present invention can comprise the technology of the various combinations for one or many person in the following: with the decoding mode of signal indication based on palette, decoding is carried out to palette, prediction palette, derive palette, and decoding based on palette decoding map and other syntactic element.
In an example, a kind of the method that video data carries out decoding to be comprised: the first palette determining the first entry with instruction first pixel value; Described first entry based on described first palette determines one or more second entry of the second pixel value of instruction second palette; And use described second palette to carry out decoding to the pixel of block of video data.
In another example, a kind of equipment for carrying out decoding to video data comprises: memory, its stored video data; And one or more processor, it is configured to: the first palette determining the first entry with instruction first pixel value; Described first entry based on described first palette determines one or more second entry of the second pixel value of instruction second palette; And use described second palette to carry out decoding to the pixel of the block of described video data.
In another example, a kind of equipment for carrying out decoding to video data comprises: for determining the device of the first palette of the first entry with instruction first pixel value; For determining one or more Article 2 destination device of the second pixel value of instruction second palette based on the described first entry of described first palette; And the device for using described second palette the pixel of block of video data to be carried out to decoding.
In another example, it stores instruction to a kind of non-transitory computer-readable media, and described instruction causes one or more processor when performing: the first palette determining the first entry with instruction first pixel value; Described first entry based on described first palette determines one or more second entry of the second pixel value of instruction second palette; And use described second palette to carry out decoding to the pixel of the block of described video data.
In another example, the method of video data being carried out to decoding comprises: determine and first index value that the first pixel in block of video data is associated, and wherein said first index value makes the position of described first pixel be related to the entry of the palette of pixel value; Determine and one or more second index value that one or more second pixel in described block of video data is associated based on described first index value, wherein said second index value makes the position of one or more the second pixel described be related to one or more entry of the described palette of pixel value; And decoding is carried out to described first pixel of described block of video data and one or more second pixel described.
In another example, a kind of equipment for carrying out decoding to video data comprises: memory, its stored video data; And one or more processor, it is configured to: determine and the first index value that the first pixel in the block of described video data is associated, and wherein said first index value makes the position of described first pixel be related to the entry of the palette of pixel value; Determine and one or more second index value that one or more second pixel in described block of video data is associated based on described first index value, wherein said second index value makes the position of one or more the second pixel described be related to one or more entry of the described palette of pixel value; And decoding is carried out to described first pixel of described block of video data and one or more second pixel described.
In another example, equipment for carrying out decoding to video data comprises: for determining a device for the first index value be associated with the first pixel in block of video data, wherein said first index value makes the position of described first pixel be related to the entry of the palette of pixel value; For determining the device of one or more the second index value be associated with one or more second pixel in described block of video data based on described first index value, wherein said second index value makes the position of one or more the second pixel described be related to one or more entry of the described palette of pixel value; And for carrying out the device of decoding to described first pixel of described block of video data and one or more second pixel described.
In another example, it stores instruction to a kind of non-transitory computer-readable media, described instruction causes one or more processor when implemented: determine and the first index value that the first pixel in the block of described video data is associated, and wherein said first index value makes the position of described first pixel be related to the entry of the palette of pixel value; Determine and one or more second index value that one or more second pixel in described block of video data is associated based on described first index value, wherein said second index value makes the position of one or more the second pixel described be related to one or more entry of the described palette of pixel value; And decoding is carried out to described first pixel of described block of video data and one or more second pixel described.
In accompanying drawing and the details setting forth one or more example of the present invention in hereafter describing.Further feature, target and advantage will from described description, graphic and claims are apparent.
Accompanying drawing explanation
Fig. 1 illustrates the block diagram that can utilize the instance video coded system of the technology described in the present invention.
Fig. 2 illustrates the block diagram can implementing the example video encoder of the technology described in the present invention.
Fig. 3 is the block diagram that graphic extension can implement the instance video decoder of the technology described in the present invention.
Fig. 4 illustrates the concept map that determine the example that video data carried out to the palette of decoding consistent with technology of the present invention.
Fig. 5 illustrates the concept map of the determination consistent with technology of the present invention to the example of the index of the palette of block of pixels.
Fig. 6 illustrates consistent with the technology of the present invention flow chart for using palette decoding mode video data to be carried out to the example procedure of decoding.
Fig. 7 illustrates consistent with the technology of the present invention flow chart for determining the example procedure of palette in based on the decoding of palette.
Fig. 8 illustrates consistent with the technology of the present invention flow chart for determining the example procedure of the index of block of video data in based on the video coding of palette.
Embodiment
The present invention comprises the technology for video coding and compression.Exactly, the present invention describes the technology of the decoding based on palette being used for video data.In traditional video coding, assuming that image is continuous tone and spatially level and smooth.Based on these hypothesis, developed various instrument, such as block-based conversion, filtering etc., and the superperformance for natural contents video shown by this little instrument.
Such as, but in the application such as such as remote desktop, cooperation work and radio display, the screen content (such as, word or computer graphical) that computer produces can be main contents to be compressed.The content of this type tends to have discrete tonal and feature sharp line, and high contrast object border.The supposition of continuous tone and smoothness no longer can be applicable to screen content, and therefore, conventional video decoding technique may not be the efficient way that compressed package contains the video data of screen content.
The present invention describes the decoding based on palette, and it can be particularly suitable for the content decoding that screen produces.For example, assuming that the specific region of video data has the color of relatively small amount.So-called " palette " can be formed as the table of the color of the video data for representing specific region (such as, given piece) by video decoder (video encoder or Video Decoder).Each pixel can be associated with the entry of the color representing pixel in palette.For example, video decoder can carry out decoding to making the index that pixel value is relevant to the appropriate value in palette.
In the above example, video encoder is by encoding to block of video data with under type: determine block palette (such as, clearly to palette carry out decoding, prediction palette), in palette the value of each pixel of normal indication entry and with the index value for pixel making pixel value relevant to palette, described piece is encoded.Video Decoder can obtain the index value of the palette of block and the pixel of described piece from encoded bit stream.Video Decoder can make the index value of pixel relevant to the entry of palette with the pixel value of reconstructed blocks.
Above example is set provides the general description of the decoding based on palette.In various example, the technology described in the present invention can comprise the technology of the various combinations for one or many person in following each: with the decoding mode of signal indication based on palette, launch palette, prediction palette, derive palette, and the decoding of launching based on palette maps and other syntactic element.This little technology can improve video coding efficiency, such as, need less bits to represent the content that screen produces.
Technology for the decoding based on palette of video data can use together with one or more other decoding technique, such as, for the technology of interframe or infra-frame prediction decoding.For example, as described in greater detail below, encoder or decoder or combined encoder-decoder (codec) can be configured to perform interframe and infra-frame prediction decoding and the decoding based on palette.
In some instances, the decoding technique based on palette can be configured to use together with one or more video coding standard.For example, high efficiency video coding (HEVC) combines by the video coding of ITU-T Video Coding Expert group (VCEG) and ISO/IEC motion characteristics planning (MPEG) the new video coding standards that cooperative groups (JCT-VC) develops.Recently HEVC text preliminary specifications is described in " high efficiency video coding (HEVC) text preliminary specifications 10 (for FDIS and agreement) (HighEfficiencyVideoCoding (HEVC) TextSpecificationDraft10 (forFDIS & Consent)) " (JCVC-L1003_v13 of the people such as Bu Luosi (Bross), JCT-VC the 12nd meeting of ITU-TSG16WP3 and ISO/IECJCT1/SC29/WG11,14 to 23 January in 2013) in (" HEVC draft 10 "), it can obtain from following address:
http://phenix.int-evry.fr/jct/doc_end_user/documents/12_Geneva/wg11/JCTVC-L1003-v13.zip。
Relative to HEVC framework, as an example, the decoding technique based on palette can be configured to be used as decoding unit (CU) pattern.In other example, the decoding technique based on palette can be configured to the PU pattern be used as in the framework of HEVC.Therefore, describe in the context of CU pattern all below the process that discloses can be applied to PU additionally or alternati.But these examples based on HEVC should not be regarded as restriction to the decoding technique based on palette described herein or restriction, because of a little technology for this reason can apply to work independently or as other existing or still leaved for development systems/standards part and apply.In these cases, the district of square block, rectangular block or even non-rectangular shape is can be for the unit of palette decoding.
Fig. 1 illustrates the block diagram that can utilize the instance video decoding system 10 of technology of the present invention.As used herein, term " video decoder " generally refers to both video encoder and Video Decoder.In the present invention, term " video coding " or " decoding " can usually refer to Video coding or video decode.The video encoder 20 of video decoding system 10 and Video Decoder 30 represent can be configured to the example of execution according to the device of the technology for the video coding based on palette of the various examples described in the present invention.For example, video encoder 20 and Video Decoder 30 can be configured to use and carry out decoding based on the decoding of palette or the non-decoding selectivity based on palette to various block of video data such as CU or PU in such as HEVC decoding.The non-decoding mode based on palette can refer to various inter prediction time decoding mode or infra-frame prediction space decoding mode, the various decoding modes of such as being specified by HEVC draft 10.
As shown in fig. 1, video decoding system 10 comprises source apparatus 12 and destination device 14.Source apparatus 12 produces encoded video data.Therefore, source apparatus 12 can be called as video coding apparatus or video encoder.Destination device 14 can be decoded to the encoded video data produced by source apparatus 12.Therefore, destination device 14 can be called as video decoder or video decoding apparatus.Source apparatus 12 and destination device 14 can be the examples of video decoding apparatus or video decoding equipment.
Source apparatus 12 and destination device 14 can comprise the device of broad range, comprise desktop PC, action calculation element, notes type (such as, on knee) computer, flat computer, Set Top Box, such as so-called " intelligence " phone expect someone's call hand-held set, TV, video camera, display unit, digital media player, video game console, car-mounted computer (in-carcomputer) or its fellow.
Destination device 14 can receive encoded video data via channel 16 from source apparatus 12.Channel 16 can comprise one or more media or the device that encoded video data can be moved to destination device 14 from source apparatus 12.In an example, channel 16 can comprise one or more communication medium making source apparatus 12 in real time encoded video data can be transmitted directly to destination device 14.In this example, source apparatus 12 can modulate encoded video data according to communication standards such as such as wireless communication protocols, and modulated video data can be transmitted into destination device 14.One or more communication medium can comprise wireless communication media and/or wired communication media, such as radio frequency (RF) frequency spectrum or one or more physical transmission line.One or more communication medium described can form a part for the network based on bag, based on network such as local area network (LAN), wide area network or the global network (such as, internet) of bag.One or more communication medium can comprise router, interchanger, base station or promote from source apparatus 12 to the miscellaneous equipment of the communication of destination device 14.
In another example, channel 16 can comprise the medium storing the encoded video data produced by source apparatus 12.In this example, destination device 14 can access medium via disk access or card access.Medium can comprise the data storage medium of multiple local access, such as Blu-ray Disc, DVD, CD-ROM, flash memory or other appropriate digital medium for storing encoded video data.
In another example, channel 16 can comprise the file server or another intermediate storage mean that store the encoded video data produced by source apparatus 12.In this example, destination device 14 can access via stream transmission or download the encoded video data being stored in file server or other intermediate storage mean place.File server can be the server that can store encoded video data and encoded video data is transmitted into the type of destination device 14.Instance file server comprises the webserver (such as, for website), file transfer protocol (FTP) (FTP) server, network attached storage (NAS) device and local drive.
Destination device 14 can connect (such as Internet connection) to access encoded video data by normal data.The example types of data cube computation can comprise be suitable for accessing the encoded video data be stored on file server wireless channel (such as, Wi-Fi connects), wired connection (such as, DSL, cable modem etc.) or both combinations.Encoded video data can be stream transmission from the transmission of file server, downloads transmission or both combinations.
Technology of the present invention is not limited to wireless application or setting.Described technology can be applied to video coding to support multiple multimedia application, such as airborne television broadcast, cable TV are launched, satellite television is launched, stream-type video is launched (such as, via internet), coding video frequency data to be stored on data storage medium, the video data be stored on data storage medium of decoding, or other application.In some instances, video decoding system 10 can be configured to support that unidirectional or bi-directional video transmission is to support the application such as such as stream video, video playback, video broadcasting and/or visual telephone.
Fig. 1 is only example, and technology of the present invention is applicable to video coding setting (such as, Video coding or video decode) of any data communication that may not comprise between code device and decoding device.In other example, data can be retrieved from local storage, network up flow type transmission, etc.Video coding apparatus can be encoded to data and data are stored into memory, and/or video decoder can from memory search data and to decoding data.In many instances, by do not communicate with one another but simply coded data to memory and/or from memory search data and the device of decoded data perform coding and decoding.
In the example of fig. 1, source apparatus 12 comprises video source 18, video encoder 20 and output interface 22.In some cases, output interface 22 can comprise modulator/demodulator (modulator-demodulator) and/or reflector.Video source 18 can comprise video capture device (such as, video camera), video archive containing the video data of previously having captured, in order to the video feed-in interface from video content provider's receiving video data and/or the computer graphics system for generation of video data, or the combination in this little source of video data.
Video encoder 20 can to the coding video data from video source 18.In some instances, encoded video data is transmitted directly to destination device 14 via output interface 22 by source apparatus 12.In other example, encoded video data can also be stored on medium or file server for being accessed for decoding and/or playback by destination device 14 after a while.
In the example of fig. 1, destination device 14 comprises input interface 28, Video Decoder 30 and display unit 32.In some instances, input interface 28 comprises receiver and/or modulator-demodulator.Input interface 28 can receive encoded video data on channel 16.Display unit 32 can integrate with destination device 14 or can in the outside of destination device 14.In general, display unit 32 shows the video data through decoding.Display unit 32 can comprise multiple display unit, such as the display unit of liquid crystal display (LCD), plasma display, Organic Light Emitting Diode (OLED) display or another type.
Video encoder 20 can be called another device some information " with signal indication " or " transmitting " being arrived such as Video Decoder 30 by the present invention substantially.Term " with signal indication " or " transmitting " can refer in order to the syntactic element of compressed video decoding data and/or the transmission of other data substantially.This communication can in real time or close to occurring in real time.Alternatively, can elapsed time span and occur this communication, such as when encode time, when syntactic element being stored into computer-readable storage medium with encoded bit stream, this communication can be there is, then, institute's syntax elements can be retrieved at any time by decoding device after being stored into these media.Therefore, although Video Decoder 30 can be called as " reception " some information, the reception of information not necessarily in real time or near occur in real time and can certain time after storing from media retrieval.
Video encoder 20 and Video Decoder 30 can be embodied as any one in the multiple appropriate circuitry of such as following each separately: one or more microprocessor, digital signal processor (DSP), application-specific integrated circuit (ASIC) (ASIC), field programmable gate array (FPGA), discrete logic, hardware or its any combination.When partly with implement software technology, the instruction of software can be stored in suitable non-transitory computer-readable storage medium and one or more processor can be used to perform instruction with hardware thus perform technology of the present invention by device.Any one (comprising the combination etc. of hardware, software, hardware and software) in foregoing teachings can be considered as one or more processor.Each in video encoder 20 and Video Decoder 30 can be included in one or more encoder or decoder, and any one accessible site in described encoder or decoder is the part of the combined encoder/decoder (codec) in related device.
In some instances, video encoder 20 and Video Decoder 30 according to video compression standard operation, such as mentioned above and HEVC standard that is that describe in HEVC draft 10.Except basic HEVC standard, there is ongoing effort and produce for the scalable video coding of HEVC, multi-view video decoding and 3D decoding expansion.In addition, can provide (such as) as described in the present invention based on the decoding mode of palette for the expansion of HEVC standard.In some instances, the technology described for the decoding based on palette in the present invention can be applicable to be configured to the encoder that operates according to other video coding standard (such as ITU-T-H.264/AVC standard or in the future standard) and decoder.Therefore, the application for the decoding mode based on palette of the decoding of the decoding unit (CU) in HEVC codec or predicting unit (PU) describes in order to the object of example.
In HEVC and other video coding standard, video sequence comprises a series of pictures usually.Picture also can be referred to as " frame ".Picture can comprise three array of samples, is expressed as S
l, S
cband S
cr.S
lit is the two-dimensional array (that is, block) of lightness sample.S
cbit is the two-dimensional array of Cb chroma sample.S
crit is the two-dimensional array of Cr chroma sample.Chroma sample can also be called as " colourity " sample in this article.In other cases, picture can be monochromatic and can only comprise lightness array of samples.
In order to produce the encoded expression of picture, video encoder 20 can produce the set of decoding tree unit (CTU).Each in CTU can be the decoding tree block of lightness sample, two of chroma sample corresponding decoding tree blocks, and in order to carry out the syntactic structure of decoding to the sample of decoding tree block.Decoding tree block can be the NxN block of sample.CTU also can be called as " tree block " or " maximum decoding unit " (LCU).The CTU of HEVC can be similar to the macro block such as H.264/AVC waiting other standard widely.But CTU may not be limited to specific size, and one or more decoding unit (CU) can be comprised.Section can be included in the CTU of the integer number of sequence continuously in raster scan.
Slice header and slice of data can be comprised through decoding section.The slice header of section can be the syntactic structure of the syntactic element comprising the information provided about section.Slice of data can comprise section through decoding CTU.
The present invention can use term " video unit " or " video block " or " block " to refer to the syntactic structure of one or more sample block and the sample for one or more sample block described in decoding.The video unit of example types or block can comprise CTU, CU, PU, converter unit (TU), macro block, macroblock partition etc.In some contexts, the discussion of PU can be exchanged with the discussion of the macro block of macroblock partition.
In order to produce the CTU through decoding, video encoder 20 recursively can perform Quadtree Partition to the decoding tree block of CTU, so that decoding tree block is divided into decode block, and therefore called after " decoding tree unit ".Decode block is the NxN block of sample.CU can be the decode block of the lightness sample of the picture with lightness array of samples, Cb array of samples and Cr array of samples, and two of chroma sample corresponding decode block, and in order to carry out the syntactic structure of decoding to the sample of decode block.The decode block of CU can be divided into one or more prediction block by video encoder 20.Prediction block can be rectangle (that is, square or the non-square) block of the sample applying identical prediction.The predicting unit (PU) of CU can be the prediction block of the lightness sample of picture, the prediction block that two of the chroma sample of picture are corresponding, and the syntactic structure in order to predict prediction block sample.For the lightness prediction block of each PU of CU, Cb, video encoder 20 can produce predicts that block and Cr predict the predictability lightness block of block, Cb block and Cr block.
Video encoder 20 can use infra-frame prediction or inter prediction to produce predictability block for PU.If video encoder 20 uses infra-frame prediction to produce the predictability block of PU, then video encoder 20 can produce the predictability block of PU based on the sample through decoding of the picture be associated with PU.
If video encoder 20 uses inter prediction to produce the predictability block of PU, then video encoder 20 can produce the predictability block of PU based on the sample through decoding of one or more picture being different from the picture be associated with PU.Video encoder 20 can use single directional prediction or bi-directional predicted with the predictability block producing PU.When video encoder 20 uses single directional prediction to produce the predictability block of PU, PU can have single MV.When video encoder 20 use bi-directional predicted produce the predictability block of PU time, PU can have two MV.
After video encoder 20 produces the predictability lightness of one or more PU of CU, Cb and Cr block, video encoder 20 can produce the lightness residual block of CU.Difference between the sample that lightness sample in one in the predictability lightness block of each sample instruction CU in the lightness residual block of CU is corresponding with the original lightness decode block of CU.In addition, video encoder 20 can produce Cb residual block for CU.Each sample in the Cb residual block of CU can indicate the difference between sample that the Cb sample in the one in the predictability Cb block of CU is corresponding with the original Cb decode block of CU.Video encoder 20 can also produce the Cr residual block of CU.Each sample in the Cr residual block of CU can indicate the difference between sample that the Cr sample in the one in the predictability Cr block of CU is corresponding with the original Cr decode block of CU.
In addition, video encoder 20 can use Quadtree Partition that the lightness of CU, Cb and Cr residual block are resolved into one or more lightness, Cb and Cr transform block.Transform block can be the rectangular block of the sample it being applied to identical conversion.The converter unit (TU) of CU can be the transform block of lightness sample, two of chroma sample corresponding transform blocks, and the syntactic structure for converting transform block sample.Therefore, each TU of CU can be associated with lightness transform block, Cb transform block and Cr transform block.The lightness transform block be associated with TU can be the sub-block of the lightness residual block of CU.Cb transform block can be the sub-block of the Cb residual block of CU.Cr transform block can be the sub-block of the Cr residual block of CU.
One or more conversion can be applied to the lightness transform block of TU thus be TU generation brightness coefficient block by video encoder 20.Coefficient block can be the two-dimensional array of conversion coefficient.Conversion coefficient can be scalar.One or more conversion can be applied to the Cb transform block of TU thus the Cb coefficient block of generation TU by video encoder 20.One or more conversion can be applied to the Cr transform block of TU thus the Cr coefficient block of generation TU by video encoder 20.
After generation coefficient block (such as, brightness coefficient block, Cb coefficient block or Cr coefficient block), video encoder 20 can quantization parameter block.Quantize to refer to substantially and conversion coefficient quantized with the amount that may reduce the data representing conversion coefficient thus the process of compression is further provided.After video encoder 20 quantization parameter block, video encoder 20 can carry out entropy code such as to the syntactic element of the conversion coefficient of instruction through quantizing, and video encoder 20 can perform context adaptive binary arithmetically decoding (CABAC) to instruction through the syntactic element of quantization transform coefficient.Video encoder 20 can export through entropy code syntactic element in bit stream.
Video encoder 20 can output packet containing through the bit stream of the syntactic element of entropy code.Bit stream can comprise the bit sequence forming the expression mapping sheet and associated data through decoding.Bit stream can comprise the sequence of network abstract layer (NAL) unit.Each in described NAL unit comprises NAL unit header and is encapsulated Raw Byte Sequence Payload (RBSP).NAL unit header can comprise the syntactic element of instruction NAL unit type code.The described NAL unit type code of being specified by the NAL unit header of NAL unit indicates the type of NAL unit.RBSP can be the syntactic structure containing the integer number byte be encapsulated in NAL unit.In some cases, RBSP comprises zero bits.
Dissimilar NAL unit can be encapsulated dissimilar RBSP.For example, the NAL unit of the first kind can be encapsulated the RBSP for image parameters collection (PPS), and the NAL unit of Second Type can be encapsulated the RBSP for cutting into slices through decoding, and the NAL unit of the 3rd type can be encapsulated the RBSP etc. for SEI.The NAL unit be encapsulated for the RBSP (compared with the RBSP for parameter set and SEI message) of video coding data can be called as video coding layer (VCL) NAL unit.
Video Decoder 30 can receive the bit stream produced by video encoder 20.In addition, Video Decoder 30 can dissect bit stream to decode to the syntactic element from bit stream.Video Decoder 30 can at least partly based on the picture of the syntactic element reconstructed video data from bitstream decoding.Process in order to reconstructed video data usually can be reciprocal with the process performed by video encoder 20.
Such as, Video Decoder 30 MV of PU can be used to think the PU of current C U determines predictability sample block.In addition, Video Decoder 30 can the transformation coefficient block that is associated with the TU of current C U of re-quantization.Video Decoder 30 can perform inverse transformation to reconstruct the transform block be associated with the TU of current C U to transformation coefficient block.Be increased to the sample of the correspondence of the transform block of the TU of current C U by the sample of the predictability sample block by the PU for current C U, Video Decoder 30 can reconstruct the decode block of current C U.Be used for the decode block of each CU of picture by reconstruct, Video Decoder 30 can reconstructed picture.
In some instances, video encoder 20 and Video Decoder 30 can be configured to perform the decoding based on palette.For example, based in the decoding of palette, and the above-mentioned infra-frame prediction of non-executing or inter prediction decoding technique, so-called palette can be decoded as the table of the color of the video data for representing specific region (such as, given piece) by video encoder 20 and Video Decoder 30.Each pixel can be associated with the entry of the color representing pixel in palette.For example, video encoder 20 can carry out decoding to making the index that pixel value is relevant to the appropriate value in palette with Video Decoder 30.
In the above example, video encoder 20 by determine block palette, in palette the value of each pixel of normal indication entry and with the index value for pixel making pixel value relevant to palette, coding is carried out to palette block of video data is encoded.Video Decoder 30 can obtain the index value of the palette of block and the pixel of described piece from encoded bit stream.Video Decoder 30 can make the index value of pixel relevant to the entry of palette with the pixel value of reconstructed blocks.
Decoding based on palette can have a certain amount of signaling consumption.For example, some positions can be needed to use characteristic (size of such as palette) and the palette self of signal indication palette.In addition, some positions can be needed to the index value of the pixel with signal indication block.In some instances, technology of the present invention can reduce the number with the position needed for this information of signal indication.For example, the technology described in the present invention can comprise the technology of the various combinations of one or many person in following each: with the decoding mode of signal indication based on palette, launch palette, prediction palette, derive palette, and the decoding of launching based on palette maps and other syntactic element.
Aspect of the present invention predicts for palette.For example, according to aspects of the present invention, video encoder 20 and/or Video Decoder 30 can determine the first palette of the first entry with instruction first pixel value.Video encoder 20 and/or Video Decoder 30 can determine one or more second entry of instruction second pixel value of the second palette subsequently based on the first entry of the first palette.Video encoder 20 and/or Video Decoder 30 also can use the second palette to carry out decoding to the pixel of block of video data.
When determining the second entry of the second palette based on first entry, video encoder 20 can be encoded to multiple syntactic element, and institute's syntax elements can be used to reconstruct the second palette by Video Decoder.For example, video encoder 20 can encode one or more syntactic element to indicate whole palette (or multiple palette in bit stream, when each color component (such as, Y, Cb, Cr or Y, U, V or R, G, B) of video data has independent palette) be copy from one or more adjacent block of the block of current positive decoding.The entry of the current color palette of current block can be called as predictability palette from the palette of its prediction (such as, copying).Predictability palette can contain to come the palette entry of one or more adjacent block of the adjacent block in the specific scanning sequence of self-contained spatial neighboring blocks and/or block.For example, adjacent block spatially can be positioned at the left side (left side adjacent block) or top (the top adjacent block) of the block of current positive decoding.In another example, video encoder 20 can use the most frequent sample value determination predictability palette entry in the cause and effect neighbor of current block.In another example, adjacent block can be adjacent according to the block of specific scanning sequence and current positive decoding in order to carry out decoding to block.That is, adjacent block can be one or more block of decoding before current block in scanning sequence.Video encoder 20 can encode the position of the adjacent block indicating palette to copy from it to one or more syntactic element.
In some instances, palette prediction can be performed by order one by one.For example, video encoder 20 can encode to indicate described palette entry whether to be included in the palette of current block for each entry of predictability palette to one or more syntactic element.If video encoder 20 does not predict the entry of the palette of current block, so video encoder 20 can encode the number of specifying non-predicted entry and this little entry to one or more extra syntactic element.
In some instances, for predict whole palette technology can with the technical combinations of one or more entry for predicting palette.For example, whether video encoder 20 can encode to indicate the current color palette to be copy completely from predictability palette to one or more syntactic element in bit stream.If situation is not like this, so whether video encoder 20 can encode to one or more syntactic element and to copy with each entry in indication predicting palette in bit stream.
In another example, be alternative in the number by signal indication entry and palette value, whether video encoder 20 can be the flag of the final palette entry of palette in order to the palette value of instruction signal indication with signal indication after by each palette value of signal indication.If palette has reached a certain largest amount, so video encoder 20 can without this " palette end " flag of signal indication.
According to aspects of the present invention, whether video encoder 20 can encode to indicate palette to predict through to enable and/or effectively to one or more syntactic element.For illustration of in the example of object, video encoder 20 can encode whether use palette to predict the palette predicting relevant block for each block (such as, CU or PU) instruction video encoder 20 to pred_palette_flag.In some instances, video encoder available signal represents the independent flag (such as, three flags of each block) of each color component.In other example, video encoder 20 available signal represents the single flag of all colours component being applicable to block.
Video Decoder 30 can obtain the information of above identification from encoded bit stream and usage data reconstructs palette.For example, whether Video Decoder 30 can receive the specific palette of instruction is the data predicted from another palette and allows Video Decoder 30 to use the information of suitable predictability palette entry.
In some instances, additionally or alternati, video encoder 20 and/or Video Decoder 30 can be in operation (that is, dynamically) structure palette.For example, entry can be added to empty palette by video encoder 20 and/or Video Decoder 30 during decoding.That is, video encoder 20 can add pixel value to palette when producing for the position in block and launching pixel value.The pixel of relatively late decoding in block with reference to the entry (such as, with index value) of comparatively early adding of palette, instead of can launch pixel value.Equally, after the new pixel value receiving the position in block, Video Decoder 30 can at once be followed the process identical with video encoder 20 and in palette, be comprised described pixel value.In this way, Video Decoder 30 constructs the palette identical with video encoder 20.Video Decoder 30 can receive for the pixel with the value be included in palette the index value identifying described value.Video Decoder 30 can use the pixel of received information (such as, the pixel value of palette and index value) reconstructed blocks.
In some instances, video encoder 20 and Video Decoder 30 can maintain the palette of fixed size.For example, video encoder 20 and Video Decoder 30 can add recently through reconstructed pixel value, and thus palette is through reconstruct.For each entry of adding palette to, abandon the entry of adding palette the earliest to.This is also sometimes referred to as first in first out (FIFO).This process upgrading palette only can be applied to use the block of pallet mode decoding or have nothing to do in decoding mode and be applied to all pieces.
Above-mentioned technology relates generally to video encoder 20 and Video Decoder 30 constructs and/or launches the palette for the decoding based on palette.Other side of the present invention relates to structure and/or launches the mapping allowing video encoder 20 and/or Video Decoder 30 to determine pixel value.For example, other side of the present invention relate to structure and/or launch make the index-mapping that specific pixel is relevant to the entry of palette.
In some instances, whether video encoder 20 pixel of indicator collet can have respective value in palette.For illustration of in the example of object, assuming that (i, the j) entry mapped corresponds to (i, the j) location of pixels in block of video data.In this example, video encoder 20 can be encoded to the flag of each location of pixels for block.The flag that video encoder 20 can equal one for the setting of (i, j) entry is the one in the multiple values in palette with the pixel value of instruction in (i, j) position.When color is included in (that is, flag equals one) in palette, to instruction, video encoder 20 also can identify that the data of the palette index of (i, j) entry of the color in palette are encoded.When the color of pixel is not included in (that is, flag equals zero) in palette, video encoder 20 also can be encoded to the data of the sample value of instruction pixel.Video Decoder 30 can obtain above-mentioned data from encoded bit stream and use described data to determine palette index and/or the pixel value of the ad-hoc location block.
In some instances, the pixel-map that can there is given position to palette index and neighbor be mapped to identity palette index probability between correlation.That is, when pixel-map is to specific palette index, one or more neighbor (with regard to locus) is mapped to the probability of identity palette index can be relatively high.
According to aspects of the present invention, video encoder 20 and/or Video Decoder 30 can be determined relative to one or more index of block of video data and one or more index of the same block of video data of decoding.For example, video encoder 20 and/or Video Decoder 30 can be configured to determine and the first index value that the first pixel in block of video data is associated, and wherein said first index value makes the value of the first pixel relevant to the entry of palette.Video encoder 20 and/or Video Decoder 30 also can be configured to determine and one or more second index value that one or more second pixel in block of video data is associated based on the first index value, and carry out decoding to first and one or more second pixel of block of video data.Therefore, in this example, the index that can map relative to one or more other index decoding mapped.
In some instances, video encoder 20 can be mapped to one or more syntactic element being in some contiguous pixels of given scanning sequence of same index value to instruction.The string of the index value of identical value can be referred to as " prolonging row " in this article.In some instances, pixel value can be associated with the index value of in palette just.Therefore, in some cases, the prolonging row and also can refer to the string of the pixel value of identical value of value.In other example, as hereafter relative to damaging described by decoding, more than one pixel value can be mapped to the same index value in palette.In this little example, the row that prolongs of value refers to the index value of identical value.
For illustration of in the example of object, if two continuity indexs being in given scanning sequence have different value, so prolong row and equal zero.If two continuity indexs being in given scanning sequence have identical value but the 3rd index in scanning sequence has different value, so prolong row and equal one.Video Decoder 30 can obtain instruction from encoded bit stream and prolongs capable syntactic element and use described data to determine to have the number of the contiguous pixels position of same index value.
Additionally or alternati, according to aspects of the present invention, the line of one or more entry that video encoder 20 and Video Decoder 30 can perform for mapping copies.Described entry due to map entry and the location of pixels of block between relation and also can be referred to as " position ".In some instances, line copies and can be depending on scanning direction.For example, video encoder 20 pixel value of ad-hoc location in indicator collet or index-mapping value can equal pixel in the row (for vertical scanning) on line (for horizontal sweep) above described ad-hoc location or the described ad-hoc location left side or index value.The number of the pixel value or index that are in scanning sequence of the row of the respective pixel value or index or the ad-hoc location left side that equal the top of ad-hoc location also can be designated as and prolong row by video encoder 20.In this example, video encoder 20 and or Video Decoder 30 can from specifying adjacent lines and from the replicate entries pixel specified number of the line of the block of current positive decoding or index value.
In some instances, be worth the line that the line copied from it directly can be adjacent to the position of current positive decoding, such as above it or the left side.In other example, some lines of block can be cushioned by video encoder 20 and/or Video Decoder 30, can be used as the predictability value of the line of the mapping of current positive decoding with any one making in some lines of mapping.For illustration of in the example of object, video encoder 20 and/or Video Decoder 30 can be configured to before carrying out decoding to the current line of pixel, store previous four lines index or pixel value.In this example, predictability row (row that index or pixel value copy from it) can indicate to block other code such as unitary code or such as unary code bit stream.Relative to brachymemma unitary code, video encoder 20 and/or Video Decoder 30 can calculate (such as, row_index-1) based on maximum row or maximum column calculates the maximum that (such as, column_index-1) determines to block unitary code.In addition, the instruction of the number of the position from predictability row be replicated also can be included in bit stream.In some instances, if just from its prediction line of current location or dependent of dead military hero in another block (such as, CU or CTU), this prediction of so can stopping using.
According to aspects of the present invention, the technology for prolonging row decoding of advancing to so-called entry can use with the above-mentioned combine with technique copied for line.For example, whether whether the value of the entry during video encoder 20 can map instruction is the value of the entry obtaining from palette or mapping is that one or more syntactic element (such as, flag) that the line through previous decoding from map obtains is encoded.Video encoder 20 also can be encoded to one or more syntactic element of the bar destination locations in the index value of instruction palette or line (row or column).Video encoder 20 also can be encoded to some one or more syntactic elements of continuous strip object of the shared identical value of instruction.Video Decoder 30 can obtain this information from encoded bit stream and use mapping and the pixel value of described signal reconstruct block.
As mentioned above, with the index that certain order scanning maps.According to aspects of the present invention, scanning direction can be vertically, level or be in diagonal (such as, 45 degree or 135 degree, paired linea angulata ground in frame).In some instances, video encoder 20 can be encoded to the instruction of each block one or more syntactic element of scanning direction for the index scanning described piece.Additionally or alternati, can come with signal indication based on so-called supplementarys such as such as block size, color space and/or color components or infer scanning direction.Video encoder 20 can specify the scanning of each color component for block.Or invisible scanning can be applicable to all colours component of block.
Technology of the present invention also comprises the other side of the decoding based on palette.For example, according to aspects of the present invention, video encoder 20 and/or Video Decoder 30 can carry out decoding to indicate described piece to one or more syntactic element for each block is use the decoding of palette decoding mode.For example, whether video encoder 20 and/or Video Decoder 30 can carry out decoding to palette mode flag (PLT_Mode_flag) and will be used for carrying out decoding to specific piece with the decoding mode of instruction based on palette.In this example, video encoder 20 codified equals the PLT_Mode_flag of is use pallet mode coding to specify current block (" current block ") of just encoding.The value of null PLT_Mode_flag specifies current block not to be use pallet mode coding.In the case, Video Decoder 30 can obtain PLT_Mode_flag from encoded bit stream and application based on the decoding mode of palette to decode to block.There is more than one decoding mode based on palette wherein can use (such as, there is more than one technology based on palette and can be used for decoding) example in, one or more syntactic element can indicate for the one in the multiple different pallet mode of block.
In some cases, the null PLT_Mode_flag of video encoder 20 codified is not use pallet mode coding to specify current block.In this little example, video encoder 20 can use in multiple inter prediction, infra-frame prediction or other decoding mode any one block is encoded.When PLT_Mode_flag equals zero, video encoder 20 can launch extraneous information (such as, syntactic element) to indicate the AD HOC for encoding to relevant block.In some instances, as described below, described pattern can be HEVC decoding mode.The use of PLT_Mode_flag describes for the object of example.In other example, such as whether other syntactic element such as multi-position code can will be used for one or more block in order to indicate the decoding mode based on palette, or instruction will use which one in multiple pattern.
When using the decoding mode based on palette, video encoder 20 such as uses one or many person in technology described herein to launch palette in the encoded video data bitstream for Video Decoder 30.Palette can be launched for each block or can share between some pieces.It is main and/or representational some pixel values that palette can refer to for block.
According to aspects of the present invention, the size of such as, palette in the number of the pixel value comprised in palette can be fixing, maybe can use one or more syntactic element signal indication in encoded bit stream.As described in greater detail below, pixel value can such as depend on for the color space of decoding by some compositions of sample.For example, pixel value can comprise lightness and chroma sample (such as, lightness, U colourity and V colourity (YUV) sample or lightness, Cb colourity and Cr colourity (YCbCr) sample).In another example, pixel value can comprise red, green and blue (RGB) sample.As described herein, term pixel value can refer to the one or many person contributed in the sample of pixel substantially.That is, term pixel value not necessarily refers to all samples contributing to pixel, and can in order to describe the single sample value contributing to pixel.
According to aspects of the present invention, palette can be launched individually for each color component of specific piece.For example, in YUV color space, the palette (expression Y value) for Y-component, the palette for U component (representing U value) and the another palette (representing V value) for V component can be there is.In another example, palette can comprise specific piece institute important.In this example, the i-th entry in palette can comprise three values (such as, Yi, Ui, Vi).According to aspects of the present invention, one or more syntactic element can indicate the size of the palette for each component (such as, Y, U, V or analog) individually.In other example, it is important that single size may be used for institute, indicates important size to make one or more syntactic element.
According to aspects of the present invention, video encoder 20 and/or Video Decoder can damage or lossless manner performs based on the decoding of palette.Namely, in some instances, video encoder 20 and/or Video Decoder 30 can use the palette entry of the pixel value being matched with block (or sending actual pixel value by not comprising described pixel value in palette) nondestructively to carry out decoding to the video data of block.In other example, as hereafter comparatively described in detail relative to Fig. 5, video encoder 20 and/or Video Decoder 30 can use the palette entry of the pixel value not being exactly matched in block to carry out decoding (damaging decoding) to the video data of block.
In some instances, for the decoding based on palette of video data technology can with such as interframe or infra-frame prediction decoding one or more other decoding technique such as technology together with use.For example, as described in greater detail below, encoder or decoder or combined encoder-decoder (codec) can be configured to perform interframe and infra-frame prediction decoding and the decoding based on palette.
Fig. 2 illustrates the block diagram can implementing the example video encoder 20 of technology of the present invention.Fig. 2 provides for illustrative purposes, and should not be regarded as technical limitations as roughly illustrating and description person in the present invention.For illustrative purposes, the present invention describes video encoder 20 in the context of HEVC decoding.But technology of the present invention goes for other coding standards or method.
Video encoder 20 represents can be configured to the example of execution according to the device of the technology for the video coding based on palette of the various examples described in the present invention.For example, video encoder 20 can be configured to use and carry out decoding based on the decoding of palette or the non-decoding selectivity based on palette to various block of video data such as CU or PU in such as HEVC decoding.The non-decoding mode based on palette can refer to various inter prediction time decoding mode or infra-frame prediction space decoding mode, the various decoding modes that such as HEVC draft 10 is specified.In an example, video encoder 20 can be configured to produce the palette of the entry with instruction pixel value, select the pixel value representing the pixel value of at least some position of block of video data in palette, and by the information that signal indication makes at least some in the position of block of video data associate with the entry corresponding respectively to selected pixel value in palette.Can use that video data is decoded by Video Decoder 30 by the information of signal indication.
In the example of figure 2, video encoder 20 comprise prediction processing unit 100, residue generation unit 102, conversion process unit 104, quantifying unit 106, inverse quantization unit 108, inversion process unit 110, reconfiguration unit 112, filter cell 114, through decode picture buffer 116 and entropy code unit 118.Prediction processing unit 100 comprises inter prediction processing unit 120 and intra-prediction process unit 126.Inter prediction processing unit 120 comprises motion estimation unit and motion compensation units (not shown).Video encoder 20 also comprises the coding unit 122 based on palette, and it is configured to the various aspects performing the decoding technique based on palette described in the present invention.In other example, video encoder 20 can comprise more, less or different functional units.
Video encoder 20 can receiving video data.Video encoder 20 can be encoded to each CTU in the section of the picture of video data.Each in CTU can be associated with the equal-sized lightness decoding tree block (CTB) of picture and corresponding CTB.As a part of encoding to CTU, prediction processing unit 100 can perform Quadtree Partition so that the CTB of CTU is divided into block less gradually.These less blocks can be the decode block of CU.Such as, the CTB be associated with CTU is divided into four equal-sized sub-blocks by prediction processing unit 100, and one or many person in sub-block is divided into four equal-sized sub-sub-blocks etc.
Video encoder 20 can encode to the CU of CTU the encoded expression (that is, through the CU of decoding) producing CU.As a part of encoding to CU, prediction processing unit 100 can split the decode block be associated with CU in the middle of one or more PU of CU.Therefore, with lightness, each PU can predict that block and corresponding colorimetric prediction block are associated.Video encoder 20 and Video Decoder 30 can support the PU with all size.As indicated above, the large I of CU refer to the size of the lightness decode block of CU and the large I of PU refer to PU lightness prediction block size.Assuming that the size of specific CU is 2Nx2N, then video encoder 20 and Video Decoder 30 can support the PU size of 2Nx2N or NxN for infra-frame prediction, and for 2Nx2N, 2NxN, Nx2N, NxN of inter prediction or the symmetrical PU size of similar size.Video encoder 20 and Video Decoder 30 can also support the asymmetric segmentation of the PU size of 2NxnU, 2NxnD, nLx2N and nRx2N for inter prediction.
Inter prediction processing unit 120 can produce by performing inter prediction to each PU of CU the predictive data being used for PU.Predictive data for PU can comprise the predictability sample block of PU and the movable information for PU.Depend on that PU is that inter prediction unit 121 can perform different operating to the PU of CU in I section, in P section or in B section.In I section, all PU are through infra-frame prediction.Therefore, if PU is that then inter prediction unit 121 does not perform inter prediction to PU in I section.Therefore, for the block of encoding in I pattern, be that use is formed from the spatial prediction of the adjacent block through previous coding in same frame through prediction block.
If PU is in P section, so whether the motion estimation unit of inter prediction processing unit 120 reference picture in searching for reference just list (such as, " RefPicList0 ") can have reference area for PU.Reference area for PU can be the district of the sample block containing the sample block the most closely corresponding to PU in reference picture.Motion estimation unit can produce instruction containing the reference key for the position of reference picture in RefPicList0 of the reference area of PU.In addition, motion estimation unit can produce the MV of the spatial displacement between the decode block of instruction PU and the reference position being associated in reference area.Such as, MV provides the bivector from current coordinate through the picture of decoding to the skew of the coordinate in reference picture.The movable information that motion estimation unit can be PU with reference to index and MV output.The motion compensation units of inter prediction processing unit 120 can based on the reality of the reference position indicated at the motion vector by PU or the predictability sample block producing PU through interpolation sample.
If PU is that then motion estimation unit can perform single directional prediction or bi-directional predicted to PU in B section.In order to perform single directional prediction to PU, whether the reference picture that motion estimation unit can search for RefPicList0 or the second reference picture list (" RefPicList1 ") has the reference area for PU.The exportable the following of motion estimation unit is as the movable information of PU: reference key, its instruction reference picture position in RefPicList0 or RefPicList1 containing reference area; MV, the space displacement between its reference position indicating the sample block of PU to be associated with same reference area; And one or more prediction direction designator, whether its instruction reference picture is in RefPicList0 or RefPicList1.The motion compensation units of inter prediction processing unit 120 can at least in part based on the reference area indicated at the motion vector by PU reality or produce the predictability sample block of PU through interpolation sample.
In order to perform bidirectional interframe predictive to PU, whether the motion estimation unit reference picture can searched in RefPicList0 has the reference area for PU, and whether the reference picture can searched in RefPicList1 has another reference area for PU.Motion estimation unit can produce the reference picture index of the reference picture position among RefPicList0 and RefPicList1 of instruction containing reference area.In addition, motion estimation unit can produce the MV of the spatial displacement between reference position and the sample block of PU indicating and be associated with reference area.The movable information of PU can comprise reference key and the MV of PU.Motion compensation units can be based, at least in part, on the reality of the reference area indicated by the motion vector of PU or produce the predictability sample block of PU through interpolation sample.
According to various example of the present invention, video encoder 20 can be configured to perform the decoding based on palette.Relative to HEVC framework, as an example, the decoding technique based on palette can be configured to be used as decoding unit (CU) pattern.In other example, the decoding technique based on palette can be configured to the PU pattern be used as in the framework of HEVC.Therefore, in the context of CU pattern, all disclosed processes (running through the present invention) described here can be applied to PU additionally or alternati.But these examples based on HEVC should not be regarded as restriction to the decoding technique based on palette described herein or restriction, because of a little technology for this reason can apply to work independently or as other existing or still leaved for development systems/standards part and apply.In these cases, the district of square block, rectangular block or even non-rectangular shape is can be for the unit of palette decoding.
When (such as) selects the coding mode based on palette for CU or PU, the coding unit 122 (such as) based on palette can perform the decoding based on palette.For example, coding unit 122 based on palette is configurable with the palette producing the entry with instruction pixel value, select the pixel value representing the pixel value of at least some position of block of video data in palette, and by the information that signal indication makes at least some in the position of block of video data associate with the entry corresponding respectively to selected pixel value in palette.Although be performed various functional description by the coding unit 122 based on palette, some or all in this little function can be performed by the combination of other processing unit or different processing units.
Coding unit 122 based on palette can be configured to produce any one in various syntactic elements described herein.Therefore, video encoder 20 can be configured to use and encode to block of video data based on the decoding mode of palette as described in the present invention.Video encoder 20 optionally uses palette decoding mode to encode to block of video data, or uses different mode (such as this HEVC inter prediction or infra-frame prediction decoding mode) to encode to block of video data.Block of video data can (such as) be CU or PU produced according to HEVC decode procedure.Video encoder 20 can inter prediction time prediction or infra-frame prediction space decoding mode be encoded to some blocks, and decodes to other block with the decoding mode based on palette.
Intra-prediction process unit 126 produces the predictive data of PU by performing infra-frame prediction to PU.Predictive data for PU can comprise predictability sample block for PU and various syntactic element.PU during intra-prediction process unit 126 can be cut into slices to I, P cuts into slices and B cuts into slices performs infra-frame prediction.
In order to perform infra-frame prediction to PU, intra-prediction process unit 126 can use multiple intra prediction mode to produce multiple set of the predictive data of PU.In order to use intra prediction mode to produce one group of predictive data for PU, sample can be extended across the sample block of PU by intra-prediction process unit 126 on the direction be associated with intra prediction mode from the sample block of adjacent PU.Assuming that adopt coding orders from left to right, from top to bottom for PU, CU and CTU, adjacent PU can above described PU, upper right side, upper left side or left.Intra-prediction process unit 126 can use the intra prediction mode of various number, such as, and 33 directional intra-prediction patterns.In some instances, the number of intra prediction mode can depend on the size in the region be associated with PU.
Prediction processing unit 100 can select the predictive data of the PU for CU in the middle of the predictive data produced by inter prediction processing unit 120 for PU or the predictive data produced by intra-prediction process unit 126 for PU.In some instances, prediction processing unit 100 measures based on the rate/distortion of the set of predictive data the predictive data selected for the PU of CU.The predictability sample block of selected predictive data can be called as selected predictability sample block in this article.
Residue generation unit 102 can based on lightness, Cb and the Cr residual block of the selected predictability lightness of the PU of the lightness of CU, Cb and Cr decode block and CU, Cb and Cr block generation CU.Such as, the residual block that residue generation unit 102 can produce CU equals the value of the sample in the decode block of CU and the difference between the corresponding corresponding sample selected in predictability sample block of the PU of CU to make each sample in residual block have.
Conversion process unit 104 can perform Quadtree Partition the residual block be associated with CU to be divided into the transform block be associated with the TU of CU.Therefore, TU can be associated with lightness transform block and two chromaticity transformation blocks.The lightness transform block of the TU of CU and the size of chromaticity transformation block and position can or can not based on the size of the prediction block of the PU of CU and positions.The quad-tree structure being called as " remaining quaternary tree " (RQT) can comprise the node be associated with each in region.The TU of CU can correspond to the leaf node of RQT.
Conversion process unit 104 can produce the transformation coefficient block of each TU being used for CU by one or more conversion is applied to the transform block of TU.Various conversion can be applied to the transform block be associated with TU by conversion process unit 104.Such as, discrete cosine transform (DCT), directional transforms or conceptive similar conversion can be applied to transform block by conversion process unit 104.In some instances, conversion is not applied to transform block by conversion process unit 104.In this type of example, transform block can be treated to transformation coefficient block.
Quantifying unit 106 can conversion coefficient in quantization parameter block.Quantizing process can reduce the bit depth be associated with some or all in conversion coefficient.Such as, during quantizing, n bit map coefficient can be rounded down to m bit map coefficient, wherein n is greater than m.Quantifying unit 106 can quantize the coefficient block be associated with the TU of CU based on the quantization parameter be associated with CU (QP) value.Video encoder 20 can adjust the degree of the quantification being applied to the coefficient block be associated with CU by adjusting the QP value be associated with CU.Quantification may make information dropout, and the conversion coefficient therefore through quantizing can have the precision lower than original transform coefficient.
Re-quantization and inverse transformation can be applied to coefficient block by inverse quantization unit 108 and inversion process unit 110 respectively, with from coefficient block reconstructed residual block.Reconfiguration unit 112 can add the corresponding sample of one or more predictability sample block carrying out free prediction processing unit 100 and produce to through reconstructed residual block, with produce be associated with TU through restructuring transformation block.By reconstructing the transform block of each TU of CU in this way, the decode block of video encoder 20 restructural CU.
Filter cell 114 can perform one or more deblocking operation to reduce the blocked false image in the decode block that is associated with CU.The decode block through reconstruct can be stored after filter unit 114 performs one or more deblocking operation to the decode block through reconstructing through decode picture buffer 116.Inter prediction processing unit 120 can use and come to perform inter prediction to the PU of other picture containing the reference picture through reconstructing decode block.In addition, intra-prediction process unit 126 can use the decode block through reconstructing in decode picture buffer 116 to perform infra-frame prediction to other PU be in the picture identical with CU.
Entropy code unit 118 can receive data from other functional unit of video encoder 20.Such as, entropy code unit 118 can receive coefficient block from quantifying unit 106, and can receive syntactic element from prediction processing unit 100.Entropy code unit 118 can perform the operation of one or more entropy code to produce the data through entropy code to data.Such as, entropy code unit 118 can perform context-adaptive variable-length decoding (CAVLC) operation to data, CABAC operates, can change to the decoded operation of variable (V2V) length, operate based on the context adaptive binary arithmetically decoding (SBAC) of grammer, probability interval splits entropy (PIPE) decoded operation, exp-Golomb coding operates or the entropy code operation of another type.Video encoder 20 can output packet containing the bit stream of the data through entropy code produced by entropy code unit 118.Such as, bit stream can comprise the data represented for the RQT of CU.
In some instances, remaining decoding does not perform together with palette decoding.Therefore, when using palette decoding mode decoding, video encoder 20 can not perform conversion or quantize.In addition, video encoder 20 can carry out entropy code to the data using palette decoding mode to produce from residual data individually.
Fig. 3 illustrates the block diagram being configured to implement the instance video decoder 30 of technology of the present invention.Fig. 3 provides for illustrative purposes, and also not as the present invention extensively illustrates and describes to limit described technology.For illustrative purposes, the Video Decoder 30 when the present invention is described in HEVC decoding.But technology of the present invention is applicable to other coding standards or method.
Video Decoder 30 represents can be configured to the example of execution according to the device of the technology for the video coding based on palette of the various examples described in the present invention.For example, Video Decoder 30 can be configured to use and decode to various block of video data such as CU or PU in such as HEVC decoding based on the decoding of palette or the non-decoding selectivity based on palette.The non-decoding mode based on palette can refer to various inter prediction time decoding mode or infra-frame prediction space decoding mode, the various decoding modes of such as being specified by HEVC draft 10.In an example, Video Decoder 30 can be configured to produce the palette of the entry with instruction pixel value, receive the information that at least some position of block of video data is associated with the entry in palette, based on the pixel value in described Information Selection palette, and based on the pixel value of selected pixel value reconstructed blocks.
In the example of fig. 3, Video Decoder 30 comprises entropy decoding unit 150, prediction processing unit 152, inverse quantization unit 154, inversion process unit 156, reconfiguration unit 158, filter cell 160 and through decode picture buffer 162.Prediction processing unit 152 comprises motion compensation units 164 and intra-prediction process unit 166.Video Decoder 30 also comprises the decoding unit 165 based on palette, and it is configured to the various aspects performing the decoding technique based on palette described in the present invention.In other example, Video Decoder 30 can comprise more, less or different functional units.
Map sheet buffer (CPB) through decoding can receive and store the encoded video data (such as, NAL unit) of bit stream.Entropy decoding unit 150 can receive encoded video data (such as, NAL unit) from CPB and dissect NAL unit with syntactic element of decoding.Entropy decoding unit 150 can carry out entropy decoding to the syntactic element through entropy code in NAL unit.Prediction processing unit 152, inverse quantization unit 154, inversion process unit 156, reconfiguration unit 158 and filter cell 160 can based on the syntactic element extracted from bit stream produce through decoding video data.
The NAL unit of bit stream can comprise the section NAL unit through decoding.Carry out a part of encoding as contraposition stream, entropy decoding unit 150 can extract syntactic element from the section NAL unit through decoding and carries out entropy decoding to institute's syntax elements.Slice header and slice of data can be comprised through each write in code section.Slice header can containing the syntactic element about section.Syntactic element in slice header can comprise the syntactic element identifying the PPS be associated with the picture containing described section.
Except decoding to the syntactic element from bit stream, Video Decoder 30 can perform reconstructed operation to not splitting CU.For performing reconstructed operation to without segmentation CU, Video Decoder 30 can perform reconstructed operation to each TU of CU.By performing reconstructed operation to each TU of CU, Video Decoder 30 can reconstruct the residual block of CU.
As the part TU of CU being performed to reconstructed operation, inverse quantization unit 154 can the coefficient block that is associated with TU of re-quantization (that is, de-quantization).Inverse quantization unit 154 can use the QP value be associated with the CU of TU to determine the degree quantized, and same inverse quantization unit 154 is by the degree of the re-quantization of application.That is, the value of the QP that can be used when quantization transform coefficient by adjustment controls compression ratio, namely in order to represent the ratio of the number of the position of original series and compressed sequence.Compression ratio can also depend on the method for adopted entropy decoding.
After inverse quantization unit 154 re-quantization coefficient block, one or more inverse transformation can be applied to coefficient block to produce the residual block be associated with TU by inversion process unit 156.For example, inverse DCT, inverse integer transform, inverse card can be neglected Nan-La Wei conversion (KLT), reverse rotation conversion, inverse directional transforms or another inverse transformation and be applied to coefficient block by inversion process unit 156.
If use infra-frame prediction to encode to PU, then intra-prediction process unit 166 can perform infra-frame prediction to produce the predictability block for PU.Intra-prediction process unit 166 can use intra prediction mode to produce the predictability lightness block of PU, Cb block and Cr block with the prediction block based on spatially adjacent PU.Intra-prediction process unit 166 can determine the intra prediction mode of PU based on one or more syntactic element from bitstream decoding.
Prediction processing unit 152 can construct the first reference picture list (RefPicList0) and the second reference picture list (RefPicList1) based on the syntactic element extracted from bit stream.In addition, if use inter prediction to encode to PU, then entropy decoding unit 150 can extract the movable information of PU.Motion compensation units 164 can determine one or more reference area of PU based on the movable information of PU.Motion compensation units 164 can based on the predictability lightness block of the sample block generation PU at one or more reference block place at PU, Cb block and Cr block.
Reconfiguration unit 158 can use lightness transform block, the predictability lightness block of PU of Cb transform block and Cr transform block and CU, Cb block and the Cr block (that is, intra-prediction data or inter prediction data) be associated with the TU of CU to reconstruct the lightness decode block of CU, Cb decode block and Cr decode block in due course.Such as, the reconfiguration unit 158 corresponding sample that the sample of lightness transform block, Cb transform block and Cr transform block can be added to predictability lightness block, Cb block and Cr block is to reconstruct the lightness decode block of CU, Cb decode block and Cr decode block.
Filter cell 160 can perform deblocking operation to reduce the blocked false image be associated with the lightness decode block of CU, Cb decode block and Cr decode block.Video Decoder 30 can store the lightness decode block of CU, Cb decode block and Cr decode block in the picture buffer 162 through decoding.Can provide that reference picture compensates for subsequent motion through decode picture buffer 162, infra-frame prediction and presenting in the display unit such as display unit 32 of such as Fig. 1.For example, Video Decoder 30 can perform infra-frame prediction or inter prediction operation based on the lightness in the picture buffer 162 of decoding, Cb and Cr block to the PU of other CU.In this way, Video Decoder 30 can extract the conversion coefficient level of important brightness coefficient block from bit stream, re-quantization conversion coefficient level, to conversion coefficient level application conversion to produce transform block, produces decode block based on transform block at least partly and exports decode block for display.
According to various example of the present invention, Video Decoder 30 can be configured to perform the decoding based on palette.When the decoding schema based on palette is through selecting (such as) for CU or PU, the decoding unit 165 (such as) based on palette can perform the decoding based on palette.For example, decoding unit 165 based on palette can be configured to produce the palette of the entry with instruction pixel value, receive the information that at least some position of block of video data is associated with the entry in palette, based on the pixel value in described Information Selection palette, and based on the pixel value of selected pixel value reconstructed blocks.Although be performed various functional description by the decoding unit 165 based on palette, some or all in this little function can be performed by the combination of other processing unit or different processing units.
Decoding unit 165 based on palette can receive palette decoding mode information, and performs aforesaid operations when palette decoding mode information instruction palette decoding mode is applied to block.When palette decoding mode information instruction palette decoding mode is not applied to block, or when the use of other pattern information instruction different mode, decoding unit 165 based on palette uses the non-decoding mode based on palette to decode to block of video data when palette decoding mode information instruction palette decoding mode is not applied to block, such as this HEVC inter prediction or infra-frame prediction decoding mode.Block of video data can (such as) be CU or PU produced according to HEVC decode procedure.Video Decoder 30 inter prediction time prediction or infra-frame prediction space decoding mode can be decoded to some blocks and decodes to other block with the decoding mode based on palette.Decoding mode based on palette can comprise multiple difference based on the one in the decoding mode of palette, maybe can there is the single decoding mode based on palette.
The palette decoding mode information received by the decoding unit 165 based on palette can comprise pallet mode syntactic element, such as flag.First value instruction palette decoding mode of pallet mode syntactic element is applied to block, and the second value instruction palette decoding mode of pallet mode syntactic element is not applied to block of video data.Decoding unit 165 based on palette can one or many person in predictability unit level, decoding unit level, section level or picture level be located to receive (such as, from video encoder 20) palette decoding mode information, maybe can receive the instruction through enabling at least one of palette decoding mode whether in image parameters collection (PPS), sequence parameter set (SPS) or video parameter collection (VPS).
In some instances, the decoding unit 165 based on palette can infer palette decoding mode information based on one or many person in the view id in the layer id in the size of decode block, frame type, color space, color component, frame sign, frame rate, scalable video coding or the multi views decoding be associated with block of video data.
Decoding unit 165 based on palette also can be configured to receive the information defining in palette at least some had in the entry of video data, and produces palette based on received information at least in part.The size of palette can be fixing or variable.In some cases, the size of palette is variable and can adjusts based on the information reinstating signal indication with video data one.The entry in palette can be specified to be whether last entry in palette by the information of signal indication.Further, in some cases, palette can have largest amount.Also can launch or infer the size of palette conditionally.Described condition can be the layer id in the size of CU, frame type, color space, color component, frame sign, frame rate, scalable video coding or the view id in multi views decoding.
Palette can be the single palette of the entry comprising the lightness component of indicator collet and the pixel value of chromatic component.In the case, each entry in palette refers to triple entries of the pixel value showing lightness component and two chromatic components.Or palette comprises the lightness palette of the entry of the pixel value of the lightness component comprising indicator collet, and comprise the colourity palette of instruction for the entry of the pixel value of the corresponding chromatic component of block.
In some instances, based on the decoding unit 165 of palette by producing palette based on the entry in previous treated data prediction palette.Previous treated data can comprise previously through the decoding palette of adjacent block or the information from palette.Decoding unit 165 based on palette can receive the prediction syntactic element indicated whether the entry in prediction palette.Prediction syntactic element can comprise the multiple prediction syntactic elements indicated whether being used for the entry of lightness and chromatic component in prediction palette respectively.
Relative to (such as) predictability palette, predictability palette can containing from the palette entry of one or more adjacent block, and described adjacent block comprises the adjacent block in the spatial neighboring blocks of block and/or specific scanning sequence.In one example, adjacent block spatially can be positioned at the left side (left side adjacent block) or top (the top adjacent block) of the block of current positive decoding.In another example, the most frequent sample value determination predictability palette entry in the cause and effect neighbor of current block can be used based on the decoding unit 165 of palette.In another example, adjacent block can be adjacent according to the block of specific scanning sequence and current positive decoding in order to carry out decoding to block.That is, adjacent block can be one or more block of decoding before current block in scanning sequence.One or more syntactic element of decoding unit 165 decodable code based on palette copies the position of the adjacent block that palette is used with instruction.
Therefore, in one example, based on palette decoding unit 165 can in some instances based on section or picture in left side adjacent block or top adjacent block palette in entry and predict at least some in the entry in palette.In the case, the entry based on the entry prediction in the palette of left side adjacent block or top adjacent block in palette can be predicted for the syntactic element of the selection of the left side adjacent block predicted or top adjacent block based on instruction by the decoding unit 165 based on palette.Institute's syntax elements can be the flag of the value of the selection with left side adjacent block or the top adjacent block indicated for predicting.
In some instances, can receive based on the decoding unit 165 of palette one or more prediction syntactic element whether at least some selected entry in instruction palette will predict based on order one by one, and correspondingly produce entry.For example, the decoding unit 165 based on palette can decode to indicate palette entry whether to be included in the palette of current block for each entry of predictability palette to one or more syntactic element.If entry is not predicted, the decoding unit 165 so based on palette can decode the number of specifying non-predicted entry and this little entry to one or more extra syntactic element.Therefore, based on palette the more measurable entries of decoding unit 165 and receive the information of directly specifying and comprising other entry of the additional entries of described number in palette.
In some instances, for predict whole palette technology can with the technical combinations of one or more entry for predicting palette.For example, whether the decoding unit 165 based on palette can decode one or more syntactic element in bit stream to indicate the current color palette to be copy from predictability palette completely.If situation is not like this, so can decodes to one or more syntactic element in bit stream based on the decoding unit 165 of palette and whether to be replicated with each entry in indication predicting palette.
In another example, whether the palette value that can receive after each palette value in order to instruction signal indication of the decoding unit 165 based on palette is the flag of the final palette entry of palette, instead of receives number and the palette value of entry.If palette has reached a certain largest amount, so this " palette end " flag can not be received based on palette decoding unit 165.
The information that at least some position of block of video data is associated with the entry in palette received by the decoding unit 165 based on palette can comprise map information, it comprises the palette index value of at least some in the position in block, and each in wherein said palette index value corresponds to the one in the entry in palette.Map information can comprise one or more and prolong row syntactic element, and it has the number of the continuous position of identical palette index value separately in indicator collet.
In some instances, the decoding unit 165 based on palette can receive the information that index line copies, and is copied by described line to copy from the pixel of another line of the position block or index value for the pixel of the position line in block or index value.Decoding unit 165 based on palette can use this information and executing line to copy to determine pixel value in the palette of the various positions of block or entry.The line of position can comprise a part for the row of the position of block, a part of going, row or row.
Based on palette decoding unit 165 can partially by receive block one or more position pixel value and described pixel value is added to entry in palette dynamically to produce palette to be in operation produce palette at least partially.Add pixel value can comprise and add pixel value to the initial palette comprising original entries set or the palette adding the sky not comprising original entries set to.In some instances, add to comprise and add pixel value new entry to be added to the initial palette comprising original entries set or the existing entry of filling in initial palette, or change or change the pixel value of the entry in initial palette.
In some instances, the fixing largest amount of palette is determined based on the decoding unit 165 of palette.After arrival largest amount, the decoding unit 165 based on palette can remove one or more entry of palette at once.In one embodiment, the decoding unit 165 based on palette can such as use fifo queue to remove the oldest entry of palette.In another example, based on the entry of the removable minimum use of decoding unit 165 of palette.In an example again, when the decoding unit 165 based on palette once can add the relative usage of palette and described entry to based on candidate entries to be removed and make about removing determining through weighting of which entry.
In some instances, palette can be the pixel value be wherein selected from for the palette of the one in the position in block be different from the actual pixel value of the position in described piece through quantizing palette, decode procedure is damaged.For example, same pixel value is optional certainly for having the palette of two diverse locations of different actual pixel value.
Fig. 4 illustrates the concept map that determine the example that video data carried out to the palette of decoding consistent with technology of the present invention.The example of Fig. 4 comprises picture 178, and it has the first decoding unit (CU) 180 be associated with the first palette 184 and the 2nd CU188 be associated with the second palette 192.As hereafter more detailed description and according to technology of the present invention, the second palette 192 is based on the first palette 184.Picture 178 also comprises with the block 196 of infra-frame prediction decoding mode decoding with the block 200 of inter prediction decoding mode decoding.
For illustrative purposes, in the context of video encoder 20 (Fig. 1 and Fig. 2) and Video Decoder 30 (Fig. 1 and Fig. 3) and relative to the technology of HEVC video coding standard to describe Fig. 4.However, it should be understood that technology of the present invention is not limited thereto mode, and can be applied in other video coding process and/or standard by other video coding processor and/or device.
In general, the CU (CU188) that palette refers to for current positive decoding is in the example in figure 4 main and/or representational some pixel values.First palette 184 and the second palette 192 are shown as and comprise multiple palette.In some instances, according to aspects of the present invention, video decoder (such as video encoder 20 or Video Decoder 30) can carry out decoding to palette individually for each color component of CU.For example, video encoder 20 can be encoded to the another palette of colourity (V) component of another palette of colourity (U) component of the palette of the lightness of CU (Y) component, CU and CU.In this example, the entry of Y palette can represent the Y value of the pixel of CU, and the entry of U palette can represent the U value of the pixel of CU, and the entry of V palette can represent the V value of the pixel of CU.
In other example, video encoder 20 can for the single palette of all colours component coding of CU.In this example, video encoder 20 can be encoded to the palette of the i-th entry of three weight values had for comprising Yi, Ui and Vi.In the case, palette comprises the value of each in the component of pixel.Therefore, the expression of palette 184 and 192 of set as the palette with multiple indivedual palette is only an example and does not wish for restrictive.
In the example in figure 4, the first palette 184 comprises three entry 202-206, and it has entry index value 1, entry index value 2 and entry index value 3 respectively.Entry 202-206 makes index value be related to comprise respectively the pixel value of pixel value A, pixel value B and pixel value C.As described in this article, not carry out decoding to the actual pixel value of a CU180, video decoder (such as video encoder 20 or Video Decoder 30) can use the decoding based on palette to carry out decoding to make index of reference 1-3 to the pixel of block.That is, for each location of pixels of a CU180, the index value of video encoder 20 codified pixel, wherein said index value is associated with the pixel value in one or many person in the first palette 184.Video Decoder 30 can obtain index value from bit stream and use one or many persons reconstructed pixel value described index value and the first palette 184.Therefore, the first palette 184 is launched by video encoder 20 and is used in based on the decoding of palette for Video Decoder 30 in encoded video data bitstream.In general, one or more palette can be launched for each CU, or one or more palette can be shared between different CU.
According to aspects of the present invention, video encoder 20 and Video Decoder 30 can determine the second palette 192 based on the first palette 184.For example, video encoder 20 can for each CU (as an example, comprise the 2nd CU188) whether the pred_palette_flag that encodes be predict from one or more palette be associated with one or more other CU to indicate the palette of described CU, the such as the most frequent sample of adjacent C U (spatially or based on scanning sequence) or cause and effect neighbor.For example, when this flag target value equals for the moment, Video Decoder 30 can determine that second palette 192 of the 2nd CU188 is from one or more palette prediction of having decoded, and the new palette that so there is no the 2nd CU188 is included in the bit stream containing described pred_palette_flag.When this flag equals zero, Video Decoder 30 can determine that the palette 192 of the 2nd CU188 is included in bit stream as new palette.In some instances, pred_palette_flag can for each different colours component decoding individually of CU (such as, for three flags of the CU in yuv video, one for Y, one for U, and one for V).In other example, can for the single pred_palette_flag of all colours component decoding of CU.
In the above example, whether every CU signal indication pred_palette_flag is to indicate any one in the entry of the palette of current block through prediction.In some instances, can based on every one or more syntactic element of entry signal indication.Whether it is for being present in flag in the current color palette for each entry signal indication of palette predictor to indicate described entry.As mentioned above, if palette entry is not predicted, so can clearly with palette entry described in signal indication.
When determining the second palette 192 (such as relative to the first palette 184, pred_palette_flag equals one) time, one or more block that video encoder 20 and/or Video Decoder 30 location prediction palette (being the first palette 184 in this example) are determined from it.Predictability palette can such as, be associated with one or more adjacent C U of the CU of current positive decoding (that is, the 2nd CU188) (such as, the most frequent sample of adjacent C U (spatially or based on scanning sequence) or cause and effect neighbor).The palette of one or more adjacent C U can be associated with predictor palette.In some examples such as such as example illustrated in fig. 4, when video encoder 20 and/or Video Decoder 30 can work as the predictability palette determining the 2nd CU188, locate left side adjacent C U, a CU180.In other example, video encoder 20 and/or Video Decoder 30 can locate one or more CU, such as top CU, CU196 in other position relative to the 2nd CU188.
Video encoder 20 and/or Video Decoder 30 can determine the CU of palette prediction based on stratum.For example, video encoder 20 and/or Video Decoder 30 initially can identify the left side adjacent C U for palette prediction, a CU180.If left side adjacent C U is not useable for prediction (such as, left side adjacent C U is with except based on the pattern decoding except the decoding mode of palette, such as intra prediction mode or intra prediction mode, or be positioned the Far Left edge of picture or section), so video encoder 20 and/or Video Decoder 30 identifiable design top adjacent C U, CU196.Video encoder 20 and/or Video Decoder 30 can continue the available CU of search according to the predesigned order of position until navigate to the CU with the palette that can be used for palette prediction.In some instances, video encoder 20 and/or Video Decoder 30 can based on multiple pieces and/or adjacent block through reconstructed sample determination predictability palette.
Although the example of Fig. 4 illustrates that the first palette 184 is as the predictability palette from single CU (CU180), but in other example, video encoder 20 and/or Video Decoder 30 can from the palette of the integrated positioning of adjacent C U for predicting.For example, video encoder 20 and/or Video Decoder can be applied one or more formula, function, rule or analog and produce palette with the palette based on the one in multiple adjacent C U or combination.
In other example again, video encoder 20 and/or Video Decoder 30 can construct the candidate list of the some potential candidates comprised for palette prediction.In this little example, the index that video encoder 20 can be encoded onto candidate list is selected for the candidate CU in the list of the current C U (such as, copying palette) of palette prediction from it with instruction.Video Decoder 30 can construct candidate list in the same manner, decoding index, and uses the palette being used for the corresponding CU used together with current C U through decoding index selection.
For illustration of in the example of object, video encoder 20 and Video Decoder 30 can construct the candidate list of a CU on the CU comprising and be positioned above current positive decoding CU and the left side being positioned current positive decoding CU.In this example, one or more syntactic element of video encoder 20 codified is selected to indicate candidate.For example, video encoder 20 codified has the flag of value zero is copy from the CU on the left side being positioned current C U to indicate the palette of current C U.The flag that video encoder 20 codified has value one is copy from the CU be positioned above current C U to indicate the palette of current C U.Video Decoder 30 decode described flag and select for palette prediction suitable CU.
In other example again, the frequency that video encoder 20 and/or Video Decoder 30 occur based on the sample value be included in one or more other palette in one or more adjacent C U and determine the palette of current positive decoding CU.For example, video encoder 20 and/or Video Decoder 30 can follow the tracks of the color be associated with the index value the most frequently used during the decoding of the CU of predetermined number.The color the most frequently used can be included in the palette of current positive decoding CU by video encoder 20 and/or Video Decoder 30.
As mentioned above, in some instances, video encoder 20 and/or Video Decoder can copy whole palette from the adjacent C U for carrying out decoding to current C U.Additionally or alternati, video encoder 20 and/or Video Decoder 30 can perform based on object palette prediction one by one.For example, video encoder 20 can be encoded one or more syntactic element for each entry of palette, and whether its instruction respective entries is predict based on predictability palette (such as, the palette of another CU).In this example, video encoder 20 can be worked as to encode for given entry when entry is predicted value (the corresponding entry of the palette such as, be associated with adjacent C U) from predictability palette and had the flag of the value equaling.The flag that video encoder 20 can have a null value for particular items coding is not predicted from the palette of another CU to indicate described particular items.In this example, to instruction, video encoder 20 also can not predict that the excessive data of the value of palette entry is encoded.
In the example in figure 4, the second palette 192 comprises four entry 208-214, and it has entry index value 1, entry index value 2, entry index value 3 and entry index 4 respectively.Entry 208-214 makes index value be related to comprise respectively the pixel value of pixel value A, pixel value B, pixel value C and pixel value D.According to aspects of the present invention, video encoder 20 and/or Video Decoder 30 can use any one location the one CU180 in technology described above to predict for palette, and the entry 1-3 entry 1-3 of the first palette 184 being copied to the second palette 192 carries out decoding for the 2nd CU188.In this way, video encoder 20 and/or Video Decoder 30 can determine the second palette 192 based on the first palette 184.In addition, video encoder 20 and/or Video Decoder 30 can carry out decoding to by the data of the entry 4 comprised together with the second palette 192.This information can comprise not from the number of the palette entry of predictor palette prediction and the pixel value corresponding to those palette entry.
In some instances, according to aspects of the present invention, one or more syntactic element can indicate the palette of such as the second palette 192 be whether completely from the particular items of predictability palette (be shown as the first palette 184 Fig. 4, but it can being made up of the entry from one or more block) prediction or the second palette 192 whether through prediction.For example, whether initial grammar element can indicate all entries through prediction.If also not all entry is through prediction (such as, having the flag of value 0) for the instruction of initial grammar element, so one or more extra syntactic element can indicate which entry of the second palette 192 to be predict from predictability palette.
According to certain aspects of the invention, predict that with palette some information be associated can be inferred from one or more characteristic of positive decoding data.That is, be not video encoder 20 syntax elements encoded (and Video Decoder 30 is decoded this little syntactic element), video encoder 20 and Video Decoder 30 can perform palette prediction based on one or more characteristic of positive decoding data.
In one example, for purposes of illustration, the value of above-mentioned pred_palette_flag can be inferred from one or many person following each (as an example): the layer id in the size of positive decoding CU, frame type, color space, color component, frame sign, frame rate, scalable video coding, or the view id in multi views decoding.That is, relative to the size of the CU as an example, video encoder 20 and/or Video Decoder 30 can determine any CU for exceeding pre-sizing, and above-mentioned pred_palette_flag equals one.In this example, do not need to use signal indication pred_palette_flag in encoded bit stream.
Although describe relative to pred_palette_flag above, but video encoder 20 and/or Video Decoder 30 or alternatively can also predict with palette the out of Memory be associated based on one or more characteristic deduction of positive decoding data, such as palette is used for the candidate CU from its prediction, or for constructing the rule of palette predicting candidate person.
According to other aspects of the invention, video encoder 20 and/or Video Decoder 30 can be in operation structure palette.For example, when initially carrying out decoding to the 2nd CU188, in palette 192, there is not entry.When at video encoder 20 and Video Decoder 30, the new value to the pixel of the 2nd CU188 carries out decoding, each new value is included in palette 192.That is, for example, video encoder 20 in CU188 position produce and add pixel value to palette 192 with during signal indication pixel value.The relatively pixel in evening when encoding in video encoder 20 couples of CU, video encoder 20 can use index value to encode to those pixels being worth identical value had be included in palette instead of with pixel value described in signal indication.Similarly, when Video Decoder 30 receives new pixel value (such as, by video encoder 20 signal indication) for the position in the 2nd CU188, Video Decoder 30 comprises described pixel value in palette 192.When the location of pixels of relatively decoding in evening in the 2nd CU188 has the pixel value adding the second palette 192 to, Video Decoder 30 can receive in identification second palette 192 such as, for reconstructing the information of the respective pixel value of the pixel value of the 2nd CU188, index value.
In some instances, as described in greater detail below, palette 184 and 192 can be maintained maximum palette size or lower than maximum palette size by video encoder 20 and/or Video Decoder 30.According to aspects of the present invention, if reach maximum palette size, such as, when the second palette 192 is in operation dynamic construction, so video encoder 20 and/or Video Decoder 30 perform same process to remove the entry of the second palette 192.Be first in first out (FIFO) technology for removing an example procedure of palette entry, wherein video encoder 20 and Video Decoder 30 remove the oldest entry of palette.In another example, video encoder 20 and Video Decoder 30 can remove the least frequent palette entry used from palette.In an example again, video encoder 20 and Video Decoder 30 can be weighted to determine to remove which entry to FIFO and frequency of utilization process.That is, removing of entry can have frequent degree that is how old and entry use based on entry.
According to some aspects, if entry (pixel value) to remove and the more late position of pixel value among positive decoding CU occurs again from palette, so video encoder 20 can be encoded to described pixel value instead of in palette, comprise entry and encode to index.Additionally or alternati, palette entry after removing, such as, when video encoder 20 and Video Decoder 30 scan the position in CU, can be input in palette by video encoder 20 again.
In some instances, can with one or more other technical combinations for determining palette for being in operation the technology deriving palette.Exactly, as an example, video encoder 20 and Video Decoder 30 upgrade the second palette 192 when can initially carry out decoding (such as, using palette prediction to predict the second palette 192 from the first palette 184) to the second palette 192 and can work as and carry out decoding to the pixel of the 2nd CU188.For example, after the initial palette of transmitting, value can be added to initial palette or be changed the value in initial palette when scanning the pixel value of the additional positions in CU by video encoder 20 at once.Similarly, after the initial palette of reception, value is added to initial palette or is changed the value in initial palette when scanning the pixel value of the additional positions in CU by Video Decoder 30 at once.
The transmitting whether video encoder 20 can use whole palette with signal indication current C U in some instances or the palette be in operation produce, or the transmitting of initial palette and the combination of the renewal of the initial palette of the derivation by being in operation.In some instances, initial palette can be the full palette being in maximum palette size, can change the value in initial palette in the case.In other example, initial palette can be less than maximum palette size, and value can be added to initial palette and/or be changed the value of initial palette by video encoder 20 and Video Decoder 30 in the case.
According to aspects of the present invention, such as the size (such as, in the number being included in the pixel value in palette) of the palette such as the first palette 184 and the second palette 192 can be fixing one or more syntactic element signal indication that maybe can use in encoded bit stream.For example, according to some aspects, video encoder 20 and Video Decoder 30 can use unitary code or block unitary code (such as, at the code that maximum restriction place of palette size is blocked) and carry out decoding to palette size.According to other side, video encoder 20 and Video Decoder 30 can use index-Columbus or Lai Si-Columbus's code to carry out decoding to palette size.
According to other side again, video encoder 20 and Video Decoder 30 can carry out decoding to the data of the size of instruction palette after each entry of palette.Relative to the second palette 192 as an example, video encoder 20 can stop flag by coding after each in entry 208-214.In this example, the stopping flag equaling can specify the entry of current positive decoding to be the final entry of the second palette 192, and null stopping flag can indicating in the second palette 192 and there is additional entries.Therefore, video encoder 20 can have the stopping flag of value zero and encode entry 214 after and have the stopping flag of value one by coding after each in entry 208-212.In some instances, after constructed palette arrives the restriction of maximum palette size, at once stopping flag can not be comprised in bit stream.Although above example discloses the technology being used for the size using signal indication palette clearly, but in other example, the size of palette also can based on so-called supplementary (such as, characteristic information, such as, layer id in the size of positive decoding CU, frame type, color space, color component, frame sign, frame rate, scalable video coding or the view id in multi views decoding, launches conditionally as mentioned above) or infers.
Technology of the present invention comprises nondestructively or carries out decoding with some loss (damaging decoding) to data.For example, relative to damaging decoding, video encoder 20 can carry out decoding to the pixel of CU and without the need to definitely by the actual pixel value of pixel value exact match in CU of palette.When technology of the present invention be applied to damage decoding time, some restrictions can be applicable to palette.For example, video encoder 20 and Video Decoder 30 can quantize palette, such as the first palette 184 and the second palette 192.That is, video encoder 20 and Video Decoder 30 can work as entry pixel value in preset range each other time merge the entry of (quantification) palette.In other words, if there is the palette value in the margin for error of new palette value, so new palette value has not been added to palette.In another example, the multiple different pixel values in block can be mapped to single palette entry, or are mapped to single palette pixel value equivalently.
Video Decoder 30 can decoded pixel value in the same manner, and no matter specific palette is harmless or damages.As an example, Video Decoder 30 can for using the index value launched by video encoder 20 to select the entry for described location of pixels in palette to pixel location in decode block, and to have nothing to do in described palette be harmless or damage.In this example, the pixel value of palette entry is used as the pixel value in decode block, and no matter whether it mates original pixel value definitely.
In the example damaging decoding, for purposes of illustration, video encoder 20 can determine bouds on error, is called residual quantity value.Candidate pixel values entry Plt_cand may correspond in waiting for the pixel value of the position in decode block at such as CU or PU.Between the tectonic epochs of palette, video encoder 20 determines the absolute difference between all existing pixel value entry in candidate pixel values entry Plt_cand and palette.If all absolute differences between the existing pixel value entry in candidate pixel values entry Plt_cand and palette are greater than residual quantity value, so video encoder 20 can add pixel value candidate to palette as entry.If the absolute difference between at least one the existing pixel value entry in pixel value entry Plt_cand and palette is equal to or less than residual quantity value, so video encoder 20 can not add candidate pixel values entry Plt_cand to palette.Therefore, when carrying out decoding to pixel value entry Plt_cand, video encoder 20 can select to have the entry as the pixel value closest to pixel value entry Plt_cand, and then some losses is incorporated in system.When palette is made up of multiple component (such as three color components), the absolute difference summation of individual component value may be used for contrast residual quantity value and compares.Or or in addition, the absolute difference of each component value can contrast the second residual quantity value and compare.
In some instances, the existing pixel value entry in palette mentioned above may use similar residual quantity comparison procedure and add.In other example, the existing pixel value in palette may use other process and add.For example, one or more original pixel values entry can be added to palette (without the need to quotient of difference comparatively) to start to construct the residual quantity comparison procedure of palette.Said process can be implemented to produce lightness and/or colourity palette by video encoder 20 and/or Video Decoder 30.
Also can be used during pixel decoding by video encoder 20 and Video Decoder 30 relative to the technology of palette structure description above.For example, when the coding of pixel value, the pixel value of the entry in the value of pixel and palette can compare by video encoder 20.If the absolute pixel value difference between the one in the entry in the value of described pixel and palette is equal to or less than residual quantity value, so described pixel value can be encoded to the entry of palette by video encoder 20.That is, in this example, video encoder 20 uses the one in the entry of palette to encode described pixel value when pixel value produces fully little (such as, in the preset range) absolute difference to palette entry.
In some instances, video encoder 20 can select the palette entry producing minimum absolute pixel value difference (compared with the pixel value of positive decoding) to carry out encoded pixel values.As an example, video encoder 20 codified index will be used for the palette entry of pixel value with instruction, such as, by the palette pixel value entry in order to reconstruct at Video Decoder 30 place through decoding pixel value.If the absolute pixel value difference between all entries in the value of pixel and palette is greater than residual quantity, so encoder can not use the one in palette entry to carry out encoded pixel values, and can change into the pixel value of pixel to be decoded is transmitted into Video Decoder 30 (and may add described pixel value to palette as entry).
In another example, video encoder 20 can select the entry of palette to encode for pixel value.Video encoder 20 can use selected entry as predictability pixel value.That is, video encoder 20 can be determined the residual value of the difference represented between actual pixel value and selected entry and encode to described remnants.Video encoder 20 can produce residual value in block by the pixel of the entry prediction of palette, and can produce the residual block of the corresponding residual pixels value comprising block of pixels.Video encoder 20 can subsequently by transform and quantization (as above relative to Fig. 2 mention) be applied to as described in residual block.In this way, video encoder 20 can produce through quantized residual conversion coefficient.
Video Decoder 30 inverible transform and re-quantization conversion coefficient are to regenerate residual block.Video Decoder 30 can the residual value reconstructed pixel value of usage forecastings palette entry value and pixel value subsequently.For example, residual value and palette entry value can be combined to reconstruct through decoding pixel value by Video Decoder 30.
In some instances, residual quantity value can be different for different CU size, picture size, color space or different colours component.Residual quantity value can be predetermined or determines based on various decoding condition.For example, video encoder 20 can use high-level syntax by residual quantity value signal indication to Video Decoder 30, the grammer in described high-level syntax PPS, SPS, VPS and/or slice header.In other example, video encoder 20 and Video Decoder 30 can through pre-configured to use identical fixed difference value.In other example again, video encoder 20 and/or Video Decoder 30 can (such as, such as CU size, color space, color component or analog, derive residual quantity value as mentioned above) adaptively based on supplementary.
In some instances, can comprise and damage decoding pallet mode as HEVC decoding mode.For example, decoding mode can comprise intra prediction mode, inter-frame forecast mode, harmless decoding pallet mode and damage decoding pallet mode.In HEVC decoding, as above relative to Fig. 2 and 3 mention, quantization parameter (QP) in order to control allow distortion.Residual quantity value for the decoding based on palette can become with QP and calculates or determine in addition.For example, above-mentioned residual quantity value can be 1<< (QP/6) or 1<< ((QP+d)/6), wherein d is constant, and " << " represents the operator that shifts left by turn.
The use described in the present invention damage the palette of decoding technique generation can by video encoder 20, Video Decoder 30 or both perform.For example, video encoder 20 can use above-mentioned residual quantity comparison techniques to produce entry in the palette of CU, and by the information for construct palette of signal indication for Video Decoder 30.That is, video encoder 20 can be configured to by the information of signal indication instruction for the pixel value of the entry in the palette of CU, and uses the pixel value that a little palette entry is associated therewith to encode to pixel value subsequently.Video Decoder 30 can use this information structuring palette, and uses entry to decode to the pixel value through decode block subsequently.In some instances, video encoder 20 available signal represents the index value identified through the palette entry of one or more location of pixels of decode block, and Video Decoder 30 can use described index value from palette retrieval correlated pixel value entry.
In other example, Video Decoder 30 can be configured to by applying above-mentioned quotient of difference comparatively technical construction palette.For example, Video Decoder 30 can receive the pixel value for the position in decode block, and determines whether the absolute difference between the existing pixel value entry in described pixel value and palette is greater than residual quantity value.If, so Video Decoder 30 can add pixel value as entry in palette, such as after a while use by the video encoder 20 manipulative indexing value of signal indication to the pixel value of other location of pixels of block based on the decoding of palette in use.In the case, video encoder 20 and Video Decoder 30 apply same or similar process to produce palette.If not, so Video Decoder 30 can not add pixel value to palette.
For illustration of in the example of object, Video Decoder 30 can receive index value for the various location of pixels in block or pixel value.If such as location of pixels reception hint value, so Video Decoder 30 can use the entry in described index value identification palette, and uses the pixel value of described palette entry for described location of pixels.If receive pixel value for location of pixels, so Video Decoder 30 can use received pixel value for described location of pixels, and application quotient of difference is comparatively to determine whether described pixel value should add palette to and subsequently after a while for palette decoding.
In coder side, if produce the absolute difference being less than or equal to residual quantity value between the existing pixel value entry in described pixel value and palette for the pixel value of the position in block, so video encoder 20 can send index value to identify the pixel value of the entry in palette for the described position of reconstruct.If produce the absolute difference being all greater than residual quantity value between the existing pixel value entry in described pixel value and palette for the pixel value of the position in block, so video encoder 20 can send described pixel value and in palette, add described pixel value as new entry.In order to construct palette, Video Decoder 30 can use the residual quantity value by encoder signal indication, depends on fixing or known residual quantity value, or infers or derive residual quantity value, such as described above.
As mentioned above, video encoder 20 and/or Video Decoder 30 can work as the decoding mode using when carrying out decoding to video data and comprise intra prediction mode, inter-frame forecast mode, harmless decoding pallet mode and damage decoding pallet mode.According to certain aspects of the invention, video encoder 20 and Video Decoder 30 can carry out decoding to instruction based on decoding one or more syntactic element whether through enabling of palette.For example, at each CU place, video encoder 20 can be encoded to syntactic elements such as such as flag PLT_Mode_flag.PLT_Mode_flag or other syntactic element can indicate the decoding mode based on palette whether will be used for given CU (or being PU in other example).For example, this flag at CU level signal indication in coded video bitstream, and can be received when decoding to described coded video bitstream by Video Decoder 30 subsequently at once.
In this example, the value equaling this PLT_Mode_flag of 1 can specify current C U to use to encode based on the decoding mode of palette.In the case, Video Decoder 30 can apply decoding mode based on palette to decode to CU.In some instances, syntactic element can indicate multiple differences for CU based on the one (such as, damage or harmless) in the decoding mode of palette.The value equaling this PLT_Mode_flag of 0 can specify current C U to be the pattern-coding used except pallet mode.For example, any one in multiple inter prediction, infra-frame prediction or other decoding mode can be used.When the value of PLT_Mode_flag is 0, video encoder 20 also can encode to indicate the AD HOC (such as, HEVC decoding mode) for the corresponding CU that encodes to excessive data.The use of PLT_Mode_flag describes for the object of example.Such as, but in other example, whether other syntactic element such as multi-position code can will be used for CU (or being PU in other example) in order to the decoding mode of instruction based on palette or indicate which one in multiple pattern to be used for decoding.
In some instances, above-mentioned flag or other syntactic element can be launched in the level higher than CU (or PU) level.For example, video encoder 20 can in this flag of section level signal indication.In the case, all CU equaled in the value instruction section of 1 use pallet mode coding.In this example, in CU level without signal indication such as the additional pattern information of pallet mode or other pattern.In another example, video encoder 20 can by this flag of signal indication in PPS, SPS or VPS.
According to certain aspects of the invention, video encoder 20 and/or Video Decoder 30 can one place in section, PPS, SPS or VPS level to one or more syntactic element (such as, such as above-mentioned flag) carry out decoding, its specify pallet mode whether for particular slice, picture, sequence or analog through enabling or stopping using, PLT_Mode_flag instruction simultaneously based on the decoding mode of palette whether for each CU.In the case, if the flag sent at section, PPS, SPS or VPS level place or other syntactic element instruction palette decoding mode are through stopping using, so in some instances, the PLT_Mode_flag with each CU of signal indication can not be needed.Or, if whether the flag sent in section, PPS, SPS or VPS level or other syntactic element instruction palette decoding mode, through enabling, so can will be used for each CU with signal indication PLT_Mode_flag to indicate the decoding mode based on palette further.Again, as above mentioning, the application being used to indicate these technology of the decoding based on palette of CU can additionally or alternati in order to indicate the decoding based on palette of PU.
In some instances, can conditionally with syntax elements on signal indication in bit stream.For example, video encoder 20 and Video Decoder 30 can distinguish only coding or decoding syntactic element based on the layer id in the size of CU, frame type, color space, color component, frame sign, frame rate, scalable video coding or the view id in multi views decoding.
Although above-mentioned example relates to such as with the explicit signaling of one or more syntactic element in bit stream, but in other example, video encoder 20 and/or Video Decoder 30 implicitly can determine that palette decoding mode is whether effectively and/or for carrying out decoding to specific piece.Video encoder 20 and Video Decoder 30 can determine based on the layer id in the size of such as CU, frame type, color space, color component, frame sign, frame rate, scalable video coding or the view id in multi views decoding that whether decoding based on palette is for block.
Although describe the technology of Fig. 4 above in the context of CU (HEVC), described technology should be understood and also can be applied in predicting unit (PU) or other video coding process and/or standard.
Fig. 5 illustrates the concept map of the determination consistent with technology of the present invention to the example of the index of the palette of block of pixels.For example, Fig. 5 comprises the mapping 240 making the relevant position of the pixel be associated with index value be related to the index value (value 1,2 and 3) of the entry of palette 244.Palette 244 can be determined with the first palette 184 described relative to Fig. 4 above and the similar mode of the second palette 192.
Again, for illustrative purposes, in the context of video encoder 20 (Fig. 1 and Fig. 2) and Video Decoder 30 (Fig. 1 and Fig. 3) and relative to the technology of HEVC video coding standard to describe Fig. 5.However, it should be understood that technology of the present invention is not limited thereto mode, and can be applied in other video coding process and/or standard by other video coding processor and/or device.
Although in the example of fig. 5 mapping 240 to be illustrated as the index value comprised for each location of pixels, should understand in other example, not all location of pixels can be associated with the index value making pixel value be related to the entry of palette 244.Namely, as mentioned above, in some instances, video encoder 20 can encode (and Video Decoder 30 can obtain described instruction from encoded bit stream) when pixel value is not included in palette 244 to the instruction of the actual pixel value (or it is through quantised versions) for mapping the position in 240.
In some instances, decoding is carried out in the extra mapping that video encoder 20 and Video Decoder 30 can be configured to which location of pixels of instruction is associated with index value.For example, assuming that (i, j) entry in described mapping corresponds to (i, j) position of CU.Video encoder 20 can be encoded one or more syntactic element for each entry of described mapping (that is, each location of pixels), and it indicates described entry whether to have associated index value.For example, the video encoder 20 codified flag with value one is one in value in palette 244 with the pixel value of instruction (i, j) position in CU.Video encoder 20 also can encode palette index (being shown as value 1-3 in the example of Fig. 5) in this example to indicate the pixel value in palette and to allow Video Decoder to reconstruct described pixel value.Comprise in the example of single entry and associated pixel value at palette 244, what video encoder 20 can skip index value uses signal indication.Flag can be encoded to that to have value zero with the pixel value of (i, j) position in CU of instruction be not one in value in palette 244 by video encoder 20.In this example, video encoder 20 also can be encoded for the instruction of the pixel value of reconstructed pixel value to for Video Decoder 30.In some instances, decoding can be carried out to pixel value in a lossy manner.
The value of the pixel in a position of CU can provide the instruction of the value of one or more other pixel in other position of described CU.For example, the adjacent pixel location that can there is CU will have the relative high probability that same pixel value maybe can be mapped to same index value (when damaging decoding, one of them above pixel value can be mapped to single index value).
Therefore, according to aspects of the present invention, one or more syntactic element of video encoder 20 codified, it indicates in given scanning sequence has the contiguous pixels of same pixel value or index value or the number of index value.As mentioned above, the pixel of identical value or the string of index value can be referred to as in this article and prolong row.For illustration of in the example of object, if two contiguous pixels in given scanning sequence or index have different value, so prolong row and equal zero.If two contiguous pixels in given scanning sequence or index have identical value but the 3rd pixel in described scanning sequence or index have different value, so prolong row and equal one.For three continuity indexs or the pixel with identical value, prolong behavior two, etc.Video Decoder 30 can obtain instruction from encoded bit stream and prolongs capable syntactic element and use described data to determine the number of the continuous position with same pixel or index value.
For illustration of in the example of object, consider the line 248 of mapping 240.Assuming that the from left to right scanning direction of level, line 248 comprises five index values " 2 " and three index values " 3 ".According to aspects of the present invention, video encoder 20 can for the primary importance code index value 2 at scanning direction center line 248.In addition, one or more syntactic element of video encoder 20 codified, its instruction have in scanning direction with the successive value with the identical index value of the index value of signal indication prolong row.In the example of online 248, video encoder 20 available signal represents and prolongs row 4, so the index value of subsequently four positions of instruction in scanning direction share with the identical index value of the index value of signal indication.Video encoder 20 can perform same process for next the different index value in line 248.That is, one or more syntactic element of row two is prolonged in video encoder 20 codified index value 3 and instruction.Video Decoder 30 can obtain the syntactic element of the number (prolonging row) of the continuity index indicating index value and have same index value in scanning direction.
As mentioned above, with the index that certain order scanning maps.According to aspects of the present invention, scanning direction can be vertically, level or be in diagonal (such as, 45 degree or 135 degree, paired linea angulata ground in frame).In some instances, video encoder 20 can be encoded to the instruction of each block one or more syntactic element of scanning direction for the index scanning described piece.Additionally or alternati, can come with signal indication based on so-called supplementarys such as such as block size, color space and/or color components or infer scanning direction.Video encoder 20 can specify the scanning of each color component for block.Or invisible scanning can be applicable to all colours component of block.
For example, relative to per-column scanning, the row 252 of mapping 240 are considered.Assuming that vertical scanning direction from top to bottom, row 252 comprise six index values " 2 " and two index values " 3 ".According to aspects of the present invention, video encoder 20 can for primary importance (opposite crests at row 252) the code index value 2 at scanning direction center line 252.In addition, video encoder 20 available signal represents and prolongs row 5, so the index value of subsequently five positions of instruction in scanning direction share with the identical index value of the index value of signal indication.Video encoder 20 also can prolong one or more syntactic element of row one for the next position code index value 3 in scanning direction and instruction.Video Decoder 30 can obtain the syntactic element of the number (prolonging row) of the continuity index indicating index value and have same index value in scanning direction.
According to aspects of the present invention, the line of one or more entry that video encoder 20 and Video Decoder 30 can perform additionally or alternati for mapping 240 copies.In some instances, described line copies and can be depending on scanning direction.For example, video encoder 20 can indicate and equal pixel in the line of (for horizontal sweep) above particular items or the row of the particular items left side (for vertical scanning) or index value for the pixel of the particular items in mapping or index value.The pixel of entry in the row on the line equaled in scanning sequence above particular items or the left side or the number of index value also can be designated as and prolong row by video encoder 20.In this example, video encoder 20 and or Video Decoder 30 can from specifying adjacent lines and from the replicate entries pixel specified number of the line of the mapping of current positive decoding or index value.
For illustration of in the example of object, consider the row 256 and 260 of mapping 240.Assuming that vertical scanning direction from top to bottom, row 256 comprise three index values " 1 ", three index values " 2 " and two index values " 3 ".Row 260 are included in scanning direction the same index value with same order.According to aspects of the present invention, video encoder 20 can be encoded one or more syntactic element for row 260, and it indicates whole row 260 to copy from row 256.One or more syntactic element described can be associated with the first entry of the opposite crests place row 260 in mapping 240.Video Decoder 30 can obtain syntactic element that index line copies and when decoding to row 260 index value of replicated columns 256 for row 260.
According to aspects of the present invention, the technology for prolonging row decoding of advancing to so-called entry can use with the above-mentioned combine with technique copied for line.For example, whether whether the value of the entry during video encoder 20 can map instruction is the value of the entry obtaining from palette or mapping is that one or more syntactic element (such as, flag) obtained from the line through previous decoding mapping 240 is encoded.Video encoder 20 also can be encoded to one or more syntactic element of the bar destination locations in the index value of instruction palette or line (row or column).Video encoder 20 also can be encoded to some one or more syntactic elements of continuous strip object of the shared identical value of instruction.Video Decoder 30 can obtain this information from encoded bit stream and use mapping and the pixel value of described signal reconstruct block.
For illustration of in the example of object, consider the row 264 and 268 of mapping 240.Assuming that the from left to right scanning direction of level, row 264 comprises three index values " 1 ", two index values " 2 " and three index values " 3 ".Row 268 comprises five index values " 1 " and three index values " 3 ".In this example, the particular items of video encoder 20 identifiable design capable 264, be then when encode data for row 268 time prolong row.For example, video encoder 20 can be encoded by one or more syntactic element identical with the primary importance of row 264 to the primary importance of indication lines 268 (leftmost position of row 268).Video encoder 20 also can in indication lines 268 in scanning direction two continuous strip objects next prolong row one or more syntactic element identical with the primary importance of row 264 and encode.
In some instances, video encoder 20 also can determine whether that carrying out decoding or use relative to the position in another row (or row) to current pixel or index value prolongs row syntactic element and carry out decoding to current pixel or index value.For example, after encoding to the primary importance of indication lines 264 and one or more syntactic element prolonging row (mentioning) above of two entries, video encoder 20 can indicate one or more syntactic element of the value 1 being used for the 4th position and instruction to prolong one or more syntactic element of row 1 for the 4th in line 268 and the 5th position (from left to right) coding.Therefore, video encoder 20 is not when encoding to these two positions with reference to another line (or row).Video encoder 20 can subsequently relative to top row 264 to the primary importance in row 268 with index value 3 carry out decoding (such as, indicate from top row 264 copy and have scanning sequence same index value continuous position prolong row).Therefore, according to aspects of the present invention, video encoder 20 can prolong row and relative to other value of line (or row), the pixel of described line (or row) or index value are carried out to decoding, carried out selecting between decoding or its combination to the pixel of line (or row) or index value relative to the value of another line (or row) such as using.In some instances, video encoder 20 can perform rate/distortion optimization to make described selection.
Video Decoder 30 can receive syntax elements and reconstruct row 268.For example, Video Decoder 30 can obtain the data of the ad-hoc location copying the associated index value of the position of the mapping 240 of current positive decoding in instruction adjacent lines from it.Video Decoder 30 also can obtain the number of the continuous position in beacon scanning order with same index value.
In some instances, the entry (as shown in the example of Fig. 5) of the line of current positive decoding directly can be adjacent to from the line of its duplicate entries.But in other example, some lines can be cushioned by video encoder 20 and/or Video Decoder 30, can be used as the predictability entry of the line of the mapping of current positive decoding with any one making in some lines of mapping.Therefore, in some instances, can be by the pixel value signal indication being used for entry row (or the row on the left side) above the next-door neighbour equaling current line or current line top two or more row (or the row on the left side) in the pixel value of entry.
For illustration of in the example of object, video encoder 20 and/or Video Decoder 30 can be configured to store the capable entry of previous n before carrying out decoding to current line entry.In this example, video encoder 20 can to block unitary code or other yard of indication predicting capable (row from its duplicate entries) in bit stream.In another example, video encoder 20 can encode (and Video Decoder 30 can be decoded to it) to the shift value between front and the predictability line being used as to carry out the mapping 240 of the reference of decoding to current line.That is, video encoder 20 can be encoded to the instruction of the certain line (or row) copying index value from it.In some instances, described shift value can be motion vector.That is, suppose c [0], the index of predictability line (such as top adjacent lines) that the index when front that c [1] ... represent mapping 240 and hypothesis u [0], u [1], u [2] ... represent mapping 240.In this example, given motion vector is d, and the index value for c [i] can be predicted from u [i+d].The value of d can use unitary, blocks unitary, index Columbus or Columbus-Lai Si code carry out decoding.
Fig. 6 illustrates consistent with the technology of the present invention flow chart for using palette decoding mode video data to be carried out to the example procedure of decoding.Relative to the method for the such as video decoder key-drawing 6 such as video encoder 20 (Fig. 1 and 2) or Video Decoder 30 (Fig. 1 and 3).However, it should be understood that, other video decoding apparatus can be configured to perform similar approach.In addition, some step in method can different order or executed in parallel.Equally, in various example, some step can be saved, and can add other step.
Whether the such as video decoder such as video encoder and/or Video Decoder 30 can initially determine the pattern that current video data block carries out decoding is decoding mode (280) based on palette.As mentioned above, one or more syntactic element (such as, PLT_Mode_flag syntactic element) of the decoding mode of instruction current block can be comprised in bit stream.In other example, video decoder can be made based on so-called supplementary and determining, as mentioned above.
Under any circumstance, if based on the decoding mode of palette not for the block (the "No" branch of step 280) of current positive decoding, so video decoder can use and carry out decoding (282) except based on the pattern except the decoding mode of palette to block of video data.For example, video decoder can use the non-pattern based on palette to carry out decoding to block of video data, such as frame mode, inter-frame mode or another decoding mode.
If the decoding mode based on palette is used for the block (the "Yes" branch of step 280) of current positive decoding, so video decoder can determine the palette (284) current block being carried out to decoding.As hereafter relative to described by Fig. 7 and 8, in some instances, video decoder can determine the palette of current block based on the palette be associated through previous decoded video data block with one or more other.
Video decoder also can determine the index value (286) of the block of current positive decoding.For example, as being hereafter described in more detail relative to Fig. 9, video decoder can determine which location of pixels of indicator collet is the mapping using index value decoding, described index value the position of pixel is related to have associated pixel value determine the entry of palette.In some cases, video decoder can determine one or more index value relative to other index value of block.For example, video decoder can be gone and/or index value based on the prolonging of index value determination index value of another line or row of being arranged in block.
Video decoder uses determined palette and index value to carry out decoding (280) to the video data of block subsequently.For example, relative to video encoder 20, video encoder 20 can be encoded to the data of instruction palette and index value in encoded bit stream.Video encoder 20 also can be encoded through quantised versions to the actual pixel value in this little position or its for any location of pixels in determined palette without respective pixel.Video Decoder 30 can dissect from encoded bit stream and decoding palette, index value and pixel value, and uses described data reconstruction block of video data.
Fig. 7 illustrates consistent with the technology of the present invention flow chart for determining the example procedure of palette in based on the decoding of palette.Relative to the method for the such as video decoder key-drawing 7 such as video encoder 20 (Fig. 1 and 2) or Video Decoder 30 (Fig. 1 and 3).However, it should be understood that, other video decoding apparatus can be configured to perform similar approach.In addition, some step in method can different order or executed in parallel.Equally, in various example, some step can be saved, and can add other step.In some instances, the technology of Fig. 7 can perform during the step 284 of Fig. 6.
In the example of figure 7, video decoder can determine the size (300) of palette.In some instances, the size of palette can be fixing.In other example, during decoding, dynamically can adjust the size (such as, by adding or removing entry from palette) of palette.Video decoder can carry out decoding to one or more syntactic element of the size of instruction palette.
Whether the palette that video decoder can determine carrying out current block decoding predicts (302) from one or more other through previous decoding palette.If described palette is through prediction (step 302 be branch), so video decoder can determine predictability palette (304).For example, video encoder 20 can be used for the palette of current block to instruction is that the data of the position (or identification prediction palette self) of the block be associated with predictability palette based on data and the instruction of the palette of one or more previous coding are encoded.Equally, Video Decoder 30 can obtain this data from encoded bit stream.As above relative to described by Fig. 4, predictability palette can be associated with the adjacent block of current positive decode block.
Video decoder can determine one or more entry (306) of current block being carried out to the palette of decoding based on one or more entry of determined predictability palette.In some instances, video decoder can copy whole palette from another block and carries out decoding for current block.In other example, video decoder can from another palette optionally duplicate entries.In some instances, the frequency that can use in previous decode block at one or more based on palette entry predicted by palette.
Video decoder also can determine one or more non-predicted entry (308) of palette.For example, in some instances, can from other only part through previous decoding palette prediction palette.In other example, can not predictably decoding palette.In this type of example, video decoder (or after which) can determine palette entry when not performing palette Predicting Technique described herein.
Fig. 8 is the flow chart of the example procedure that the index for determine block of video data consistent with technology of the present invention is described.Relative to the method for the such as video decoder key-drawing 8 such as video encoder 20 (Fig. 1 and 2) or Video Decoder 30 (Fig. 1 and 3).However, it should be understood that, other video decoding apparatus can be configured to perform similar approach.In addition, some step in method can different order or executed in parallel.Equally, in various example, some step can be saved, and can add other step.In some instances, the technology of Fig. 8 can perform during the step 286 of Fig. 6.
Video decoder can initially be determined such as the scanning direction (320) of the mapping of scan indices in based on the decoding of palette.In some instances, as above relative to Fig. 5 example mentioned, video decoder can through pre-configured to use specific scanning direction.In other example, video decoder can carry out decoding to one or more syntactic element of the described scanning direction of instruction and determine scanning direction based on institute's syntax elements.In other example again, video decoder can determine scanning direction based on so-called supplementary.
Video decoder also can determine the mapping with the position of the associated index value of palette of current positive decoded video data block.For example, one or more pixel value of current positive decode block can not represent in from the palette of described piece.In this little example, video decoder directly can carry out decoding (instead of carrying out decoding to the index value of the pixel value identified in palette) to pixel value.Which location of pixels can be indicated to be associated with index value for described mapping and which location of pixels is not associated with index value.
Video decoder determines one or more the first index value (324).For the given pixel in block, video decoder can be determined to make the pixel value of described given pixel be related to the index value of the entry in the palette of described piece.
Video decoder also determines one or more the second index value (326) based on the first index value.For example, as above relative to described by Fig. 5, video decoder can determine one or more index value relative to the index value of other decoding.In some instances, video decoder can determine the second index value based on the prolonging row of the index value of consecutive identical values.In other example, video decoder can determine the second index value based on one or more first index value be included in the line different from the line of current positive decoding pixel.In other example, video decoder can the combination of application technology to determine the second index value.
Will be appreciated that, depend on example, some action of any one in technology described herein or event can perform by different order, can add, merge or all omit (such as, put into practice described technology not necessarily all described actions or event).In addition, in certain embodiments, can simultaneously (such as, by multiple threads, interrupt processing or multiple processor) and non-sequentially performs an action or event.In addition, although for clarity, some aspect of the present invention is described as be performed by individual module or unit, should be understood that technology of the present invention can be performed by the combination of the unit be associated with video decoder or module.
Relative to exploitation HEVC standard, some aspect of the present invention is described for purposes of illustration.But the technology described in the present invention can have for other video coding process, comprise other standard still undeveloped or proprietary video coding process.
Technology as described above can be performed by video encoder 20 (Fig. 1 and 2) and/or Video Decoder 30 (Fig. 1 and 3), can be referred to as video decoder substantially both it.Similarly, at where applicable, video coding can refer to Video coding or video decode.
Although describe the particular combination of the various aspects of described technology above, provide these combinations only in order to the example of the technology described in the present invention is described.Therefore, technology of the present invention should not be limited to the combination of these examples and can contain any combination imagined of the various aspects of the technology described in the present invention.
In one or more example, described function can be implemented in hardware, software, firmware or its any combination.If with implement software, then described function can store as one or more instruction or code or transmit on computer-readable media, and is performed by hardware based processing unit.Computer-readable media can comprise computer-readable storage medium, it corresponds to the tangible medium of such as data storage medium, or comprise the communication medium promoting any media (such as, according to communication protocol) computer program being sent to another place from.In this way, computer-readable media can correspond to the tangible computer readable memory medium of (1) nonvolatile shape substantially, or (2) communication medium, such as signal or carrier wave.Data storage medium can be can by one or more computer or one or more processor access with retrieval for implementing any available media of the instruction of the technology described in the present invention, code and/or data structure.Computer program can comprise computer-readable media.
To illustrate with example and unrestricted, this type of computer-readable storage medium can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage apparatus, disk storage device or other magnetic storage device, flash memory, or can in order to store in instruction or data structure form want program code and can by other media any of computer access.And, any connection is called computer-readable media rightly.For example, if use the wireless technology such as coaxial cable, Connectorized fiber optic cabling, twisted-pair feeder, digital subscribe lines (DSL) or such as infrared ray, radio and microwave from website, server or other remote source instruction, so the wireless technology such as coaxial cable, Connectorized fiber optic cabling, twisted-pair feeder, DSL or such as infrared ray, radio and microwave is included in the definition of media.However, it should be understood that described computer-readable storage medium and data storage medium do not comprise be connected, carrier wave, signal or other temporary transient media, but be in fact directed to non-transitory tangible storage medium.As used herein, disk and case for computer disc are containing compact disk (CD), laser-optical disk, optical compact disks, digital versatile disc (DVD), floppy discs and Blu-ray Disc, wherein disk is usually with magnetic means rendering data, and CD laser rendering data to be optically.Combination above also should be included in the scope of computer-readable media.
Can by such as one or more digital signal processor (DSP), general purpose microprocessor, application-specific integrated circuit (ASIC) (ASIC), field programmable logic array (FPGA) or other equivalence be integrated or one or more processor such as discrete logic performs instruction.Therefore, " processor " cocoa refers to said structure or is suitable for implementing any one in other structure any of technology described herein as used herein, the term.In addition, in certain aspects, as herein describedly functionally being configured for providing in the specialized hardware of Code And Decode and/or software module, or can be incorporated in combination codec.And, described technology can be implemented in one or more circuit or logic element completely.
Technology of the present invention can be implemented in extensive multiple device or equipment, and described device or equipment comprise wireless handset, integrated circuit (IC) or one group of IC (such as, chipset).Describe in the present invention various assembly, module or unit be in order to emphasize to be configured to perform disclose the function aspects of the device of technology, but necessarily not realized by different hardware unit.In fact, as described above, various unit in conjunction with suitable software and/or firmware combinations in codec hardware unit, or can be provided by the set of interoperability hardware cell, and described hardware cell comprises one or more processor as described above.
Various example has been described.These and other example within the scope of the appended claims.
Claims (30)
1. video data is carried out to a method for decoding, described method comprises:
Determine the first palette of the first entry with instruction first pixel value;
Described first entry based on described first palette determines one or more second entry of the second pixel value of instruction second palette; And
Described second palette is used to carry out decoding to the pixel of block of video data.
2. method according to claim 1, wherein said piece is included in and has in the picture of multiple pieces, and wherein determines that the entry that described first palette comprises based on being associated with more than in described multiple pieces block determines described first entry.
3. method according to claim 1, it comprises further:
When one or more second entry described comprises all described first entry, all described Article 1 destination datas are comprised to instruction one or more second entry described and carries out decoding; And
When one or more second entry described comprise be less than all described first entry time, for each entry of described second palette whether data in described first palette are included in the described respective entries of instruction and instruction carries out decoding for the data of the pixel value not being included in the entry in described first palette.
4. method according to claim 1, it comprises further:
The number with the pixel of respective pixel values based on the presumptive area of described video data determines the probability of happening of the described respective pixel values of described video data; And
Based on described probability of happening, index value is assigned to the entry of described second palette, is included as the entry with relatively high probability of happening and assigns relatively low index value.
5. method according to claim 1, it comprises further:
The number with the pixel of respective pixel values based on the presumptive area of described video data determines the probability of happening of the described respective pixel values of described video data; And
Be included in described first palette based on one or more second entry described in described second palette and index value be assigned to one or more second entry described, comprise and index value relatively low compared with other entry any of described second palette is assigned to one or more second entry described; And
6. method according to claim 1, wherein said first palette is associated with the first block of video data, and described method comprises further:
Relative to the data of the position using the described block of video data of described second palette decoding, decoding is carried out to described first piece of instruction.
7. method according to claim 6, wherein to the described data of the described position of instruction carry out decoding comprise to instruction described first piece be whether be positioned at the described block of video data using described second palette decoding above top adjacent block or described first piece whether be positioned to use the data of the left side adjacent block on the described block of video data left side of described second palette decoding to carry out decoding.
8. method according to claim 1, wherein said first palette is in order to carry out decoding to from described piece of described second palette decoding different block of pixels, and described method comprises further:
Determine that each entry of described first palette is in order to the frequency of carrying out decoding with the described block of pixels of described first palette decoding;
Decoding is carried out based on the data of described frequency to each identified in one or more second entry described.
9. method according to claim 1, it comprises further:
Carry out decoding based on the prediction syntactic element of described first entry to instruction one or more second entry described.
10. method according to claim 1, it comprises further by combining one or more entry described in described second palette and quantizes one or more entry described in described second palette.
11. methods according to claim 1, it comprises further:
Based on each in one or more entry described in the pixel value of entry to be added and described second palette pixel value between difference whether be greater than residual quantity value and add described entry to described second palette.
12. methods according to claim 1, it comprises the size determining described second palette further, wherein determines that the described size of described second palette comprises at least one in the following:
The described size determining described second palette is fixing;
Determine that the described size of described second palette is variable and based on the data of described video data block decoding;
Characteristic based on described block of video data determines the described size of described palette; Or
Based on the described size determining described palette with the final bar destination data in described second palette of the instruction of described video data block decoding.
13. methods according to claim 1, wherein carry out decoding to the described pixel of described piece and comprise and decoding to described pixel, and wherein carry out decoding to described pixel and comprise:
The respective index value for one or many person described pixel is obtained, the entry of the second palette described in wherein said respective index value identification from encoded bit stream; And
The value of one or more pixel described is determined by described respective index value is matched with at least one in the described entry of described second palette.
14. methods according to claim 1, wherein carry out decoding to the described pixel of described piece and comprise and encoding to described pixel, and wherein carry out coding to described pixel and comprise:
Determine the respective index value of one or many person in described pixel, the respective entries of the second palette described in wherein said respective index value identification; And
In encoded bit stream, described index value is encoded.
15. 1 kinds for carrying out the equipment of decoding to video data, described equipment comprises:
Memory, its stored video data; And
One or more processor, it is configured to:
Determine the first palette of the first entry with instruction first pixel value;
Described first entry based on described first palette determines one or more second entry of the second pixel value of instruction second palette; And
Described second palette is used to carry out decoding to the pixel of the block of described video data.
16. equipment according to claim 15, wherein said piece is included in and has in the picture of multiple pieces, and wherein in order to determine described first palette, the entry that one or more processor described is configured to based on being associated with more than in described multiple pieces block determines described first entry.
17. equipment according to claim 15, one or more processor wherein said is configured to further:
When one or more second entry described comprises all described first entry, all described Article 1 destination datas are comprised to instruction one or more second entry described and carries out decoding; And
When one or more second entry described comprise be less than all described first entry time, for each entry of described second palette whether data in described first palette are included in the described respective entries of instruction and instruction carries out decoding for the data of the pixel value not being included in the entry in described first palette.
18. equipment according to claim 1, one or more processor wherein said is configured to further:
The number with the pixel of respective pixel values based on the presumptive area of described video data determines the probability of happening of the described respective pixel values of described video data; And
Based on described probability of happening, index value is assigned to the entry of described second palette, is included as the entry with relatively high probability of happening and assigns relatively low index value.
19. equipment according to claim 15, one or more processor wherein said is configured to further:
The number with the pixel of respective pixel values based on the presumptive area of described video data determines the probability of happening of the described respective pixel values of described video data; And
Be included in described first palette based on one or more second entry described in described second palette and index value be assigned to one or more second entry described, comprise and index value relatively low compared with other entry any of described second palette is assigned to one or more second entry described; And
20. equipment according to claim 15, wherein said first palette is associated with the first block of video data, and one or more processor wherein said is configured to carry out decoding to described first piece of instruction relative to the data of the position using the described block of video data of described second palette decoding further.
21. equipment according to claim 20, wherein in order to carry out decoding to the described data of the described position of instruction, whether whether one or more processor described to be configured to instruction described first piece be top adjacent block above being positioned at the described block of video data using described second palette decoding or described first piece is be positioned to use the data of the left side adjacent block on the described block of video data left side of described second palette decoding to carry out decoding.
22. equipment according to claim 15, wherein said first palette is in order to carry out decoding to from described piece of described second palette decoding different block of pixels, and one or more processor wherein said is configured to further:
Determine that each entry of described first palette is in order to the frequency of carrying out decoding with the described block of pixels of described first palette decoding;
Decoding is carried out based on the data of described frequency to each identified in one or more second entry described.
23. equipment according to claim 15, it is carry out decoding based on the prediction syntactic element of described first entry that one or more processor wherein said is configured to further to instruction one or more second entry described.
24. equipment according to claim 15, one or more processor wherein said is configured to further by combining one or more entry described in described second palette and quantizes one or more entry described in described second palette.
25. equipment according to claim 15, whether the difference between one or more processor wherein said is configured to further based on the pixel value of each in one or more entry described in the pixel value of entry to be added and described second palette is greater than residual quantity value and adds described entry to described second palette.
26. equipment according to claim 15, one or more processor wherein said is configured to the size determining described second palette further, and wherein in order to determine the described size of described second palette, one or more processor described is configured to:
The described size determining described second palette is fixing;
Determine that the described size of described second palette is variable and based on the data of described video data block decoding;
Characteristic based on described block of video data determines the described size of described palette; Or
Based on the described size determining described palette with the final bar destination data in described second palette of the instruction of described video data block decoding.
27. equipment according to claim 15, wherein in order to carry out decoding to the described pixel of described piece, one or more processor described is configured to decode to described pixel, and wherein in order to decode to described pixel, one or more processor described is configured to:
The respective index value for one or many person described pixel is obtained, the entry of the second palette described in wherein said respective index value identification from encoded bit stream; And
The value of one or more pixel described is determined by described respective index value is matched with at least one in the described entry of described second palette.
28. equipment according to claim 15, wherein in order to carry out decoding to the described pixel of described piece, one or more processor described is configured to encode to described pixel, and wherein in order to encode to described pixel, one or more processor described is configured to:
Determine the respective index value of one or many person in described pixel, the respective entries of the second palette described in wherein said respective index value identification; And
In encoded bit stream, described index value is encoded.
29. 1 kinds for carrying out the equipment of decoding to video data, described equipment comprises:
For determining the device of the first palette of the first entry with instruction first pixel value;
For determining one or more Article 2 destination device of the second pixel value of instruction second palette based on the described first entry of described first palette; And
For the device using described second palette the pixel of block of video data to be carried out to decoding.
30. 1 kinds of non-transitory computer-readable medias, it stores instruction, and described instruction causes one or more processor when performing:
Determine the first palette of the first entry with instruction first pixel value;
Described first entry based on described first palette determines one or more second entry of the second pixel value of instruction second palette; And
Described second palette is used to carry out decoding to the pixel of the block of described video data.
Applications Claiming Priority (7)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201361809236P | 2013-04-05 | 2013-04-05 | |
| US61/809,236 | 2013-04-05 | ||
| US201361810649P | 2013-04-10 | 2013-04-10 | |
| US61/810,649 | 2013-04-10 | ||
| US14/244,688 | 2014-04-03 | ||
| US14/244,688 US11259020B2 (en) | 2013-04-05 | 2014-04-03 | Determining palettes in palette-based video coding |
| PCT/US2014/033019 WO2014165789A1 (en) | 2013-04-05 | 2014-04-04 | Determining palettes in palette-based video coding |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105191320A true CN105191320A (en) | 2015-12-23 |
| CN105191320B CN105191320B (en) | 2018-10-19 |
Family
ID=51654451
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201480019157.7A Active CN105075272B (en) | 2013-04-05 | 2014-04-04 | The method and apparatus of palette index is determined in the video coding based on palette |
| CN201480019413.2A Active CN105191320B (en) | 2013-04-05 | 2014-04-04 | The method and apparatus that video data is coded and decoded |
Family Applications Before (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201480019157.7A Active CN105075272B (en) | 2013-04-05 | 2014-04-04 | The method and apparatus of palette index is determined in the video coding based on palette |
Country Status (9)
| Country | Link |
|---|---|
| US (2) | US11259020B2 (en) |
| EP (2) | EP2982125B1 (en) |
| JP (2) | JP6297671B2 (en) |
| KR (2) | KR102205328B1 (en) |
| CN (2) | CN105075272B (en) |
| BR (2) | BR112015025459B1 (en) |
| ES (1) | ES2727635T3 (en) |
| HU (1) | HUE043455T2 (en) |
| WO (2) | WO2014165789A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107409215A (en) * | 2015-01-29 | 2017-11-28 | 高通股份有限公司 | Pallet mode for video coding decodes |
| CN114128277A (en) * | 2019-08-13 | 2022-03-01 | 高通股份有限公司 | Palette predictor update for local dual trees |
Families Citing this family (99)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11259020B2 (en) | 2013-04-05 | 2022-02-22 | Qualcomm Incorporated | Determining palettes in palette-based video coding |
| WO2015006169A1 (en) * | 2013-07-08 | 2015-01-15 | Sony Corporation | Improvement for palette coding mode |
| US9558567B2 (en) | 2013-07-12 | 2017-01-31 | Qualcomm Incorporated | Palette prediction in palette-based video coding |
| EP3058740B1 (en) * | 2013-10-14 | 2020-06-03 | Microsoft Technology Licensing, LLC | Features of base color index map mode for video and image coding and decoding |
| US10291827B2 (en) * | 2013-11-22 | 2019-05-14 | Futurewei Technologies, Inc. | Advanced screen content coding solution |
| GB201321851D0 (en) * | 2013-12-10 | 2014-01-22 | Canon Kk | Run length parameters coding for palette mode |
| KR101897378B1 (en) * | 2013-12-10 | 2018-09-10 | 캐논 가부시끼가이샤 | Method and apparatus for encoding or decoding a palette in palette coding mode |
| CA2934116C (en) * | 2013-12-18 | 2019-07-30 | Tzu-Der Chuang | Method and apparatus for palette table prediction |
| US10321141B2 (en) * | 2013-12-18 | 2019-06-11 | Hfi Innovation Inc. | Method and apparatus for palette initialization and management |
| EP3085068A4 (en) * | 2013-12-18 | 2016-12-21 | Hfi Innovation Inc | Palette prediction and sharing in video coding |
| GB2523076B (en) * | 2013-12-19 | 2016-08-10 | Canon Kk | Improved palette mode in HEVC for the encoding process |
| WO2015091879A2 (en) * | 2013-12-19 | 2015-06-25 | Canon Kabushiki Kaisha | Improved encoding process using a palette mode |
| WO2015096812A1 (en) | 2013-12-27 | 2015-07-02 | Mediatek Inc. | Method and apparatus for palette coding with cross block prediction |
| EP3087742B1 (en) | 2013-12-27 | 2020-09-23 | HFI Innovation Inc. | Method and apparatus for syntax redundancy removal in palette coding |
| CN110225345B (en) | 2013-12-27 | 2022-07-19 | 寰发股份有限公司 | Method and apparatus for primary color index map encoding |
| KR101951083B1 (en) | 2014-01-02 | 2019-05-10 | 브이아이디 스케일, 인크. | Two-dimensional palette coding for screen content coding |
| CN105850132B (en) | 2014-01-07 | 2019-10-22 | 联发科技股份有限公司 | Method and apparatus for encoding and decoding video data block |
| US9699468B2 (en) * | 2014-02-10 | 2017-07-04 | Microsoft Technology Licensing, Llc | Adaptive screen and video coding scheme |
| CN104853209B (en) * | 2014-02-16 | 2020-09-29 | 同济大学 | Image coding and decoding method and device |
| US10542274B2 (en) | 2014-02-21 | 2020-01-21 | Microsoft Technology Licensing, Llc | Dictionary encoding and decoding of screen content |
| KR101864979B1 (en) * | 2014-03-11 | 2018-06-05 | 에이치에프아이 이노베이션 인크. | Method and apparatus of single sample mode for video coding |
| WO2015138936A1 (en) * | 2014-03-13 | 2015-09-17 | Huawei Technologies Co., Ltd. | Improved method for screen content coding |
| PL3117617T3 (en) | 2014-03-14 | 2022-08-22 | Vid Scale, Inc. | Palette coding for screen content coding |
| JP6164360B2 (en) * | 2014-03-14 | 2017-07-19 | 富士通株式会社 | Image encoding device, image decoding device, image encoding method, and image decoding method |
| US9826242B2 (en) | 2014-03-14 | 2017-11-21 | Qualcomm Incorporated | Palette-based video coding |
| WO2015135509A1 (en) | 2014-03-14 | 2015-09-17 | Mediatek Inc. | Method for palette table initialization and management |
| US10638143B2 (en) * | 2014-03-21 | 2020-04-28 | Futurewei Technologies, Inc. | Advanced screen content coding with improved color table and index map coding methods |
| US10362336B2 (en) | 2014-03-25 | 2019-07-23 | Qualcomm Incorporated | Palette predictor signaling with run length code for video coding |
| US10264285B2 (en) | 2014-05-22 | 2019-04-16 | Qualcomm Incorporated | Coding runs in palette-based video coding |
| US10750198B2 (en) | 2014-05-22 | 2020-08-18 | Qualcomm Incorporated | Maximum palette parameters in palette-based video coding |
| US10038915B2 (en) | 2014-05-22 | 2018-07-31 | Qualcomm Incorporated | Escape sample coding in palette-based video coding |
| US10091512B2 (en) | 2014-05-23 | 2018-10-02 | Futurewei Technologies, Inc. | Advanced screen content coding with improved palette table and index map coding methods |
| US10142642B2 (en) | 2014-06-04 | 2018-11-27 | Qualcomm Incorporated | Block adaptive color-space conversion coding |
| CN105282550A (en) * | 2014-06-13 | 2016-01-27 | 财团法人工业技术研究院 | Encoding method and decoding method, encoding and decoding system, encoder and decoder |
| CA2950964C (en) * | 2014-06-19 | 2018-10-30 | Hfi Innovation Inc. | Method and apparatus of candidate generation for single sample mode in video coding |
| CN106416246B (en) * | 2014-06-20 | 2019-09-17 | 寰发股份有限公司 | Method and apparatus for binarization of syntax and context adaptive coding in video coding |
| JP6328277B2 (en) * | 2014-06-20 | 2018-05-23 | 寰發股▲ふん▼有限公司ＨＦＩ Ｉｎｎｏｖａｔｉｏｎ Ｉｎｃ． | Pallet predictor signaling method for video coding |
| JP2017522839A (en) * | 2014-06-20 | 2017-08-10 | シャープ株式会社 | Alignment palette encoding |
| US9906799B2 (en) * | 2014-06-20 | 2018-02-27 | Qualcomm Incorporated | Copy from previous rows for palette mode coding |
| CN106537910B (en) * | 2014-07-07 | 2019-07-12 | 寰发股份有限公司 | Method for using jumping-out pixel as predictor in index image coding |
| US9955157B2 (en) | 2014-07-11 | 2018-04-24 | Qualcomm Incorporated | Advanced palette prediction and signaling |
| US9544607B2 (en) * | 2014-08-25 | 2017-01-10 | Hfi Innovation Inc. | Method of palette index signaling for image and video coding |
| WO2016034117A1 (en) * | 2014-09-03 | 2016-03-10 | Mediatek Inc. | Method for color index coding using a generalized copy previous mode |
| KR20160037111A (en) | 2014-09-26 | 2016-04-05 | 주식회사 케이티 | Method and apparatus for processing a video signal |
| WO2016049839A1 (en) | 2014-09-30 | 2016-04-07 | Microsoft Technology Licensing, Llc | Rules for intra-picture prediction modes when wavefront parallel processing is enabled |
| WO2016057444A2 (en) * | 2014-10-06 | 2016-04-14 | Vid Scale, Inc. | Improved palette coding for screen content coding |
| US9596479B2 (en) * | 2014-10-07 | 2017-03-14 | Hfi Innovation Inc. | Method of pulse-code modulation and palette coding for video coding |
| US10397611B2 (en) * | 2014-10-08 | 2019-08-27 | Lg Electronics Inc. | Method and device for encoding/decoding 3D video |
| US10477218B2 (en) | 2014-10-20 | 2019-11-12 | Kt Corporation | Method and apparatus for predicting and restoring a video signal using palette entry |
| EP3198875B1 (en) * | 2014-11-06 | 2021-04-21 | MediaTek Inc. | Method for palette predictor update |
| CN114630131B (en) * | 2014-11-12 | 2023-11-07 | 寰发股份有限公司 | Jump out pixel encoding and decoding method in index mapping encoding and decoding |
| US10666974B2 (en) * | 2014-11-12 | 2020-05-26 | Hfi Innovation Inc. | Methods of escape pixel coding in index map coding |
| JP6532222B2 (en) | 2014-11-28 | 2019-06-19 | キヤノン株式会社 | Image encoding device, image encoding method and program, image decoding device, image decoding method and program |
| WO2016100424A1 (en) * | 2014-12-19 | 2016-06-23 | Mediatek Inc. | Methods of palette based prediction for non-444 color format in video and image coding |
| WO2016114583A1 (en) * | 2015-01-15 | 2016-07-21 | 주식회사 케이티 | Method and device for processing video signal |
| CN105828080B (en) | 2015-01-26 | 2020-02-14 | 同济大学 | Image coding and decoding method and device |
| JP6122516B2 (en) | 2015-01-28 | 2017-04-26 | 財團法人工業技術研究院Ｉｎｄｕｓｔｒｉａｌ Ｔｅｃｈｎｏｌｏｇｙ Ｒｅｓｅａｒｃｈ Ｉｎｓｔｉｔｕｔｅ | Encoding method and encoder |
| CN107211153B (en) * | 2015-01-29 | 2020-10-02 | 株式会社Kt | Method and apparatus for processing video signal |
| WO2016122251A1 (en) * | 2015-01-29 | 2016-08-04 | 주식회사 케이티 | Method and apparatus for processing video signals |
| CN107211122B (en) * | 2015-01-29 | 2020-05-05 | 佳能株式会社 | Palette predictor initialization program for encoding or decoding self-contained coding structure |
| EP3251353A1 (en) * | 2015-01-29 | 2017-12-06 | VID SCALE, Inc. | Palette coding modes and palette flipping |
| EP3251352A1 (en) * | 2015-01-29 | 2017-12-06 | VID SCALE, Inc. | Escape color coding for palette coding mode |
| US10425659B2 (en) * | 2015-01-30 | 2019-09-24 | Qualcomm Incorporated | Coding escape pixels for palette coding |
| WO2016119196A1 (en) * | 2015-01-30 | 2016-08-04 | 富士通株式会社 | Image coding method, apparatus, and image processing device |
| US9900624B2 (en) | 2015-02-05 | 2018-02-20 | Mediatek Inc. | Methods and apparatus of decoding process for palette syntax |
| WO2016127889A1 (en) * | 2015-02-13 | 2016-08-18 | Mediatek Inc. | Method and apparatus for palette index coding in video and image compression |
| EP3248377B1 (en) * | 2015-02-16 | 2022-05-04 | HFI Innovation Inc. | Method and apparatus for palette predictor initialization for palette coding in video and image compression |
| WO2016131417A1 (en) * | 2015-02-17 | 2016-08-25 | Mediatek Inc. | Method and apparatus for palette coding of monochrome contents in video and image compression |
| WO2016146076A1 (en) * | 2015-03-18 | 2016-09-22 | Mediatek Inc. | Method and apparatus for index map coding in video and image compression |
| KR102691340B1 (en) * | 2015-04-02 | 2024-08-02 | 주식회사 케이티 | Method and apparatus for processing a video signal |
| KR102060871B1 (en) | 2015-04-08 | 2019-12-30 | 에이치에프아이 이노베이션 인크. | Palette Mode Context Coding and Binarization in Video Coding |
| SG11201708026TA (en) | 2015-04-08 | 2017-10-30 | Hfi Innovation Inc | Methods of palette mode coding in video coding |
| EP3306933A4 (en) * | 2015-05-27 | 2019-05-08 | KT Corporation | Method and device for processing video signal |
| WO2016192662A1 (en) * | 2015-06-03 | 2016-12-08 | Mediatek Inc. | Method and apparatus for resource sharing between intra block copy mode and inter prediction mode in video coding systems |
| WO2016192677A1 (en) | 2015-06-03 | 2016-12-08 | Mediatek Inc. | Method and apparatus of error handling for video coding using intra block copy mode |
| US20160360205A1 (en) * | 2015-06-08 | 2016-12-08 | Industrial Technology Research Institute | Video encoding methods and systems using adaptive color transform |
| WO2016197312A1 (en) * | 2015-06-09 | 2016-12-15 | 富士通株式会社 | Method and apparatus for encoding pixel index and image processing device |
| CN106303526A (en) * | 2015-06-09 | 2017-01-04 | 富士通株式会社 | Method for encoding images, device and image processing equipment |
| CN106664405B (en) | 2015-06-09 | 2020-06-09 | 微软技术许可有限责任公司 | Robust encoding/decoding of escape-coded pixels with palette mode |
| US11146788B2 (en) | 2015-06-12 | 2021-10-12 | Qualcomm Incorporated | Grouping palette bypass bins for video coding |
| US9743092B2 (en) * | 2015-10-13 | 2017-08-22 | Nokia Technologies Oy | Video coding with helper data for spatial intra-prediction |
| JP6662123B2 (en) * | 2016-03-14 | 2020-03-11 | 富士通株式会社 | Image encoding apparatus, image encoding method, and image encoding program |
| CN106331716B (en) * | 2016-08-31 | 2019-05-24 | 西安万像电子科技有限公司 | Video-frequency compression method and device |
| US10110914B1 (en) * | 2016-09-15 | 2018-10-23 | Google Llc | Locally adaptive warped motion compensation in video coding |
| CN106791883A (en) * | 2016-11-18 | 2017-05-31 | 上海兆芯集成电路有限公司 | Texture brick compresses and decompression method and the device using the method |
| CN106534874B (en) * | 2016-11-18 | 2019-11-29 | 上海兆芯集成电路有限公司 | The compression of texture brick and decompression method and the device using this method |
| US10555006B2 (en) | 2016-12-22 | 2020-02-04 | Qualcomm Incorporated | Deriving bilateral filter information based on a prediction mode in video coding |
| US11695919B2 (en) | 2018-08-28 | 2023-07-04 | Google Llc | Lossy image compression using palettization of locally mixed colors |
| US10848787B2 (en) | 2018-08-28 | 2020-11-24 | Google Llc | Lossy image compression using palettization of locally mixed colors |
| CN112672132B (en) * | 2018-09-30 | 2023-12-26 | Oppo广东移动通信有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN109257609B (en) * | 2018-09-30 | 2021-04-23 | Oppo广东移动通信有限公司 | Data processing method and device, electronic device and storage medium |
| FR3092719A1 (en) * | 2019-02-07 | 2020-08-14 | Orange | Methods and devices for encoding and decoding a data stream representative of at least one image. |
| US11134258B2 (en) | 2019-02-11 | 2021-09-28 | Qualcomm Incorporated | Predictive coefficient coding |
| EP4011080A4 (en) * | 2019-09-12 | 2023-04-12 | ByteDance Inc. | Using palette predictor in video coding |
| EP4561074A3 (en) * | 2019-10-05 | 2025-07-23 | LG Electronics Inc. | Image or video coding based on signaling of transform skip - and palette coding-related information |
| CN118612435A (en) * | 2019-12-02 | 2024-09-06 | 现代自动车株式会社 | Video encoding and decoding method using differential modulation |
| CN111372124B (en) * | 2020-03-09 | 2022-04-15 | 格兰菲智能科技有限公司 | Character and image coding and decoding method |
| KR102589351B1 (en) * | 2020-03-25 | 2023-10-16 | 주식회사 케이티 | Method and device for processing a video |
| KR102384793B1 (en) * | 2021-09-24 | 2022-04-11 | 티오더 주식회사 | system for delivery service online platform |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1411664A (en) * | 1999-12-22 | 2003-04-16 | 爱慕特尔有限公司 | System for moving image transmitting/receiving using wireless communication and method of same |
| US6748116B1 (en) * | 2000-08-22 | 2004-06-08 | Airzip, Inc. | Apparatus and method for compressing and decompressing image files |
| US7343037B1 (en) * | 2004-02-04 | 2008-03-11 | Microsoft Corporation | Dynamic, locally-adaptive, lossless palettization of color and grayscale images |
| US20110110416A1 (en) * | 2009-11-12 | 2011-05-12 | Bally Gaming, Inc. | Video Codec System and Method |
Family Cites Families (52)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5686961A (en) | 1992-01-27 | 1997-11-11 | Harris Corporation | Automatic video image data reduction and prioritization system and method |
| US5408542A (en) | 1992-05-12 | 1995-04-18 | Apple Computer, Inc. | Method and apparatus for real-time lossless compression and decompression of image data |
| JP2000511363A (en) | 1994-07-14 | 2000-08-29 | ジョンソン、グレイス、カンパニー | Method and apparatus for compressing images |
| US5930390A (en) | 1996-03-28 | 1999-07-27 | Intel Corporation | Encoding/decoding signals using a remap table |
| US6008816A (en) * | 1996-04-25 | 1999-12-28 | Microsoft Corporation | Method and system for managing color specification using attachable palettes and palettes that refer to other palettes |
| US5781483A (en) | 1996-12-31 | 1998-07-14 | Micron Technology, Inc. | Device and method for repairing a memory array by storing each bit in multiple memory cells in the array |
| US6157746A (en) | 1997-02-12 | 2000-12-05 | Sarnoff Corporation | Apparatus and method for encoding wavelet trees generated by a wavelet-based coding method |
| US6064771A (en) | 1997-06-23 | 2000-05-16 | Real-Time Geometry Corp. | System and method for asynchronous, adaptive moving picture compression, and decompression |
| JPH11308465A (en) * | 1998-04-17 | 1999-11-05 | Seiko Epson Corp | Color image encoding method and encoding device thereof, color image decoding method and decoding device thereof |
| US6256415B1 (en) | 1998-06-10 | 2001-07-03 | Seiko Epson Corporation | Two row buffer image compression (TROBIC) |
| CN1148975C (en) | 1998-07-07 | 2004-05-05 | C-立方微系统股份有限公司 | MPEG start code compatible overlay graphics encoding |
| JP2000115782A (en) | 1998-10-06 | 2000-04-21 | Canon Inc | Coder, coding method and storage medium |
| US6909804B2 (en) | 2000-01-27 | 2005-06-21 | Marger Johnson & Mccollom, P.C. | Image compression usable with animated images |
| KR100708074B1 (en) | 2000-04-12 | 2007-04-16 | 삼성전자주식회사 | Text output method using gray scale table and apparatus therefor |
| JP4011278B2 (en) | 2000-11-06 | 2007-11-21 | 株式会社デンソー | Image compression apparatus and recording medium |
| US6819793B1 (en) | 2000-06-30 | 2004-11-16 | Intel Corporation | Color distribution for texture and image compression |
| AUPR212600A0 (en) | 2000-12-18 | 2001-01-25 | Canon Kabushiki Kaisha | Efficient video coding |
| JP2002369011A (en) | 2001-06-06 | 2002-12-20 | Canon Inc | Image processing apparatus, image processing method, and image processing program |
| US7319720B2 (en) | 2002-01-28 | 2008-01-15 | Microsoft Corporation | Stereoscopic video |
| US6898313B2 (en) | 2002-03-06 | 2005-05-24 | Sharp Laboratories Of America, Inc. | Scalable layered coding in a multi-layer, compound-image data transmission system |
| US7433526B2 (en) | 2002-04-30 | 2008-10-07 | Hewlett-Packard Development Company, L.P. | Method for compressing images and image sequences through adaptive partitioning |
| US7302006B2 (en) | 2002-04-30 | 2007-11-27 | Hewlett-Packard Development Company, L.P. | Compression of images and image sequences through adaptive partitioning |
| GB2388502A (en) | 2002-05-10 | 2003-11-12 | Chris Dunn | Compression of frequency domain audio signals |
| US20040013297A1 (en) | 2002-07-18 | 2004-01-22 | Roger Lo | Method for performing color gamut compression |
| WO2004034323A2 (en) | 2002-10-07 | 2004-04-22 | Summus, Inc. | System for graphics compression and display |
| US20040151394A1 (en) | 2003-01-31 | 2004-08-05 | Mikael Soderberg | Symmetrical, highly deterministic, low complexity, temporal transform video codec and vehicle distribution system incorporating same |
| WO2006022741A1 (en) | 2004-08-23 | 2006-03-02 | Marger Johnson & Mccollom, P.C. | Image compression usable with animated images |
| KR101058010B1 (en) | 2004-09-07 | 2011-08-19 | 삼성전자주식회사 | Apparatus, Method and Information Storage Media for Graphic Data Generation |
| US8073275B2 (en) | 2005-02-09 | 2011-12-06 | Mobixell Networks Ltd. | Image adaptation with target size, quality and resolution constraints |
| JP2006229583A (en) | 2005-02-17 | 2006-08-31 | Eastman Kodak Co | Communication system and digital camera and dock apparatus |
| US8243340B2 (en) | 2006-02-23 | 2012-08-14 | Microsoft Corporation | Pre-processing of image data for enhanced compression |
| US8848789B2 (en) | 2006-03-27 | 2014-09-30 | Qualcomm Incorporated | Method and system for coding and decoding information associated with video compression |
| US7343837B1 (en) | 2006-09-15 | 2008-03-18 | James Anthony Domanico | Multi-plane flexible handle for ratchets and wrenches |
| US8170101B2 (en) | 2006-10-27 | 2012-05-01 | Sharp Laboratories Of America, Inc. | Methods and systems for low-complexity data compression |
| AU2006246497B2 (en) | 2006-11-30 | 2010-02-11 | Canon Kabushiki Kaisha | Method and apparatus for hybrid image compression |
| CN101211341A (en) | 2006-12-29 | 2008-07-02 | 上海芯盛电子科技有限公司 | Image intelligent mode recognition and searching method |
| US8213511B2 (en) | 2007-04-30 | 2012-07-03 | Texas Instruments Incorporated | Video encoder software architecture for VLIW cores incorporating inter prediction and intra prediction |
| US9232226B2 (en) | 2008-08-19 | 2016-01-05 | Marvell World Trade Ltd. | Systems and methods for perceptually lossless video compression |
| US8134573B2 (en) | 2009-02-23 | 2012-03-13 | Sharp Laboratories Of America, Inc. | Encoding CMYK data for display using indexed RGB |
| RU2420021C2 (en) | 2009-03-24 | 2011-05-27 | Корпорация "САМСУНГ ЭЛЕКТРОНИКС Ко., Лтд." | Method to compress images and video sequences |
| US9024862B2 (en) * | 2009-07-02 | 2015-05-05 | Ricoh Co., Ltd. | Dynamic creation of waveform palette |
| EP3349360B1 (en) | 2011-01-14 | 2019-09-04 | GE Video Compression, LLC | Entropy encoding and decoding scheme |
| US20120294353A1 (en) | 2011-05-16 | 2012-11-22 | Mediatek Inc. | Apparatus and Method of Sample Adaptive Offset for Luma and Chroma Components |
| US20130101033A1 (en) | 2011-10-14 | 2013-04-25 | Qualcomm Incorporated | Coding non-symmetric distributions of data |
| US8615138B2 (en) | 2011-11-03 | 2013-12-24 | Google Inc. | Image compression using sub-resolution images |
| CN107071420B (en) | 2012-04-13 | 2021-07-16 | 佳能株式会社 | Method, apparatus and system for encoding and decoding subsets of transform units of video data |
| US11259020B2 (en) | 2013-04-05 | 2022-02-22 | Qualcomm Incorporated | Determining palettes in palette-based video coding |
| US9558567B2 (en) | 2013-07-12 | 2017-01-31 | Qualcomm Incorporated | Palette prediction in palette-based video coding |
| US9794579B2 (en) | 2013-07-15 | 2017-10-17 | Qualcomm Incorporated | Decoded picture buffer operations for video coding |
| US10321141B2 (en) | 2013-12-18 | 2019-06-11 | Hfi Innovation Inc. | Method and apparatus for palette initialization and management |
| KR101951083B1 (en) | 2014-01-02 | 2019-05-10 | 브이아이디 스케일, 인크. | Two-dimensional palette coding for screen content coding |
| US9906799B2 (en) | 2014-06-20 | 2018-02-27 | Qualcomm Incorporated | Copy from previous rows for palette mode coding |
-
2014
- 2014-04-03 US US14/244,688 patent/US11259020B2/en active Active
- 2014-04-03 US US14/244,711 patent/US9654777B2/en active Active
- 2014-04-04 CN CN201480019157.7A patent/CN105075272B/en active Active
- 2014-04-04 WO PCT/US2014/033019 patent/WO2014165789A1/en active Application Filing
- 2014-04-04 JP JP2016506656A patent/JP6297671B2/en active Active
- 2014-04-04 EP EP14720461.4A patent/EP2982125B1/en active Active
- 2014-04-04 KR KR1020157031433A patent/KR102205328B1/en active Active
- 2014-04-04 KR KR1020157031417A patent/KR102252843B1/en active Active
- 2014-04-04 CN CN201480019413.2A patent/CN105191320B/en active Active
- 2014-04-04 WO PCT/US2014/033013 patent/WO2014165784A1/en active Application Filing
- 2014-04-04 BR BR112015025459-4A patent/BR112015025459B1/en active IP Right Grant
- 2014-04-04 HU HUE14720462A patent/HUE043455T2/en unknown
- 2014-04-04 ES ES14720462T patent/ES2727635T3/en active Active
- 2014-04-04 JP JP2016506659A patent/JP6463730B2/en active Active
- 2014-04-04 BR BR112015025439-0A patent/BR112015025439B1/en active IP Right Grant
- 2014-04-04 EP EP14720462.2A patent/EP2982126B1/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1411664A (en) * | 1999-12-22 | 2003-04-16 | 爱慕特尔有限公司 | System for moving image transmitting/receiving using wireless communication and method of same |
| US6748116B1 (en) * | 2000-08-22 | 2004-06-08 | Airzip, Inc. | Apparatus and method for compressing and decompressing image files |
| US7343037B1 (en) * | 2004-02-04 | 2008-03-11 | Microsoft Corporation | Dynamic, locally-adaptive, lossless palettization of color and grayscale images |
| US20110110416A1 (en) * | 2009-11-12 | 2011-05-12 | Bally Gaming, Inc. | Video Codec System and Method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107409215A (en) * | 2015-01-29 | 2017-11-28 | 高通股份有限公司 | Pallet mode for video coding decodes |
| CN107409215B (en) * | 2015-01-29 | 2020-01-17 | 高通股份有限公司 | Palette mode coding for video coding |
| CN114128277A (en) * | 2019-08-13 | 2022-03-01 | 高通股份有限公司 | Palette predictor update for local dual trees |
| CN114128277B (en) * | 2019-08-13 | 2022-12-13 | 高通股份有限公司 | Palette Predictor Updates for Partial Dual Trees |
Also Published As
| Publication number | Publication date |
|---|---|
| BR112015025439A2 (en) | 2017-07-18 |
| JP2016521043A (en) | 2016-07-14 |
| KR102252843B1 (en) | 2021-05-14 |
| BR112015025459A2 (en) | 2017-07-18 |
| JP2016521042A (en) | 2016-07-14 |
| EP2982125A1 (en) | 2016-02-10 |
| EP2982126A1 (en) | 2016-02-10 |
| KR20150139902A (en) | 2015-12-14 |
| EP2982126B1 (en) | 2019-02-27 |
| WO2014165784A1 (en) | 2014-10-09 |
| US9654777B2 (en) | 2017-05-16 |
| US20140301475A1 (en) | 2014-10-09 |
| HUE043455T2 (en) | 2019-08-28 |
| US20140301474A1 (en) | 2014-10-09 |
| US11259020B2 (en) | 2022-02-22 |
| EP2982125B1 (en) | 2022-04-27 |
| JP6463730B2 (en) | 2019-02-06 |
| ES2727635T3 (en) | 2019-10-17 |
| WO2014165789A1 (en) | 2014-10-09 |
| KR102205328B1 (en) | 2021-01-19 |
| CN105075272B (en) | 2018-05-22 |
| KR20150140722A (en) | 2015-12-16 |
| CN105075272A (en) | 2015-11-18 |
| CN105191320B (en) | 2018-10-19 |
| BR112015025439B1 (en) | 2023-03-28 |
| BR112015025459B1 (en) | 2023-04-11 |
| JP6297671B2 (en) | 2018-03-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105191320A (en) | Determining palettes in palette-based video coding | |
| CN105556974A (en) | Palette prediction in palette-based video coding | |
| EP3158747B1 (en) | Single color palette mode in video coding | |
| JP6625644B2 (en) | Palette mode for subsampling format | |
| KR102142181B1 (en) | Restriction on palette block size in video coding | |
| CN105264891A (en) | Residual differential pulse code modulation (DPCM) extensions and harmonization with transform skip, rotation, and scans | |
| CN105247866A (en) | Infrared video display eyewear | |
| CN104335586A (en) | Motion vector rounding | |
| CN104365105A (en) | External pictures in video coding | |
| CN104471946A (en) | Unification of signaling lossless coding mode and pulse code modulation (PCM) mode in video coding | |
| CN104185990A (en) | Coding of coefficients in video coding | |
| CN104205846A (en) | View synthesis mode for three-dimensional video coding | |
| CN104737538A (en) | Performing quantization to facilitate deblocking filtering | |
| CN105075267A (en) | Disabling inter-view prediction for reference picture list in video coding | |
| CN104380748A (en) | Grouping of bypass-coded bins for SAO syntax elements | |
| CN103975595A (en) | Fragmented parameter set for video coding | |
| US9961351B2 (en) | Palette mode coding |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |