CN109889829A

CN109889829A - Fast sample adaptive compensation for 360 video

Info

Publication number: CN109889829A
Application number: CN201910094768.2A
Authority: CN
Inventors: 张萌萌; 刘志; 岳�文
Original assignee: North China University of Technology
Current assignee: Shenzhen Xiaoyu Interactive Co.,Ltd.
Priority date: 2019-01-23
Filing date: 2019-01-23
Publication date: 2019-06-14
Anticipated expiration: 2039-01-23
Also published as: CN109889829B

Abstract

A method for SAO for 360-degree video in High Efficiency Video Coding (HEVC), comprising: performing ERP projection on the 360-degree video to obtain an ERP projection video; performing intra-frame on a CTU of a current frame in the ERP projection video prediction or inter prediction to determine the optimal RD-cost; comparing the RD-cost of the CTU to a threshold to determine whether to perform SAO, wherein the threshold is based at least in part on quantization for the ERP projection video parameters and ERP projection weights, and wherein the ERP projection weights are based at least in part on the number of CTUs in the height of the ERP projection video and the position of the CTUs in the current frame of the ERP projection video definite.

Description

The point self-adapted compensation of the quick sample product of 360 degree of videos

Technical field

The present invention relates to images and field of video processing, more specifically, are related to the needle in efficient video coding (HEVC) Compensation (SAO) point self-adapted to the quick sample product of 360 degree of videos.

Background technique

In April, 2010, the two big world video encoding standard tissue VCEG and MPEG set up video compress joint group JCT- VC (Joint collaborative Team on Video Coding) develops efficient video coding HEVC (High together Efficiency video coding) standard, also referred to as H.265.HEVC standard main target is and previous generation standard H.264/AVC the raising for realizing significantly code efficiency, in particular for high-resolution video sequence.Its target is identical Code rate is reduced to the 50% of H.264 standard under video quality (PSNR).

With regard to the current stage, HEVC still continues to use the hybrid encoding frame for H.264 beginning to use.Interframe and intra prediction are compiled Code: the correlation of time-domain and spatial domain is eliminated.Transition coding: transition coding is carried out to eliminate spatial coherence to residual error.Entropy Coding: statistical redundancy is eliminated.HEVC will put forth effort to study new encoding tool or technology, mention in hybrid encoding frame High video compression efficiency.

Currently, the new features of many codings proposed in the discussion of JCT-VC tissue, it is possible to HEVC mark can be added In standard, the specific document of each discussion can be obtained from http://wftp3.itu.int.

The first edition of HEVC standard [4] is completed in the January, 2013.And in April, 2013, in October, 2014 With 4 phases of the moons in 2015 after 3 versions of publication, these versions can be obtained easily from network, and the application will be upper Three versions for stating HEVC standard are incorporated in this specification as background technique of the invention.

In HEVC, due to still using block-based hybrid encoding frame, there is still a need for processing blocking artifacts, vibration Bell effect etc..In order to reduce influence of such distortion to video quality, HEVC uses loop filtering technology (In-loop Filtering) comprising deblocking filtering (Deblocking filtering) and pixel samples adaptive equalization (Dample Adaptive Offset, SAO).SAO is one of many new technologies of HEVC [5].As shown in Figure 1, SAO is located at deblocking filtering After device.SAO is classified and is counted, offset value calculation to each pixel of each coding tree unit (CTU), and selection is best SAO type, and will be in SAO type and offset write-in code stream.Then, deviant is added to each pixel of reconstructed frame, To reduce the distortion between reconstructed frame and primitive frame.SAO significant can improve subjective and objective video quality [5].SAO is mainly by three It is grouped as: statistical collection, SAO type decision and SAO filtering, as shown in Figure 2.

Statistical collection: there are mainly two types of the offset types for needing statistical collection process by SAO: boundary compensation (EO) and sideband It compensates (BO).For EO type, there are four types of EO subtype (0 ° of EO, 90 ° of EO, 45 ° of 135 ° of EO and EO).According to classifying rules Classify to each EO subtype, and calculates the pixel number in each classification and distortion summation.For BO type, image pixel intensities 32 sidebands are divided into, and are classified according to classifying rules to 32 sidebands, and calculate the pixel in each classification Number and distortion summation.The classifying rules of EO and BO is as shown in Figure 2.

SAO type decision: four kinds of SAO types: EO, BO, OFF and MERGE be can choose, wherein OFF expression is not applied SAO realizes that MERGE indicates that SAO parameter directly makes for a block in video code flow by a switch parameter Which with the SAO of the block of top or left side, at this moment only need to identify the SAO parameter for using adjacent block.It is received according to statistics The information of collection, SAO type decision calculate the optimal compensation of each SAO type by quick rate-distortion optimization (RDO) process [6] Value, and select optimal SAO type.

SAO filtering: each pixel of CTU is classified and is compensated according to the best SAO type of acquisition and deviant.

Fig. 2 shows that SAO process consists of three parts: statistical collection, SAO type decision and SAO filtering.[16] it has studied The computation complexity of each section.The result shows that statistical collection accounts for about 82%, the SAO type decision and SAO of SAO total processing time Filtering is respectively 11% and 7%.Complicated statistical collection process is to restrict the principal element of SAO processing speed.

In virtual reality system, multiple cameras are spliced balling-up for capturing 360 degree of scenes, the scene then shot 360 degree of videos of shape format.Any scene that user can freely be watched in 360 degree of scenes by headset equipment is shown (HMD) and experience on the spot in person [1] is obtained.360 degree of videos are a kind of new video coding contents.Although 360 degree of videos are Just popular and 360 degree of videos are spherical videos after HEVC standard proposition, and still [2] have been proposed under HEVC standard 360 degree of video coding frameworks.In typical 360 degree of video compress frames, spherical video needs to be converted to before encoding flat Plane video, planar video need to be converted to spherical video [3] in encoded.Conversion regime is known as projecting.It has been proposed a variety of Projection format, for example, rectangular projections (ERP) is waited, the equivalent projection (AEP) of adjustment, cubic projection (CMP), isogonism cube Figure projection (EAC), truncation square pyramid projection (TSP), compact octahedra projection (COHP), compact icosahedron projection (CISP) etc..When selecting ERP as projection format, the cataloged procedure of 360 degree of videos includes: that original video is projected as ERP Projection format, and encoding and decoding are executed to ERP projection video, it is to rebuild by the reconstruction video of ERP projection format again back projection Video.Projection process is essential for 360 degree of Video codings.Projection format as intermediate form influences 360 degree of views The coding efficiency of frequency.In fact, not yet determining which kind of projection format has forced coding performance at present.But ERP is made extensively With being the default form of 360 degree of videos.Therefore, the characteristics of mainly studying ERP projection format herein.

Compared with planar video, 360 degree of videos have different characteristics, the optimal parameter of existing SAO fast algorithm and Process is not suitable for 360 degree of videos.In this application, the characteristics of being based on 360 degree of videos proposes a kind of for 360 degree of videos Quick SAO algorithm.

The application is the improvement to existing HEVC agreement, in order to enable those skilled in the art can fully understand this hair It is bright, the citation of a variety of concepts referred in the application is appended below, these documents are incorporated herein on the whole and conduct A part of present specification.

1.B.Luo, F.Xu, C.Richardt and J.Yong, " 360 ° of Parallax360:Stereoscopic Scene Representation for Head-Motion Parallax, " in IEEE Transactions on Visualization and Computer Graphics, vol.24, no.4, pp.1545-1553, April 2018.

2.Y.Y, E.Alshina, J.Boyce, " Algorithm descriptions of projection format Conversion and video quality metrics in 360Lib ", Joint Video Exploration Team 16 WP of of ITU-T SG 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-H1004,7th Meeting, July 2017.

3.W.Zou, F.Yang and S.Wan, " Perceptual video quality metric for Compression artefacts:from two-dimensional to omnidirectional, " in IET Inage Processing, vol.12, no.3, pp. 374-381,3 2018.

4.Sullivan, Gary J., et al. " Overview of the high efficiency video Coding (HEVC) standard. " Circuits and Systems for Video Technology, IEEE Transactions on 22.12 (2012): 1649-1668.

5.C.-M.Fu, E.Alshina, A.Alshin, Y.-W.Huang, C.-Y.Chen, C.-Y.Tsai, C.-W.Hsu, S.-M. Lei, J.-H.Park, and W.-J.Han, " Sample adaptive offset in the HEVC Standard, " Circuits and Systems for Video Technology, IEEE Transactions on, Vol.22, no.12, pp.1755-1764,2012.

6.Zhang M, Bai H, Lin C, et al.Texture Characteristics Based Fast Coding Unit Partition in HEVC Intra Coding, Data Compression Conference.IEEE, 2015: 477-477.

7.Z.Zhengyong, C.Zhiyun and P.Peng, " A fast SAO algorithm based on Coding unit partition for HEVC, " 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, 2015, pp.392-395.

8.J.Joo, Y.Choi and K.Lee, " Fast sample adaptive offset encoding Algorithm for HEVC based on intra prediction mode, " 2013 IEEE Third International Conference on Consumer Electronics Berlin (ICCE-Berlin), Berlin, 2013, pp.50-53.

9.T.Y.Kuo, H.Chiu and F.Amirul, " Fast sample adaptive offset encoding For HEVC, " 2016 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Nantou, 2016, pp.1-2.

10.S.Yin, X.Zhang and Z.Gao, " Efficient SAO coding algorithm for x265 Encoder, " 2015 Visual Communications and Image Processing (VCIP), Singapore, 2015, pp.1-4.

11.S.E.Gendy, A.Shalaby and M.S.Sayed, " Fast parameter estimation Algorithm for sample adaptive offset in HEVC encoder, " 2015 Visual Communications and Image Processing (VCIP), Singapore, 2015, pp.1-4.

12.K.Yang, S.Wan, Y.Gong, Y.Yang and Y.Feng, " Fast sample adaptive offset For H.265/HEVC based on temporal dependency, " 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Jeju, 2016, pp.1-4.

13.Sungjei Kim, Jinwoo Jeong, Jeong-Mee Moon and Yong-HWan Kim, " Fast sample adaptive offset parameter estimation algorithm based on early Termination for HEVC encoder, " 2017 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, 2017, pp.241-242.

14.W.Zhang and C.Guo, " Design and implementation of parallel Algorithms for sample adaptive offset in HEVC based on GPU, " 2016 Sixth International Conference on Information Science and Technology (ICIST), Dalian, 2016, pp.181-187.

15.Y.Wang, X.Guo, Y.Lu, X.Fan and D.Zhao, " GPU-based optimization for Sample adaptive offset in HEVC, " 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, 2016, pp.829-833.

16.Y.Choi and J.Joo, " Exploration of Practical HEVC/H.265 Sample Adaptive offset Encoding Policies, " in IEEE Signal Processing Letters, vol.22, No.4, pp.465-468, April 2015.

17.Y.Li, J.Xu and Z.Chen, " Spherical domain rate-distortion Optimization for 360-degree video coding, " 2017 IEEE International Conference On Multimedia and Expo (ICME), Hong Kong, 2017, pp.709-714.

18.Y.Sun and L.Yu, " Coding optimization based on weighted-to- Spherically-uniform quality metric for 360 video, " 2017 IEEE Visual Communications and Image Processing (VCIP), St.Petersburg, FL, 2017, pp.1-4.

19.Jill Boyce, Elena Alshina, Adeel Abbas, " JVET common test conditions And evaluation procedures for 360 ° of video ", Joint Video Exploration Team (JVET) of 16 WP of ITU-T SG 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-H1030,8th Meeting, Oct.2017.

20.X.Xiu, Y.He, Y.Ye and B.Vishwanath, " An evaluation framework for 360- Degtee video compression, " 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, 2017, pp.1-4.

21.H.Bai, C.Zhu and Y.Zhao, " Optimized Multiple Description Lattice Vector Quantization for Wavelet Image Coding, " in IEEE Transactions on Circuits and Systems for Video Technology, vol.17, no.7, pp.912-917, July 2007.

22.C.Yeh, Z.Zhang, M.Chen and C.Lin, " HEVC Intra Frame Coding Based on Convolutional Neural Network, " in IEEE Access, vol.6, pp.50087-50095,2018.

23.L.Chang, Z.Liu, L.Wang and X.Li, " Enhance the HEVC Fast Intra CU Mode Decision Based on Convolutional Neural Network by Corner Power Estimation, " 2018 Data Compression Conference, Snowbird, UT, 2018, pp.400-400.

24.T.Katayama, K.Kuroda, W.Shi, T.Song and T.Shimamoto, " Low-complexity Intra coding algorithm based on convolutional neural network for HEVC, " 2018 International Conference on Information and Computer Technologies (ICICT), DeKalb, IL, 2018, pp. 115-118.

25.G.Bjontegaard, " Calculation of average PSNR differences between RD- Curves, " ITU-T VCEG-M33,2001.

Summary of the invention

The present invention is directed to the characteristic of 360 degree of videos, proposes a kind of quick SAO method for 360 degree of videos.It is mentioned The SAO process of algorithm improvement out on the basis of retaining entire SAO process increases simplified SAO process.By being based on After the SAO of threshold value executes decision, simplified SAO process can be used, this will greatly reduce the time of statistic gathering, thus Reduce the computation complexity of SAO.

According to an aspect of the present invention, it proposes one kind to be directed to 360 degree of videos in efficient video coding (HEVC) The method of sampling point adaptive equalization (SAO), this method comprises:

Projection is executed to 360 degree of videos, to obtain projection video；

Intra prediction or inter-prediction are executed to the coding tree unit (CTU) of the present frame in the ERP projection video, with Determine best RD-cost；

The RD-cost of the CTU is compared with threshold value to judge whether to execute SAO,

Wherein, the threshold value is at least partially based on quantization parameter and projection weight for the projection video come really Fixed, and wherein, the projection weight is at least partially based on CTU quantity and institute in the height of the projection video Position of the CTU in the present frame of the projection video is stated to determine.

According to a further aspect of the present invention, the method further includes: if it is determined that not executing SAO, then at least Not to CTU exercise boundary compensation (EO) and sideband compensation (BO)；If it is determined that executing SAO, then executed for the CTU One of OFF or MERGE operation.

According to a further aspect of the present invention, wherein the threshold value is based at least partially at least one of following: described Project weight with 2 for the e of the logarithm or quantization parameter at bottom power, or combinations thereof.

According to a further aspect of the present invention, wherein only for the top 1/4 and lower section 1/4 in the projection video Height, the RD-cost of the CTU is compared with threshold value to judge whether to execute SAO.

A kind of efficient video coding (HEVC) hardware coder according to another aspect, is suitable for being directed to 360 degree of videos Sampling point adaptive equalization (SAO), the encoder are configured as executing the above method.

The invention proposes one kind to using method as mentioned or encoder as mentioned to compile according to another aspect, The decoder that 360 video flowings of code are decoded.

The invention proposes a kind of for executing the computer program product of the above method according to another aspect,.

According to another aspect, the invention proposes a kind of equipment that can be used for coding and decoding video, the equipment include: one or Multiple processors；Memory, wherein being stored with computer code, the computer code is real when being executed by the processor The existing above method.

It is described according to another aspect, to be projected as waiting rectangular projections (ERP).

Detailed description of the invention

Fig. 1 shows one embodiment of the encoder block diagram of HEVC.

Fig. 2 shows the brief block diagrams of the SAO in HEVC.

Fig. 3 shows the weight distribution of ERP projection.

Fig. 4 shows the method flow diagram of the various aspects according to present disclosure.

Fig. 5 shows the schematic diagram of the equipment for coding and decoding video of the various aspects according to present disclosure.

Specific embodiment

Various schemes are described referring now to the drawings.In the following description, it in order to explain, elaborates multiple specific thin Section is in order to provide the thorough understanding to one or more schemes.It may be evident, however, that also can without these specific details Enough realize these schemes.

As used in this specification, term " component ", " module ", " system " etc. are intended to refer to related to computer Entity, such as, but not limited to, hardware, firmware, the combination of hardware and software, software or software in execution.For example, Component can be but not limited to: process, processor, object, the executable (executable), execution run on a processor Thread, program, and/or computer.For example, the application program and the calculating equipment run on the computing device is ok It is component.One or more components can be located in executive process and/or execution thread, and component can be located at a meter On calculation machine and/or it is distributed on two or more computers.In addition, these components can from have be stored thereon it is each The various computer-readable mediums of kind data structure execute.Component can be communicated by means of locally and/or remotely process, example If basis has the signal of one or more data groupings, for example, from by means of signal and local system, distributed system In another component interaction and/or with interacted by means of signal with other systems on the network of such as internet etc one The data of a component.

The present invention is directed to the characteristic of 360 degree of videos, proposes a kind of quick SAO method for 360 degree of videos.It is mentioned The SAO process of algorithm improvement out on the basis of retaining entire SAO process increases simplified SAO process.By being based on After the SAO of threshold value executes decision, it can choose and execute routine SAO process as defined in HEVC or select not executing SAO or only hold Row MERGE processing, to realize simplified SAO process, this will greatly reduce the time of statistic gathering, to reduce The computation complexity of SAO.

I. algorithm is summarized

Weight (weight) in (1) 360 degree of video

360 degree of videos are a kind of spherical videos, are that 360 degree of videos and the maximum of conventional video are distinguished.In order to be marked in HEVC Standard is lower to encode 360 degree of videos, it is necessary to project to 360 degree of videos in planar video.Although projection video and conventional video are all Planar video, but projection video has the distortion and stretching of spherical video.Therefore, the evaluating objective quality index of conventional video PSNR is not suitable for projection video.WS-PSNR is proposed as the objective quality assessment index [18] of projection video.WS-PSNR is Projection video designs weight, and projection video is smaller in the weight of distortion and stretch zones, and vice versa, then passes through weighted average Method calculates WS-PSNR.WS-PSNR is commented by the objective quality that joint video exploration team (JVET) is recognized as 360 degree of video qualities Estimate index.Therefore, weight is the maximum difference between projection video and conventional video.

Wherein (i, j) indicates that location of pixels, height indicate the height of video.Fig. 3 shows ERP weight distribution.Color It is deeper, closer to 0.Color is more shallow, closer to 1.Region0 is defined as the two poles of the earth near zone, and weight is small；Region1 is defined as The region of equator, weight are big.As shown in figure 3, Region0 includes upper 1/4 region and lower 1/4 region of video； Region1 represents 1/2 region of centre of video.

(2)RD-cost

In HEVC, through-rate aberration optimizing (RDO) [17] recursively calculates intra prediction and frame in HEVC standard Between the optimum prediction mode predicted and best CU divide.

J=D+ λ R (2)

Wherein D indicates that the distortion in current prediction mode, R expression encode all information in current prediction mode Required bit number, λ is Lagrange factor, and J indicates Lagrangian cost (RD-cost).RD-cost is smaller, prediction The code efficiency of mode is higher, and RD-cost is bigger, and the code efficiency of prediction mode is lower.

(3) algorithm proposed

In one embodiment of the invention, setting threshold value (Threshold) is to predefine whether execute SAO process. SAO process is executed as RD-cost > Threshold does not otherwise execute SAO process.By being skipped at SAO based on threshold value Reason, can greatly reduce encoding calculation amount.

In one embodiment of the invention, it is used to determine whether that the threshold value for executing SAO process is at least partially based on It is determined for quantization parameter and ERP the projection weight of 360 degree of videos.

In one embodiment of the invention, quantization parameter has influence for the judgement for whether executing SAO.According to experiment Statistics, with the increase of quantization parameter, CTU does not apply the probability of SAO to increase；With the increase of quantization parameter, same CTU's RD-cost also increases.Therefore, quantization parameter is considered in threshold value, whether threshold value that can be set is to executing SAO and sentence It is fixed more accurate.

In the embodiment of an example and not restrictive, it may be considered that the power of the e of quantization parameter.

In one embodiment of the invention, the weight of projection video has influence for the judgement for whether executing SAO.By It is projected in ERP used herein, therefore the weight is also referred to as ERP projection weight.Under study for action, inventor notices pair In the weight of ERP projection, in Region0 the weight of CTU close to 0, therefore in Region0 CTU distortion to final video quality It influences little.Therefore, consider that the weight of projection video has met the projection of ERP projection in the judgement for whether executing SAO The characteristics of video, so that the judgement not will cause the decline of coded video quality.

In the embodiment of an example of the invention and not restrictive, inventor proposes following weight set-up mode:

Wherein (x, y) indicates the position of CTU, and m is the CTU quantity in video height, and weight (x, y) indicates the institute of CTU There is the average weight of pixel.Therefore, the weight of different location is different, and the distortion of CTU is to video quality in Region1 Influence will influence of the distortion to video quality much smaller than CTU in Region0.Therefore, the CTU in Region1 is carried out detailed Thin decision simultaneously carries out a kind of method that rough decision is adaptation projection video characteristic to the CTU in Region0.

In one embodiment of the invention, inventor is according to the weight modification threshold value of different latitude (correspond to y value).Hair Bright people usesTo indicate the scale factor of different latitude.Latitude is bigger, and the factor is bigger.

In the embodiment of an example and not restrictive, threshold value setting as shown in the following Table 1 is given.

Table 1.ERP projection video improved threshold value at different Q P

Wherein, α and 1- α respectively indicates the percentage of fixed threshold and variable thresholding, and QP is quantization parameter.

In one embodiment of the invention, inventor considers the power of the e of quantization parameter.For example, in another example and In non-limiting embodiment, threshold value can determine as follows:

In one embodiment of the invention, as described above, inventor notices the distortion pair of the CTU in Region0 The influence of video quality will be much smaller than influence of the distortion of CTU in Region1 to video quality.Therefore in order to reduce by SAO Ensure video quality while caused calculation amount, it can be only for the Region0 in ERP projection video (i.e. above present frame 1/4 and lower section 1/4 height) Lai Jinhang aforesaid operations, and to the Region1 region of 1/2 height (present frame among) still into The processing of SAO as defined in row HEVC agreement.

In one embodiment of the invention, since the calculation amount of MERGE operation is smaller, aforesaid operations are being carried out When, it may be considered that when meeting threshold condition (being, for example, less than threshold value), executes MERGE operation or do not execute SAO operation (OFF Operation).

Fig. 4 shows the method flow diagram of the various aspects according to present disclosure.This method is used to compile in efficient video The sampling point adaptive equalization (SAO) of 360 degree of videos is directed in code (HEVC).

According to one embodiment, this method comprises: to rectangular projections (ERP) such as 360 degree of videos execution, to obtain ERP projection video.It should be readily apparent to one skilled in the art that other projecting methods other than ERP can be executed, specifically Projecting method is not focus of the invention.

According to another embodiment, this method further include: to the coding tree unit of the present frame in the ERP projection video (CTU) intra prediction or inter-prediction are executed, with the best RD-cost of determination.

According to another embodiment, this method further include: being compared the RD-cost of CTU with threshold value to judge whether Execute SAO, wherein the threshold value be at least partially based on for ERP projection video quantization parameter and ERP projection weight come Determining, and wherein, the ERP projection weight is at least partially based on the CTU number in the height of the ERP projection video Amount and CTU position in the present frame of the ERP projection video determines.

According to another embodiment, if it is decided that do not execute SAO, then at least do not compensate (EO) to the CTU exercise boundary (BO) is compensated with sideband；If it is determined that and execute SAO, then for the CTU execute OFF or MERGE operation one of.

According to another embodiment, the threshold value is based at least partially at least one of following: the ERP projects weight With 2 for the e of the logarithm or quantization parameter at bottom power, or combinations thereof.

It will be described only for the height of top 1/4 and lower section 1/4 in ERP projection video according to another embodiment The RD-cost of CTU is compared to judge whether to execute SAO with threshold value.

Fig. 5 shows the schematic diagram of the equipment for coding and decoding video of the various aspects according to present disclosure.Such as Fig. 5 Shown, which may include one or more processors and memory, and computer code is stored in the memory, described Computer code is realized when being executed by the processor and is directed to 360 in efficient video coding (HEVC) as described herein Spend the method for the sampling point adaptive equalization (SAO) of video.

The encoder that present disclosure can also relate to for realizing above-mentioned coding method according to another aspect,.The coding Device can be specialized hardware.

Present disclosure can also relate to what corresponding 360 video flowings to after coding were decoded according to another aspect, Decoder.

Present disclosure can also relate to execute the computer program product of methods described herein according to another aspect,.

When implemented in hardware, video encoder can use general processor, digital signal processor (DSP), dedicated collection At circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device Part, discrete hardware components are designed as executing any combination thereof of function described herein, Lai Shixian or execution.General processor It can be microprocessor, but alternatively, which is also possible to any conventional processor, controller, microcontroller Or state machine.Processor also can be implemented as calculating the combination of equipment, for example, the combination of DSP and microprocessor, multiple micro- places Manage the combination of device, the combination or any other such structure of one or more microprocessors and DSP core.In addition, at least one A processor may include that can operate to execute one or more modules of above-mentioned one or more steps and/or operation.

When with hardware circuits such as ASIC, FPGA to realize video encoder, may include be configured as executing it is various The various circuit blocks of function.Those skilled in the art can be according to the various constraint conditions applied over the whole system come with various These circuits of mode design and implementation, to realize various functions disclosed in this invention.

Although aforementioned open file discusses exemplary arrangement and/or embodiment, it should be noted that being wanted without departing substantially from by right In the case where the scheme for the description for asking book to define and/or the range of embodiment, many change and modification can be made herein.And And although in the singular describe or require the scheme and/or embodiment element, it is also contemplated that plural number feelings Condition is limited to odd number unless expressly stated.In addition, all or part of any scheme and/or embodiment can with it is any its Its scheme and/or all or part of of embodiment are used in combination, unless showing different.

Claims

1. A method of sample adaptive compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC), comprising:

performing projection on the 360-degree video to obtain a projection video;

performing intra-frame prediction or inter-frame prediction on the coding tree unit (CTU) of the current frame in the projected video to determine the best RD-cost;

comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO,

wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projected video, and wherein the projection weight is based at least in part on the number of CTUs in the height of the projected video and the The position of the CTU in the current frame of the projected video is determined.

2. The method of claim 1, further comprising:

If it is determined not to perform SAO, at least not perform boundary compensation (EO) and sideband compensation (BO) for the CTU;

If it is determined to perform SAO, one of OFF or MERGE operations is performed for the CTU.

3. The method of claim 1 or 2, wherein the threshold is based, at least in part, on at least one of: the base-2 logarithm of the projection weight, or the quantization parameter raised to a power of e, or a combination thereof.

4. The method of any one of claims 1-3, wherein the RD-cost of the CTU is compared with a threshold for only the upper 1/4 and lower 1/4 heights in the projected video. Compare to determine whether to perform SAO.

5. A High Efficiency Video Coding (HEVC) hardware encoder adapted for Sample Adaptive Compensation (SAO) for 360-degree video, the encoder being configured to:

performing projection on the 360-degree video to obtain a projection video;

6. A decoder for decoding a 360 video stream encoded using the method of any of claims 1-4 or the encoder of claim 5.

7. A computer program product for performing the method of any of claims 1-4.

8. A device that can be used for video encoding and decoding, the device comprising:

one or more processors;

a memory having stored therein computer code that, when executed by the processor, implements a method of Sample Adaptive Compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC), the method comprising :

performing projection on the 360-degree video to obtain a projection video;

performing intra-frame prediction or inter-frame prediction on the coding tree unit (CTU) of the current frame in the ERP projection video to determine the best RD-cost;

9. The method of any of claims 1-4, wherein the projection is an equirectangular projection (ERP).