CN109889829B - Fast sample adaptive compensation for 360 degree video - Google Patents

Abstract

A method for SAO for 360-degree video in High Efficiency Video Coding (HEVC), comprising: performing ERP projection on the 360-degree video to obtain an ERP projection video; performing intra-frame on a CTU of a current frame in the ERP projection video prediction or inter prediction to determine the optimal RD-cost; comparing the RD-cost of the CTU to a threshold to determine whether to perform SAO, wherein the threshold is based at least in part on quantization for the ERP projection video parameters and ERP projection weights, and wherein the ERP projection weights are based at least in part on the number of CTUs in the height of the ERP projection video and the position of the CTUs in the current frame of the ERP projection video definite.

Description

Fast sample adaptive compensation for 360 video

technical field

The present invention relates to the field of image and video processing, and more particularly, to fast sample adaptive compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC).

Background technique

In April 2010, the two major international video coding standards organizations, VCEG and MPEG, established a joint video compression group, JCT-VC (Joint collaborative Team on Video Coding), to jointly develop the HEVC (Highefficiency video coding) standard for high-efficiency video coding, also known as H .265. The main goal of the HEVC standard is to achieve a substantial improvement in coding efficiency with the previous generation standard H.264/AVC, especially for high-resolution video sequences. The goal is to reduce the bit rate to 50% of the H.264 standard at the same video quality (PSNR).

At the current stage, HEVC still uses the hybrid coding framework adopted by H.264. Inter and Intra Predictive Coding: De-correlation between temporal and spatial domains. Transform coding: Transform coding the residuals to remove spatial correlations. Entropy coding: Eliminate statistical redundancy. HEVC will focus on researching new coding tools or technologies within the hybrid coding framework to improve video compression efficiency.

At present, many new features of coding that have been proposed in the discussions organized by JCT-VC may be added to the HEVC standard. The specific documents of each discussion can be obtained from http://wftp3.itu.int.

The first version of the HEVC standard [4] was completed in January 2013. And three versions were successively released in April 2013, October 2014 and April 2015, these versions can be easily obtained from the Internet, and this application incorporates the three versions of the above HEVC standard into this specification as the background art of the present invention.

In HEVC, since the block-based hybrid coding framework is still used, there is still a need to deal with blocking, ringing, etc. In order to reduce the influence of such distortion on video quality, HEVC adopts in-loop filtering technology, which includes deblocking filtering (Deblocking filtering) and pixel sample adaptive compensation (Dample Adaptive Offset, SAO). SAO is one of many new technologies for HEVC [5]. As shown in Figure 1, SAO is located after the deblocking filter. The SAO classifies and counts each pixel of each coding tree unit (CTU), calculates the compensation value, selects the best SAO type, and writes the SAO type and the compensation value into the code stream. Then, an offset value is added to each pixel of the reconstructed frame to reduce the distortion between the reconstructed frame and the original frame. SAO can significantly improve the subjective and objective video quality [5]. SAO mainly consists of three parts: statistics collection, SAO type decision and SAO filtering, as shown in Figure 2.

Statistics Collection: SAO has two main types of excursions that require a statistics collection process: boundary compensation (EO) and sideband compensation (BO). For the EO type, there are four EO subtypes (EO 0°, EO 90°, EO 135° and EO 45°). Each EO subtype is classified according to the classification rules and the sum of the number of pixels and distortion in each class is calculated. For the BO type, the pixel intensities were equally divided into 32 sidebands, and the 32 sidebands were classified according to the classification rule, and the number of pixels and the sum of distortions in each class were calculated. The classification rules for EO and BO are shown in Figure 2.

SAO type decision: Four SAO types can be selected: EO, BO, OFF and MERGE, where OFF means that SAO is not applied, which is implemented by a switch parameter in the video stream, and MERGE means that for a block, its SAO parameter is directly To use the SAO of the block above or to the left, it is only necessary to identify which SAO parameter of the adjacent block is used. Based on the statistically collected information, the SAO type decision calculates the optimal compensation value for each SAO type through a rapid rate-distortion optimization (RDO) process [6], and selects the optimal SAO type.

SAO filtering: Classifies and compensates each pixel of the CTU according to the obtained best SAO type and offset value.

Figure 2 shows that the SAO process consists of three parts: statistics collection, SAO type decision and SAO filtering. [16] studied the computational complexity of each part. The results show that statistics collection accounts for about 82% of the total processing time of SAO, and SAO type decision and SAO filtering account for 11% and 7%, respectively. The complex statistical collection process is the main factor restricting the processing speed of SAO.

In a virtual reality system, multiple cameras are used to capture a 360-degree scene, which is then stitched into a spherical format 360-degree video. Users can freely watch any scene display (HMD) in a 360-degree scene through a head-mounted device and get an immersive experience [1]. 360-degree video is a new video encoding content. Although 360-degree video became popular after the HEVC standard was proposed and 360-degree video is spherical video, [2] has proposed a 360-degree video coding framework under the HEVC standard. In a typical 360-degree video compression framework, spherical video needs to be converted to flat video before encoding, and flat video needs to be converted to spherical video after encoding [3]. The way of transformation is called projection. Various projection formats have been proposed, such as equirectangular projection (ERP), adjusted equal area projection (AEP), cube projection (CMP), equirectangular projection (EAC), truncated square pyramid projection (TSP), compact The octahedral projection (COHP), the compact icosahedral projection (CISP), etc. When ERP is selected as the projection format, the encoding process of the 360-degree video includes: projecting the original video into the ERP projection format, performing encoding and decoding on the ERP projection video, and re-projecting the reconstructed video in the ERP projection format into the reconstructed video. The projection process is essential for 360-degree video encoding. The projection format as an intermediate format affects the encoding performance of 360 video. In fact, it has not yet been determined which projection format has the best encoding performance. However, ERP is widely used and is the default format for 360-degree video. Therefore, this paper mainly studies the characteristics of ERP projection format.

Compared with flat videos, 360-degree videos have different characteristics, and the optimal parameters and procedures of existing SAO fast algorithms are not suitable for 360-degree videos. In this application, based on the characteristics of 360-degree video, a fast SAO algorithm for 360-degree video is proposed.

This application is an improvement to the existing HEVC protocol. In order to enable those skilled in the art to fully understand the present invention, citations for various concepts mentioned in this application are attached below, which are incorporated herein in their entirety and incorporated herein by reference. as part of the specification of this application.

1.B.Luo, F.Xu, C.Richardt and J.Yong, "Parallax360: Stereoscopic 360°Scene Representation for Head-Motion Parallax," in IEEE Transactions onVisualization and Computer Graphics, vol.24, no.4, pp .1545-1553, April 2018.

2.Y.Y, E.Alshina, J.Boyce, "Algorithm descriptions of projection formatconversion and video quality metrics in 360Lib", Joint Video Exploration Teamof ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-H1004, 7th Meeting, July2017.

3. W. Zou, F. Yang and S. Wan, "Perceptual video quality metric for compression artefacts: from two-dimensional to omnidirectional," in IET InageProcessing, vol.12, no.3, pp.374-381, 3 2018 .

4. Sullivan, Gary J., et al. "Overview of the high efficiency videocoding (HEVC) standard." Circuits and Systems for Video Technology, IEEE Transactions on 22.12 (2012): 1649-1668.

5.C.-M.Fu, E.Alshina, A.Alshin, Y.-W.Huang, C.-Y.Chen, C.-Y.Tsai, C.-W.Hsu, S.-M. Lei, J.-H.Park, and W.-J.Han, "Sample adaptive offset in the HEVCstandard," Circuits and Systems for Video Technology, IEEE Transactions on, vol.22, no.12, pp.1755-1764 , 2012.

6. Zhang M, Bai H, Lin C, et al. Texture Characteristics Based Fast CodingUnit Partition in HEVC Intra Coding, Data Compression Conference. IEEE, 2015: 477-477.

7.Z.Zhengyong, C.Zhiyun and P.Peng, "A fast SAO algorithm based on coding unit partition for HEVC," 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS), Beijing, 2015, pp.392-395 .

8. J. Joo, Y. Choi and K. Lee, "Fast sample adaptive offset encoding algorithm for HEVC based on intra prediction mode," 2013 IEEE Third International Conference on Consumer Electronics Berlin (ICCE-Berlin), Berlin, 2013, pp.50 -53.

9. T.Y.Kuo, H.Chiu and F.Amirul, "Fast sample adaptive offset encoding for HEVC," 2016 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Nantou, 2016, pp.1-2.

10. S. Yin, X. Zhang and Z. Gao, "Efficient SAO coding algorithm for x265encoder," 2015 Visual Communications and Image Processing (VCIP), Singapore, 2015, pp.1-4.

11. S.E.Gendy, A.Shalaby and M.S.Sayed, "Fast parameter estimation algorithm for sample adaptive offset in HEVC encoder," 2015 Visual Communications and Image Processing (VCIP), Singapore, 2015, pp.1-4.

12. K. Yang, S. Wan, Y. Gong, Y. Yang and Y. Feng, "Fast sample adaptive offset for H.265/HEVC based on temporal dependency," 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), Jeju, 2016, pp.1-4.

13. Sungjei Kim, Jinwoo Jeong, Jeong-Mee Moon and Yong-HWan Kim, "Fastsample adaptive offset parameter estimation algorithm based on earlytermination for HEVC encoder," 2017 IEEE International Conference on Consumer Electronics (ICCE), Las Vegas, NV, 2017, pp.241-242.

14. W. Zhang and C. Guo, "Design and implementation of parallelalgorithms for sample adaptive offset in HEVC based on GPU," 2016 Sixth International Conference on Information Science and Technology (ICIST), Dalian, 2016, pp.181-187.

15.Y.Wang, X.Guo, Y.Lu, X.Fan and D.Zhao, "GPU-based optimization forsample adaptive offset in HEVC," 2016 IEEE International Conference on ImageProcessing(ICIP), Phoenix, AZ, 2016, pp.829-833.

16. Y. Choi and J. Joo, "Exploration of Practical HEVC/H.265 SampleAdaptive offset Encoding Policies," in IEEE Signal Processing Letters, vol. 22, no. 4, pp. 465-468, April 2015.

17. Y. Li, J. Xu and Z. Chen, "Spherical domain rate-distortion optimization for 360-degree video coding," 2017 IEEE International Conferenceon Multimedia and Expo (ICME), Hong Kong, 2017, pp.709-714.

18. Y. Sun and L. Yu, "Coding optimization based on weighted-to-spherically-uniform quality metric for 360 video," 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, 2017, pp.1 -4.

19. Jill Boyce, Elena Alshina, Adeel Abbas, “JVET common test conditions and evaluation procedures for 360° video”, Joint Video Exploration Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11 , JVET-H1030, 8th Meeting, Oct.2017.

20. X. Xiu, Y. He, Y. Ye and B. Vishwanath, "An evaluation framework for 360-degtee video compression," 2017 IEEE Visual Communications and ImageProcessing (VCIP), St. Petersburg, FL, 2017, pp. 1-4.

21. H. Bai, C. Zhu and Y. Zhao, "Optimized Multiple Description LatticeVector Quantization for Wavelet Image Coding," in IEEE Transactions on Circuits and Systems for Video Technology, vol.17, no.7, pp.912-917, July 2007.

22. C. Yeh, Z. Zhang, M. Chen and C. Lin, "HEVC Intra Frame Coding Based on Convolutional Neural Network," in IEEE Access, vol.6, pp.50087-50095, 2018.

23. L. Chang, Z. Liu, L. Wang and X. Li, "Enhance the HEVC Fast Intra CU ModeDecision Based on Convolutional Neural Network by Corner Power Estimation," 2018 Data Compression Conference, Snowbird, UT, 2018, pp. 400-400.

24. T. Katayama, K. Kuroda, W. Shi, T. Song and T. Shimamoto, "Low-complexityintra coding algorithm based on convolutional neural network for HEVC," 2018International Conference on Information and Computer Technologies (ICICT), DeKalb, IL, 2018, pp.115-118.

25. G. Bjontegaard, "Calculation of average PSNR differences between RD-curves," ITU-T VCEG-M33, 2001.

SUMMARY OF THE INVENTION

Aiming at the characteristics of 360-degree video, the present invention proposes a fast SAO method for 360-degree video. The proposed algorithm improves the SAO process by adding a simplified SAO process on the basis of retaining the entire SAO process. After the threshold-based SAO execution decision, a simplified SAO process can be used, which will greatly reduce the time for statistical data collection and thus reduce the computational complexity of SAO.

According to one aspect of the present invention, a method for sample adaptive compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC) is proposed, the method comprising:

performing projection on the 360-degree video to obtain a projection video;

performing intra-frame prediction or inter-frame prediction on the coding tree unit (CTU) of the current frame in the ERP projection video to determine the best RD-cost;

comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO,

wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projected video, and wherein the projection weight is based at least in part on the number of CTUs in the height of the projected video and the The position of the CTU in the current frame of the projected video is determined.

According to a further aspect of the present invention, the method further comprises: if it is determined not to perform SAO, at least not performing boundary compensation (EO) and sideband compensation (BO) on the CTU; if it is determined to perform SAO, The CTU performs one of the OFF or MERGE operations.

According to a further aspect of the invention, wherein the threshold is based, at least in part, on at least one of: the base-2 logarithm of the projection weight, or the quantization parameter raised to a power of e, or a combination thereof.

According to a further aspect of the present invention, the RD-cost of the CTU is compared with a threshold to determine whether to perform SAO only for the upper 1/4 and lower 1/4 heights in the projected video.

According to another aspect, a High Efficiency Video Coding (HEVC) hardware encoder adapted for sample adaptive compensation (SAO) for 360 degree video, the encoder configured to perform the above method.

According to another aspect, the present invention proposes a decoder for decoding a 360 video stream encoded using the method as described or the encoder as described.

According to another aspect, the present invention proposes a computer program product for performing the above method.

According to another aspect, the present invention proposes a device usable for video encoding and decoding, the device comprising: one or more processors; a memory in which computer code is stored, the computer code when executed by the processor , to implement the above method.

According to another aspect, the projection is an equirectangular projection (ERP).

Description of drawings

Figure 1 shows one embodiment of an encoder block diagram for HEVC.

Figure 2 shows a simplified block diagram of SAO in HEVC.

Figure 3 shows the weight distribution of ERP projections.

4 illustrates a method flow diagram in accordance with various aspects of the present disclosure.

5 shows a schematic diagram of an apparatus for video encoding and decoding according to various aspects of the present disclosure.

Detailed ways

Various schemes are now described with reference to the figures. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. Obviously, however, these schemes can be implemented without these specific details.

As used in this application, the terms "component," "module," "system," and the like are intended to refer to computer-related entities such as, but not limited to, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. For example, both an application running on a computing device and the computing device can be components. One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed across two or more computers. In addition, these components can execute from various computer readable media having various data structures stored thereon. Components may communicate by means of local and/or remote processes, for example, based on signals having one or more data packets, for example, from interacting with another component in a local system, a distributed system by means of signals, and/or with another component in a distributed system. Data for a component on a network such as the Internet that interacts with other systems by means of signals.

Aiming at the characteristics of 360-degree video, the present invention proposes a fast SAO method for 360-degree video. The proposed algorithm improves the SAO process by adding a simplified SAO process on the basis of retaining the entire SAO process. After a threshold-based SAO execution decision, one can choose to execute the regular SAO process specified by HEVC or choose not to execute SAO or only execute MERGE processing, thus realizing a simplified SAO process, which will greatly reduce the time for statistical data collection, thereby reducing SAO computational complexity.

I. Algorithm Overview

(1) Weight in 360-degree video

360-degree video is a spherical video, which is the biggest difference between 360-degree video and traditional video. In order to encode a 360-degree video under the HEVC standard, the 360-degree video must be projected into a flat video. While both projected video and traditional video are flat videos, projected video has the distortion and stretching of spherical video. Therefore, the objective quality evaluation index PSNR of traditional video is not suitable for projection video. WS-PSNR was proposed as an objective quality assessment metric for projected videos [18]. WS-PSNR designs weights for the projected video, and the projected video has less weight in the distorted and stretched regions, and vice versa, and then calculates the WS-PSNR by the weighted average method. WS-PSNR is recognized by the Joint Video Exploration Team (JVET) as an objective quality assessment metric for 360-degree video quality. Therefore, weight is the biggest difference between projected video and traditional video.

where (i, j) represents the pixel position and height represents the height of the video. Figure 3 shows the ERP weight distribution. The darker the color, the closer it is to 0. The lighter the color, the closer to 1. Region0 is defined as the area near the poles with a small weight; Region1 is defined as the area near the equator with a heavy weight. As shown in Figure 3, Region0 includes the upper 1/4 area and the lower 1/4 area of the video; Region1 represents the middle 1/2 area of the video.

(2)RD-cost

In HEVC, the optimal prediction modes and optimal CU partitions for intra prediction and inter prediction in the HEVC standard are recursively calculated by Rate Distortion Optimization (RDO) [17].

J=D+λ·R (2)

where D represents the distortion in the current prediction mode, R represents the number of bits required to encode all the information in the current prediction mode, λ is the Lagrangian factor, and J represents the Lagrangian cost (RD-cost). The smaller the RD-cost, the higher the coding efficiency of the prediction mode, and the larger the RD-cost, the lower the coding efficiency of the prediction mode.

(3) The proposed algorithm

In one embodiment of the present invention, a threshold (Threshold) is set to predetermine whether to perform the SAO process. When RD-cost>Threshold, the SAO process is performed, otherwise, the SAO process is not performed. By skipping SAO processing based on a threshold, the amount of encoding computation can be greatly reduced.

In one embodiment of the invention, the threshold for determining whether to perform the SAO process is determined based at least in part on the quantization parameters and ERP projection weights for the 360-degree video.

In one embodiment of the present invention, the quantization coefficients have an influence on the decision whether to perform SAO. According to experimental statistics, with the increase of quantization coefficient, the probability of CTU not applying SAO increases; with the increase of quantization coefficient, the RD-cost of the same CTU also increases. Therefore, the quantization coefficient is considered when determining the threshold value, and the determination of whether to execute SAO can be more accurate with the set threshold value.

In one exemplary and non-limiting embodiment, the quantization parameter raised to the power of e may be considered.

In one embodiment of the present invention, the weight of the projected video has an impact on the decision whether to perform SAO. Since ERP projections are used in this paper, the weights are also referred to as ERP projection weights. During the research, the inventors noticed that for the weight of the ERP projection, the weight of the CTU in Region0 is close to 0, so the distortion of the CTU in Region0 has little effect on the final video quality. Therefore, the weight of the projected video is considered in the determination of whether to execute the SAO, which conforms to the characteristics of the projected video projected by the ERP, so that the determination will not cause degradation of the quality of the encoded video.

In an exemplary but non-limiting embodiment of the present invention, the inventor proposes the following weight setting methods:

where (x, y) represents the position of the CTU, m is the number of CTUs in the video height, and weight(x, y) represents the average weight of all pixels of the CTU. Therefore, the weights of different positions are different, and the influence of the distortion of the CTU in Region1 on the video quality is much smaller than the influence of the distortion of the CTU in Region0 on the video quality. Therefore, making detailed decisions for CTUs in Region1 and coarse decisions for CTUs in Region0 is a way to adapt to the characteristics of projected video.

来表示不同纬度的比例因子。纬度越大，因子越大。In one embodiment of the present invention, the inventor modifies the threshold according to the weight of different latitudes (corresponding to the y value). Inventor uses

to represent scale factors for different latitudes. The larger the latitude, the larger the factor.

In an exemplary, non-limiting embodiment, threshold settings are given as shown in Table 1 below.

Table 1. Improved thresholds for ERP projection videos at different QPs

Among them, α and 1-α represent the percentage of fixed threshold and variable threshold, respectively, and QP is the quantization parameter.

In one embodiment of the present invention, the inventors consider the quantization parameter raised to the power of e. For example, in another exemplary and non-limiting embodiment, the threshold may be determined as follows:

In one embodiment of the present invention, as described above, the inventor noticed that the influence of the distortion of the CTU in Region0 on the video quality is much smaller than the influence of the distortion of the CTU in the Region1 on the video quality. Therefore, in order to ensure the video quality while reducing the amount of computation caused by SAO, the above operations can be performed only for Region0 in the ERP projection video (that is, the height above 1/4 and below the current frame), while for Region1 (The area of 1/2 height in the middle of the current frame) still performs the SAO processing specified by the HEVC protocol.

In an embodiment of the present invention, since the calculation amount of the MERGE operation is small, when the above operation is performed, it may be considered that when the threshold condition (for example, less than the threshold) is satisfied, the MERGE operation is performed or the SAO operation (OFF operation) is not performed. .

4 illustrates a method flow diagram in accordance with various aspects of the present disclosure. This method is used for Sample Adaptive Compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC).

According to one embodiment, the method includes performing an equirectangular projection (ERP) on the 360-degree video to obtain an ERP projection video. Those skilled in the art can easily understand that other projection methods other than ERP can be implemented, and the specific projection method is not the focus of the present invention.

According to another embodiment, the method further comprises: performing intra-frame prediction or inter-frame prediction on the coding tree unit (CTU) of the current frame in the ERP projection video to determine the best RD-cost.

According to another embodiment, the method further includes comparing the RD-cost of the CTU to a threshold value determined based at least in part on quantization parameters for the ERP projection video and ERP projection weights to determine whether to perform SAO , and wherein the ERP projection weight is determined based at least in part on the number of CTUs in the height of the ERP projection video and the position of the CTU in the current frame of the ERP projection video.

According to another embodiment, if it is determined not to perform SAO, at least boundary compensation (EO) and sideband compensation (BO) are not performed on the CTU; and if it is determined to perform SAO, then at least the CTU is not performed with an OFF or MERGE operation. one.

According to another embodiment, the threshold is based, at least in part, on at least one of: a base-2 logarithm of the ERP projection weight, or a power of e of the quantization parameter, or a combination thereof.

According to another embodiment, the RD-cost of the CTU is compared with a threshold to determine whether to perform SAO only for the upper 1/4 and lower 1/4 heights in the ERP projection video.

5 shows a schematic diagram of an apparatus for video encoding and decoding according to various aspects of the present disclosure. As shown in FIG. 5, the apparatus may include one or more processors and a memory having computer code stored in the memory that, when executed by the processor, implements high-efficiency video as described herein. A method of sample adaptive compensation (SAO) for 360-degree video in coding (HEVC).

According to another aspect, the present disclosure may also relate to an encoder for implementing the above encoding method. The encoder can be dedicated hardware.

According to another aspect, the present disclosure may also relate to a corresponding decoder for decoding the encoded 360 video stream.

According to another aspect, the present disclosure may also relate to a computer program product for performing the methods described herein.

When implemented in hardware, the video encoder may use a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic A device, discrete hardware component, or any combination thereof designed to perform the functions described herein is implemented or performed. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, eg, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors and a DSP core, or any other such configuration. Additionally, at least one processor may include one or more modules operable to perform one or more of the steps and/or operations described above.

When the video encoder is implemented with hardware circuits such as ASIC, FPGA, etc., it may include various circuit blocks configured to perform various functions. Those skilled in the art can design and implement these circuits in various ways according to various constraints imposed on the entire system to achieve various functions disclosed in the present invention.

While the foregoing disclosure discusses exemplary aspects and/or embodiments, it should be noted that many changes and/or changes may be made herein without departing from the scope of the described aspects and/or embodiments as defined by the claims. Revise. Furthermore, although elements of the described aspects and/or embodiments are described or claimed in the singular, the plural is contemplated unless limitation to the singular is expressly stated. Additionally, all or a portion of any aspect and/or embodiment may be used in conjunction with all or a portion of any other aspect and/or embodiment, unless indicated to the contrary.

Claims (6)

1. A method of sample adaptive compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC), comprising: performing projection on the 360-degree video to obtain a projection video; performing intra-frame prediction or inter-frame prediction on the coding tree unit (CTU) of the current frame in the projected video to determine the optimal RD-cost; comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO, wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projected video, and wherein the projection weight is based at least in part on the number of CTUs in the height of the projected video and the The position of the CTU in the current frame of the projected video is determined. 2. The method of claim 1, further comprising: If it is determined not to perform SAO, at least not perform boundary compensation (EO) and sideband compensation (BO) for the CTU; If it is determined to perform SAO, one of OFF or MERGE operations is performed for the CTU. 3. The method of claim 1 or 2, wherein the threshold is based, at least in part, on at least one of the base-2 logarithms of the projection weights, or the quantization parameter raised to a power of e, or a combination thereof. 4. The method of claim 1 or 2, wherein the RD-cost of the CTU is compared with a threshold only for the height of the upper 1/4 and the lower 1/4 in the projected video to determine whether Execute SAO. 5. The method of claim 1 or 2, wherein the projection is an equirectangular projection (ERP). 6. A device that can be used for video encoding and decoding, the device comprising: one or more processors; a memory having stored therein computer code that, when executed by the processor, implements a method of Sample Adaptive Compensation (SAO) for 360-degree video in High Efficiency Video Coding (HEVC), the method comprising : performing projection on the 360-degree video to obtain a projection video; performing intra-frame prediction or inter-frame prediction on the coding tree unit (CTU) of the current frame in the projected video to determine the optimal RD-cost; comparing the RD-cost of the CTU with a threshold to determine whether to perform SAO, wherein the threshold is determined based at least in part on a quantization parameter and a projection weight for the projected video, and wherein the projection weight is based at least in part on the number of CTUs in the height of the projected video and the The position of the CTU in the current frame of the projected video is determined.

Images