[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

GPU-based real-time super-resolution system for high-quality UHD video up-conversion

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Super-resolution (SR) is a technique that reconstructs high-resolution images using the information present in low-resolution images. Due to their potentials of being used in wide range of image and video applications, various SR algorithms have been studied and proposed in the literature until recently. However, many of the algorithms provide insufficient perceptual quality, possess high computational complexity, or have high memory requirement, which make them hard to apply on consumer-level products. Therefore, in this paper we propose an effective super-resolution method that not only provides an excellent visual quality but also a high-speed performance suitable for video conversion applications. The proposed super-resolution adopts self-similarity framework, which reconstructs the high-frequency (HF) information of the high-resolution image by referring to the image pairs generated from self-similar regions. The method further enhances the perceptual sharpness of the video through region-adaptive HF enhancement algorithm and applies iterative back projection to maintain its consistency with the input image. The proposed method is suitable for parallel processing and therefore is able to provide its superb visual quality on a high conversion speed through GPU-based acceleration. The experimental results show that the proposed method has superior HF reconstruction performance compared to other state-of-the-art upscaling solutions and is able to generate videos that are visually as sharp as the original high-resolution videos. On a single PC with four GPUs, the proposed method can convert Full HD resolution video into UHD resolution with real-time conversion speed. Due to its fast and high-quality conversion capability, the proposed method can be applied on various consumer products such as UHDTV, surveillance system, and mobile devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Hardie RC, Barnard KJ, Armstrong EA (1997) Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans Image Process 6:1621–1633

    Article  Google Scholar 

  2. Farsiu S, Robinson MD, Elad M, Milanfar P (2004) Fast and robust multiframe super-resolution. IEEE Trans Image Process 13:1327–1344

    Article  Google Scholar 

  3. Ng MK, Shen H, Lam EY, Zhang L (2007) A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP J Adv Signal Process 2007:74585-1–74585-16

    Article  MATH  Google Scholar 

  4. Yang J, Wang Z, Lin Z, Cohen S, Huang T (2012) Coupled dictionary training for image super-resolution. IEEE Trans Image Process 21(8):3467–3478

    Article  MathSciNet  MATH  Google Scholar 

  5. Peleg T, Elad M (2014) A statistical prediction model based on sparse representations for single image super-resolution. IEEE Trans Image Process 23(6):2569–2582

    Article  MathSciNet  MATH  Google Scholar 

  6. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: Curves and surfaces, pp 711–730

  7. Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Proceedings of Asian Conference on Computer Vision, Singapore

  8. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38:295–307

  9. Choi J-S, Kim M (2016) Super-interpolation with edge-orientation-based mapping kernels for low complex upscaling. IEEE Trans Image Process 25:469–483

    Article  MathSciNet  Google Scholar 

  10. Dong W, Zhang L, Shi G, Wu X (2011) Image deblurring and superresolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans Image Process 20(7):1838–1857

    Article  MathSciNet  MATH  Google Scholar 

  11. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    Article  MathSciNet  MATH  Google Scholar 

  12. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

  13. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. In: 12th International Conference on Computer Vision, pp 349–356

  14. Freedman G, Fattal R (2011) Image and video upscaling from local self-examples. ACM Trans Graph 30, Article No. 12, April 2011

  15. Yang J, Lin Z, Cohen S (2013) Fast image super-resolution based on in-place example regression. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, 23–28 June 2013, pp 1059–1066

  16. Jun JH, Choi JH, Lee DY, Jeong S, Cho SH, Kim HY, Kim JO (2015) Accelerating self-similarity-based image super-resolution using OpenCL. IEIE Trans Smart Process Comput 4(1):10–15

    Article  Google Scholar 

  17. Chen S, Gong H, Li C (2011) Super resolution from a single image based on self-similarity. In: IEEE International Conference on Computational and Information Sciences, Oct 2011, pp 91–94

  18. Khronos OpenCL Working Group (2012) The OpenCL Specification Version 1.2. Khronos Group. http://www.khronos.org/opencl

  19. Version 2.1.0, July 2016. http://www.infognition.com/videoenhancer/

  20. Asuni N, Giachetti A (2008) Accuracy improvements and artifacts removal in edge based image interpolation. In: Proceedings Third International Conference Computer Vision Theory and Applications (VISAPP’08)

  21. Giachetti A, Asuni N (2011) Real-time artifact-free image upscaling. IEEE Trans Image Process 20(10):2760–2768

    Article  MathSciNet  MATH  Google Scholar 

  22. Yang S, Kim Y, Jeong J (2008) Fine edge-preserving technique for display devices. IEEE Trans Consum Electron 54(4):1761–1769

    Article  Google Scholar 

  23. Kang W et al (2013) Real-time super-resolution for digital zooming using finite kernel-based edge orientation estimation and truncated image restoration. In: Proceedings of 20th IEEE International Conference on Image Processing, Melbourne, VIC, Sept 2013, pp 1311–1315

  24. Park SJ, Lee OY, Kim JO (2013) Self-similarity based image super-resolution on frequency domain. In: Proceedings of APSIPA ASC 2014, Nov 2013, pp 1–4

  25. Wang Z, Bovik AC, Evans BI (2000) Blind measurement of blocking artifacts in images. In: IEEE International Conference on Image Processing, Sep 2000, pp 981–984

  26. Bae S-H, Kim M (2014) A novel generalized DCT-based JND profile based on an elaborate CM-JND model for variable block-sized transforms in monochrome images. IEEE Trans Image Process 23(8):3227–3240

    Article  MathSciNet  MATH  Google Scholar 

  27. Alam MM (2014) Local masking in natural images: a database and analysis. J Vis 14(8):22–22

    Article  Google Scholar 

  28. Ponomarenko N et al (2013) Color image database TID2013: peculiarities and preliminary results. In: Proceedings of 4th European Workshop on Visual Information Processing, June 2013, pp 106–111

  29. Song L, Tang X, Zhang W, Yang X, Xia P (2013) The SJTU 4K video sequence dataset. In: The Fifth International Workshop on Quality of Multimedia Experience (QoMEX2013), Klagenfurt, 3rd–5th July 2013

  30. Chen Q, Montesinos P, Sun Q, Heng P, Xia D (2010) Adaptive total variation denoising based on difference curvature. Image Comput 28(3):298–306

    Article  Google Scholar 

  31. European Broadcast Union (2013) EBU UHD-1 Test Set (online). http://tech.ebu.ch/testsequences/uhd-1

  32. Blender Foundation (2012) Tears of Steel, Mango Open Movie Project. http://tearsofsteel.org

  33. Ulichney RA (1988) Dithering with blue noise. Proc IEEE 76(1):56–79

    Google Scholar 

  34. ITU-R BT.500-13 (2012) ITU, Methodology for the subjective assessment of the quality of television pictures

Download references

Acknowledgements

This work was supported by ICT R&D program of MSIP/IITP. [B0101-16-1280, Development of Cloud Computing Based Realistic Media Production Technology].

Author information

Authors and Affiliations

  1. Electronics and Telecommunications Research Institute, Daejeon, Republic of Korea

    Dae Yeol Lee, Jooyoung Lee, Hui Yong Kim & Jin Soo Choi

  2. School of Electrical Engineering, Korea University, Seoul, Republic of Korea

    Ji-Hoon Choi & Jong-Ok Kim

Authors
  1. Dae Yeol Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jooyoung Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ji-Hoon Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jong-Ok Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hui Yong Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jin Soo Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dae Yeol Lee.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, D.Y., Lee, J., Choi, JH. et al. GPU-based real-time super-resolution system for high-quality UHD video up-conversion. J Supercomput 74, 456–484 (2018). https://doi.org/10.1007/s11227-017-2136-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-017-2136-1

Keywords

Search

Navigation