[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

A posterior mean approach for MRF-based spatially adaptive multi-frame image super-resolution

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Multi-frame image super-resolution (SR) has been intensively studied in recent years, aiming at reconstructing high-resolution images from several degraded ones (e.g., shift, blurred, aliased, and noisy). In the literature, one of the most popular SR frameworks is the maximum a posteriori model, where a spatially homogeneous image prior and manually adjusted regularization parameter are commonly used for the entire high-resolution image, thus ignoring local spatially adaptive properties of natural images. In this paper, a posterior mean approach is proposed for spatially adaptive multi-frame image super-resolution. First, a flexible Laplacian prior is proposed incorporating both the gradient and Hessian information of images, not only able to better preserve image structures, e.g., edge, texture, but also to suppress staircase effects in the flat regions. In the subsequent, a fully Bayesian SR framework is formulated, wherein the variational Bayesian method is utilized to simultaneously estimate the high-resolution image and unknown hyper-parameters for the image prior and noise. The final experimental results show that the proposed approach is highly competitive against existing algorithms, producing a super-resolved image with higher peak signal-to-noise ratio and better visual perception.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Afonso, M.V., Bioucas-Dias, J.M., Figueiredo, M.A.T.: Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19(9), 2345–2356 (2010)

    Article  MathSciNet  Google Scholar 

  2. Babacan, S.D., Molina, R., Katsaggelos, A.K.: Parameter estimation in TV image restoration using variational distribution approximation. IEEE Trans. Image Process. 17(3), 326–339 (2008)

    Google Scholar 

  3. Babacan, S.D., Molina, R., Katsaggelos, A.K.: Variational Bayesian super resolution. IEEE Trans. Image Process. 20(4), 984–999 (2011)

    Google Scholar 

  4. Dogan, Z., Lefkimmiatis, S., Bourquard, A., Unser, M.: A second-order extension of TV regularization for image deblurring. In: 18th IEEE International Conference on Image Processing (ICIP), Brussels, Belgium, pp. 705–708 (2011)

  5. Farsiu, S., Robinson, M.D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)

    Google Scholar 

  6. Hardie, R.C., Barnard, K.J., Armstrong, E.E.: Join MAP registration and high resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6(12), 1621–1633 (1997)

    Article  Google Scholar 

  7. He, H., Kondi, L.P.: MAP based resolution enhancement of video sequences using a Huber–Markov random field image prior model. Int. Conf. Image Process. 3, 933–936 (2003)

    Google Scholar 

  8. Kanemura, A., Maeda, S.-I., Ishii, S.: Superresolution with compound Markov random fields via the variational EM Algorithm. Neural Netw. 22(7), 1025–1034 (2009)

    Article  Google Scholar 

  9. Kang, M., Chaudhuri, S., (eds.) : Super-resolution image reconstruction. IEEE Signal Process. Mag. 20(3), 19–20 (2003)

    Google Scholar 

  10. Katsaggelos, A.K., Molina, R., Mateos, J.: Super Resolution of Images and Video. Morgan and Claypool, San Rafael (2007)

    Google Scholar 

  11. Lefkimmiatis, S., Bourquard, A., Unser, M.: Hessian-based norm regularization for image restoration with biomedical applications. IEEE Trans. Image Process. 21(3), 983–995 (2012)

    Article  MathSciNet  Google Scholar 

  12. Li, F., Shen, C., Fan, J., Shen, C.: Image restoration combining a total variational filter and a fourth-order filter. J. Vis. Commun. Image Represent 18(4), 322–330 (2007)

    Google Scholar 

  13. Lysaker, M., Lundervold, A., Tai, X.-C.: Noise removal using fourth-order partial differential equation with applications to medical magnetic resonance images in space and time. IEEE Trans. Image Processing 12(12), 1579–1590 (2003)

    Article  MATH  Google Scholar 

  14. Lysaker, M., Tai, X.-C.: Iterative image restoration combining total variation minimization and a second-order functional. Int. J. Comput. Vis. 66(1), 5–18 (2006)

    Article  MATH  Google Scholar 

  15. Milanfar, P.: Super-Resolution Imaging. CRC Press, New York (2011)

    Google Scholar 

  16. Mohammad-Djafari, A.: Bayesian approach with prior models which enforce sparsity in signal and image processing. EURASIP J. Adv. Signal Process. 52, 1–19 (2012)

    Google Scholar 

  17. Ng, M.K., Shen, H., Lam, E.Y., Zhang, L.: A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP J. Adv. Signal Process. 74585, 1–16 (2007)

    Article  Google Scholar 

  18. Oliveira, J., Figueiredo, M., Bioucas-Dias, J.: Adaptative total variation image deblurring: a majorization-minimization approach. Signal Process. 89(9), 1683–1693 (2009)

    Article  MATH  Google Scholar 

  19. Pickup, L.C., Capel, D.P., Roberts, S.J., Zisserman, A.: Bayesian methods for image super-resolution. Comput. J. 52(1), 101–113 (2009)

    Article  Google Scholar 

  20. Shao, W.-Z., Deng, H.-S., Wei, Z.-H.: Multi-Parseval frame-based nonconvex sparse image deconvolution. Opt. Eng. 51(6), 067008-1–067008-12 (2012)

    Article  Google Scholar 

  21. Smidl, V., Quinn, A.: The Variational Bayes Method in Signal Processing. Springer, New York (2005)

    Google Scholar 

  22. Tipping, M.E., Bishop, C.M.: Bayesian image super-resolution. Adv. Neural Inf. Process. Syst. 15, 1587–1594 (2003)

    Google Scholar 

  23. Tzikas, D.G., Likas, A.C., Galatsanos, N.P.: The variational approximation for Bayesian inference. IEEE Signal Process. Mag. 25(6), 131–146 (2008)

    Article  Google Scholar 

  24. Weickert, J.: Anisotropic Diffusion in Image Processing. Teubner-Verlag, Stuttgart (1998)

    MATH  Google Scholar 

  25. Zeng, W.L., Lu, X.B.: A robust variational approach to super-resolution with nonlocal TV regularisation term. Imaging Sci. J. 61(2), 268–278 (2013)

    Google Scholar 

  26. Zomet, A., Rav-Acha, A., Peleg, S.: Robust super-resolution. In: IEEE Computer Society Conference on Computer Vision and, Pattern Recognition, pp. 645–650 (2001)

Download references

Acknowledgments

Many thanks are given to the three anonymous reviewers for their detailed and insightful comments, which helped to significantly strengthen this manuscript. Wen-Ze Shao is very grateful to Professor Yi-Zhong Ma and Dr. Min Wu for their kind supports in the past years. He also shows many thanks to those kind people for helping him through his lost and sad years. The work is supported in part by the Talent Introduction Project of Nanjing University of Posts and Telecommunications (NY212014).

Author information

Authors and Affiliations

  1. College of Telecommunications and Information Engineering, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China

    Wen-Ze Shao

  2. School of Mathematics and Statistics, Nanjing Audit University, Nanjing, 211815, China

    Hai-Song Deng

  3. School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing, 210094, China

    Zhi-Hui Wei

Authors
  1. Wen-Ze Shao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hai-Song Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhi-Hui Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wen-Ze Shao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shao, WZ., Deng, HS. & Wei, ZH. A posterior mean approach for MRF-based spatially adaptive multi-frame image super-resolution. SIViP 9, 437–449 (2015). https://doi.org/10.1007/s11760-013-0458-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-013-0458-x

Keywords