[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Single image deraining using local rain distribution map

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Existing image deraining methods work on the entire domain of images causing the deraining results to contain rain streaks or over-smoothing. To alleviate these problems, we propose a local pixel rain streak removal method based on the observation that rain streaks are sparsely distributed in images when rain levels are slight and middle. We first propose a discriminative anisotropic gradient prior to efficiently extract rain streaks from the background and generate a rain streak distribution (RSD) mask. Then, we recover the information in the rain-distorted regions with the RSD mask using a simple multi-layer image inpainting method. Experiments on both synthesized and real-world images demonstrate that our method outperforms the state-of-the-art methods in terms of rain streak detection, removal, and information restoration.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Algorithm 2
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

Code available on request from the authors.

References

  1. Yang W, Tan R, Feng J, Liu J, Yan S, Guo Z (2020) Joint rain detection and removal from a single image with contextualized deep networks. IEEE Trans Pattern Anal Mach Intell 42(6):1377–1393

    Article  Google Scholar 

  2. Li S, Araujo I, Ren W et al (2019) Single image deraining: a comprehensive benchmark analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp 3838–3847

  3. Yang W, Tan R, Wang S, Fang Y, Liu J (2020) Single image deraining: from model-based to data-driven and beyond. IEEE Trans Pattern Anal Mach Intell 43:4059–4077

    Article  Google Scholar 

  4. Wang H, Wu Y, Li M, Zhao Q, Meng D (2022) Survey on rain streak from video or a single image. Sci China Inf Sci 65:111101

    Article  Google Scholar 

  5. Xu J, Zhao W, Liu P, Tang X (2012) An improved guidance image based method to remove rain and snow in a single image. Comput Inform Sci 5(3):49–55

    Article  Google Scholar 

  6. Zheng X, Liao Y, Guo W, Fu X, Ding X (2013) Single-image based rain and snow removal using multi-guided filter. In: Neural information processing. Lecture Notes in Computer Science, vol 8228, pp 258–265

  7. Ding X, Chen L, Zheng X, Huang Y, Zeng D (2016) Single image rain and snow removal via guided L0 smoothing filter. Multimedia Tools Appl 75(5):2697–2712

    Article  Google Scholar 

  8. Kim JH, Lee C, Sim JY, Kim CS (2013) Single-image deraining using an adaptive nonlocal means filter. In: Proceedings of the IEEE International Conference on Image Processing (ICIP), pp 914–917

  9. Fu YH, Kang LW, Lin CW, Hsu CT (2011) Single-frame-based rain removal via image decomposition. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 1453–1456

  10. Kang L, Lin C, Fu Y (2012) Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans Image Process 21(4):1742–1755

    Article  MathSciNet  Google Scholar 

  11. Wang Y, Liu S, Chen C, Zeng B (2017) A hierarchical approach for rain or snow removing in a single color image. IEEE Trans Image Process 26(8):3936–3950

    Article  MathSciNet  Google Scholar 

  12. Luo Y, Xu Y, Ji H (2015) Removing rain from a single image via discriminative sparse coding. In: Proceedings of the IEEE International Conference on Computer Vision(ICCV), pp 3397–3405

  13. Li Y, Tan R, Guo X, Lu J, Brown M (2016) Rain streak removal using layer priors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pp 2736–2744

  14. Zhu L, Fu C, Lischinski D, Heng P (2017) Joint bi-layer optimization for single-image rain streak removal. In: Proceedings of the IEEE international conference on computer vision (ICCV), pp 2526–2534

  15. Gu S, Meng D, Zuo W, Zhang L (2017) Joint convolutional analysis and synthesis sparse representation for single image layer separation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp 1717–1725

  16. Deng L, Huang T, Zhao X, Jiang T (2018) A directional global sparse model for single image rain removal. Appl Math Model 59:662–679

    Article  MathSciNet  Google Scholar 

  17. Fu X, Huang J, Ding X, Liao Y, Paisley J (2017) Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans Image Process 26(6):2944–2956

    Article  MathSciNet  Google Scholar 

  18. Fu X, Huang J, Zeng D, Huang Y, Ding X, Paisley J (2017) Removing rain from single images via a deep detail network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3855–3863

  19. Zhang H, Sindagi V, Patel V (2020) Image de-raining using a conditional generative adversarial network. IEEE Trans Circuits Syst Video Technol 30(11):3943–3956

    Article  Google Scholar 

  20. Yang W, Tan R, Feng J, Liu J, Guo Z, Yan S (2017) Deep joint rain detection and removal from a single image. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1357–1366

  21. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze and-excitation context aggregation net for single image deraining. In: Proceedings of the European Conference on Computer Vision (ECCV), pp 254–269

  22. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3937–3946

  23. Zhang K, Zuo W, Zhang L (2018) FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans Image Process 27(9):4608–4622

    Article  MathSciNet  Google Scholar 

  24. Zhang X, Li H, Qi Y, Leow W, Ng T (2006) Rain removal in video by combining temporal and chromatic properties. In: 2006 IEEE International Conference on Multimedia and Expo, pp 461–464

  25. Xu L, Jia J (2010) Two-phase kernel estimation for robust motion deblurring. Eur Conf on Computer Vision (ECCV) 6311:157–170

    Google Scholar 

  26. Mu P, Chen J, Liu R, Fan X, Luo Z (2019) Learning bilevel layer priors for single image rain streaks removal. IEEE Signal Process Lett 26(2):307–311

    Article  Google Scholar 

  27. Liu R, Jiang Z, Ma L, Fan X, Li H, Luo Z (2018) Deep layer prior optimization for single image rain streaks removal. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 1408–1412

  28. Xu L, Lu C, Xu Y, Jia J (2011) Image smoothing via L0 gradient minimization. In: Proceedings of the 2011 SIGGRAPH Asia Conference, vol 174, pp 1–12

  29. Moll J (2005) The anisotropic total variation flow. Math Ann 332:177–218

    Article  MathSciNet  Google Scholar 

  30. Chen H, Fan Y, Wang Q, Li Z (2020) Morphological component image restoration by employing Bregmanized sparse regularization and anisotropic total variation. Circuits Systems Signal Process 39:2507–2532

    Article  Google Scholar 

  31. Chen H, Xu Z, Feng Q, Fan Y, Li Z (2020) An L0 regularized cartoon-texture decomposition model for restoring images corrupted by blur and impulse noise. Signal Proc: Image Commun 82:115762

    Google Scholar 

  32. Xu L, Zheng S, Jia J (2013) Unnatural L0 sparse representation for natural image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 1107–1114

  33. Liang Z, Xu J, Zhang D, Cao Z, Zhang L (2018) A hybrid L1-L0 layer decomposition model for tone mapping. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 4758–4766

  34. Chen H, Wang C, Song Y, Li Z (2015) Split Bregmanized anisotropic total variation model for image deblurring. J Vis Commun Image R 31:282–293

    Article  Google Scholar 

  35. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends®. Mach Learn 3(1):1–122

  36. Esser E (2009) Applications of lagrangian-based alternating direction methods and connections to split Bregman. CAM reports 09–31, UCLA

  37. Hale E, Yin W, Zhang Y (2008) Fixed-point continuation for L1-minimization: methodology and convergence. SIAM J Optim 19:1107–1130

    Article  MathSciNet  Google Scholar 

  38. Beck A, Teboulle M (2009) A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imaging Sci 2:183–202

    Article  MathSciNet  Google Scholar 

  39. Wen Z, Yin W, Goldfarb D, Zhang Y (2010) A fast algorithm for sparse reconstruction based on shrinkage, subspace optimization, and continuation. SIAM J Sci Comput 32:1832–1857

    Article  MathSciNet  Google Scholar 

  40. Wright S, Nowak R, Figueiredo M (2009) Sparse reconstruction by separable approximation. IEEE Trans Signal Process 57:2479–2493

    Article  MathSciNet  Google Scholar 

  41. Wang Y, Yang J, Yin W, Zhang Y (2008) A new alternating minimization algorithm for total variation image reconstruction. SIAM J Imag Sci 1(3):248–272

    Article  MathSciNet  Google Scholar 

  42. Jung M, Kang M (2015) Simultaneous cartoon and texture image restoration with higher-order regularization. SIAM J Imaging Sci 8(1):721–756

    Article  MathSciNet  Google Scholar 

  43. Tang L, Zhang H, He C, Fang Z (2019) Non-convex and non-smooth variational decomposition for image restoration. Appl Math Model 69:355–377

    Article  MathSciNet  Google Scholar 

  44. Wei W, Meng D, Zhao Q, Xu Z, Wu Y (2019) Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3877–3886

  45. Wang Z, Bovik A, Sheikh H, Simoncelli E (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  46. Elad M, Starck J, Querre P, Donoho D (2005) Simultaneous cartoon and texture image inpainting usingmorphological component analysis (MCA). Appl Comput Harmon Anal 19:340–358

    Article  MathSciNet  Google Scholar 

  47. Bertalmio M, Vese L, Sapiro G, Osher S (2003) Simultaneous structure and texture image inpainting. IEEE Trans Image Proc 12(8):882–889

    Article  Google Scholar 

  48. Arias P, Facciolo G, Caselles V, Sapiro G (2011) A variational framework for exemplar-based image inpainting. Int J Comput Vision 93:319–347

    Article  MathSciNet  Google Scholar 

  49. Chen H, Xu Z, Zhang Y, Fan Y, Li Z (2021) An L0-regularized global anisotropic gradient prior for single-image de-raining. Appl Math Model 98:628–651

    Article  MathSciNet  Google Scholar 

  50. Li R, Cheong L, Tan R (2019) Heavy rain image restoration: integrating physics model and conditional adversarial learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 1633–1642

  51. Fu X, Qi Q, Zha Z, Ding X, Wu F, Paisley J (2021) Successive graph convolutional network for image de-raining. Int J Comput Vision 129:1691–1711

    Article  Google Scholar 

  52. Zhou M, Xiao J, Chang Y et al (2021) Image de-raining via continual learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 4907–4916

  53. Wei W, Meng D, Zhao Q, Xu Z, Wu Y (2019) Semi-supervised transfer learning for image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 3877–3886

  54. Wang H, Xie Q, Zhao Q, Meng D (2020) A model-driven deep neural network for single image rain removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 3103–3112

Download references

Acknowledgements

The authors would like to thank the support by Natural Science Foundation of Huaian (HABZ202116). In addition, they would like to thank Dr. Xueyang Fu, Dr. Liangjian Deng, Yu Luo, and Wenhan Yang for sharing their codes online.

Author information

Authors and Affiliations

  1. Faculty of Mathematics and Physics, Huaiyin Institute of Technology, Huai’an, 223001, China

    Huasong Chen, Jing Wu, Qiansheng Feng & Yuanyuan Fan

  2. School of Science, Nanjing University of Science and Technology, Nanjing, 210094, China

    Zhenhua Xu & Zhenhua Li

Authors
  1. Huasong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhenhua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qiansheng Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuanyuan Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhenhua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huasong Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, H., Wu, J., Xu, Z. et al. Single image deraining using local rain distribution map. Multimed Tools Appl 83, 50349–50380 (2024). https://doi.org/10.1007/s11042-023-16972-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16972-9

Keywords

Search

Navigation