[go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Physical-prior-guided single image dehazing network via unpaired contrastive learning

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Image dehazing aims to restore high fidelity clear images from hazy ones. It has wide applications on many intelligent image analysis systems in computer vision area. Many prior-based and learning-based methods have already made significant progress in this field. However, the domain gap between synthetic and real hazy images still negatively impacts model’s generalization performance in real-world scenarios. In this paper, we have proposed an effective physical-prior-guided single image dehazing network via unpaired contrastive learning (PDUNet). The learning process of PDUNet consists of pre-training stage on synthetic data and fine-tuning stage on real data. Mixed-prior modules, controllable zero-convolution modules, and unpaired contrastive regularization with hybrid transmission maps have been proposed to fully utilize complementary advantages of both prior-based and learning-based strategies. Specifically, mixed-prior module provides precise haze distributions. Zero-convolution modules serving as controllable bypass supplement pre-trained model with additional real-world haze information, as well as mitigate the risk of catastrophic forgetting during fine-tuning. Hybrid prior-generated transmission maps are employed for unpaired contrastive regularization. Through leveraging physical prior statistics and vast of unlabel real-data, the proposed PDUNet exhibits excellent generalization and adaptability on handling real-world hazy scenarios. Extensive experiments on public dataset have demonstrated that the proposed method improves PSNR,NIQE and BRISQUE values by an average of 0.33, 0.69 and 2.3, respectively, with comparable model efficiency compared to SOTA. Related codes and model parameters will be publicly available on Github https://github.com/Jotra9872/PDU-Net.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

No datasets were generated or analysed during the current study.

References

  1. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    Google Scholar 

  2. Chen, Z., Wang, Y., Yang, Y., Liu, D.: Psd: principled synthetic-to-real dehazing guided by physical priors. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7180–7189 (2021)

  3. Song, Y., He, Z., Qian, H., Du, X.: Vision transformers for single image dehazing. IEEE Trans. Image Process. 32, 1927–1941 (2023)

    Article  Google Scholar 

  4. Zheng, Y., Zhan, J., He, S., Dong, J., Du, Y.: Curricular contrastive regularization for physics-aware single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5785–5794 (2023)

  5. Hautière, N., Tarel, J.-P., Aubert, D.: Towards fog-free in-vehicle vision systems through contrast restoration. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007). IEEE

  6. Fattal, R.: Single image dehazing. ACM Trans. Graph. (TOG) 27(3), 1–9 (2008)

    Article  Google Scholar 

  7. Tan, R.T.: Visibility in bad weather from a single image. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008). IEEE

  8. Meng, G., Wang, Y., Duan, J., Xiang, S., Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 617–624 (2013)

  9. Fattal, R.: Dehazing using color-lines. ACM Trans. Graph. (TOG) 34(1), 1–14 (2014)

    Article  Google Scholar 

  10. Berman, D., Avidan, S., et al.: Non-local image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1674–1682 (2016)

  11. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  Google Scholar 

  12. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., Hua, G.: Gated context aggregation network for image dehazing and deraining. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383 (2019). IEEE

  13. Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., Yang, M.-H.: Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2157–2167 (2020)

  14. Guo, C.-L., Yan, Q., Anwar, S., Cong, R., Ren, W., Li, C.: Image dehazing transformer with transmission-aware 3d position embedding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5812–5820 (2022)

  15. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H.: Restormer: Efficient transformer for high-resolution image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5728–5739 (2022)

  16. Jiang, K., Wang, Z., Yi, P., Chen, C., Wang, Z., Wang, X., Jiang, J., Lin, C.-W.: Rain-free and residue hand-in-hand: a progressive coupled network for real-time image deraining. IEEE Trans. Image Process. 30, 7404–7418 (2021)

    Article  Google Scholar 

  17. Xiao, Y., Yuan, Q., Jiang, K., He, J., Lin, C.-W., Zhang, L.: Ttst: A top-k token selective transformer for remote sensing image super-resolution. IEEE Trans. Image Process. 33, 738–752 (2024)

    Article  Google Scholar 

  18. Chen, Z., He, Z., Lu, Z.-M.: Dea-net: single image dehazing based on detail-enhanced convolution and content-guided attention. IEEE Trans. Image Process. 33, 1002–1015 (2024)

    Article  Google Scholar 

  19. Yi, W., Dong, L., Liu, M., Hui, M., Kong, L., Zhao, Y.: Priors-assisted dehazing network with attention supervision and detail preservation. Neural Netw. 173, 106165 (2024)

    Article  Google Scholar 

  20. Yi, W., Dong, L., Liu, M., Hui, M., Kong, L., Zhao, Y.: Frequency-guidance collaborative triple-branch network for single image dehazing. Displays 80, 102577 (2023)

    Article  Google Scholar 

  21. Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.-H.: Gated fusion network for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3253–3261 (2018)

  22. Ren, W., Pan, J., Zhang, H., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks with holistic edges. Int. J. Comput. Vis. 128, 240–259 (2020)

    Article  Google Scholar 

  23. Wang, Y., Yan, X., Wang, F.L., Xie, H., Yang, W., Wei, M., Qin, J.: Ucl-dehaze: towards real-world image dehazing via unsupervised contrastive learning. IEEE Trans. Image Process. 33, 1361–1374 (2024)

    Article  Google Scholar 

  24. Wang, N., Cui, Z., Su, Y., He, C., Lan, Y., Li, A.: Prior-guided multiscale network for single-image dehazing. IET Image Proc. 15(13), 3368–3379 (2021)

    Article  Google Scholar 

  25. Wang, N., Cui, Z., Li, A., Su, Y., Lan, Y.: Multi-priors guided dehazing network based on knowledge distillation. In: Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 15–26 (2022). Springer

  26. Zhu, Q., Mai, J., Shao, L.: A fast single image haze removal algorithm using color attenuation prior. IEEE Trans. Image Process. 24(11), 3522–3533 (2015)

    Article  MathSciNet  Google Scholar 

  27. McCartney, E.J., Hall, F.F.: Optics of the atmosphere: scattering by molecules and particles. Phys. Today 30(5), 76–77 (1977)

    Article  Google Scholar 

  28. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pp. 154–169 (2016). Springer

  29. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28(1), 492–505 (2018)

    Article  MathSciNet  Google Scholar 

  30. Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7314–7323 (2019)

  31. Qin, X., Wang, Z., Bai, Y., Xie, X., Jia, H.: Ffa-net: Feature fusion attention network for single image dehazing. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 11908–11915 (2020)

  32. Wang, C., Pan, J., Wang, W., Dong, J., Wang, M., Ju, Y., Chen, J.: Promptrestorer: a prompting image restoration method with degradation perception. Adv. Neural. Inf. Process. Syst. 36, 8898–8912 (2023)

    Google Scholar 

  33. Wang, C., Pan, J., Wang, W., Fu, G., Liang, S., Wang, M., Wu, X.-M., Liu, J.: Correlation matching transformation transformers for uhd image restoration. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 5336–5344 (2024)

  34. Tran, L.-A., Moon, S., Park, D.-C.: A novel encoder-decoder network with guided transmission map for single image dehazing. Procedia Comput. Sci. 204, 682–689 (2022)

    Article  Google Scholar 

  35. Henaff, O.: Data-efficient image recognition with contrastive predictive coding. In: International Conference on Machine Learning, pp. 4182–4192 (2020). PMLR

  36. Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607 (2020). PMLR

  37. Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P., Buchatskaya, E., Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al.: Bootstrap your own latent-a new approach to self-supervised learning. Adv. Neural. Inf. Process. Syst. 33, 21271–21284 (2020)

    Google Scholar 

  38. Guo, Y., Xu, M., Li, J., Ni, B., Zhu, X., Sun, Z., Xu, Y.: Hcsc: hierarchical contrastive selective coding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9706–9715 (2022)

  39. Chen, X., Pan, J., Jiang, K., Li, Y., Huang, Y., Kong, C., Dai, L., Fan, Z.: Unpaired deep image deraining using dual contrastive learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2017–2026 (2022)

  40. Wu, H., Qu, Y., Lin, S., Zhou, J., Qiao, R., Zhang, Z., Xie, Y., Ma, L.: Contrastive learning for compact single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10551–10560 (2021)

  41. Xiao, Y., Yuan, Q., Jiang, K., He, J., Wang, Y., Zhang, L.: From degrade to upgrade: learning a self-supervised degradation guided adaptive network for blind remote sensing image super-resolution. Inf. Fusion 96, 297–311 (2023)

    Article  Google Scholar 

  42. Wang, L., Wang, Y., Dong, X., Xu, Q., Yang, J., An, W., Guo, Y.: Unsupervised degradation representation learning for blind super-resolution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10581–10590 (2021)

  43. Park, T., Efros, A.A., Zhang, R., Zhu, J.-Y.: Contrastive learning for unpaired image-to-image translation. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part IX 16, pp. 319–345 (2020). Springer

  44. Han, J., Shoeiby, M., Petersson, L., Armin, M.A.: Dual contrastive learning for unsupervised image-to-image translation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 746–755 (2021)

  45. Yi, W., Dong, L., Liu, M., Hui, M., Kong, L., Zhao, Y.: Sid-net: single image dehazing network using adversarial and contrastive learning. Multimed. Tools Appl. 83(28), 71619–71638 (2024)

    Article  Google Scholar 

  46. Yi, W., Dong, L., Liu, M., Hui, M., Kong, L., Zhao, Y.: Towards compact single image dehazing via task-related contrastive network. Expert Syst. Appl. 235, 121130 (2024)

    Article  Google Scholar 

  47. Yi, W., Dong, L., Liu, M., Hui, M., Kong, L., Zhao, Y.: Semi-supervised progressive dehazing network using unlabeled contrastive guidance. Neurocomputing 551, 126494 (2023)

    Article  Google Scholar 

  48. Chen, X., Fan, Z., Li, P., Dai, L., Kong, C., Zheng, Z., Huang, Y., Li, Y.: Unpaired deep image dehazing using contrastive disentanglement learning. In: European Conference on Computer Vision, pp. 632–648 (2022). Springer

  49. Shen, Y., Deng, S., Yang, W., Wei, M., Xie, H., Zhang, X., Qin, J., Wang, M.: Semi-drdnet semi-supervised detail-recovery image deraining network via unpaired contrastive learning. CoRR (2022)

  50. Ye, Y., Yu, C., Chang, Y., Zhu, L., Zhao, X.-l., Yan, L., Tian, Y.: Unsupervised deraining: Where contrastive learning meets self-similarity. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5821–5830 (2022)

  51. Zhang, L., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: Proceedings of IEEE/CVF International Conference on Computer Vision, pp. 3813–3824 (2023)

  52. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: all-in-one dehazing network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4770–4778 (2017)

  53. Deng, Z., Zhu, L., Hu, X., Fu, C.-W., Xu, X., Zhang, Q., Qin, J., Heng, P.-A.: Deep multi-model fusion for single-image dehazing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2453–2462 (2019)

  54. Tu, Z., Talebi, H., Zhang, H., Yang, F., Milanfar, P., Bovik, A., Li, Y.: Maxim: multi-axis mlp for image processing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5769–5780 (2022)

  55. Hong, M., Liu, J., Li, C., Qu, Y.: Uncertainty-driven dehazing network. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 906–913 (2022)

  56. Aly, H.A., Dubois, E.: Image up-sampling using total-variation regularization with a new observation model. IEEE Trans. Image Process. 14(10), 1647–1659 (2005)

    Article  Google Scholar 

  57. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

  58. Ancuti, C.O., Ancuti, C., Sbert, M., Timofte, R.: Dense-haze: a benchmark for image dehazing with dense-haze and haze-free images. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1014–1018 (2019). IEEE

  59. Ancuti, C.O., Ancuti, C., Vasluianu, F.-A., Timofte, R.: Ntire 2021 nonhomogeneous dehazing challenge report. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 627–646 (2021)

  60. He, T., Zhang, Z., Zhang, H., Zhang, Z., Xie, J., Li, M.: Bag of tricks for image classification with convolutional neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 558–567 (2019)

  61. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  62. Mittal, A., Moorthy, A.K., Bovik, A.C.: No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  Google Scholar 

  63. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2012)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China under Grand No. 62366021.

Funding

This article is funded by National Natural Science Foundation of China (62366021).

Author information

Authors and Affiliations

  1. School of Computer and Information Engineering, Jiangxi Normal University, Ziyang Ave. 99, Nanchang, 330022, Jiangxi, China

    Mawei Wu, Aiwen Jiang, Hourong Chen & Jihua Ye

  2. School of Digital Industry, Jiangxi Normal University, Wuchu Ave., Shangrao, 334000, Jiangxi, China

    Aiwen Jiang

Authors
  1. Mawei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aiwen Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hourong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jihua Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Mawei Wu and Aiwen Jiang wrote the main manuscript. Aiwen Jiang funded the work and provided main concept and ideas for the manuscript. Mawei Wu conducted all experiments. Hourong Chen prepared ablational experiements and Figures and prepare the revised version. Jihua Ye discuss the idea for improvements. All authors have reviewed the manuscript.

Corresponding author

Correspondence to Aiwen Jiang.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, M., Jiang, A., Chen, H. et al. Physical-prior-guided single image dehazing network via unpaired contrastive learning. Multimedia Systems 30, 261 (2024). https://doi.org/10.1007/s00530-024-01462-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01462-1

Keywords