[go: up one dir, main page]
Skip to main content
SpringerLink
Log in

Dehaze on small-scale datasets via self-supervised learning

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Real-world dehazing datasets usually suffer from small scales because of high collection costs. If networks are trained with such insufficient data, it leads to not only low performance on objective metrics, but also visually insufficient contrast enhancement. Self-supervised learning helps networks learn useful knowledge from unlabeled data and further achieve better performance on small-scale data, which has achieved great success on high-level vision tasks. However, there are rare works to develop self-supervised learning on low-level vision tasks, such as dehazing. In this paper, we propose a simple but effective self-supervised learning method for dehazing, to improve networks’ performance on small-scale real-world datasets. Our useful observations are twofold. First, generating visually pleasing haze-free images from real-world hazy images is very difficult, but generating visually pleasing denser hazy images is much easier. Second, forcing networks to reduce dense haze will enhance the contrast enhancement capability of networks, and it is beneficial for further dehazing. Therefore, we generate numerous denser hazy images rehazy from a real-world hazy image. With pretraining on image pairs [rehazy, hazy], networks learn key capabilities of enhancing contrast. Experiments show that it stably outperforms directly supervised learning by a considerable margin, but only spends a cheap extra pretraining time cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

All data used in this paper is public and available [7, 9, 10].

References

  1. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    Google Scholar 

  2. Tan, R.T.: Visibility in bad weather from a single image. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2008). IEEE

  3. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4770–4778 (2017)

  4. Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.-H.: Gated fusion network for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3253–3261 (2018)

  5. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  6. Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7314–7323 (2019)

  7. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28(1), 492–505 (2018)

    Article  MathSciNet  Google Scholar 

  8. Li, L., Dong, Y., Ren, W., Pan, J., Gao, C., Sang, N., Yang, M.-H.: Semi-supervised image dehazing. IEEE Trans. Image Process. 29, 2766–2779 (2019)

    Article  Google Scholar 

  9. Ancuti, C., Ancuti, C.O., Timofte, R., De Vleeschouwer, C.: I-haze: a dehazing benchmark with real hazy and haze-free indoor images. In: International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 620–631 (2018). Springer

  10. Ancuti, C.O., Ancuti, C., Timofte, R., De Vleeschouwer, C.: O-haze: a dehazing benchmark with real hazy and haze-free outdoor images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 754–762 (2018)

  11. Morales, P., Klinghoffer, T., Jae Lee, S.: Feature forwarding for efficient single image dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0–0 (2019)

  12. Liu, J., Wu, H., Xie, Y., Qu, Y., Ma, L.: Trident dehazing network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 430–431 (2020)

  13. Guo, T., Li, X., Cherukuri, V., Monga, V.: Dense scene information estimation network for dehazing. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 0–0 (2019)

  14. Zhang, M., Zhang, J., Lu, Z., Xiang, T., Ding, M., Huang, S.: Iept: Instance-level and episode-level pretext tasks for few-shot learning. In: International Conference on Learning Representations (2020)

  15. An, Y., Xue, H., Zhao, X., Zhang, L.: Conditional self-supervised learning for few-shot classification. In: Zhou, Z.-H. (ed.) Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pp. 2140–2146. International Joint Conferences on Artificial Intelligence Organization, (2021). Main Track

  16. Su, J.-C., Maji, S., Hariharan, B.: When does self-supervision improve few-shot learning? In: European Conference on Computer Vision, pp. 645–666 (2020). Springer

  17. Gidaris, S., Bursuc, A., Komodakis, N., Pérez, P., Cord, M.: Boosting few-shot visual learning with self-supervision. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8059–8068 (2019)

  18. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012)

  19. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4700–4708 (2017)

  20. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  Google Scholar 

  21. Dong, C., Loy, C.C., He, K., Tang, X.: Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2015)

    Article  Google Scholar 

  22. Tao, X., Gao, H., Shen, X., Wang, J., Jia, J.: Scale-recurrent network for deep image deblurring. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8174–8182 (2018)

  23. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks. In: European Conference on Computer Vision, pp. 154–169 (2016). Springer

  24. Ren, W., Pan, J., Zhang, H., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks with holistic edges. Int. J. Comput. Vision 128(1), 240–259 (2020)

    Article  Google Scholar 

  25. Dong, H., Pan, J., Xiang, L., Hu, Z., Zhang, X., Wang, F., Yang, M.-H.: Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2157–2167 (2020)

  26. Ren, W., Pan, J., Zhang, H., Cao, X., Yang, M.-H.: Single image dehazing via multi-scale convolutional neural networks with holistic edges. Int. J. Comput. Vision 128(1), 240–259 (2020)

    Article  Google Scholar 

  27. Zheng, Z., Ren, W., Cao, X., Hu, X., Wang, T., Song, F., Jia, X.: Ultra-high-definition image dehazing via multi-guided bilateral learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16180–16189 (2021). IEEE

  28. Zhang, S., Ren, W., Tan, X., Wang, Z.-J., Liu, Y., Zhang, J., Zhang, X., Cao, X.: Semantic-aware dehazing network with adaptive feature fusion. IEEE Trans. Cybernet. 51, 457 (2021)

    Google Scholar 

  29. Zhang, J., Ren, W., Zhang, S., Zhang, H., Nie, Y., Xue, Z., Cao, X.: Hierarchical density-aware dehazing network. IEEE Trans. Cybernet. 52, 11187 (2021)

    Article  Google Scholar 

  30. Xiao, C., Gan, J.: Fast image dehazing using guided joint bilateral filter. Vis. Comput. 28(6), 713–721 (2012)

    Article  Google Scholar 

  31. Manu, C.M., Sreeni, K.G.: GANID: a novel generative adversarial network for image dehazing. Vis. Comput. 39, 3923–3936 (2023)

  32. Zhang, S., He, F.: Drcdn: learning deep residual convolutional dehazing networks. Vis. Comput. 36(9), 1797–1808 (2020)

    Article  Google Scholar 

  33. Guo, C.-L., Yan, Q., Anwar, S., Cong, R., Ren, W., Li, C.: Image dehazing transformer with transmission-aware 3d position embedding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5812–5820 (2022)

  34. Valanarasu, J.M.J., Yasarla, R., Patel, V.M.: Transweather: Transformer-based restoration of images degraded by adverse weather conditions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2353–2363 (2022)

  35. Chen, H., Wang, Y., Guo, T., Xu, C., Deng, Y., Liu, Z., Ma, S., Xu, C., Xu, C., Gao, W.: Pre-trained image processing transformer. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12299–12310 (2021)

  36. Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., et al.: A survey on vision transformer. IEEE Trans. Pattern Anal. Mach. Intell. 45(1), 87 (2022)

    Article  Google Scholar 

  37. Ren, W., Zhang, J., Xu, X., Ma, L., Cao, X., Meng, G., Liu, W.: Deep video dehazing with semantic segmentation. IEEE Trans. Image Process. 28(4), 1895–1908 (2018)

    Article  MathSciNet  Google Scholar 

  38. Zhang, J., Li, L., Zhang, Y., Yang, G., Cao, X., Sun, J.: Video dehazing with spatial and temporal coherence. Vis. Comput. 27(6), 749–757 (2011)

    Article  Google Scholar 

  39. Chen, C., Do, M.N., Wang, J.: Robust image and video dehazing with visual artifact suppression via gradient residual minimization. In: European Conference on Computer Vision, pp. 576–591 (2016). Springer

  40. Kim, J.-H., Jang, W.-D., Sim, J.-Y., Kim, C.-S.: Optimized contrast enhancement for real-time image and video dehazing. J. Vis. Commun. Image Represent. 24(3), 410–425 (2013)

    Article  Google Scholar 

  41. Zhang, J., Li, L., Zhang, Y., Yang, G., Cao, X., Sun, J.: Video dehazing with spatial and temporal coherence. Vis. Comput. 27(6), 749–757 (2011)

    Article  Google Scholar 

  42. Ancuti, C.O., Ancuti, C., Sbert, M., Timofte, R.: Dense-haze: A benchmark for image dehazing with dense-haze and haze-free images. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1014–1018 (2019). https://doi.org/10.1109/ICIP.2019.8803046

  43. Engin, D., Genc, A., Ekenel, H.K.: Cycle-dehaze: Enhanced cyclegan for single image dehazing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 938–9388 (2018). https://doi.org/10.1109/CVPRW.2018.00127

  44. Fathony, R., Goela, N.: Discrete wasserstein generative adversarial networks (dwgan) (2018)

  45. Singh, A., Bhave, A., Prasad, D.K.: Single image dehazing for a variety of haze scenarios using back projected pyramid network. In: European Conference on Computer Vision, pp. 166–181 (2020). Springer

  46. Nathan Silberman, P.K. Derek Hoiem, Fergus, R.: Indoor segmentation and support inference from rgbd images. In: ECCV (2012)

  47. Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings., vol. 1, p. (2003). IEEE

  48. Yi, Q., Li, J., Fang, F., Jiang, A., Zhang, G.: Efficient and accurate multi-scale topological network for single image dehazing. IEEE Trans. Multimed. 24, 3114–3128 (2021)

    Article  Google Scholar 

  49. Yu, H., Zheng, N., Zhou, M., Huang, J., Xiao, Z., Zhao, F.: Frequency and spatial dual guidance for image dehazing. In: Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XIX, pp. 181–198 (2022). Springer

Download references

Acknowledgements

This work was supported by the Key Research Project of Zhejiang Lab (No.2021MH0AC01), Civil Aerospace Pre-Research Project (No. D040104) and National Natural Science Foundation of China (No. 61975175). We also thank Meijuan Bian from the facility platform of optical engineering of Zhejiang University for instrument support.

Author information

Authors and Affiliations

  1. Research Center for Intelligent Sensing Systems, Zhejiang Laboratory, Hangzhou, 311100, China

    Zhaojie Chen & Tingting Jiang

  2. State Key Laboratory of Modern Optical Instrumentation, Zhejiang University, Hangzhou, 310027, China

    Zhaojie Chen, Qi Li, Huajun Feng, Zhihai Xu & Yueting Chen

Authors
  1. Zhaojie Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huajun Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhihai Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yueting Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Tingting Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tingting Jiang.

Ethics declarations

Conflict of interest

There are no conflicts of interest in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Z., Li, Q., Feng, H. et al. Dehaze on small-scale datasets via self-supervised learning. Vis Comput 40, 4235–4249 (2024). https://doi.org/10.1007/s00371-023-03079-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-03079-3

Keywords

Search

Navigation