[go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Retinal vessel segmentation based on self-distillation and implicit neural representation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Segmenting retinal blood vessels from retinal images is a crucial step in ocular disease diagnosis. It is also one of the most important applications and research in ophthalmic image analysis. However, the contrast between the retinal vessels and background in fundus images is low. The size and shape of retinal vessels vary significantly, and the width of some small vessels is often below 10 pixels or even 1 pixel. Moreover, some blood vessels are discontinuous owing to illumination, which complicates the segmentation of retinal blood vessels. To address these problems, this paper innovatively proposes a novel retinal vessel segmentation network framework based on self-distillation and implicit neural representation, which predicts retinal vessels in two stages. First, the self-distillation method extracts the main features of retinal images using the properties of Vision Transformer (ViT) to obtain preliminary images for the blood vessel segmentation. Second, implicit neural representation improves the resolution of the original retinal image, and the details of blood vessels are enhanced through the texture enhancement module to obtain accurate results of the blood vessel segmentation. Furthermore, we adopted an improved centerline dice (clDice) loss function to constrain the topology of blood vessels. We experimented on two benchmark retinal datasets (i.e., Drive and Chase) to quantitatively and qualitatively analyze the proposed method. The results indicate that the proposed outperformed the mainstream baseline. The visual segmentation results also show that this method can segment thin blood vessels more accurately and ensure the continuity of blood vessels.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ur Rehman M, Abbas Z, Khan SH, Ghani SH et al (2018) Diabetic retinopathy fundus image classification using discrete wavelet transform. In: 2018 2nd international conference on engineering innovation (ICEI). IEEE, pp 75–80

  2. Sahlsten J, Jaskari J, Kivinen J, Turunen L, Jaanio E, Hietala K, Kaski K (2019) Deep learning fundus image analysis for diabetic retinopathy and macular edema grading. Sci Rep 9(1):1–11

    Article  Google Scholar 

  3. Syahputra M, Amalia C, Rahmat R, Abdullah D, Napitupulu D, Setiawan M, Albra W, Andayani U et al (2018) Hypertensive retinopathy identification through retinal fundus image using backpropagation neural network. In: Journal of physics: conference series. IOP Publishing, vol 978, p 012106

  4. Nagpal D, Panda SN, Malarvel M (2021) Hypertensive retinopathy screening through fundus images-a review. In: 2021 6th international conference on inventive computation technologies (ICICT). IEEE, pp 924–929

  5. Singh LK, Garg H, Khanna M, Bhadoria RS, et al. (2021) An enhanced deep image model for glaucoma diagnosis using feature-based detection in retinal fundus. Med Bio Eng Comput 59(2):333–353

    Article  Google Scholar 

  6. Shabbir A, Rasheed A, Shehraz H, Saleem A, Zafar B, Sajid M, Ali N, Dar SH, Shehryar T (2021) Detection of glaucoma using retinal fundus images: a comprehensive review. Math Biosci Eng 18(3):2033–2076

    Article  Google Scholar 

  7. Jin Q, Meng Z, Pham TD, Chen Q, Wei L, Su R (2019) Dunet: a deformable network for retinal vessel segmentation. Knowl-Based Syst 178:149–162

    Article  Google Scholar 

  8. Vidya BS, Chandra E (2019) Entropy based local binary pattern (elbp) feature extraction technique of multimodal biometrics as defence mechanism for cloud storage. Alexandria Eng J 58(1):103–114

    Article  Google Scholar 

  9. Oloumi F, Rangayyan RM, Oloumi F, Eshghzadeh-Zanjani P, Ayres FJ (2007) Detection of blood vessels in fundus images of the retina using gabor wavelets. In: 2007 29th annual international conference of the ieee engineering in medicine and biology society. IEEE, pp 6451–6454

  10. Niemeijer M, Staal J, van Ginneken B, Loog M, Abramoff MD (2004) Comparative study of retinal vessel segmentation methods on a new publicly available database. In: Medical imaging 2004: image processing. International Society for Optics and Photonics, vol 5370, pp 648–656

  11. You X, Peng Q, Yuan Y, Cheung Y-m, Lei J (2011) Segmentation of retinal blood vessels using the radial projection and semi-supervised approach. Pattern Recognit 44(10-11):2314–2324

    Article  Google Scholar 

  12. Lesage D, Angelini ED, Bloch I, Funka-Lea G (2009) A review of 3d vessel lumen segmentation techniques: models, features and extraction schemes. Med Image Anal 13(6):819–845

    Article  Google Scholar 

  13. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  14. Strudel R, Garcia R, Laptev I, Schmid C (2021) Segmenter: transformer for semantic segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 7262–7272

  15. Sagar A (2021) Vitbis: vision transformer for biomedical image segmentation. In: Clinical image-based procedures, distributed and collaborative learning, artificial intelligence for combating COVID-19 and secure and privacy-preserving machine learning: 10th workshop, CLIP 2021, second workshop, DCL 2021, first workshop, LL-COVID19 2021, and first workshop and tutorial, PPML 2021, held in conjunction with MICCAI 2021, strasbourg, France, 27 September and 1, 2021 October. Proceedings, pp 34–45

  16. Lin A, Chen B, Xu J, Zhang Z, Lu G, Zhang D (2022) Ds-transunet: dual swin transformer u-net for medical image segmentation. IEEE Trans Instrument Measure

  17. Hatamizadeh A, Tang Y, Nath V, Yang D, Myronenko A, Landman B, Roth HR, Xu D (2022) Unetr: transformers for 3d medical image segmentation. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 574–584

  18. Lin Z, Huang J, Chen Y, Zhang X, Zhao W, Li Y, Lu L, Zhan M, Jiang X, Liang X (2021) A high resolution representation network with multi-path scale for retinal vessel segmentation. Comput Methods Prog Biomed 208:106206

    Article  Google Scholar 

  19. Jiang Y, Yao H, Wu C, Liu W (2020) A multi-scale residual attention network for retinal vessel segmentation. Symmetry 13(1):24

    Article  Google Scholar 

  20. Yin P, Cai H, Wu Q (2022) Df-net: deep fusion network for multi-source vessel segmentation. Inform Fusion 78:199–208

    Article  Google Scholar 

  21. Jiang Y, Wu C, Wang G, Yao H-X, Liu W-H (2021) Mfi-net: a multi-resolution fusion input network for retinal vessel segmentation. Plos one 16(7):0253056

    Article  Google Scholar 

  22. Kamran SA, Hossain KF, Tavakkoli A, Zuckerbrod SL, Sanders KM, Baker SA (2021) Rv-gan: segmenting retinal vascular structure in fundus photographs using a novel multi-scale generative adversarial network. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 34–44

  23. Boudegga H, Elloumi Y, Akil M, Bedoui MH, Kachouri R, Abdallah AB (2021) Fast and efficient retinal blood vessel segmentation method based on deep learning network. Comput Med Imaging Graph 90:101902

    Article  Google Scholar 

  24. Yu Y, Zhu H (2021) Retinal vessel segmentation with constrained-based nonnegative matrix factorization and 3d modified attention u-net. EURASIP J Image Video Process 2021(1):1–21

    Article  Google Scholar 

  25. Sun X, Fang H, Yang Y, Zhu D, Wang L, Liu J, Xu Y (2021) Robust retinal vessel segmentation from a data augmentation perspective. In: International workshop on ophthalmic medical image analysis. Springer, pp 189–198

  26. Naveed K, Abdullah F, Madni HA, Khan MA, Khan TM, Naqvi SS (2021) Towards automated eye diagnosis: an improved retinal vessel segmentation framework using ensemble block matching 3d filter. Diagnostics 11(1):114

    Article  Google Scholar 

  27. Dash S, Senapati MR, Sahu PK, Chowdary P (2021) Illumination normalized based technique for retinal blood vessel segmentation. Int J Imaging Syst Technol 31(1):351–363

    Article  Google Scholar 

  28. Tavakoli M, Mehdizadeh A, Pourreza Shahri R, Dehmeshki J (2021) Unsupervised automated retinal vessel segmentation based on radon line detector and morphological reconstruction. IET Image Process

  29. Toptaş B, Hanbay D (2021) Retinal blood vessel segmentation using pixel-based feature vector. Biomedical Signal Process Cont 103053:70

    Google Scholar 

  30. Wang J, Zhao Y, Qian L, Yu X, Gao Y (2021) Ear-net: error attention refining network for retinal vessel segmentation. In: 2021 Digital Image Computing: Techniques and Applications (DICTA). IEEE, pp 1–7

  31. Zhang J, Zhang Y, Xu X (2021) Pyramid u-net for retinal vessel segmentation. In: ICASSP 2021-2021 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 1125–1129

  32. Wu H, Wang W, Zhong J, Lei B, Wen Z, Qin J (2021) Scs-net: a scale and context sensitive network for retinal vessel segmentation. Med Image Anal 102025:70

    Google Scholar 

  33. Palanivel DA, Natarajan S, Gopalakrishnan S (2020) Retinal vessel segmentation using multifractal characterization. Appl Soft Comput 106439:94

    Google Scholar 

  34. Zhang M, Yu F, Zhao J, Zhang L, Li Q (2020) Befd: boundary enhancement and feature denoising for vessel segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 775–785

  35. Caron M, Touvron H, Misra I, Jégou H, Mairal J, Bojanowski P, Joulin A (2021) Emerging properties in self-supervised vision transformers. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 9650–9660

  36. Zhao H, Li H, Cheng L (2020) Improving retinal vessel segmentation with joint local loss by matting. Pattern Recogn 107068:98

    Google Scholar 

  37. Shit S, Paetzold JC, Sekuboyina A, Ezhov I, Unger A, Zhylka A, Pluim JP, Bauer U (2021) Menze, BH: cldice-a novel topology-preserving loss function for tubular structure segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 16560–16569

  38. Staal J, Abràmoff M D, Niemeijer M, Viergever MA, Van Ginneken B (2004) Ridge-based vessel segmentation in color images of the retina. IEEE Trans Med Imaging 23(4):501–509

    Article  Google Scholar 

  39. Fraz MM, Remagnino P, Hoppe A, Uyyanonvara B, Rudnicka AR, Owen CG, Barman SA (2012) An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Trans Biomed Eng 59(9):2538–2548

    Article  Google Scholar 

  40. Ding Y, Yu X, Yang Y (2021) Rfnet: region-aware fusion network for incomplete multi-modal brain tumor segmentation. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3975–3984

  41. Yang D, Myronenko A, Wang X, Xu Z, Roth HR, Xu D (2021) T-automl: automated machine learning for lesion segmentation using transformers in 3d medical imaging. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3962–3974

  42. Zhang J, Xie Y, Xia Y, Shen C (2021) Dodnet: learning to segment multi-organ and tumors from multiple partially labeled datasets. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1195–1204

  43. Reiß S, Seibold C, Freytag A, Rodner E, Stiefelhagen R (2021) Every annotation counts: multi-label deep supervision for medical image segmentation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9532–9542

  44. Viedma IA, Alonso-Caneiro D, Read SA, Collins MJ (2022) Oct retinal and choroidal layer instance segmentation using mask r-cnn. Sensors 22(5):2016

    Article  Google Scholar 

  45. Gour M, Jain S, Sunil Kumar T (2020) Residual learning based cnn for breast cancer histopathological image classification. Int J Imaging Syst Technol 30(3):621–635

    Article  Google Scholar 

  46. Wang W, Xie E, Li X, Fan D-P, Song K, Liang D, Lu T, Luo P, Shao L (2021) Pyramid vision transformer: a versatile backbone for dense prediction without convolutions. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 568–578

  47. Hinton G, Vinyals O, Dean J, et al. (2015) Distilling the knowledge in a neural network. vol 2(7), arXiv:1503.02531

  48. Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv:2010.11929

  49. Zhou Y, Yu H, Shi H (2021) Study group learning: improving retinal vessel segmentation trained with noisy labels. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 57–67

  50. Zhan X, Pan X, Dai B, Liu Z, Lin D, Loy CC (2020) Self-supervised scene de-occlusion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3784–3792

  51. Chen Y, Liu S, Wang X (2021) Learning continuous image representation with local implicit image function. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 8628–8638

  52. Owen CG, Rudnicka AR, Mullen R, Barman SA, Monekosso D, Whincup PH, Ng J, Paterson C (2009) Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (caiar) program. Investig Ophthalmol Vis Sci 50(5):2004–2010

    Article  Google Scholar 

  53. Zhuo Z, Huang J, Lu K, Pan D, Feng S (2020) A size-invariant convolutional network with dense connectivity applied to retinal vessel segmentation measured by a unique index. Computer Methods Programs Biomedicine 196:105508

    Article  Google Scholar 

  54. Zhou C, Zhang X, Chen H (2020) A new robust method for blood vessel segmentation in retinal fundus images based on weighted line detector and hidden markov model. Computer Methods Programs Biomedicine 187:105231

    Article  Google Scholar 

  55. Li X, Jiang Y, Li M, Yin S (2020) Lightweight attention convolutional neural network for retinal vessel image segmentation. IEEE Trans Indust Inform 17(3):1958–1967

    Article  Google Scholar 

  56. Li L, Verma M, Nakashima Y, Nagahara H, Kawasaki R (2020) Iternet: retinal image segmentation utilizing structural redundancy in vessel networks. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 3656– 3665

  57. Lv Y, Ma H, Li J, Liu S (2020) Attention guided u-net with atrous convolution for accurate retinal vessels segmentation. IEEE Access 8:32826–32839

    Article  Google Scholar 

  58. Yin P, Yuan R, Cheng Y, Wu Q (2020) Deep guidance network for biomedical image segmentation. IEEE Access 8:116106–116116

    Article  Google Scholar 

  59. Wang K, Zhang X, Huang S, Wang Q, Chen F (2020) Ctf-net: retinal vessel segmentation via deep coarse-to-fine supervision network. In: 2020 IEEE 17th international symposium on biomedical imaging (ISBI). IEEE, pp 1237–1241

  60. Guo C, Szemenyei M, Yi Y, Wang W, Chen B, Fan C (2021) Sa-unet: spatial attention u-net for retinal vessel segmentation. In: 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, pp. 1236–1242

  61. Khan TM, Alhussein M, Aurangzeb K, Arsalan M, Naqvi SS, Nawaz SJ (2020) Residual connection-based encoder decoder network (rced-net) for retinal vessel segmentation. IEEE Access 8:131257–131272

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Division of Computer Science and Engineering, Jeonbuk National University, 567 Baekje-daero, Jeonju, 54896, Republic of Korea

    Jia Gu, Fangzheng Tian & Il-Seok Oh

  2. Center for Advanced Image Information Technology, Jeonbuk National University, 567 Baekje-daero, Jeonju, 54896, Republic of Korea

    Il-Seok Oh

Authors
  1. Jia Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fangzheng Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Il-Seok Oh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Il-Seok Oh.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gu, J., Tian, F. & Oh, IS. Retinal vessel segmentation based on self-distillation and implicit neural representation. Appl Intell 53, 15027–15044 (2023). https://doi.org/10.1007/s10489-022-04252-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04252-2

Keywords