[go: up one dir, main page]
Skip to main content

Advertisement

SpringerLink
Log in

Hierarchical deep network with uncertainty-aware semi-supervised learning for vessel segmentation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The analysis of organ vessels is essential for computer-aided diagnosis and surgical planning. But it is not an easy task since the fine-detailed connected regions of organ vessel bring a lot of ambiguity in vessel segmentation and sub-type recognition, especially for the low-contrast capillary regions. Furthermore, recent two-staged approaches would accumulate and even amplify these inaccuracies from the first-stage whole vessel segmentation into the second-stage sub-type vessel pixel-wise classification. Moreover, the scarcity of manual annotation in organ vessels poses another challenge. In this paper, to address the above issues, we propose a hierarchical deep network where an attention mechanism localizes the low-contrast capillary regions guided by the whole vessels, and enhance the spatial activation in those areas for the sub-type vessels. In addition, we propose an uncertainty-aware semi-supervised training framework to alleviate the annotation-hungry limitation of deep models. The proposed method achieves the state-of-the-art performance in the benchmarks of both retinal artery/vein segmentation in fundus images and liver portal/hepatic vessel segmentation in CT images. Our implementation is publicly available at https://github.com/XGGNet/Vessel-Seg.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Abràmoff MD, Garvin MK, Sonka M (2010) Retinal imaging and image analysis. IEEE Rev Biomed Eng 3:169–208

    Article  Google Scholar 

  2. AlBadawi S, Fraz MM (2018) Arterioles and venules classification in retinal images using fully convolutional deep neural network. In: Int. Conf. on Img. Anal. and Recog., Springer, pp. 659–668

  3. Bi L, Kim J, Kumar A, Feng D (2017) Automatic liver lesion detection using cascaded deep residual networks. arXiv preprint arXiv:1704.02703

  4. Bruno P, Zaffino P, Scaramuzzino S, De Rosa S, Indolfi C, Calimeri F, Spadea MF (2018) Using cnns for designing and implementing an automatic vascular segmentation method of biomedical images. In: International Conference of the Italian Association for Artificial Intelligence. pp. 60–70. Springer

  5. Chew SK, Xie J, Wang JJ (2012) Retinal arteriolar diameter and the prevalence and incidence of hypertension: a systematic review and meta-analysis of their association. Curr Hypertens Rep 14(2):144–151

    Article  Google Scholar 

  6. Chi Y, Liu J, Venkatesh SK, Huang S, Zhou J, Tian Q, Nowinski WL (2010) Segmentation of liver vasculature from contrast enhanced ct images using context-based voting. IEEE Trans Biomed Eng 58(8):2144–2153

    Google Scholar 

  7. Ching T, Himmelstein DS, Beaulieu-Jones BK, Kalinin AA, Do BT, Way GP, Ferrero E, Agapow PM, Zietz M, Hoffman MM et al (2018) Opportunities and obstacles for deep learning in biology and medicine. J R Soc Interface 15(141):20170387

    Article  Google Scholar 

  8. Clavien PA, Petrowsky H, DeOliveira ML, Graf R (2007) Strategies for safer liver surgery and partial liver transplantation. New Engl J Med 356(15):1545–1559

    Article  Google Scholar 

  9. Dashtbozorg B, Mendonça AM, Campilho A (2014) An automatic graph-based approach for artery/vein classification in retinal images. IEEE Trans Image Process 23(3):1073–1083

    Article  MathSciNet  MATH  Google Scholar 

  10. Der Kiureghian A, Ditlevsen O (2009) Aleatory or epistemic? does it matter? Struct Saf 31(2):105–112

    Article  Google Scholar 

  11. DeVries T, Taylor GW (2018) Leveraging uncertainty estimates for predicting segmentation quality. arXiv preprint arXiv:1807.00502

  12. Dou Q, Chen H, Jin Y, Yu L, Qin J, Heng P (2016) 3d deeply supervised network for automatic liver segmentation from CT volumes pp. 149–157

  13. Estrada R, Allingham MJ, Mettu PS, Cousins SW, Tomasi C, Farsiu S (2015) Retinal artery-vein classification via topology estimation. IEEE Trans Med Imaging 34(12):2518–2534

    Article  Google Scholar 

  14. Fu H, Xu Y, Wong DWK, Liu J (2016) Retinal vessel segmentation via deep learning network and fully-connected conditional random fields. In: 2016 IEEE 13th international symposium on biomedical imaging, IEEE, pp. 698–701

  15. Gal Y, Ghahramani Z (2016) Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059

  16. Galdran A, Meyer M, Costa P, Campilho A, et al. (2019) Uncertainty-aware artery/vein classification on retinal images. In: 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), IEEE, pp. 556–560

  17. Guo X, Xiao R, Zhang T, Chen C, Wang J, Wang Z (2020) A novel method to model hepatic vascular network using vessel segmentation, thinning, and completion. Med Biol Eng Comput pp. 1–16

  18. Hong S, Noh H, Han B (2015) Decoupled deep neural network for semi-supervised semantic segmentation. Adv Neural Inf Process Syst pp. 1495–1503

  19. Hu Q, Abramoff MD, Garvin MK (2013) Automated separation of binary overlapping trees in low-contrast color retinal images. In: MICCAI, pp. 436–443

  20. Huang Q, Sun J, Ding H, Wang X, Wang G (2018) Robust liver vessel extraction using 3d u-net with variant dice loss function. Comput Biol Med 101:153–162

    Article  Google Scholar 

  21. Hung WC, Tsai YH, Liou YT, Lin YY, Yang MH (2018) Adversarial learning for semi-supervised semantic segmentation. arXiv preprint arXiv:1802.07934

  22. Ikram MK, de Jong FJ, Vingerling JR, Witteman JC, Hofman A, Breteler MM, de Jong PT (2004) Are retinal arteriolar or venular diameters associated with markers for cardiovascular disorders? The Rotterdam Study. Investig Ophthamol Vis Sci 45(7):2129–2134

    Article  Google Scholar 

  23. Jerman T, Pernuš F, Likar B, Špiclin Ž (2016) Enhancement of vascular structures in 3d and 2d angiographic images. IEEE Trans Med Imaging 35(9):2107–2118

    Article  Google Scholar 

  24. Kawasaki R, Xie J, Cheung N et al (2012) Retinal microvascular signs and risk of stroke: the Multi-Ethnic Study of Atherosclerosis (MESA). Stroke 43(12):3245–3251

    Article  Google Scholar 

  25. Kendall A, Gal Y (2017) What uncertainties do we need in bayesian deep learning for computer vision?. Adv Neural Inf Process Syst pp. 5574–5584

  26. Kohl S, Romeraparedes B, Meyer C, De Fauw J, Ledsam JR, Maierhein KH, Eslami SMA, Rezende DJ, Ronneberger O (2018) A probabilistic U-Net for segmentation of ambiguous images pp. 6965–6975

  27. Laine S, Aila T (2016) Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242

  28. Lebre MA, Vacavant A, Grand-Brochier M, Rositi H, Abergel A, Chabrot P, Magnin B (2019) Automatic segmentation methods for liver and hepatic vessels from CT and MRI volumes, applied to the Couinaud scheme. Comput Biol Med 110:42–51

    Article  Google Scholar 

  29. Lee DH (2013) Pseudo-label : the simple and efficient semi-supervised learning method for deep neural networks. In: ICMLW, pp. 2,3,7

  30. Leibig C, Allken V, Ayhan MS, Berens P, Wahl S (2017) Leveraging uncertainty information from deep neural networks for disease detection. Sci Rep 7(1):1–14

    Article  Google Scholar 

  31. Li Q, Arnab A, Torr PH (2018) Weakly-and semi-supervised panoptic segmentation. In: Proceedings of the European Conference on Computer Vision, pp. 102–118

  32. Lin TY, Goyal P, Girshick R, He K, Dollar P (2017) Focal loss for dense object detection. In: ICCV. pp. 2980–2988

  33. Liskowski P, Krawiec K (2016) Segmenting retinal blood vessels with deep neural networks. IEEE Trans Med Imaging 35(11):2369–2380

    Article  Google Scholar 

  34. Ma W, Yu S, Ma K, Wang J, Ding X, Zheng Y (2019) Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 769–778

  35. Merveille O, Talbot H, Najman L, Passat N (2017) Curvilinear structure analysis by ranking the orientation responses of path operators. IEEE Trans Pattern Anal Mach Intell 40(2):304–317

    Article  MATH  Google Scholar 

  36. Meyer MI, Galdran A, Costa P, Mendonça AM, Campilho A (2018) Deep convolutional artery/vein classification of retinal vessels. In: Int. Conf. on Img. Anal. and Recog, pp. 622–630

  37. Mo J, Zhang L (2017) Multi-level deep supervised networks for retinal vessel segmentation. Int J Comput Assist Radiol Surg 12(12):2181–2193

    Article  Google Scholar 

  38. Nguyen TT, Wang JJ, Islam FA, Mitchell P, Tapp RJ, Zimmet PZ, Simpson R, Shaw J, Wong TY (2008) Retinal arteriolar narrowing predicts incidence of diabetes: the australian diabetes, obesity and lifestyle (AusDiab) study. Diabetes 57(3):536–539

    Article  Google Scholar 

  39. Nguyen UT, Bhuiyan A, Park LA, Ramamohanarao K (2013) An effective retinal blood vessel segmentation method using multi-scale line detection. Pattern Recognit 46(3):703–715

    Article  Google Scholar 

  40. Niemeijer M, van Ginneken B, Abràmoff M (2009) Automatic classification of retinal vessels into arteries and veins. In: Med. Img. 2009: Comp-Aided Diag 7260: 72601F

  41. O’Donohue J, Ng C, Catnach S, Farrant P, Williams R (2004) Diagnostic value of doppler assessment of the hepatic and portal vessels and ultrasound of the spleen in liver disease. Eur J Gastroenterol Hepatol 16(2):147–155

    Article  Google Scholar 

  42. Odstrcilik J, Kolar R, Budai A, Hornegger J, Jan J, Gazarek J, Kubena T, Cernosek P, Svoboda O, Angelopoulou E (2013) Retinal vessel segmentation by improved matched filtering: evaluation on a new high-resolution fundus image database. IET Image Process 7(4):373–383

    Article  MathSciNet  Google Scholar 

  43. Papandreou G, Chen LC, Murphy KP, Yuille AL (2015) Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1742–1750

  44. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-assisted Intervention, Springer, pp. 234–241

  45. Sajjadi M, Javanmardi M, Tasdizen T (2016) Mutual exclusivity loss for semi-supervised deep learning. In: 2016 IEEE International Conference on Image Processing

  46. Selle D, Preim B, Schenk A, Peitgen HO (2002) Analysis of vasculature for liver surgical planning. IEEE Trans Med Imaging 21(11):1344–1357

    Article  Google Scholar 

  47. Shi W, Gong Y, Ding C, Ma Z, Tao X, Zheng N (2018) Transductive semi-supervised deep learning using min-max features. In: European Conference on Computer Vision

  48. Soares JV, Leandro JJ, Cesar RM, Jelinek HF, Cree MJ (2006) Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification. IEEE Trans Med Imaging 25(9):1214–1222

    Article  Google Scholar 

  49. Soler L, Hostettler A, Agnus V, Charnoz A, Fasquel J, Moreau J, Osswald A, Bouhadjar M, Marescaux J (2010) 3d image reconstruction for comparison of algorithm database: a patient specific anatomical and medical image database. Tech, Rep, IRCAD, Strasbourg, France

    Google Scholar 

  50. Tarvainen A, Valpola H (2017) Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. Adv Neural Inf Process Syst pp. 1195–1204

  51. Vorontsov E, Tang A, Pal C, Kadoury S (2018) Liver lesion segmentation informed by joint liver segmentation. In: 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), IEEE, pp. 1332–1335

  52. Wei Y, Xiao H, Shi H, Jie Z, Feng J, Huang TS (2018) Revisiting dilated convolution: A simple approach for weakly-and semi-supervised semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7268–7277

  53. Wong TY, Klein R, Sharrett AR, Duncan BB, Couper DJ, Tielsch JM, Klein BE, Hubbard LD (2002) Retinal arteriolar narrowing and risk of coronary heart disease in men and women: the atherosclerosis risk in communities study. JAMA 287(9):1153–1159

    Article  Google Scholar 

  54. Wu Y, Xia Y, Song Y, Zhang Y, Cai W (2018) Multiscale network followed network model for retinal vessel segmentation. In: MICCAI, pp. 119–126

  55. Xu X, Ding W, Abràmoff MD, Cao R (2017) An improved arteriovenous classification method for the early diagnostics of various diseases in retinal image. Comp. Methods Prog. Biomed. 141:3–9

    Article  Google Scholar 

  56. Yu , Wang S, Li X, Fu CW, Heng PA (2019) Uncertainty-aware self-ensembling model for semi-supervised 3d left atrium segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, pp. 605–613

  57. Yves Grandvalet YB (2005) Semi-supervised learning by entropy minimization. Adv Neural Inf Process Syst 17

  58. Zhang Z, Zhang X, Peng C, Xue X, Sun J (2018) Exfuse: Enhancing feature fusion for semantic segmentation. In: ECCV, pp. 269–284

  59. Zhao Y, Xie J, Su P, Zheng Y, Liu Y, Cheng J, Liu J (2018) Retinal artery and vein classification via dominant sets clustering-based vascular topology estimation. In: MICCAI, pp. 56–64

Download references

Acknowledgements

This work is supported in part by National Key Research and Development Program of China (No. 2019YFC0118100), in part of ZheJiang Province Key Research Development Program (No. 2020-C03073), in part by National Natural Science Foundation of China under Grants 81671766, 61971369, U19B2031, U1605252, 61671309, in part by Open Fund of Science and Technology on Automatic Target Recognition Laboratory 6142503190202, in part by Fundamental Research Funds for the Central Universities 20720180059, 20720190116, 20720200003, and in part by Tencent Open Fund.

Author information

Authors and Affiliations

  1. School of Informatics, Xiamen University, Xiamen, 361005, China

    Chenxin Li, Wenao Ma, Liyan Sun, Xinghao Ding & Yue Huang

  2. Department of Radiology, The Third Medical Centre, Chinese PLA General Hospital, Beijing, China

    Guisheng Wang

  3. Deepwise AI Laboratory, Beijing, 100125, China

    Yizhou Yu

Authors
  1. Chenxin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenao Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liyan Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinghao Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yue Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guisheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yizhou Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xinghao Ding.

Ethics declarations

Conflict of interest

Authors declare that there is no conflict of interests regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, C., Ma, W., Sun, L. et al. Hierarchical deep network with uncertainty-aware semi-supervised learning for vessel segmentation. Neural Comput & Applic 34, 3151–3164 (2022). https://doi.org/10.1007/s00521-021-06578-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06578-3

Keywords

Search

Navigation