Diabetic Retinal Grading Using Attention-Based Bilinear Convolutional Neural Network and Complement Cross Entropy
<p>Five types of lesion grades in the data set.</p> "> Figure 2
<p>Fine-grained image classification of gulls.</p> "> Figure 3
<p>Fine-grained image classification of diabetic retinal fundus disease.</p> "> Figure 4
<p>The structure of the proposed neural network model.</p> "> Figure 5
<p>Schematic diagram of the attention module.</p> "> Figure 6
<p>The distribution of images of each category in the training set and test sets.</p> "> Figure 7
<p>Comparison of compact bilinear pooling model.</p> "> Figure 8
<p>Comparison of homologous bilinear pooling and non-homologous bilinear pooling.</p> "> Figure 9
<p>Comparison of different convolutional neural network structures.</p> "> Figure 10
<p>Comparison of different loss functions.</p> "> Figure 11
<p>Comparison of attention mechanism modules.</p> "> Figure 12
<p>The visualization of attention mechanisms.</p> ">
Abstract
:1. Introduction
2. Related Work
3. Materials and Methods
3.1. Non-Homologous Convolutional Neural Network and Compact Bilinear Pooling Operation
3.2. Attention Mechanism Module
3.3. The Problem of Unbalanced Sample Distribution
4. Experiments and Results
4.1. Data Set
4.2. Ablation Test
4.3. Evaluation Index and the Experimental Evaluation Result of Our Network
4.4. The Experimental Environment and the Experimental Results
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Conflicts of Interest
References
- Araújo, T.; Aresta, G.; Mendonça, L.; Penas, S.; Maia, C.; Carneiro, Â.; Mendonça, A.M.; Campilho, A. DR|GRADUATE: Uncertainty-aware deep learning-based diabetic retinopathy grading in eye fundus images—ScienceDirect. Med. Image Anal. 2020, 63, 101715. [Google Scholar] [CrossRef] [PubMed]
- Peng, J.; Wang, Y. Medical Image Segmentation with Limited Supervision: A Review of Deep Network Models. IEEE Access 2021, 9, 36827–36851. [Google Scholar] [CrossRef]
- Wah, C.; Branson, S.; Welinder, P.; Perona, P.; Belongie, S. The Caltech-UCSD Birds-200-2011 Dataset. 2011. Available online: http://www.vision.caltech.edu/visipedia/CUB-200-2011.html (accessed on 23 June 2021).
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal Loss for Dense Object Detection. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2999–3007. [Google Scholar]
- Nguyen, H.; Butler, M.; Roychoudhry, A.; Shannon, A.; Flack, J.; Mitchell, P. Classification of diabetic retinopathy using neural networks. In Proceedings of the 18th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Amsterdam, The Netherlands, 31 October–3 November 1996; Volume 4, pp. 1548–1549. [Google Scholar]
- Xiaohui, Z.; Chutatape, A. Detection and classification of bright lesions in color fundus images. In Proceedings of the 2004 International Conference on Image Processing (ICIP ’04), Singapore, 24–27 October 2004; Volume 1, pp. 139–142. [Google Scholar]
- Zhang, X.; Chutatape, O. A SVM approach for detection of hemorrhages in background diabetic retinopathy. In Proceedings of the 2005 IEEE International Joint Conference on Neural Networks, Montreal, QC, Canada, 31 July–4 August 2005; Volume 4, pp. 2435–2440. [Google Scholar]
- Zhang, X.; Chutatape, O. Top-down and bottom-up strategies in lesion detection of background diabetic retinopathy. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR ’05), San Diego, CA, USA, 20–25 June 2005; Volume 2, pp. 422–428. [Google Scholar]
- Barriga, E.S.; Murray, V.; Agurto, C.; Pattichis, M.S.; Bauman, W.; Zamora, G.; Soliz, P. Automatic system for diabetic retinopathy screening based on AM-FM, partial least squares, and support vector machines. In Proceedings of the 2010 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Rotterdam, The Netherlands, 14–17 April 2010. [Google Scholar]
- Xu, L.; Luo, S. Support vector machine based method for identifying hard exudates in retinal images. In Proceedings of the IEEE Youth Conference on Information, Computing & Telecommunication, Beijing, China, 20–21 September 2009. [Google Scholar]
- Sinthanayothin, C.; Kongbunkiat, V.; Phoojaruenchanachai, S.; Singalavanija, A. Automated screening system for diabetic retinopathy. In Proceedings of the 3rd International Symposium on Image and Signal Processing and Analysis (ISPA 2003), Rome, Italy, 18–20 September 2003; Volume 2, pp. 915–920. [Google Scholar]
- Acharya, R.; Chua, C.K.; Ng, E.Y.; Yu, W.; Chee, C. Application of higher order spectra for the identification of diabetes retinopathy stages. J. Med. Syst. 2008, 32, 481–488. [Google Scholar] [CrossRef] [PubMed]
- Pratt, H.; Coenen, F.; Broadbent, D.M.; Harding, S.P.; Zheng, Y. Convolutional Neural Networks for Diabetic Retinopathy. Procedia Comput. Sci. 2016, 90, 200–205. [Google Scholar] [CrossRef] [Green Version]
- Kori, A.; Chennamsetty, S.S.; Alex, V. Ensemble of convolutional neural networks for automatic grading of diabetic retinopathy and macular edema. arXiv 2018, arXiv:1809.04228. [Google Scholar]
- Olivas, E.; Guerrero, J.; Martinez-Sober, M.; Magdalena-Benedito, J.; Serrano L¢pez, A. Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques; IGI Global: Hershey, PA, USA, 2010; pp. 242–264. [Google Scholar]
- Wang, Z.; Yang, J. Diabetic Retinopathy Detection via Deep Convolutional Networks for Discriminative Localization and Visual Explanation. In Proceedings of the Workshops at the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA, 2–7 February 2018. [Google Scholar]
- Bravo, M.A.; Arbeláez, P.A. Automatic diabetic retinopathy classification. In Proceedings of the 13th International Symposium on Medical Information Processing and Analysis, San Andres Island, Colombia, 5–7 October 2017. [Google Scholar]
- Fu, J.; Zheng, H.; Mei, T. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4438–4446. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-Excitation Networks. arXiv 2017, arXiv:1709.01507. [Google Scholar]
- Huang, G.; Liu, Z.; Laurens, V.D.M.; Weinberger, K.Q. Densely Connected Convolutional Networks. arXiv 2016, arXiv:1608.06993. [Google Scholar]
- Simonyan, K.; Zisserman, A.J. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Rabinovich, A. Going Deeper with Convolutions. arXiv 2014, arXiv:1409.4842. [Google Scholar]
- Targ, S.; Almeida, D.; Lyman, K. Resnet in Resnet: Generalizing Residual Architectures. arXiv 2016, arXiv:1603.08029. [Google Scholar]
- Lin, T.Y.; Roychowdhury, A.; Maji, S. Bilinear Convolutional Neural Networks for Fine-grained Visual Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 40, 1309–1322. [Google Scholar] [CrossRef] [PubMed]
- Kim, Y.; Lee, Y.; Jeon, M. Imbalanced Image Classification with Complement Cross Entropy. arXiv 2020, arXiv:2009.02189. [Google Scholar]
- Lin, T.Y.; Roychowdhury, A.; Maji, S. Bilinear CNN Models for Fine-grained Visual Recognition. arXiv 2015, arXiv:1504.07889. [Google Scholar]
- Kang, B.; Xie, S.; Rohrbach, M.; Yan, Z.; Gordo, A.; Feng, J.; Kalantidis, Y. Decoupling representation and classifier for long-tailed recognition. arXiv 2019, arXiv:1910.09217. [Google Scholar]
- Tang, K.; Huang, J.; Zhang, H. Long-tailed classification by keeping the good and removing the bad momentum causal effect. arXiv 2020, arXiv:2009.12991. [Google Scholar]
- Zhou, B.; Cui, Q.; Wei, X.S.; Chen, Z.M. BBN: Bilateral-branch network with cumulative learning for long-tailed visual recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 9719–9728. [Google Scholar]
- Zhao, Z.; Xu, X. BiRA-Net: Bilinear Attention Net for Diabetic Retinopathy Grading. In Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP), Taipei, Taiwan, 22–25 September 2019. [Google Scholar]
- Zhao, Z.; Chopra, K.; Zeng, Z.; Li, X. Sea-Net: Squeeze-And-Excitation Attention Net for Diabetic Retinopathy Grading. In Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates, 25–28 October 2020; pp. 2496–2500. [Google Scholar]
Evaluation Index | Accuracy | Recall Rate | Precision | (F1-Score) |
---|---|---|---|---|
Result | 0.823 | 0.819 | 0.828 | 0.823 |
Hardware or Software Environment | Name |
---|---|
Sytstem Version | Ubuntu 16.04 |
CPU | Intel(R) Xeon(R) W-2150B CPU @ 3.00 GHz |
GPU | NVIDIA RTX2080TI |
Framework | Pytorch 1.4.0 |
Driver | 440.82 |
Cuda | 10.2 |
Method | Accuracy | Precision | Recall Rate | (F1-Score) |
---|---|---|---|---|
ResNet50 | 0.740 | 0.723 | 0.791 | 0.755 |
DenseNet121 | 0.797 | 0.714 | 0.793 | 0.751 |
BIRA-Net | 0.815 | 0.749 | 0.816 | 0.781 |
SEA-Net | 0.819 | 0.787 | 0.824 | 0.805 |
Ours | 0.823 | 0.819 | 0.828 | 0.823 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, P.; Yang, X.; Jin, B.; Zhou, Q. Diabetic Retinal Grading Using Attention-Based Bilinear Convolutional Neural Network and Complement Cross Entropy. Entropy 2021, 23, 816. https://doi.org/10.3390/e23070816
Liu P, Yang X, Jin B, Zhou Q. Diabetic Retinal Grading Using Attention-Based Bilinear Convolutional Neural Network and Complement Cross Entropy. Entropy. 2021; 23(7):816. https://doi.org/10.3390/e23070816
Chicago/Turabian StyleLiu, Pingping, Xiaokang Yang, Baixin Jin, and Qiuzhan Zhou. 2021. "Diabetic Retinal Grading Using Attention-Based Bilinear Convolutional Neural Network and Complement Cross Entropy" Entropy 23, no. 7: 816. https://doi.org/10.3390/e23070816