Adenocarcinoma Recognition in Endoscopy Images Using Optimized Convolutional Neural Networks
<p>An illustration of the architecture of AlexNet.</p> "> Figure 2
<p>Residual learning: a building block in ResNet.</p> "> Figure 3
<p>A five-layer dense block in DenseNet.</p> "> Figure 4
<p>Proposed network architecture. Note that each long rectangle means “convolution layer”.</p> "> Figure 5
<p>Influence of the number of convolution layers. (<b>a</b>) Influence on neural networks; (<b>b</b>) influence on 2D images.</p> "> Figure 6
<p>Influence area of the convolution product according to the size of the 2D image.</p> "> Figure 7
<p>Endoscopy image type in this experiment. (<b>a</b>) Normal images; (<b>b</b>) adenoma images; (<b>c</b>) adenocarcinoma images.</p> "> Figure 8
<p>The inference time per image is estimated for the same batch size of 8 for all methods.</p> ">
Abstract
:1. Introduction
2. Image Classification Using Deep Learning
3. Proposed Network Architecture
3.1. Structure
3.2. Number of Convolution Layers
4. Experiments
4.1. Experimental Data
4.2. Experiments of Convolution Layer
4.3. Experimental Evaluation
4.4. Network Training
4.5. Performance Evaluation
5. Discussion
6. Conclusions
7. Data Availability
Author Contributions
Funding
Conflicts of Interest
References
- Watson, J.D.; Crick, F. A structure for deoxyribose nucleic acid. Nature 1953, 171, 737–738. [Google Scholar] [CrossRef]
- Hixson, L.; Fennerty, M.B.; Sampliner, R.; McGee, D.; Garewal, H. Prospective study of the frequency and size distribution of polyps missed by colonoscopy. J. Natl. Cancer Inst. 1990, 82, 1769–1772. [Google Scholar] [CrossRef]
- Litjens, G.; Kooi, T.; Bejnordi, B.E.; Setio, A.A.A.; Ciompi, F.; Ghafoorian, M.; Van Der Laak, J.A.; Van Ginneken, B.; Sánchez, C.I. A survey on deep learning in medical image analysis. Med. Image Anal. 2017, 42, 60–88. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Häfner, M.; Tamaki, T.; Tanaka, S.; Uhl, A.; Wimmer, G.; Yoshida, S. Local fractal dimension based approaches for colonic polyp classification. Med. Image Anal. 2015, 26, 92–107. [Google Scholar] [CrossRef] [PubMed]
- Wimmer, G.; Tamaki, T.; Tischendorf, J.J.; Häfner, M.; Yoshida, S.; Tanaka, S.; Uhl, A. Directional wavelet based features for colonic polyp classification. Med. Image Anal. 2016, 31, 16–36. [Google Scholar] [CrossRef]
- Tamaki, T.; Yoshimuta, J.; Kawakami, M.; Raytchev, B.; Kaneda, K.; Yoshida, S.; Takemura, Y.; Onji, K.; Miyaki, R.; Tanaka, S. Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med. Image Anal. 2013, 17, 78–100. [Google Scholar] [CrossRef] [PubMed]
- Stehle, T.; Auer, R.; Gross, S.; Behrens, A.; Wulff, J.; Aach, T.; Winograd, R.; Trautwein, C.; Tischendorf, J. Classification of colon polyps in NBI endoscopy using vascularization features. In Medical Imaging 2009: Computer-Aided Diagnosis; International Society for Optics and Photonics: Bellingham, WA, USA, 2009; Volume 7260, p. 72602S. [Google Scholar]
- LeCun, Y.; Bottou, L.; Bengio, Y.; Haffner, P. Gradient-based learning applied to document recognition. Proc. IEEE 1998, 86, 2278–2324. [Google Scholar] [CrossRef] [Green Version]
- Takiyama, H.; Ozawa, T.; Ishihara, S.; Fujishiro, M.; Shichijo, S.; Nomura, S.; Miura, M.; Tada, T. Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks. Sci. Rep. 2018, 8, 7497. [Google Scholar] [CrossRef] [PubMed]
- Itoh, T.; Kawahira, H.; Nakashima, H.; Yata, N. Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images. Endosc. Int. Open 2018, 6, E139–E144. [Google Scholar] [CrossRef] [Green Version]
- Shichijo, S.; Nomura, S.; Aoyama, K.; Nishikawa, Y.; Miura, M.; Shinagawa, T.; Takiyama, H.; Tanimoto, T.; Ishihara, S.; Matsuo, K.; et al. Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images. EBioMedicine 2017, 25, 106–111. [Google Scholar] [CrossRef] [Green Version]
- Hirasawa, T.; Aoyama, K.; Tanimoto, T.; Ishihara, S.; Shichijo, S.; Ozawa, T.; Ohnishi, T.; Fujishiro, M.; Matsuo, K.; Fujisaki, J.; et al. Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 2018, 21, 653–660. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Komeda, Y.; Handa, H.; Watanabe, T.; Nomura, T.; Kitahashi, M.; Sakurai, T.; Okamoto, A.; Minami, T.; Kono, M.; Arizumi, T.; et al. Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: Preliminary experience. Oncology 2017, 93, 30–34. [Google Scholar] [CrossRef] [PubMed]
- Byrne, M.F.; Chapados, N.; Soudan, F.; Oertel, C.; Pérez, M.L.; Kelly, R.; Iqbal, N.; Chandelier, F.; Rex, D.K. Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model. Gut 2019, 68, 94–100. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zhang, R.; Zheng, Y.; Mak, T.W.C.; Yu, R.; Wong, S.H.; Lau, J.Y.; Poon, C.C. Automatic detection and classification of colorectal polyps by transferring low-level CNN features from nonmedical domain. IEEE J. Biomed. Health Inform. 2016, 21, 41–47. [Google Scholar] [CrossRef]
- Yu, L.; Chen, H.; Dou, Q.; Qin, J.; Heng, P.A. Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J. Biomed. Health Inform. 2016, 21, 65–75. [Google Scholar] [CrossRef]
- Esteva, A.; Kuprel, B.; Novoa, R.A.; Ko, J.; Swetter, S.M.; Blau, H.M.; Thrun, S. Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017, 542, 115. [Google Scholar] [CrossRef]
- Gulshan, V.; Peng, L.; Coram, M.; Stumpe, M.C.; Wu, D.; Narayanaswamy, A.; Venugopalan, S.; Widner, K.; Madams, T.; Cuadros, J.; et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016, 316, 2402–2410. [Google Scholar] [CrossRef]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826. [Google Scholar]
- Antony, J.; McGuinness, K.; O’Connor, N.E.; Moran, K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. In Proceedings of the 2016 23rd International Conference on Pattern Recognition (ICPR), Cancún, Mexico, 4–8 December 2016; pp. 1195–1200. [Google Scholar]
- Roth, H.R.; Lu, L.; Seff, A.; Cherry, K.M.; Hoffman, J.; Wang, S.; Liu, J.; Turkbey, E.; Summers, R.M. A new 2.5 D representation for lymph node detection using random sets of deep convolutional neural network observations. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Boston, MA, USA, 14–18 September 2014; Springer: Cham, Switzerland, 2014; pp. 520–527. [Google Scholar]
- Havaei, M.; Davy, A.; Warde-Farley, D.; Biard, A.; Courville, A.; Bengio, Y.; Pal, C.; Jodoin, P.M.; Larochelle, H. Brain tumor segmentation with deep neural networks. Med. Image Anal. 2017, 35, 18–31. [Google Scholar] [CrossRef] [Green Version]
- Pereira, S.; Pinto, A.; Alves, V.; Silva, C.A. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Trans. Med Imaging 2016, 35, 1240–1251. [Google Scholar] [CrossRef]
- Shen, W.; Zhou, M.; Yang, F.; Yang, C.; Tian, J. Multi-scale convolutional neural networks for lung nodule classification. In Proceedings of the International Conference on Information Processing in Medical Imaging, Isle of Skye, UK, 28 June–3 July 2015; Springer: Cham, Switzerland, 2015; pp. 588–599. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012; pp. 1097–1105. [Google Scholar]
- Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on Machine Learning (ICML-10), Haifa, Israel, 21–24 June 2010; pp. 807–814. [Google Scholar]
- Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 2014, 15, 1929–1958. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Huang, G.; Liu, Z.; Van Der Maaten, L.; Weinberger, K.Q. Densely connected convolutional networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 4700–4708. [Google Scholar]
- Tieleman, T.; Hinton, G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA Neural Netw. Mach. Learn. 2012, 4, 26–31. [Google Scholar]
The Number of Original Images | The Number of Rotated Images | Total Training Images | |
---|---|---|---|
Normal | 773 | 15,460 | 16,233 |
Adenoma | 626 | 15,990 | 16,616 |
Adenocarcinoma | 449 | 16,160 | 16,609 |
The Number of Test Images | |
---|---|
Normal | 128 |
Adenoma | 142 |
Adenocarcinoma | 140 |
Layers | 21-Layer | 30-Layer | 41-Layer | 43-Layer-a | 43-Layer-b | 43-Layer-c | 43-Layer-d |
---|---|---|---|---|---|---|---|
conv1 | [3 × 3128] × 4 | [3 × 3128] × 6 | [3 × 3128] × 8 | [3 × 3128] × 15 | [3 × 3128] × 9 | [3 × 3128] × 6 | [3 × 3128] × 6 |
Pooling | 2 × 2 ma × pool, stride 2 | ||||||
conv2 | [3 × 3128] × 4 | [3 × 3128] × 6 | [3 × 3128] × 8 | [3 × 3128] × 9 | [3 × 3128] × 15 | [3 × 3128] × 9 | [3 × 3128] × 8 |
Pooling | 2 × 2 ma × pool, stride 2 | ||||||
conv3 | [3 × 3128] × 4 | [3 × 3,128] × 6 | [3 × 3128] × 8 | [3 × 3128] × 6 | [3 × 3128] × 6 | [3 × 3128] × 15 | [3 × 3128] × 9 |
Pooling | 2 × 2 ma × pool, stride 2 | ||||||
conv4 | [3 × 3128] × 4 | [3 × 3128] × 6 | [3 × 3128] × 8 | [3 × 3128] × 8 | [3 × 3128] × 8 | [3 × 3128] × 8 | [3 × 3128] × 15 |
Pooling | 2 × 2 ma × pool, stride 2 | ||||||
conv5 | [3 × 3128] × 4 | [3 × 3128] × 6 | [3 × 3128] × 8 | [3 × 3128] × 5 | [3 × 3128] × 5 | [3 × 3128] × 5 | [3 × 3128] × 5 |
Pooling | 7 × 7 average pool, stride 2 | ||||||
Pooling | 1024D fully-connected | ||||||
Pooling | 3D fully-connected |
Layers | 21-Layer | 30-Layer | 41-Layer | 43-Layer-a | 43-Layer-b | 43-Layer-c | 43-Layer-d |
---|---|---|---|---|---|---|---|
Accuracy | 62% | 68% | 80% | 83% | 86% | 91% | 88% |
Model | Processed Images per Second | Parameters (in Million) |
---|---|---|
VGG | 96 | 138 |
ResNet | 102 | 25.5 |
DenseNet | 104 | 27.2 |
Proposed Method | 38 | 6.5 |
Actual Class | Predicted Class | ||
---|---|---|---|
Normal | Adenoma | Adenocarcinoma | |
Normal | 120 | 8 | 0 |
Adenoma | 3 | 135 | 4 |
Adenocarcinoma | 3 | 5 | 132 |
Actual Class | Predicted Class | Error Images | |
---|---|---|---|
(a) | Normal | Adenocarcinoma | |
(b) | Adenoma | Normal | |
(c) | Adenoma | Adenocarcinoma | |
(d) | Adenocarcinoma | Normal | |
(e) | Adenocarcinoma | Adenoma |
Class | Sensitivity (%) | Precision (%) | Accuracy (%) |
---|---|---|---|
Normal | 93.37 | 95.23 | |
Adenoma | 95.07 | 91.21 | 94.39 (%) |
Adenocarcinoma | 94.28 | 97.05 |
CNN Networks | Accuracy (%) |
---|---|
VGG19 | 87 |
ResNet | 90 |
DenseNet | 89 |
Proposed Architecture | 94 |
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Park, H.-C.; Kim, Y.-J.; Lee, S.-W. Adenocarcinoma Recognition in Endoscopy Images Using Optimized Convolutional Neural Networks. Appl. Sci. 2020, 10, 1650. https://doi.org/10.3390/app10051650
Park H-C, Kim Y-J, Lee S-W. Adenocarcinoma Recognition in Endoscopy Images Using Optimized Convolutional Neural Networks. Applied Sciences. 2020; 10(5):1650. https://doi.org/10.3390/app10051650
Chicago/Turabian StylePark, Hyun-Cheol, Yoon-Jae Kim, and Sang-Woong Lee. 2020. "Adenocarcinoma Recognition in Endoscopy Images Using Optimized Convolutional Neural Networks" Applied Sciences 10, no. 5: 1650. https://doi.org/10.3390/app10051650