Breast Tumor Tissue Image Classification Using DIU-Net
<p>System architecture of the proposed method. For the binary classification case, the class output corresponds to benign/malignant, while for the multi-classification case, the class output corresponds to the eight subcategories.</p> "> Figure 2
<p>(<b>a</b>) A pathological tissue patch. (<b>b</b>) Hard segmentation result of (<b>a</b>). (<b>c</b>) Soft segmentation result of (<b>a</b>).</p> "> Figure 3
<p>Dilated Inception block.</p> "> Figure 4
<p>The architecture of the DIU-Net, where convolutional layers are replaced by DIs and the numbers indicate the number of channels.</p> "> Figure 5
<p>The architecture of the BHCNet-3.</p> "> Figure 6
<p>Sample images of four sub-datasets for segmentation: (<b>a</b>) UCSB; (<b>b</b>) TNBC; (<b>c</b>) 2018DSB; (<b>d</b>) MoNuSeg.</p> "> Figure 7
<p>The segmentation results using different models without the C3S for a sample image of the 2018 DSB sub-dataset. The white pixel represents FP (false positive), while the yellow pixel represents FN (false negative). (<b>a</b>) Image; (<b>b</b>) Ground Truth; (<b>c</b>) U-Net; (<b>d</b>) Att-U-Net; (<b>e</b>) R2U-Net; (<b>f</b>) U-Net++; (<b>g</b>) Ours.</p> "> Figure 8
<p>The segmentation results using different models with the C3S during the training phase for the same image in <a href="#sensors-22-09838-f007" class="html-fig">Figure 7</a>.</p> "> Figure 9
<p>The multi-class confusion matrix of the proposed model for (<b>a</b>) 200× test set and (<b>b</b>) 400× test set.</p> "> Figure 10
<p>The segmentation results of the proposed method: (<b>a</b>) the input images; (<b>b</b>) segmented results of our method.</p> "> Figure 11
<p>The impact of choosing different <math display="inline"><semantics> <mrow> <msub> <mi>λ</mi> <mi>c</mi> </msub> </mrow> </semantics></math> on accuracy for binary classification.</p> "> Figure 12
<p>The impact of choosing different <math display="inline"><semantics> <mrow> <msub> <mi>λ</mi> <mi>c</mi> </msub> </mrow> </semantics></math> on accuracy for multi-class classification.</p> ">
Abstract
:1. Introduction
- (1)
- A segmentation–classification joint training mechanism was proposed to enhance the classification performance for breast cancer pathology images.
- (2)
- A Dilated Inception U-Net model with better cross-scale description ability was proposed to enhance the performance of nuclei segmentation.
- (3)
- A Complementary Color Conversion Scheme was proposed to enhance the generalization ability of the nuclei segmentation.
- (4)
- The proposed model can provide visual information of the model to the pathologist so that the pathologist can understand where the area concerned by the model is.
2. Method
2.1. System Architecture
2.2. Segmentation Network
2.3. Classification Network
2.4. Training
3. Experiments
3.1. Datasets
3.2. Evaluating the Segmentation Performance
3.3. Evaluating the Classification Performance
3.4. Ablation Study
4. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Chun, M.C. Breast Cancer: Symptoms, Risk Factors, and Treatment, Medical News Today. 2018. Available online: https://www.medicalnewstoday.com/articles/37136.php (accessed on 10 March 2018).
- World Health Organization. Available online: http://www.who.int/en/ (accessed on 10 March 2018).
- Boyle, P.; Levin, B. World Cancer Report. 2008. Available online: https://publications.iarc.fr/Non-Series-Publications/World-Cancer-Reports/World-Cancer-Report-2008 (accessed on 16 October 2022).
- Lim, C.N.; Suliong, C.; Rao, C.V.; Aung, T.; Sieman, J.; Tin, W. Recent Advances in Breast Cancer Diagnosis Entering an Era of Precision Medicine. Borneo J. Med. Sci. (BJMS) 2019, 13. [Google Scholar] [CrossRef]
- Chen, D.R.; Lin, Y.C.; Huang, Y.L. 3D Contouring for Breast Tumor in Sonography. arXiv 2019, arXiv:190109407. [Google Scholar]
- Horvat, J.V.; Keating, D.M.; Rodrigues-Duarte, H.; Morris, E.A.; Mango, V.L. Calcifications at Digital Breast Tomosynthesis: Imaging Features and Biopsy Techniques. RadioGraphics 2019, 39, 307. [Google Scholar] [CrossRef] [PubMed]
- He, L.; Long, L.R.; Antani, S.; Thoma, G. Computer assisted diag nosis in histopathology, Sequence Genome Anal. Methods Appl. 2010, 3, 271–287. [Google Scholar]
- He, L.; Long, L.R.; Antani, S.; Thoma, G.R. Histology image analysis for carcinoma detection and grading. Comput. Methods Programs Biomed. 2012, 107, 538–556. [Google Scholar] [CrossRef] [Green Version]
- Das, K.; Conjeti, S.; Roy, A.G.; Chatterjee, J.; Sheet, D. Multiple instance learning of deep convolutional neural networks for breast histopathology whole slide classification. In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI), Washington, DC, USA, 4–7 April 2018; pp. 578–581. [Google Scholar]
- Araújo, T.; Aresta, G.; Castro, E.; Rouco, J.; Aguiar, P.; Eloy, C.; Campilho, A. Classification of breast cancer histology images using convolutional neural networks. PLoS ONE 2017, 12, e0177544. [Google Scholar] [CrossRef]
- Spanhol, F.A.; Oliveira, L.S.; Petitjean, C.; Heutte, L. A dataset for breast cancer histopathological image classification. IEEE Trans. Biomed. Eng. 2016, 63, 1455–1462. [Google Scholar] [CrossRef]
- Ojala, T.; Pietikainen, M.; Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. In IEEE Transactions on Pattern Analysis and Machine Intelligence; IEEE Computer Society: Washington, DC, USA, 2002; Volume 24, pp. 971–987. [Google Scholar]
- Guo, Z.; Zhang, L.; Zhang, D. A completed modeling of local binary pattern operator for texture classification. IEEE Trans. Image Process. 2010, 19, 1657–1663. [Google Scholar]
- Ojansivu, V.; Heikkilä, J. Blur insensitive texture classification using local phase quantization. In Proceedings of the 3rd International Conference on Image and Signal Processing (ICISP), Cherbourg-Octeville, France, 1–3 July 2008; Volume 5099, pp. 236–243. [Google Scholar]
- Haralick, R.; Shanmugam, K.; Dinstein, I. Textural features for image classification. IEEE Trans. Syst. Man Cybern. 1973, 3, 610–621. [Google Scholar] [CrossRef] [Green Version]
- Hamilton, N.A.; Pantelic, R.S.; Hanson, K.; Teasdale, R.D. Fast automated cell phenotype image classification. BMC Bioinform. 2007, 8, 110. [Google Scholar] [CrossRef] [Green Version]
- Rublee, E.; Rabaud, V.; Konolige, K.; Bradski, G. ORB: An efficient alternative to SIFT or SURF. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 2564–2571. [Google Scholar]
- Chan, A.; Tuszynski, J.A. Automatic prediction of tumour malignancy in breast cancer with fractal dimension. Open Sci. 2016, 3, 160558. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kahya, M.A.; Al-Hayani, W.; Algamal, Z.Y. Classification of breast cancer histopathology images based on adaptive sparse support vector machine. J. Appl. Math. Bioinform. 2017, 7, 49. [Google Scholar]
- Bardou, D.; Zhang, K.; Ahmad, S.M. Classification of Breast Cancer Based on Histology Images Using Convolutional Neural Networks. IEEE Access 2018, 6, 24680–24693. [Google Scholar] [CrossRef]
- Jannesari, M.; Habibzadeh, M.; Aboulkheyr, H.; Khosravi, P.; Elemento, O.; Totonchi, M.; Hajirasouliha, I. Breast Cancer Histopathological Image Classification: A Deep Learning Approach. In Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain, 3–6 December 2018; pp. 2405–2412. [Google Scholar]
- Jiang, Y.; Chen, L.; Zhang, H.; Xiao, X. Breast cancer histopathological image classification using convolutional neural networks with small SE-ResNet module. PLoS ONE 2019, 14, e0214587. [Google Scholar] [CrossRef] [PubMed]
- Adeshina, S.A.; Adedigba, A.P.; Adeniyi, A.A.; Aibinu, A.M. Breast cancer histopathology image classification with deep convolutional neural networks. In Proceedings of the 2018 14th International Conference on Electronics Computer and Computation (ICECCO), Kaskelen, Kazakhstan, 29 November–1 December 2018; pp. 206–212. [Google Scholar]
- George, K.; Sankaran, P.; Joseph, K. Computer assisted recognition of breast cancer in biopsy images via fusion of nucleus-guided deep convolutional features. Comput. Methods Programs Biomed. 2020, 194, 105531. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional Networks for Biomedical Image Segmentation. In 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Oktay, O.; Schlemper, J.; Folgoc, L.L.; Lee, M.; Heinrich, M.; Misawa, K.; Rueckert, D. Attention U-Net: Learning where to look for the pancreas. arXiv 2018, arXiv:1804.03999. [Google Scholar]
- Alom, M.Z.; Hasan, M.; Yakopcic, C.; Taha, T.M.; Asari, V.K. Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation. arXiv 2018, arXiv:1802.06955. [Google Scholar]
- Zhou, Z.; Rahman Siddiquee, M.M.; Tajbakhsh, N.; Liang, J. Unet++: A nested u-net architecture for medical image segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Springer: Berlin/Heidelberg, Germany, 2018; pp. 3–11. [Google Scholar]
- Szegedy, C.; Liu, W.; Jia, Y.; Sermanet, P.; Reed, S.; Anguelov, D.; Rabinovich, A. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 1–9. [Google Scholar]
- Yu, F.; Koltun, V. Multi-scale context aggregation by dilated convolutions. In Proceedings of the 4th International Conference on Learning Representations, San Juan, Puerto Rico, 2–4 May 2016. [Google Scholar]
- Gour, M.; Jain, S.; Kumar, T.S. Residual learning based CNN for breast cancer histopathological image classification. Int. J. Imaging Syst. Technol. 2020, 30, 621–635. [Google Scholar] [CrossRef]
- Hu, C.; Sun, X.; Yuan, Z.; Wu, Y. Classification of breast cancer histopathological image with deep residual learning. Int. J. Imaging Syst. Technol. 2021, 31, 1583–1594. [Google Scholar] [CrossRef]
- Gelasca, E.D.; Byun, J.; Obara, B.; Manjunath, B.S. Evaluation and benchmark for biological image segmentation. In Proceedings of the 2008 15th IEEE International Conference on Image Processing, San Diego, CA, USA, 12–15 October 2008. [Google Scholar]
- Naylor, P.; Lae, M.; Reyal, F.; Walter, T. Segmentation of nuclei in histopathology images by deep regression of the distance map. IEEE Trans. Med. Imaging 2018, 38, 448–459. [Google Scholar] [CrossRef]
- Booz Allen Hamilton. 2018 Data Science Bowl. Kaggle. Available online: https://www.kaggle.com/c/data-science-bowl-2018 (accessed on 16 October 2022).
- Kumar, N.; Verma, R.; Sharma, S.; Bhargava, S.; Vahadane, A.; Sethi, A. A Dataset and a Technique for Generalized Nuclear Segmentation for Computational Pathology. IEEE Trans. Med. Imaging 2017, 36, 1550–1560. [Google Scholar] [CrossRef] [PubMed]
U-Net | Att-U-Net | R2U-Net | U-Net++ | Ours | ||
---|---|---|---|---|---|---|
Validation Set | ||||||
Dice Coefficient | 0.69 | 0.68 | 0.35 | 0.68 | 0.70 | |
Test Set | ||||||
Dice Coefficient | 0.39 | 0.40 | 0.36 | 0.36 | 0.41 |
U-Net | Att-U-Net | R2U-Net | U-Net++ | Ours | ||
---|---|---|---|---|---|---|
Validation Set | ||||||
Dice Coefficient | 0.84 | 0.84 | 0.24 | 0.85 | 0.85 | |
Test Set | ||||||
Dice Coefficient | 0.50 | 0.53 | 0.33 | 0.52 | 0.54 |
Focal Loss | Dice Loss | Dice Coefficient (without C3S) | Dice Coefficient (with C3S) |
---|---|---|---|
Validation Set | |||
✓ | ✗ | 0.66 | 0.84 |
✗ | ✓ | 0.66 | 0.82 |
✓ | ✓ | 0.70 | 0.85 |
Test Set | |||
✓ | ✗ | 0.41 | 0.47 |
✗ | ✓ | 0.34 | 0.44 |
✓ | ✓ | 0.41 | 0.54 |
Methods | 200× | 400× | Total |
---|---|---|---|
CNN [20] | 96.36 | 95.97 | 96.16 |
ResNet [21] | 93.64 | 93.16 | 93.4 |
BHCNet [22] | 97.2 | 96.96 | 97.04 |
NucDeep [24] | 96.21 | ✗ | 96.21 |
ResHist [31] | 91.15 | 86.27 | ✗ |
myResNet-34 [32] | 90.47 | 88.79 | ✗ |
Ours | 97.24 | 98.19 | 97.74 |
Methods | 200× | 400× | Total |
---|---|---|---|
CNN [20] | 80.83 | 81.03 | 80.93 |
ResNet [21] | 76.54 | 79.58 | 78.06 |
BHCNet [22] | 92.27 | 91.15 | 91.71 |
NucDeep [24] | 63.3 | ✗ | 63.3 |
Ours | 93.75 | 94.43 | 94.09 |
Methods | Binary Classification | Multi-Class Classification |
---|---|---|
Without C3S | 97.18 | 92.67 |
Without DI | 97.04 | 91.71 |
Separated training version | 91.42 | 75.98 |
Ours | 97.74 | 94.09 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Lee, J.-S.; Wu, W.-K. Breast Tumor Tissue Image Classification Using DIU-Net. Sensors 2022, 22, 9838. https://doi.org/10.3390/s22249838
Lee J-S, Wu W-K. Breast Tumor Tissue Image Classification Using DIU-Net. Sensors. 2022; 22(24):9838. https://doi.org/10.3390/s22249838
Chicago/Turabian StyleLee, Jiann-Shu, and Wen-Kai Wu. 2022. "Breast Tumor Tissue Image Classification Using DIU-Net" Sensors 22, no. 24: 9838. https://doi.org/10.3390/s22249838