Abstract
Image classification is one of the most important areas in computer vision. Hierarchical multi-label classification applies when a multi-class image classification problem is arranged into smaller ones based upon a hierarchy or taxonomy. Thus, hierarchical classification modes generally provide multiple class predictions on each instance, whereby these are expected to reflect the structure of image classes as related to one another. In this paper, we propose a multi-label capsule network (ML-CapsNet) for hierarchical classification. Our ML-CapsNet predicts multiple image classes based on a hierarchical class-label tree structure. To this end, we present a loss function that takes into account the multi-label predictions of the network. As a result, the training approach for our ML-CapsNet uses a coarse to fine paradigm while maintaining consistency with the structure in the classification levels in the label-hierarchy. We also perform experiments using widely available datasets and compare the model with alternatives elsewhere in the literature. In our experiments, our ML-CapsNet yields a margin of improvement with respect to these alternative methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bahadori, M.T.: Spectral capsule networks. In: International Conference on Learning Representations Workshops (2018)
Chen, B., Huang, X., Xiao, L., Jing, L.: Hyperbolic capsule networks for multi-label classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3115–3124 (2020)
Davis, J., Liang, T., Enouen, J., Ilin, R.: Hierarchical classification with confidence using generalized logits. In: 2020 25th International Conference on Pattern Recognition (ICPR), pp. 1874–1881 (2021)
Dempster, A., Laird, N., Rubin, D.: Maximum-likehood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B Methodol. 39, 1–38 (1977)
Deng, J., Krause, J., Berg, A.C., Fei-Fei, L.: Hedging your bets: optimizing accuracy-specificity trade-offs in large scale visual recognition. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3450–3457 (2012)
Deng, L.: The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process. Mag. 29(6), 141–142 (2012)
Dhall, A., Makarova, A., Ganea, O., Pavllo, D., Greeff, M., Krause, A.: Hierarchical image classification using entailment cone embeddings. In: Computer Vision and Pattern Recognition Workshops, pp. 836–837 (2020)
Dimitrovski, I., Kocev, D., Loskovska, S., Džeroski, S.: Hierarchical annotation of medical images. Pattern Recogn. 44(10), 2436–2449 (2011)
Hahn, T., Pyeon, M., Kim, G.: Self-routing capsule networks. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
Hinton, G.E., Krizhevsky, A., Wang, S.D.: Transforming auto-encoders. In: International Conference on Artificial Neural Networks, pp. 44–51 (2011)
Hinton, G.E., Sabour, S., Frosst, N.: Matrix capsules with EM routing. In: International Conference on Learning Representations (2018)
Hussain, Z., et al.: Automatic understanding of image and video advertisements. In: Computer Vision and Pattern Recognition, pp. 1705–1715 (2017)
Jampour, M., Abbaasi, S., Javidi, M.: Capsnet regularization and its conjugation with resnet for signature identification. Pattern Recogn. 120, 107851 (2021)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images (2009)
Meng, Y., Shen, J., Zhang, C., Han, J.: Weakly-supervised hierarchical text classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 6826–6833 (2019)
Neill, J.O.: Siamese capsule networks. arXiv E-preprints (2018)
Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Computer Vision and Pattern Recognition, pp. 7263–7271 (2017)
Ren, H., Yu, X., Zou, L., Zhou, Y., Wang, X., Bruzzone, L.: Extended convolutional capsule network with application on SAR automatic target recognition. Signal Process. 183, 108021 (2021)
Rousu, J., Saunders, C., Szedmak, S., Shawe-Taylor, J.: Kernel-based learning of hierarchical multilabel classification models. J. Mach. Learn. Res. 7, 1601–1626 (2006)
Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Seo, Y., Shin, K.S.: Hierarchical convolutional neural networks for fashion image classification. Expert Syst. Appl. 116, 328–339 (2019)
Ubaru, S., Dash, S., Mazumdar, A., Günlük, O.: Multilabel classification by hierarchical partitioning and data-dependent grouping. In: Advances in Neural Information Processing Systems (2020)
Upadhyay, Y., Schrater, P.: Generative adversarial network architectures for image synthesis using capsule networks. arXiv E-preprint (2018)
Vens, C., Struyf, J., Schietgat, L., Džeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Mach. Learn. 73(2), 185–214 (2008)
Wang, M., Deng, W.: Deep face recognition: a survey. Neurocomputing 429, 215–244 (2021)
Wang, Z., Zhan, J., Duan, C., Guan, X., Lu, P., Yang, K.: A review of vehicle detection techniques for intelligent vehicles. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Wehrmann, J., Cerri, R., Barros, R.: Hierarchical multi-label classification networks. In: International Conference on Machine Learning, pp. 5075–5084 (2018)
Xiang, C., Zhang, L., Tang, Y., Zou, W., Xu, C.: MS-CapsNet: a novel multi-scale capsule network. IEEE Signal Process. Lett. 25(12), 1850–1854 (2018)
Xiao, H., Rasul, K., Vollgraf, R.: Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)
Yan, Z., et al.: HD-CNN: hierarchical deep convolutional neural networks for large scale visual recognition. In: International Conference on Computer Vision, pp. 2740–2748 (2015)
Zhang, H., Cisse, M., Dauphin, Y.N., Lopez-Paz, D.: mixup: beyond empirical risk minimization. In: International Conference on Learning Representations (2018)
Zhang, Z., Xie, Y., Xing, F., McGough, M., Yang, L.: MDNet: a semantically and visually interpretable medical image diagnosis network. In: Computer Vision and Pattern Recognition, pp. 6428–6436 (2017)
Zhao, Y., Birdal, T., Deng, H., Tombari, F.: 3D point capsule networks. In: Computer Vision and Pattern Recognition (2019)
Zhu, X., Bain, M.: B-CNN: branch convolutional neural network for hierarchical classification. arXiv E-prints pp. arXiv-1709 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Noor, K.T., Robles-Kelly, A., Kusy, B. (2022). A Capsule Network for Hierarchical Multi-label Image Classification. In: Krzyzak, A., Suen, C.Y., Torsello, A., Nobile, N. (eds) Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2022. Lecture Notes in Computer Science, vol 13813. Springer, Cham. https://doi.org/10.1007/978-3-031-23028-8_17
Download citation
DOI: https://doi.org/10.1007/978-3-031-23028-8_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-23027-1
Online ISBN: 978-3-031-23028-8
eBook Packages: Computer ScienceComputer Science (R0)