Abstract
Partial label learning (PLL) is a form of weakly supervised learning which aims to train a deep network from training instances and its corresponding label set. The label set, also known as the candidate set, is a group of labels associated with each training instance, out of which only one label is the ground truth for the training instance. Contrastive learning is one of the popular techniques used to learn from a partially labeled dataset, intending to reduce intra-class while maximizing inter-class distance. In this paper, we suggest improving the contrastive technique used in PiCO. The proposed Contrastive Representation via Angle and Distance based Loss (CRADL) segregates the contrastive loss into two parts, the angle based loss and the distance based loss. The former angle based loss covers the angular separation between two contrastive vectors. However, we showcase a scenario where such angular loss prefers one contrastive vector over the other despite having the same angle. Thus, the second loss term built on distance fixes the issue. We show experiments on CIFAR-10 and CIFAR-100, where the corresponding PLL databases are generated using uniform noise. The experiments show that the PLL algorithms learn better using the proposed CRADL-based learning and generate distinguishing representations, as observed by compact cluster formation with CRADL. This eventually results in CRADL outperforming the current state-of-the-art studies in PLL setup at different uniform noise rates.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abhigna, B.S., Soni, N., Dixit, S.: Crowdsourcing - a step towards advanced machine learning. Procedia Comput. Sci. 132, 632–642 (2018)
Chen, C.H., Patel, V.M., Chellappa, R.: Learning from ambiguously labeled face images. Trans. Pattern Anal. Mach. Intell. 40(7), 1653–1667 (2017)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International conference on machine learning. PMLR (2020)
Cour, T., Sapp, B., Taskar, B.: Learning from partial labels. J. Mach. Learn. Res. 12, 1501–1536 (2011)
Cubuk, E.D., Zoph, B., Shlens, J., Le, Q.V.: Randaugment: practical automated data augmentation with a reduced search space (2019)
Feng, L., An, B.: Leveraging latent label distributions for partial label learning. In: International Joint Conferences on Artificial Intelligence (2018)
Feng, L., Kaneko, T., Han, B., Niu, G., An, B., Sugiyama, M.: Learning with multiple complementary labels. In: International Conference on Machine Learning. PMLR (2020)
Feng, L., et al.: Provably consistent partial-label learning. In: Advances in Neural Information Processing Systems (2020)
Feng, L., et al.: Provably consistent partial-label learning (2020)
Feng, L., Shu, S., Lin, Z., Lv, F., Li, L., An, B.: Can cross entropy loss be robust to label noise? In: International Joint Conferences on Artificial Intelligence (2021)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Computer Vision and Pattern Recognition (2020)
Hüllermeier, E., Beringer, J.: Learning from ambiguously labeled examples. Intell. Data Anal. 10(5), 419–439 (2006)
Jin, R., Ghahramani, Z.: Learning with multiple labels. In: Advances in Neural Information Processing Systems (2002)
Jing, L., Vincent, P., LeCun, Y., Tian, Y.: Understanding dimensional collapse in contrastive self-supervised learning (2022)
Khosla, P., et al.: Supervised contrastive learning. In: Advances in Neural Information Processing Systems (2020)
Krizhevsky, A.: Learning multiple layers of features from tiny images. Master’s thesis, University of Tront (2009)
Luo, J., Orabona, F.: Learning from candidate labeling sets. In: Advances in Neural Information Processing Systems (2010)
Lv, J., Xu, M., Feng, L., Niu, G., Geng, X., Sugiyama, M.: Progressive identification of true labels for partial-label learning. In: International Conference on Machine Learning. PMLR (2020)
Malhotra, A., Sankaran, A., Vatsa, M., Singh, R., Morris, K.B., Noore, A.: Understanding ACE-V latent fingerprint examination process via eye-gaze analysis. IEEE Trans. Biom. Behav. Identity Sci. 3(1), 44–58 (2020)
Sheng, V., Zhang, J.: Machine learning with crowdsourcing: a brief summary of the past research and future directions. In: Proceedings of the AAAI Conference on Artificial Intelligence (2019)
Sohn, K.: Improved deep metric learning with multi-class n-pair loss objective. In: Advances in Neural Information Processing Systems (2016)
Su, G., Chen, W., Xu, M.: Positive-unlabeled learning from imbalanced data. In: International Joint Conferences on Artificial Intelligence (2021)
van der Maaten, L., Hinton, G.: Visualizing high-dimensional data using t-SNE. J. Mach. Learn. Res. (2008)
Van Engelen, J.E., Hoos, H.H.: A survey on semi-supervised learning. Mach. Learn. 109(2), 373–440 (2020)
Wang, H., et al.: PiCO: contrastive label disambiguation for partial label learning. In: International Conference on Learning Representations (2022)
Wen, H., Cui, J., Hang, H., Liu, J., Wang, Y., Lin, Z.: Leveraged weighted loss for partial label learning. In: International Conference on Machine Learning. PMLR (2021)
Wu, D.D., Wang, D.B., Zhang, M.L.: Revisiting consistency regularization for deep partial label learning. In: International Conference on Machine Learning. PMLR (2022)
Xu, N., Lv, J., Geng, X.: Partial label learning via label enhancement. In: Conference on Artificial Intelligence. AAAI (2019)
Xu, N., Qiao, C., Geng, X., Zhang, M.L.: Instance-dependent partial label learning. In: Advances in Neural Information Processing Systems (2021)
Yao, Y., Gong, C., Deng, J., Yang, J.: Network cooperation with progressive disambiguation for partial label learning. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds.) ECML PKDD 2020. LNCS (LNAI), vol. 12458, pp. 471–488. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67661-2_28
Yu, F., Zhang, M.L.: Maximum margin partial label learning. In: Asian Conference on Machine Learning. PMLR (2016)
Zeng, Z., et al.: Learning by associating ambiguously labeled images. In: Computer Vision and Pattern Recognition. IEEE (2013)
Zhang, M.L., Zhou, B.B., Liu, X.Y.: Partial label learning via feature-aware disambiguation. In: Conference on Knowledge Discovery and Data Mining. ACM SIGKDD (2016)
Zhang, Z., Sabuncu, M.: Generalized cross entropy loss for training deep neural networks with noisy labels. In: Advances in Neural Information Processing Systems (2018)
Zhou, Z.H.: A brief introduction to weakly supervised learning. Natl. Sci. Rev. 5(1), 44–53 (2018)
Zhuang, C., Zhai, A.L., Yamins, D.: Local aggregation for unsupervised learning of visual embeddings. In: International Conference on Computer Vision (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Ethics Statement
Our proposed algorithm does not raise any ethical concerns. No database is collected in this research, and the experiments shown are on publicly available datasets. However, it is essential to note that partial label learning (PLL) algorithms, including the proposed one, aim to disambiguate and learn from labels that have some noise. Hence, while PLL aims to reduce its performance gap from fully supervised correct labels, the performance is typically lower. Therefore, while PLL research can benefit from our work, it is crucial to exercise caution and responsibility to ensure positive and socially beneficial outcomes of machine learning algorithms.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Chudasama, P., Kadam, T., Patel, R., Malhotra, A., Magam, M. (2023). Contrastive Representation Through Angle and Distance Based Loss for Partial Label Learning. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14172. Springer, Cham. https://doi.org/10.1007/978-3-031-43421-1_40
Download citation
DOI: https://doi.org/10.1007/978-3-031-43421-1_40
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43420-4
Online ISBN: 978-3-031-43421-1
eBook Packages: Computer ScienceComputer Science (R0)
