Abstract
Partial Label Learning (PLL) aims to train a classifier when each training instance is associated with a set of candidate labels, among which only one is correct but is not accessible during the training phase. The common strategy dealing with such ambiguous labeling information is to disambiguate the candidate label sets. Nonetheless, existing methods ignore the disambiguation difficulty of instances and adopt the single-trend training mechanism. The former would lead to the vulnerability of models to the false positive labels and the latter may arouse error accumulation problem. To remedy these two drawbacks, this paper proposes a novel approach termed “Network Cooperation with Progressive Disambiguation” (NCPD) for PLL. Specifically, we devise a progressive disambiguation strategy of which the disambiguation operations are performed on simple instances firstly and then gradually on more complicated ones. Therefore, the negative impacts brought by the false positive labels of complicated instances can be effectively mitigated as the disambiguation ability of the model has been strengthened via learning from the simple instances. Moreover, by employing artificial neural networks as the backbone, we utilize a network cooperation mechanism which trains two networks collaboratively by letting them interact with each other. As two networks have different disambiguation ability, such interaction is beneficial for both networks to reduce their respective disambiguation errors, and thus is much better than the existing algorithms with single-trend training process. Extensive experimental results on various benchmark and practical datasets demonstrate the superiority of our NCPD approach to other state-of-the-art PLL methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The notion of “multi-birth group” will be detailed later in Sect. 3.1.
- 2.
These datasets are available at http://palm.seu.edu.cn/zhangml.
References
Arpit, D., et al.: A closer look at memorization in deep networks. In: Proceedings of the International Conference on Machine Learning, pp. 233–242 (2017)
Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 92–100 (1998)
Briggs, F., Fern, X.Z., Raich, R.: Rank-loss support instance machines for MIML instance annotation. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, pp. 534–542 (2012)
Chen, C.H., Patel, V.M., Chellappa, R.: Learning from ambiguously labeled face images. IEEE Trans. Pattern Anal. Mach. Intell. 40(7), 1653–1667 (2018)
Chen, Z.S., Wu, X., Chen, Q.G., Hu, Y., Zhang, M.L.: Multi-view partial multi-label learning with graph-based disambiguation. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3553–3560 (2020)
Cour, T., Sapp, B., Jordan, C., Taskar, B.: Learning from ambiguously labeled images. In: Proceedings of the Computer Vision and Pattern Recognition, pp. 919–926 (2009)
Feng, L., An, B.: Leveraging latent label distributions for partial label learning. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 2107–2113 (2018)
Feng, L., An, B.: Partial label learning with self-guided retraining. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3542–3549 (2019)
Gong, C., Liu, T., Tang, Y., Yang, J., Yang, J., Tao, D.: A regularization approach for instance based superset label learning. IEEE Trans. Cybern. 48(3), 967–978 (2018)
Gong, C., Shi, H., Liu, T., Zhang, C., Yang, J., Tao, D.: Loss decomposition and centroid estimation for positive and unlabeled learning. IEEE Trans. Pattern Anal. Mach. Intell. 43, 918–932 (2019)
Gong, C., Zhang, H., Yang, J., Tao, D.: Learning with inadequate and incorrect supervision. In: Proceedings of the International Conference on Data Mining, pp. 889–894 (2017)
Guillaumin, M., Verbeek, J., Schmid, C.: Multiple instance metric learning from automatically labeled bags of faces. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6311, pp. 634–647. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15549-9_46
Han, B., et al.: Co-teaching: robust training of deep neural networks with extremely noisy labels. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 8527–8537 (2018)
Hassoun, M.H., et al.: Fundamentals of Artificial Neural Networks. MIT Press, Cambridge (1995)
Hüllermeier, E., Beringer, J.: Learning from ambiguously labeled examples. Intell. Data Anal. 10(5), 419–439 (2006)
Jin, R., Ghahramani, Z.: Learning with multiple labels. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 921–928 (2003)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Liu, L., Dietterich, T.: Learnability of the superset label learning problem. In: Proceedings of the International Conference on Machine Learning, pp. 1629–1637 (2014)
Liu, L., Dietterich, T.G.: A conditional multinomial mixture model for superset label learning. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 548–556 (2012)
Liu, T., Tao, D.: Classification with noisy labels by importance reweighting. IEEE Trans. Pattern Anal. Mach. Intell. 38(3), 447–461 (2015)
Liu, W., Xu, D., Tsang, I.W., Zhang, W.: Metric learning for multi-output tasks. IEEE Trans. Pattern Anal. Mach. Intell. 41(2), 408–422 (2018)
Luo, J., Orabona, F.: Learning from candidate labeling sets. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 1504–1512 (2010)
Lyu, G., Feng, S., Lang, C., Wang, T.: A self-paced regularization framework for partial-label learning. arXiv preprint arXiv:1804.07759 (2018)
Nguyen, N., Caruana, R.: Classification with partial labels. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, pp. 551–559 (2008)
Tang, C.Z., Zhang, M.L.: Confidence-rated discriminative partial label learning. In: Proceedings of the AAAI Conference on Artificial Intelligence (2017)
Wan, S., Gong, C., Zhong, P., Du, B., Zhang, L., Yang, J.: Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Trans. Geosci. Remote Sens. 58(5), 3162–3177 (2019)
Wang, D.B., Li, L., Zhang, M.L.: Adaptive graph guided disambiguation for partial label learning. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, pp. 83–91 (2019)
Wu, X., Zhang, M.L.: Towards enabling binary decomposition for partial label learning. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 2868–2874 (2018)
Yao, Y., Deng, J., Chen, X., Gong, C., Wu, J., Yang, J.: Deep discriminative CNN with temporal ensembling for ambiguously-labeled image classification. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 12669–12676 (2020)
Yi, K., Wu, J.: Probabilistic end-to-end noise correction for learning with noisy labels. In: Proceedings of the Computer Vision and Pattern Recognition, pp. 7017–7025 (2019)
Yu, F., Zhang, M.-L.: Maximum margin partial label learning. Mach. Learn. 106(4), 573–593 (2016). https://doi.org/10.1007/s10994-016-5606-4
Yu, X., Han, B., Yao, J., Niu, G., Tsang, I.W., Sugiyama, M.: How does disagreement help generalization against label corruption? arXiv preprint arXiv:1901.04215 (2019)
Zeng, Z., et al.: Learning by associating ambiguously labeled images. In: Proceedings of the Computer Vision and Pattern Recognition. pp. 708–715 (2013)
Zhang, C., Bengio, S., Hardt, M., Recht, B., Vinyals, O.: Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.03530 (2016)
Zhang, C., Ren, D., Liu, T., Yang, J., Gong, C.: Positive and unlabeled learning with label disambiguation. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 4250–4256 (2019)
Zhang, M.L., Yu, F.: Solving the partial label learning problem: an instance-based approach. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 4048–4054 (2015)
Zhang, M.L., Yu, F., Tang, C.Z.: Disambiguation-free partial label learning. IEEE Trans. Knowl. Data Eng. 29(10), 2155–2167 (2017)
Zhang, M.L., Zhou, B.B., Liu, X.Y.: Partial label learning via feature-aware disambiguation. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining, pp. 1335–1344 (2016)
Acknowledgments
This research is supported by NSF of China (Nos: 61973162, U1713208), NSF of Jiangsu Province (No: BK20171430), the Fundamental Research Funds for the Central Universities (No: 30920032202), the “Young Elite Scientists Sponsorship Program” by Jiangsu Province and CAST (No: 2018QNRC001), and the Program for Changjiang Scholars.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Yao, Y., Gong, C., Deng, J., Yang, J. (2021). Network Cooperation with Progressive Disambiguation for Partial Label Learning. In: Hutter, F., Kersting, K., Lijffijt, J., Valera, I. (eds) Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2020. Lecture Notes in Computer Science(), vol 12458. Springer, Cham. https://doi.org/10.1007/978-3-030-67661-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-67661-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-67660-5
Online ISBN: 978-3-030-67661-2
eBook Packages: Computer ScienceComputer Science (R0)
