Abstract
Semantic interpretability and visual expressivity are important objectives in exploratory analysis of text. On the one hand, while some documents may have explicit categories, we could develop a better understanding of a corpus by studying its finer-grained structures, which may be latent. By inferring latent topics and discovering keywords associated with each topic, one obtains a semantic interpretation of the corpus. One the other hand, by visualizing documents, latent topics, and category labels on the same plot, one gains a bird’s eye view of the relationships among documents, topics, and various categories. Semantic visualization is a class of methods that unify both topic modeling and visualization. In this paper, we propose a novel semantic visualization model for networked documents that incorporates partial labels. We introduce coordinate-based label distribution and label-dependent topic distribution to visualize documents, topics, and labels in a semi-supervised way. We further derive three variants for singly-labeled, multi-labeled, and hierarchically-labeled documents. The focus on semi-supervision that employs variants of labeling structures is particularly novel. Experiments verify the efficacy of our model against baselines.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
Source code and datasets are available at https://github.com/cezhang01/semivn.
References
Bai, H., Chen, Z., Lyu, M.R., King, I., Xu, Z.: Neural relational topic models for scientific article analysis. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 27–36 (2018)
Bishop, C. M.: Pattern Recognition and Machine Learning, Springer, Heidelberg (2006). ISBN 978-0-387-31073-2
Bouma, G.: Normalized (pointwise) mutual information in collocation extraction. In: Proceedings of GSCL, pp. 31–40 (2009)
Chang J., Blei D.: Relational topic models for document networks. In: Artificial Intelligence and Statistics, pp. 81–88 (2009)
Choo, J., Lee, C., Reddy, C.K., Park, H.: Utopian: user-driven topic modeling based on interactive nonnegative matrix factorization. IEEE Trans. Visual Comput. Graph. 19(12), 1992–2001 (2013)
Evert, S.: Google web 1t 5-grams made easy (but not for the computer). In: Proceedings of the NAACL HLT 2010 Sixth Web as Corpus Workshop, pp. 32–40 (2010)
Blei, D.M., Griffiths, T.L., Jordan, M.I., Tenenbaum, J.B.: Hierarchical topic models and the nested Chinese restaurant process. In: Advances in Neural Information Processing Systems, pp. 17–24 (2004)
Iwata, T., Yamada, T., Ueda, N.: Probabilistic latent semantic visualization: topic model for visualizing documents. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 363–371 (2008)
Jang, E., Gu, S., Poole, B.: Categorical reparameterization with Gumbel-Softmax. In: Proceedings of International Conference on Learning Representations (2017)
Kingma, D.P., Welling, M.: Auto-encoding variational Bayes. arXiv preprint arXiv:1312.6114 (2013)
Kingma, D.P., Rezende, D.J., Mohamed, S., Welling, M.: Semi-supervised learning with deep generative models. In: Proceedings of Advances in Neural Information Processing Systems, pp. 3581–3589 (2014)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Lacoste-Julien, S., Sha, F., Jordan, M.I.: DiscLDA: discriminative learning for dimensionality reduction and classification. In: Proceedings of Advances in Neural Information Processing Systems, pp. 897–904. (2009)
Le, T., Akoglu, L.: ContraVis: contrastive and visual topic modeling for comparing document collections. In: Proceedings of The World Wide Web Conference, pp. 928–938 (2019)
Le, T.M., Lauw, H.W.: Probabilistic latent document network embedding. In: 2014 IEEE International Conference on Data Mining, pp. 270–279 (2014)
Lee, H., Kihm, J., Choo, J., Stasko, J., Park, H.: iVisClustering: an interactive visual document clustering via topic modeling. In: Computer graphics forum, vol. 31, No. 3pt3, pp. 1155–1164. Oxford, UK: Blackwell Publishing Ltd. (2012)
Leow, Y.Y., Laurent, T., Bresson, X.: GraphTSNE: a visualization technique for graph-structured data. arXiv preprint arXiv:1904.06915 (2019)
Van der Maaten, L., Hinton, G.: Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2579–2605 (2008)
Maddison, C.J., Mnih, A., Teh, Y.W.: The concrete distribution: a continuous relaxation of discrete random variables. In: Proceedings of International Conference on Learning Representations (2017)
Blei, D.M., McAuliffe, J.D.: Supervised topic models. In: Proceedings of Advances in Neural Information Processing Systems, pp. 121–128 (2008)
McCallum, A.K., Nigam, K., Rennie, J., Seymore, K.: Automating the construction of internet portals with machine learning. Inf. Retrieval 3(2), 127–163 (2000)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
Morin, F., Bengio, Y.: Hierarchical probabilistic neural network language model. In: International Workshop on Artificial Intelligence and Statistics, pp. 246–252 (2005)
Ramage, D., Hall, D., Nallapati, R., Manning, C.D.: Labeled LDA: a supervised topic model for credit attribution in multi-labeled corpora. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pp. 248–256 (2009)
Ramage, D., Manning, C.D., Dumais, S.: Partially labeled topic models for interpretable text mining. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 457–465 (2011)
Rozemberczki, B., Davies, R., Sarkar, R., Sutton, C.: GEMSEC: graph embedding with self clustering. In: Proceedings of the 2019 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, pp. 65–72 (2019)
Srivastava, A., Sutton, C.: Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488 (2017)
Tang, J., Liu, J., Zhang, M., Mei, Q.: Visualizing large-scale and high-dimensional data. In: Proceedings of the 25th International Conference on World Wide Web, pp. 287–297 (2016)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. In: Proceedings of International Conference on Learning Representations (2018)
Wu, M., Goodman, N.: Multimodal generative models for scalable weakly-supervised learning. In: Proceedings of Advances in Neural Information Processing Systems, pp. 5575–5585 (2018)
Zhang, C., Lauw, H.W.: Topic modeling on document networks with adjacent-encoder. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34(04), pp. 6737–6745 (2020)
Acknowledgments
This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its NRF Fellowship Programme (Award No. NRF-NRFF2016-07).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, D.C., Lauw, H.W. (2021). Semi-supervised Semantic Visualization for Networked Documents. In: Oliver, N., Pérez-Cruz, F., Kramer, S., Read, J., Lozano, J.A. (eds) Machine Learning and Knowledge Discovery in Databases. Research Track. ECML PKDD 2021. Lecture Notes in Computer Science(), vol 12977. Springer, Cham. https://doi.org/10.1007/978-3-030-86523-8_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-86523-8_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86522-1
Online ISBN: 978-3-030-86523-8
eBook Packages: Computer ScienceComputer Science (R0)
