Abstract
With the rapid development of multimedia data such as text, image, cross-media retrieval has become increasingly important, because users can retrieve the results with various types of media by submitting a query of any media type. The measure of relevance among different media is a basic problem. Existing methods usually only consider the original media instances (such as images, texts) but ignore their patches. In fact, cross-media patches can emphasize the important parts and improve the precision of cross-media correlation. What’s more, existing cross-media retrieval methods often focus on modeling the pairwise correlation with the similarity matrix is a constant matrix, while the similarity matrix which is not a constant matrix can improve the accuracy. In this paper, we propose a novel algorithm for cross-media data, called cross-media correlation analysis with semi-supervised graph regularization (CMCA), which can not only take full advantage of both the media instances and their patches in one graph, but also explore the similarity matrix which can improve the correlation between data. CMCA explores the sparse and semi-supervised regularization for different media types, and integrates them into a unified optimization matter, which increases the performance of the algorithm. Comparing with the current state-of-the-art methods on two datasets (i.e., Wikipedia, XMedia), the comprehensive experimental results demonstrate the effectiveness of our proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ranjan, V., Rasiwasia, N., Jawahar, C.V.: Multi-label cross-modal retrieval. In: IEEE International Conference on Computer Vision (ICCV), pp. 4094–4102 (2015)
Hua, Y., Wang, S., Liu, S., Cai, A., Huang, Q.: Cross-modal correlation learning by adaptive hierarchical semantic aggregation. IEEE Trans. Multimedia (TMM) 18(6), 1201–1216 (2016)
Peng, Y., Ngo, C.-W.: Clip-based similarity measure for query-dependent clip retrieval and video summarization. IEEE Trans. Circ. Syst. Video Technol. 16(5), 612–627 (2006)
Escalante, H.J., Hérnadez, C.A., Sucar, L.E., Montes, M.: Late fusion of heterogeneous methods for multimedia image retrieval. In: Proceedings of 1st ACM International Conference on Multimedia Information Retrieval, pp. 172–179 (2008)
Liu, J., Xu, C., Luo, H.: Cross-media retrieval: state-of-the-art and open issues. Int. J. Multimedia Intell. Secur. 1(1), 33–52 (2010)
Hotelling, H.: Relations between two sets of variates. Biometrika 28(3–4), 321–377 (1936)
Blaschko, M., Lampert, C.: Correlational spectral clustering. In: Proceedings of IEEE International Conference on Computer Vision Pattern Recognition, pp. 1–8, June 2008
Zhai, X., Peng, Y., Xiao, J.: Cross-modality correlation propagation for cross-media retrieval. In: Proceedings of International Conference on Acoustics, Speech, Signal Processing, pp. 2337–2340 (2012)
Pereira, J.C., et al.: On the role of correlation and abstraction in cross-modal multimedia retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 521–535 (2014)
Zhai, X., Peng, Y., Xiao, J.: Learning cross-media joint representation with sparse and semisupervised regularization. IEEE Trans. Circ. Syst. Video Technol. 24(6), 965–978 (2014)
Li, D., Dimitrova, N., Li, M., Sethi, I.: Multimedia content processing through cross-modal association. In: Proceedings of ACM International Conference on Multimedia, pp. 604–611 (2003)
Peng, Y., Zhai, X., Zhao, Y., Huang, X.: Semi-supervised cross-media feature learning with unified patch graph regularization. IEEE Trans. Circ. Syst. Video Technol. (TCSVT) 26(3), 583–596 (2016)
Wang, K., He, R., Wang, L., Wang, W., Tan, T.: Joint feature selection and subspace learning for cross-modal retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 38(10), 2010–2023 (2016)
Rasiwasia, N., et al.: A new approach to cross-modal multimedia retrieval. In: Proceedings of ACM International Conference on Multimedia (ACM-MM), pp. 251–260 (2010)
Rasiwasia, N., Pereira, J.C., Coviello, E., Doyle, G., Lanckriet, G., Levy, R., Vasconcelos, N.: A new approach to cross-modal multimedia retrieval. In: Proceedings of 18th ACM International Conference on Multimedia, pp. 251–260 (2010)
Zhai, X., Peng, Y., Xiao, J.: Heterogeneous metric learning with joint graph regularization for cross-media retrieval. In: Proceedings of 27th AAAI Conference on Artificial Intelligence, pp. 1198–1204 (2013)
Zhai, X., Peng, Y., Xiao, J.: Cross-modality correlation propagation for cross-media retrieval. IEEE Int. Conf. Acoust. 22(10), 2337–2340 (2012)
Acknowledgement
This research is supported by the National Natural Science Foundation of China (No. 61003127, No. 61373109).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhang, H., Qi, T. (2018). Cross-Media Correlation Analysis with Semi-supervised Graph Regularization. In: Huang, DS., Bevilacqua, V., Premaratne, P., Gupta, P. (eds) Intelligent Computing Theories and Application. ICIC 2018. Lecture Notes in Computer Science(), vol 10954. Springer, Cham. https://doi.org/10.1007/978-3-319-95930-6_24
Download citation
DOI: https://doi.org/10.1007/978-3-319-95930-6_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95929-0
Online ISBN: 978-3-319-95930-6
eBook Packages: Computer ScienceComputer Science (R0)