Abstract
We present a highly accurate and very efficient approach for personality traits prediction based on video. Unlike the traditional method, we proposed a decision-level information fusion method based on deep learning. We have separated the video modal into two parts, visual modal and audio model. The two models were processed by improved VGG-16 and LSTM network, respectively, and combined with an Extreme Learning Machine (ELM) to architecture decision-level information fusion. Experiments on challenging Youtube-8M dataset show that our proposed approach significantly outperforms traditional decision-level fusion method in terms of both efficiency and accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Sun, S.L., Deng, Z.L.: Multisensor information fusion with applications. Automatica 40(6), 1017–1023 (2004)
Kapoor, A., Picard, R.W.: Multimodal affect recognition in learning environments. In: ACM International Conference on Multimedia, pp. 677–682. ACM (2005)
Xing, X., Wang, K., Yan, T., Lv, Z.: Complete canonical correlation analysis with application to multi-view gait recognition. Pattern Recogn. 50, 107–117 (2016)
Xing, X., Wang, K.: Couple manifold discriminant analysis with bipartite graph embedding for face recognition at a distance. Sig. Process. 125, 329–335 (2016)
Piras, L., Giacinto, G.: Information fusion in content based image retrieval: a comprehensive overview. Inf. Fusion 37, 50–60 (2017)
Xing, X., Wang, K., Lv, Z.: Fusion of gait and facial features using coupled projections for people identification at a distance. IEEE Sig. Process. Lett. 22(12), 2349–2353 (2015)
Eid, M., Diener, E.: Intraindividual variability in affect: reliability, validity, and personality correlates. J. Pers. Soc. Psychol. 76(4), 662–676 (1999)
Reeves, B., Nass, C.: The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places. Cambridge University Press, New York (1996)
Lee, M.J., Choi, S.Y., Jeong, H.J., et al.: A precise image crawling system with image classification based on deep learning. Adv. Sci. Lett. 23(3), 1623–1626 (2017)
Hu, G., Yang, Y., Yi, D., et al.: When face recognition meets with deep learning: an evaluation of convolutional neural networks for face recognition, pp. 384–392 (2015)
Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: Proceedings of IEEE International Conference of Computer Vision, pp. 1520–1528 (2015)
Lin, X.F., Yacou, B., Sherif, B.J., et al.: Performance analysis of pattern classifier combination by plurality voting. Pattern Recogn. Lett. 24(12), 1959–1969 (2003)
Fauvel, M., Chanussot, J., Benediktsson, J.A.: Decision fusion for the classification of urban remote sensing images. IEEE Trans. Geosci. Remote Sens. 44(10), 2828–2838 (2006)
Pan, A., Zhou, J., Zhang, P., et al.: Research on power quality evaluation based on radar chart method and fuzzy membership degree. Energy Power Eng. 09(4), 725–734 (2017)
Liang, L.K., Jing-Bo, L.I.: Discussion on the extraction and standardization of TCM symptom based on maximum probability method. China J. Tradit. Chin. Med. Pharm. 5, 060 (2017)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feed-ward neural networks. In: Proceedings of IEEE International Joint Conference on Neural Networks, vol. 2, pp. 985–990. IEEE (2005)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)
Graves, A.: Long short-term memory. In: Graves, A. (ed.) Supervised Sequence Labelling with Recurrent Neural Networks. Springer, Heidelberg (2012). doi:10.1007/978-3-642-24797-2_4
Greff, K., Srivastava, R.K., Koutnik, J., et al.: LSTM: a search space odyssey. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–11 (2016)
Ma, X., Hovy, E.: End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF (2016)
Abu-El-Haija, S., Kothari, N., Lee, J., et al.: YouTube-8M: A Large-Scale Video Classification Benchmark (2016)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition, pp. 770–778 (2015)
Naseem, I., Togneri, R., Bennamoun, M.: Linear regression for face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 2106–2112 (2010)
Tabib Mahmoudi, F., Samadzadegan, F., Reinartz, P.: A decision level fusion method for object recognition using multi-angular imagery. Proc. SPIE XL1/W3(1), 62795F-62795F-6 (2013)
Vedaldi, A., Lenc, K.: MatConvNet: Convolutional Neural Networks for MATLAB. Eprint Arxiv, pp. 689–692 (2014)
Acknowledgments
This work was supported by the Fundamental Research Funds for the Central Universities of China, Natural Science Foundation of China, and Natural Science Fund of Heilongjiang Province of China under Grand Nos. HEUCFJ170404, 61573114, 61703119, F2015033 and QC2017070.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wang, K., Liu, M., Hao, X., Xing, X. (2017). Decision-Level Fusion Method Based on Deep Learning. In: Zhou, J., et al. Biometric Recognition. CCBR 2017. Lecture Notes in Computer Science(), vol 10568. Springer, Cham. https://doi.org/10.1007/978-3-319-69923-3_72
Download citation
DOI: https://doi.org/10.1007/978-3-319-69923-3_72
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-69922-6
Online ISBN: 978-3-319-69923-3
eBook Packages: Computer ScienceComputer Science (R0)