Abstract
Automatic music video editing is still a challenging task due to the lack of knowledge of how music and video are matched to produce attractive effects. Previous works usually matches music and video following assumption or empirical knowledge. In this paper, we use a dual-wing harmonium model to learn and represent the underlying music video editing rules from a large dataset of music videos. The editing rules are extracted by clustering the low dimensional representation of music video clips. In the experiments, we give an intuitive visualization for the discovered editing rules. These editing rules partially reflect professional music video editor’s skills and can be used to further improve the quality of automatically generated music video.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Foote, J., Cooper, M., et al.: Creating music videos using automatic media analysis. In: Proceedings of the tenth ACM international conference on Multimedia, pp. 553–560 (2002)
Wang, J., Xu, C., et al.: Automatic generation of personalized music sports video. In: Proceedings of the 13th annual ACM international conference on Multimedia, pp. 735–744 (2005)
Xian-Sheng, H.U.A., Lie, L.U., et al.: Automatic music video generation based on temporal pattern analysis. In: Proceedings of the 12th annual ACM international conference on Multimedia. ACM, New York (2004)
Yuya, M., Miki, A., et al.: Mining video editing rules in video streams. In: Proceedings of the tenth ACM international conference on Multimedia, Juan-les-Pins, France. ACM, New York (2002)
Xing, E., Yan, R., et al.: Mining associated text and images with dual-wing harmoniums. In: Proceedings of the 21th Annual Conf. on Uncertainty in Artificial Intelligence (UAI 2005). AUAI press (2005)
Welling, M., Rosen-Zvi, M., et al.: Exponential family harmoniums with an application to information retrieval. Advances in Neural Information Processing Systems 17, 1481–1488 (2005)
Smolensky, P.: Information processing in dynamical systems: foundations of harmony theory. Computational Models Of Cognition And Perception Series, pp. 194–281. Mit Press, Cambridge (1986)
Ellis, D.: Comparing features statistics: MFCCs, MSGs, etc (1999), http://www.icsi.berkeley.edu/~dpwe/respite/multistream/msgmfcc.html
Hinton, G.E.: Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation 14(8), 1771–1800 (2002)
Yang, J., Liu, Y., et al.: Harmonium Models for Semantic Video Representation and Classification. In: SIAM Conf. Data Mining (2007)
Ngo, C.W., Pong, T.C., et al.: Motion analysis and segmentation through spatio-temporal slices processing. IEEE Transactions on Image Processing 12(3), 341–355 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liao, C., Wang, P.P., Zhang, Y. (2008). Exploring Music Video Editing Rules with Dual-Wing Harmonium Model. In: Huang, YM.R., et al. Advances in Multimedia Information Processing - PCM 2008. PCM 2008. Lecture Notes in Computer Science, vol 5353. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-89796-5_67
Download citation
DOI: https://doi.org/10.1007/978-3-540-89796-5_67
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-89795-8
Online ISBN: 978-3-540-89796-5
eBook Packages: Computer ScienceComputer Science (R0)