Abstract
In this paper, we propose a novel disocclusion region recovery approach for two-view to n-view conversion system. Although the topic of view synthesis has been exhaustively studied for decades, a reliable disocclusion region recovery approach, an indispensable issue in synthesizing realistic content of virtual view, is still under research. The most common concept used for predicting these unknown pixels is inpainting-related method, which fills the disocclusion region with the information of mated exemplars in self-defined searching domain. In spite of widely taken in making up the missing values generated among the synthesis procedures, the result quality of inpainting-based approach is sensitive to the filling priority and also unstable in recovering large disocclusion region. Therefore, we propose a hierarchical interpolation-based approach to calculate the desired lost information under coarse-to-fine manner accompanied with the joint bilateral upsampling technology, applied for enlarging the estimation from small dimension to higher-resolution. Proposed hierarchical interpolation-based scheme is more robust in restoring the value of missing region and also induces fewer artifacts. We demonstrate the superior quality of the synthesized virtual views under the proposed recovery algorithm over the traditional inpainting-based method through experiments on several benchmarking video datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. In: Proceedings of 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. vol. 2, pp. II-721. IEEE (2003)
Do, L., Zinger, S., et al.: Quality improving techniques for free-viewpoint DIBR. In: IS&T/SPIE Electronic Imaging. pp. 75240I–75240I. International Society for Optics and Photonics (2010)
Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Trans. Graph. (TOG). vol. 26, Article no. 96. ACM (2007)
Lin, S.-J., Cheng, C.-M., Lai, S.-H.: Spatio-temporally consistent multi-view video synthesis for autostereoscopic displays. In: Muneesawang, P., Wu, F., Kumazawa, I., Roeksabutr, A., Liao, M., Tang, X. (eds.) PCM 2009. LNCS, vol. 5879, pp. 532–542. Springer, Heidelberg (2009)
Luo, H.L., Shen, C.T., Chen, Y.C., Wu, R.H., Hung, Y.P.: Automatic multi-resolution joint image smoothing for depth map refinement. In: 2013 2nd IAPR Asian Conference on Pattern Recognition (ACPR), pp. 284–287. IEEE (2013)
MPEG ISO/IEC JTC1/SC29/WG11: View Synthesis Software Manual (VSRS), release 3.5 (2009)
Oh, K.J., Yea, S., Ho, Y.S.: Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3D video. In: Picture Coding Symposium, 2009. PCS 2009. pp. 1–4. IEEE (2009)
Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR 2007. pp. 1–8. IEEE (2007)
Solh, M., AlRegib, G.: Hierarchical hole-filling for depth-based view synthesis in FTV and 3D video. IEEE J. Select. Topics Signal Process. 6(5), 495–504 (2012)
Wang, L., Jin, H., Yang, R., Gong, M.: Stereoscopic inpainting: joint color and depth completion from stereo images. In: IEEE Conference on Computer Vision and Pattern Recognition, 2008, CVPR 2008. pp. 1–8. IEEE (2008)
Wei, C.-H., Chiang, C.-K., Sun, Y.-W., Lin, M.-H., Lai, S.-H.: Novel multi-view synthesis from a stereo image pair for 3D display on mobile phone. In: Park, J.-I., Kim, J. (eds.) ACCV 2012. LNCS, vol. 7729, pp. 568–579. Springer, Heidelberg (2013)
Yang, Q.: A non-local cost aggregation method for stereo matching. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1402–1409. IEEE (2012)
Zhao, Y., Zhu, C., Chen, Z., Yu, L.: Depth no-synthesis-error model for view synthesis in 3D video. IEEE Trans. Image Process. 20(8), 2221–2228 (2011)
Acknowledgements
This work was partially supported by Ministry of Science and Technology, Taiwan, R.O.C., under the grant MOST 101-2221-E-007-129-MY3.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Lin, WT., Yeh, CT., Lai, SH. (2015). Hierarchical Interpolation-Based Disocclusion Region Recovery for Two-View to N-View Conversion System. In: Ho, YS., Sang, J., Ro, Y., Kim, J., Wu, F. (eds) Advances in Multimedia Information Processing -- PCM 2015. PCM 2015. Lecture Notes in Computer Science(), vol 9315. Springer, Cham. https://doi.org/10.1007/978-3-319-24078-7_66
Download citation
DOI: https://doi.org/10.1007/978-3-319-24078-7_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24077-0
Online ISBN: 978-3-319-24078-7
eBook Packages: Computer ScienceComputer Science (R0)