[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Variational Multi-Valued Velocity Field Estimation for Transparent Sequences

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Motion estimation in sequences with transparencies is an important problem in robotics and medical imaging applications. In this work we propose a variational approach for estimating multi-valued velocity fields in transparent sequences. Starting from existing local motion estimators, we derive a variational model for integrating in space and time such a local information in order to obtain a robust estimation of the multi-valued velocity field. With this approach, we can indeed estimate multi-valued velocity fields which are not necessarily piecewise constant on a layer—each layer can evolve according to a non-parametric optical flow. We show how our approach outperforms existing methods; and we illustrate its capabilities on challenging experiments on both synthetic and real sequences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Toro, J., Owens, F., Medina, R.: Using known motion fields for image separation in transparency. Pattern Recognit. Lett. 24, 597–605 (2003)

    Article  Google Scholar 

  2. Oppenheim, A.V.: Superposition in a class of nonlinear systems. In: Proceedings of IEEE International Convention, New York, USA, pp. 171–177 (1964)

    Google Scholar 

  3. Guenther, R.D.: Modern Optics. Wiley, New York (1990)

    Google Scholar 

  4. Qian, N., Andersen, R., Adelson, E.: Transparent motion perception as detection of unbalanced motion signals. III. Modeling. J. Neurosci. 14(12), 7381–7392 (1994)

    Google Scholar 

  5. Ramirez-Manzanares, A., Rivera, M., Kornprobst, P., Lauze, F.: A variational approach for multi-valued velocity field estimation in transparent sequences. In: Proceedings of International Conference on Scale Space and Variational Methods in Computer Vision, Ischia, Italy. LNCS, vol. 4485, pp. 227–238 (2007)

    Chapter  Google Scholar 

  6. Horn, B., Schunck, B.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  7. Papenberg, N., Bruhn, A., Brox, T., Didas, S., Weickert, J.: Highly accurate optical flow computations with theoretically justified warping. Int. J. Comput. Vis. 67(2), 141–158 (2006)

    Article  Google Scholar 

  8. Nir, T., Bruckstein, A., Kimmel, R.: Over-parameterized variational optical flow. Int. J. Comput. Vis. 76(2), 205–216 (2008)

    Article  Google Scholar 

  9. Barron, J., Fleet, D., Beauchemin, S.: Performance of optical flow techniques. Int. J. Comput. Vis. 12(1), 43–77 (1994)

    Article  Google Scholar 

  10. Baker, S., Scharstein, D., Lewis, J.P., Roth, K., Black, M.J., Szeliski, R.: A database and evaluation methodology for optical flow. In: Proceeding of the 11th International Conference on Computer Vision, IEEE, Rio de Janeiro, Brazil (2007)

    Google Scholar 

  11. Bergen, J.R., Burt, P.J., Hingorani, R., Peleg, S.: Computing two motions from three frames. In: Third International Conference on Computer Vision, Osaka, Japan, pp. 27–32 (1990)

    Chapter  Google Scholar 

  12. Burt, P.J., Hingorani, R., Kolczynski, R.J.: Mechanisms for isolating component patterns in the sequential analysis of multiple motion. In: Proceedings of IEEE Workshop on Visual Motion, Princeton, NJ, pp. 187–193 (1991)

    Chapter  Google Scholar 

  13. Irani, M., Peleg, S.: Motion analysis for image enhancement: resolution, occlusion, and transparency. J. Vis. Commun. Image Represent. 4(4), 324–335 (1993)

    Article  Google Scholar 

  14. Irani, M., Rousso, B., Peleg, S.: Computing occluding and transparent motions. Int. J. Comput. Vis. 12(1), 5–16 (1994)

    Article  Google Scholar 

  15. Shizawa, M., Mase, K.: Simultaneous multiple optical flow estimation. In: Proceedings of 10th International Conference on Pattern Recognition, pp. 274–278 (1990)

    Chapter  Google Scholar 

  16. Shizawa, M., Mase, K.: Principle of superposition: a common computational framework for analysis of multiple motion. In: Proceedings of IEEE Workshop on Visual Motion, pp. 164–172 (1991)

    Chapter  Google Scholar 

  17. Shizawa, M., Mase, K.: A unified computational theory for motion transparency and motion boundaries based on eigenergy analysis. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, IEEE, Lahaina, Hawai, pp. 289–295 (1991)

    Chapter  Google Scholar 

  18. Förstner, W.: A feature based corresponding algorithm for image matching. Int. Arch. Photogramm. Remote Sens. 26, 150–166 (1986)

    Google Scholar 

  19. Bigun, J., Granlund, G.H., Wiklund, J.: Multidimensional orientation estimation with applications to texture analysis and optical flow. IEEE Trans. Pattern Anal. Mach. Intell. 13(8), 775–790 (1991). Report LiTH-ISY-I-0828 1986 and Report LiTH-ISY-I-1148 1990, both at Computer Vision Laboratory, Linköping University, Sweden

    Article  Google Scholar 

  20. Mota, C., Stuke, I., Aach, T., Barth, E.: Divide-and-Conquer strategies for estimating multiple transparent motions. In: Proceedings of 1st International Workshop on Complex Motion, Schloss Reisensburg, Germany. LNCS, vol. 3417, pp. 66–78 (2005)

    Google Scholar 

  21. Mühlich, M., Aach, T.: A theory of multiple orientation estimation. In: Proceedings of the European Conference on Computer Vision. LNCS, vol. 3952, pp. 69–82 (2006)

    Google Scholar 

  22. Vernon, D.: Decoupling Fourier components of dynamic image sequences: a theory of signal separation, image segmentation and optical flow estimation. In: European Conference on Computer Vision, vol. 2, pp. 69–85 (1998)

    Google Scholar 

  23. Zhou, W., Kambhamettu, C.: Separation of reflection by Fourier decoupling. In: Asian Conference on Computer Vision, Jeju Island, Korea (2004)

    Google Scholar 

  24. Liu, H.C., Hong, T.H., Herman, M., Chellappa, R.: Spatio-temporal filters for transparent motion segmentation. In: Proceedings of the International Conference on Image Processing, Washington, USA, pp. 464–468 (1995)

    Google Scholar 

  25. Darrell, T., Simoncelli, E.: Separation of transparent motion into layers using velocity-tuned mechanisms. Tech. Rep. 244, MIT Media Laboratory Vision and Modeling Group (1993)

  26. Stuke, I., Aach, T., Barth, E., Mota, C.: Estimation of multiple motions by block matching. In: Proceedings in 4th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing, pp. 358–362 (2003)

    Google Scholar 

  27. Stuke, I., Aach, T., Barth, E., Mota, C.: Multiple-motion-estimation by block matching using MRF. Int. J. Comput. Inform. Sci. 26, 141–152 (2004)

    Google Scholar 

  28. Auvray, V., Bouthemy, P., Lienard, J.: Motion estimation in x-ray image sequence with bi-distributed transparency. In: Proceedings of IEEE International Conference on Image Processing, Atlanta, USA, pp. 1057–1060 (2006)

    Google Scholar 

  29. Fitzpatrick, J.M.: The existence of geometrical density-image transformations corresponding to object motion. Comput. Vis. Graph. Image Process. 44(2), 155–174 (1988)

    Article  MathSciNet  Google Scholar 

  30. Pingault, M., Bruno, E., Pellerin, D.: A robust multiscale b-spline function decomposition for estimating motion transparency. IEEE Trans. Image Process. 12(11), 1416–1426 (2003)

    Article  Google Scholar 

  31. Black, M.J., Anandan, P.: The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. CVGIP, Image Underst. 63(1), 75–104 (1996)

    Google Scholar 

  32. Jepson, A., Black, M.J.: Mixture models for optical flow computation. In: Proceedings of Intl. Conf. on Computer Vision and Pattern Recognition, pp. 760–761 (1993)

    Chapter  Google Scholar 

  33. Ju, S.X., Black, M.J., Jepson, A.D.: Skin and bones: Multi-layer, locally affine, optical flow and regularization with transparency. In: Proceedings of Intl. Conf. on Computer Vision and Pattern Recognition, San Francisco, CA, pp. 307–314 (1996)

    Google Scholar 

  34. Black, M.J., Fleet, D.J., Yacoob, Y.: Robustly estimating changes in image appearance. Comput. Vis. Image Underst. 78, 8–31 (2000)

    Article  Google Scholar 

  35. Jojic, N., Frey, B.: Learning flexible sprites in video layers. In: Proceedings in IEEE Conf. on Computer Vision and Pattern Recognition, pp. 199–206 (2001)

    Google Scholar 

  36. Weiss, Y., Adelson, E.H.: A unified mixture framework for motion segmentation: incorporating spatial coherence and estimating the number of models. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, IEEE, San Francisco, CA, pp. 321–326 (1996)

    Google Scholar 

  37. Rivera, M., Ocegueda, O., Marroquin, J.L.: Entropy-controlled quadratic Markov measure field models for efficient image segmentation. IEEE Trans. Image Process. 16(12), 3047–3057 (2007)

    Article  MathSciNet  Google Scholar 

  38. Ramirez-Manzanares, A., Rivera, M., Kornprobst, P., Lauze, F.: Multi-valued motion fields estimation for transparent sequences with a variational approach. Tech. Rep. RR-5920, INRIA (Also, Reporte Técnico CIMAT, (CC)I-06-12) (2006)

  39. Szeliski, R., Avidan, S., Anandan, P.: Layer extraction from multiple images containing reflections and transparency. In: Proceedings in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 246–253 (2000)

    Google Scholar 

  40. Nicolescu, M., Medioni, G.: Layered 4d representation and voting for grouping from motion. IEEE Trans. Pattern Anal. Mach. Intell. 25(4), 492–501 (2003)

    Article  Google Scholar 

  41. Sarel, B., Irani, M.: Separating transparent layers of repetitive dynamic behaviors. In: Proceedings of the Tenth International Conference on Computer Vision, vol. 1, pp. 26–32. IEEE Computer Society, Beijing (2005)

    Google Scholar 

  42. Oo, T., Kawasaki, H., Ohsawa, Y., Ikeuchi, K.: The separation of reflected and transparent layers from real-world image sequences. Mach. Vis. Appl. 18(1), 17–24 (2007)

    Article  Google Scholar 

  43. Black, M.J., Rangarajan, P.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Comput. Vis. 19(1), 57–91 (1996)

    Article  Google Scholar 

  44. Ramirez-Manzanares, A., Rivera, M.: Brain nerve boundless estimation by restoring and filtering intra-voxel information in DT-MRI. In: Proceedings of Second Workshop on Variational and Level Sets Methods, pp. 71–80 (2003)

    Google Scholar 

  45. Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996)

    Article  Google Scholar 

  46. Blake, A., Zisserman, A.: Visual Reconstruction, 1st edn. MIT Press, Cambridge (1987)

    Google Scholar 

  47. Grossberg, S., Mingolla, E., Viswanathan, L.: Neural dynamics of motion integration and segmentation within and across apertures. Vis. Res. 41, 2521–2553 (2001)

    Article  Google Scholar 

  48. Braddick, O.: Local and global representations of velocity; transparency opponency and global direction perception. Perception 26, 995–1010 (1997)

    Article  Google Scholar 

  49. Weiss, Y., Simoncelli, E., Adelson, E.: Motion illusions as optimal percepts. Nat. Neurosci. 5(6), 598–604 (2002)

    Article  Google Scholar 

  50. Fleet, D.J., Weiss, Y.: Optical flow estimation. In: Paragios, N., Chen, Y., Faugeras, O. (eds.) Mathematical Models for Computer Vision: The Handbook. Springer, Berlin (2005)

    Google Scholar 

  51. Bayerl, P., Neumann, H.: A fast biologically inspired algorithm for recurrent motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. 29(2), 246–260 (2007)

    Article  Google Scholar 

  52. Sarel, B., Irani, M.: Separating transparent layers through layer information exchange. In: Pajdla, T., Matas, J. (eds.) Proceedings of the 8th European Conference on Computer Vision, pp. 328–341. Springer, Berlin (2004)

    Google Scholar 

  53. Ramirez-Manzanares, A., Rivera, M.: Basis tensor decomposition for restoring intra-voxel structure and stochastic walks for inferring brain connectivity in DT-MRI. Int. J. Comput. Vis. 69(1), 77–92 (2006)

    Article  Google Scholar 

  54. Field, D.J.: Scale-invariance and self-similar ‘wavelet’ transforms: an analysis of natural scenes and mammalian visual systems. In: O.U. Press (ed.) Wavelets, Fractals and Fourier Transforms: New Developments and New Applications. Oxford University Press, London (1993)

    Google Scholar 

  55. Field, D.J.: What is the goal of sensory coding? Neural Comput. 6, 559–601 (1994)

    Article  Google Scholar 

  56. Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by v1? Vis. Res. 37, 3311–3325 (1997)

    Article  Google Scholar 

  57. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 1st edn. Springer, New York (2003)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Guanajuato, AP 402, Guanajuato, Gto., C.P. 36000, Mexico

    Alonso Ramírez-Manzanares

  2. Centro de Investigacion en Matematicas A.C., AP 402, Guanajuato, Gto., 36000, Mexico

    Mariano Rivera

  3. INRIA, Odyssée Lab., 2004 route des Lucioles BP 93, 06902, Sophia Antipolis, France

    Pierre Kornprobst

  4. Institute of Computer Science, University of Copenhagen, Universitetsparken 5, 2100, Kbh Ø, Denmark

    François Lauze

Authors
  1. Alonso Ramírez-Manzanares
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mariano Rivera
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Pierre Kornprobst
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. François Lauze
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alonso Ramírez-Manzanares.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ramírez-Manzanares, A., Rivera, M., Kornprobst, P. et al. Variational Multi-Valued Velocity Field Estimation for Transparent Sequences. J Math Imaging Vis 40, 285–304 (2011). https://doi.org/10.1007/s10851-011-0260-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-011-0260-8

Keywords

Search

Navigation