Abstract
Optical motion capture systems suffer from marker occlusions resulting in loss of useful information. This paper addresses the problem of real-time joint localisation of legged skeletons in the presence of such missing data. The data is assumed to be labelled 3d marker positions from a motion capture system. An integrated framework is presented which predicts the occluded marker positions using a Variable Turn Model within an Unscented Kalman filter. Inferred information from neighbouring markers is used as observation states; these constraints are efficient, simple, and real-time implementable. This work also takes advantage of the common case that missing markers are still visible to a single camera, by combining predictions with under-determined positions, resulting in more accurate predictions. An Inverse Kinematics technique is then applied ensuring that the bone lengths remain constant over time; the system can thereby maintain a continuous data-flow. The marker and Centre of Rotation (CoR) positions can be calculated with high accuracy even in cases where markers are occluded for a long period of time. Our methodology is tested against some of the most popular methods for marker prediction and the results confirm that our approach outperforms these methods in estimating both marker and CoR positions.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Notes
Neighbours are the markers belonging to the same limb segment.
Markers in a clique have constant distances between each other.
A sub-base joint is a joint which connects 2 or more chains.
References
Aristidou, A., Cameron, J., Lasenby, J.: Predicting missing markers to drive real-time centre of rotation estimation. In: Proceedings of the V Conference on Articulated Motion and Deformable Objects, Mallorca, Spain. LNCS, vol. 5098, pp. 238–247 (2008)
Aristidou, A., Lasenby, J.: Inverse kinematics: a review of existing techniques and introduction of a new fast iterative solver. Tech. Rep. F-INFENG/TR. 632, CUED (2009)
Aristidou, A., Lasenby, J.: FABRIK: a fast, iterative solver for the inverse kinematics problem. Graphical Models 73(5), 243–260 (2011)
Aristidou, A., Lasenby, J., Cameron, J.: Methods for real-time restoration and estimation in optical motion capture. Tech. Rep. F-INFENG/TR. 619, CUED (2009)
Asseo, S.J., Ardila, R.J.: Sensor independent target state estimator design and evaluation. In: Proceedings of the National Aerospace and Electronics Conference (NAECON), pp. 916–924 (1982)
Backer, A.S.: Estimating missing motion capture data with accelerometers. MPhil thesis, Cambridge University Engineering Department. Cambridge, UK (August 2009)
Baillieul, J.: Kinematic programming alternatives for redundant manipulators. In: Proceedings of the IEEE International Conference on Robotics and Automation, vol. 2, pp. 722–728 (1985)
Balestrino, A., De Maria, G., Sciavicco, L.: Robust control of robotic manipulators. In: Proceedings of the 9th IFAC World Congress, vol. 5, pp. 2435–2440 (1984)
Broeren, J., Sunnerhagen, K.S., Rydmark, M.: A kinematic analysis of a haptic handheld stylus in a virtual environment: a study in healthy subjects. Journal of NeuroEngineering and Rehabilitation 4, 13 (2007)
Brown, J., Latombe, J.C., Montgomery, K.: Real-time knot-tying simulation. The Visual Computer 20(2), 165–179 (2004)
Buss, S.R., Kim, J.S.: Selectively damped least squares for inverse kinematics. Journal of Graphics Tools 10(3), 37–49 (2005)
Cameron, J., Lasenby, J.: A real-time sequential algorithm for human joint localization. In: ACM SIGGRAPH Posters, p. 107. ACM, New York (2005)
Chai, J., Hodgins, J.K.: Performance animation from low-dimensional control signals. ACM Transactions on Graphics (TOG) 24(3), 686–696 (2005)
Chang, L.Y., Pollard, N.: Constrained least-squares optimization for robust estimation of center of rotation. Journal of Biomechanics 40(1), 1392–1400 (2007)
Courty, N., Arnaud, E.: Inverse kinematics using sequential Monte Carlo methods. In: Proceedings of the V Conference on Articulated Motion and Deformable Objects, Mallorca, Spain. LNCS, vol. 5098, pp. 1–10 (2008)
Courty, N., Cuzol, A.: Conditional stochastic simulation for character animation. Computer Animation and Virtual Worlds—CASA’ 2010 21(3–4), 443–452 (2010)
Der, K.G., Sumner, R.W., Popović, J.: Inverse kinematics for reduced deformable models. In: ACM SIGGRAPH Papers, pp. 1174–1179. ACM, New York (2006)
Dessai, S.S., Hornung, A., Kobbelt, L.: Automatic data normalization and parameterization for optical motion tracking. J. Virtual Real. Broadcast. 3(3) (2006)
Doran, C., Lasenby, A.: Geometric Algebra for Physicists. Cambridge University Press, Cambridge (2003)
Dorfmüller-Ulhaas, K.: Robust optical user motion tracking using a Kalman filter. Tech. Rep. TR-2003-6, Institut fuer Informatik, Universitatsstr. 2, 86159 Augsburg (2003)
Doucet, A., De Freitas, N., Gordon, N.: Sequential Monte Carlo Methods in Practice. Springer, New York (2001)
Ehrig, R.M., Taylor, W.R., Duda, G.N., Heller, M.O.: A survey of formal methods for determining the centre of rotation of ball joints. Journal of Biomechanics 39(15), 2798–2809 (2006)
Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley-Interscience, New York (1987)
Gamage, S.S.H.U., Lasenby, J.: New least squares solutions for estimating the average centre of rotation and the axis of rotation. Journal of Biomechanics 35(1), 87–93 (2002)
Grochow, K., Martin, S.L., Hertzmann, A., Popović, Z.: Style-based inverse kinematics. In: ACM Transactions on Graphics (TOG), pp. 522–531. ACM, New York (2004)
Halvorsen, K.: Bias compensated least squares estimate of the center of rotation. Journal of Biomechanics 36, 999–1008 (2003)
Halvorsen, K., Lesser, M., Lundberg, A.: A new method for estimating the axis of rotation and the center of rotation. Journal of Biomechanics 32, 1221–1227 (1999)
Hashiguchi, J., Nivomiya, H., Tanaka, H., Nakamura, M., Nobuhara, K.: Biomechanical analysis of a golf swing using motion capture system. In: Proceedings of Annual Meeting of Japanese Society for Orthopaedic Biomechanics, vol. 27, pp. 325–330 (2006)
Hecker, C., Raabe, B., Enslow, R.W., Deweese, J., Maynard, J., van Prooijen, K.: Real-time motion retargeting to highly varied user-created morphologies. ACM Transactions on Graphics (TOG) 27(3), 1–11 (2008)
Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Skeleton-based motion capture for robust reconstruction of human motion. In: Proceedings of the IEEE Computer Animation (CA’00), pp. 77–86 (2000)
Herda, L., Fua, P., Plänkers, R., Boulic, R., Thalmann, D.: Using skeleton-based tracking to increase the reliability of optical motion capture. Human Movement Science Journal 20(3), 313–341 (2001)
Hestenes, D., Sobczyk, G.: Clifford Algebra to Geometric Calculus: A Unified Language for Mathematics and Physics. Reidel, Dordrecht (1984)
Holzreiter, S.: Calculation of the instantaneous centre of rotation for a rigid body. Journal of Biomechanics 24(7), 643–647 (1991)
Horn, B.: Closed-form solution of absolute orientation using unit quaternions. Journal of the Optical Society of America A 4, 629–642 (1987)
Hornung, A., Sar-Dessai, S.: Self-calibrating optical motion tracking for articulated bodies. In: Proceedings of the IEEE Conference on Virtual Reality, VR’05, pp. 75–82. IEEE Computer Society, Washington (2005)
Hsu, E., Gentry, S., Popović, J.: Example-based control of human motion. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’04), pp. 69–77. Eurographics Association, Grenoble (2004)
Ishigaki, S., White, T., Zordan, V.B., Liu, C.K.: Performance-based control interface for character animation. ACM Transaction on Graphics (TOG) 28(3), 1–8 (2009)
Jazwinski, A.H.: Stochastic Processes and Filtering Theory. Academic Press, San Diego (1970)
Julier, S.J., Uhlmann, J.K.: A new extension of the kalman filter to nonlinear systems. In: Proceedings of the International Symposium on Aerospace/Defense Sensing, Simululation and Controls, vol. Acquisition, Tracking and Pointing XI, Florida, USA, pp. 182–193 (1997)
Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME, J. Basic Eng., 35–45 (1960)
Li, L., McCann, J., Pollard, N.S., Faloutsos, C.: Dynammo: mining and summarization of coevolving sequences with missing values. In: Proceedings of the 15th International Conference on Knowledge Discovery and Data Mining, pp. 507–516. ACM, Paris (2009)
Li, L., McCann, J., Pollard, N.S., Faloutsos, C.: Bolero: a principled technique for including bone length constraints in motion capture occlusion filling. In: Proceedings of the ACM Symposium on Computer Animation, Madrid, Spain (2010)
Li, X.R., Jilkov, V.P.: Survey of maneuvering target tracking. Part I: Dynamic models. IEEE Transactions on Aerospace and Electronic Systems 39(4), 1333–1364 (2003)
Lin, M.C., Gottschalk, S.: Collision detection between geometric models: a survey. In: Proceedings of IMA Conference on Mathematics of Surfaces, pp. 37–56 (1998)
Liu, G., McMillan, L.: Estimation of missing markers in human motion capture. The Visual Computer 22(9–11), 721–728 (2006)
Liu, G., Zhang, J., Wang, W., McMillan, L.: Human motion estimation from a reduced marker set. In: I3D’06: Proceedings of the Symposium on Interactive 3D Graphics and Games, pp. 35–42. ACM, New York (2006)
Maidi, M., Ababsa, F., Mallem, M.: Handling occlusions for robust augmented reality systems. EURASIP J. Image Video Process. 2010 (2010)
Menache, A.: Understanding Motion Capture for Computer Animation and Video Games. Morgan Kaufmann, San Francisco (1999)
Merwe, R.V.D., Doucet, A., Freitas, N.D., Wan, E.: The unscented particle filter. Tech. Rep. F-INFENG/TR. 380, CUED (2000)
Nakamura, Y., Hanafusa, H.: Inverse kinematic solutions with singularity robustness for robot manipulator control. Trans. ASME Journal of Dynamic Systems, Measurement, and Control 108(3), 163–171 (1986)
O’Brien, J.F., Bodenheimer, R.E., Brostow, G.J., Hodgins, J.K.: Automatic joint parameter estimation from magnetic motion capture data. In: Proceedings of Graphic Interface, pp. 53–60 (2000)
Park, S.I., Hodgins, J.K.: Capturing and animating skin deformation in human motion. ACM Transaction on Graphics (TOG) 25(3), 881–889 (2006)
Pechev, A.N.: Inverse kinematics without matrix invertion. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA’08), Pasadena, CA, USA, pp. 2005–2012 (2008)
Phasespace inc: Optical motion capture systems. http://www.phasespace.com
Piazza, T., Lundström, J., Hugestrand, A., Kunz, A., Fjeld, M.: Towards solving the missing marker problem in realtime motion capture. In: Proceedings of the International Design Engineering Technical Conference (2009)
Rhijn, A.V., Mulder, J.D.: Optical tracking and automatic model estimation of composite interaction devices. In: IEEE Virtual Reality Conference, pp. 135–142 (2006)
Ringer, M., Lasenby, J.: A procedure for automatically estimating model parameters in optical motion capture. In: Proceedings of the British Machine Vision Conference, pp. 747–756 (2002)
Rose, C., Cohen, M., Bodenheimer, B.: Verbs and adverbs: multidimensional motion interpolation. IEEE Computer Graphics and Applications 18(5), 32–40 (1998)
Silaghi, M.C., Plänkers, R., Boulic, R., Fua, P., Thalmann, D.: Local and global skeleton fitting techniques for optical motion capture. In: Proceedings of the International Workshop on Modelling and Motion Capture Techniques for Virtual Environments, London, UK, pp. 26–40 (1998)
Singer, R.A.: Estimating optimal tracking filter performance for manned maneuvering targets. IEEE Transactions on Aerospace and Electronic Systems 6(1), 473–483 (1970)
Singer, R.A., Behnke, K.W.: Real-time tracking filter evaluation and selection for tactical applications. IEEE Transactions on Aerospace and Electronic Systems 7(1), 100–110 (1971)
Sumner, R.W., Zwicker, M., Gotsman, C., Popović, J.: Mesh-based inverse kinematics. ACM Transactions on Graphics (TOG) 24(3), 488–495 (2005)
Tak, S., Ko, H.S.: A physically-based motion retargeting filter. ACM Transactions on Graphics (TOG) 24(1), 98–117 (2005)
Taylor, G.W., Hinton, G.E., Roweis, S.T.: Modeling human motion using binary latent variables. In: Advances in Neural Information Processing Systems, pp. 1345–1352. MIT Press, Cambridge (2007)
Wampler, C.W.: Manipulator inverse kinematic solutions based on vector formulations and damped least-squares methods. IEEE Transactions on Systems, Man and Cybernetics 16(1), 93–101 (1986)
Wang, J.M., Fleet, D.J., Hertzmann, A.: Gaussian process dynamical models for human motion. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(2), 283–298 (2008)
Wang, L.C.T., Chen, C.C.: A combined optimization method for solving the inverse kinematics problems of mechanical manipulators. IEEE Transactions on Robotics and Automation 7(4), 489–499 (1991)
Welch, G., Bishop, G., Vicci, L., Brumback, S., Keller, K., Colucci, D.: The HiBall tracker: high-performance wide-area tracking for virtual and augmented environments. In: Virtual Reality Software and Technology, VRST, pp. 1–10. ACM, New York (1999)
Wiley, D.J., Hahn, J.K.: Interpolation synthesis of articulated figure motion. IEEE Computer Graphics and Applications 17(6), 39–45 (1997)
Wolovich, W.A., Elliott, H.: A computational technique for inverse kinematics. IEEE Conference on Decision and Control 23, 1359–1363 (1984)
Yu, Q., Li, Q., Deng, Z.: Online motion capture marker labeling for multiple interacting articulated targets. Computer Graphics Forum (Proceedings of Eurographics) 27(7), 477–483 (2007)
Zordan, V.B., Van Der Horst, N.C.: Mapping optical motion capture data to skeletal motion using a physical model. In: Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA’03), pp. 245–250. Eurographics Association, San Diego (2003)
Author information
Authors and Affiliations
Corresponding author
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
(AVI 21.9 MB)
Rights and permissions
About this article
Cite this article
Aristidou, A., Lasenby, J. Real-time marker prediction and CoR estimation in optical motion capture. Vis Comput 29, 7–26 (2013). https://doi.org/10.1007/s00371-011-0671-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-011-0671-y