[go: up one dir, main page]

Skip to main content
Log in

Parameter estimation in high dimensional Gaussian distributions

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

In order to compute the log-likelihood for high dimensional Gaussian models, it is necessary to compute the determinant of the large, sparse, symmetric positive definite precision matrix. Traditional methods for evaluating the log-likelihood, which are typically based on Cholesky factorisations, are not feasible for very large models due to the massive memory requirements. We present a novel approach for evaluating such likelihoods that only requires the computation of matrix-vector products. In this approach we utilise matrix functions, Krylov subspaces, and probing vectors to construct an iterative numerical method for computing the log-likelihood.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Al-Mohy, A., Higham, N.: Computing the action of the matrix exponential, with an application to exponential integrators. Technical report, University of Manchester (2011)

  • Anitescu, M., Chen, J., Wang, L.: A matrix-free approach for solving the Gaussian process maximum likelihood problem. SIAM J. Sci. Comput. 34, A240–A262 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  • Aune, E., Eidsvik, J., Pokern, Y.: Iterative numerical methods for sampling from high dimensional Gaussian distributions. Stat. Comput. (2012, forthcoming). doi:10.1007/s11222-012-9326-8

  • Bai, Z., Fahey, G., Golub, G.: Some large-scale matrix computation problems. J. Comput. Appl. Math. 74(1), 71–89 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  • Bai, Z., Golub, G.: Bounds for the trace of the inverse and the determinant of symmetric positive definite matrices. Ann. Numer. Math. 4, 29–38 (1997)

    MATH  MathSciNet  Google Scholar 

  • Bekas, C., Kokiopoulou, E., Saad, Y.: An estimator for the diagonal of a matrix. Appl. Numer. Math. 57(11–12), 1214–1229 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  • Benzi, M., Golub, G.: Bounds for the entries of matrix functions with applications to preconditioning. BIT Numer. Math. 39(3), 417–438 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  • Benzi, M., Razouk, N.: Decay bounds and \(\mathcal{O}(n)\) algorithms for approximating functions of sparse matrices. Electron. Trans. Numer. Anal. 28, 16–39 (2007)

    MATH  MathSciNet  Google Scholar 

  • Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. B 36, 192–236 (1974)

    MATH  MathSciNet  Google Scholar 

  • Bolin, D., Lindgren, F.: Spatial models generated by nested stochastic partial differential equations, with an application to global ozone mapping. Ann. Appl. Stat. 5(1), 523–550 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  • Buland, A., Omre, H.: Bayesian linearized AVO inversion. Geophysics 68, 185–198 (2003)

    Article  Google Scholar 

  • Candès, E., Demanet, L., Donoho, D., Ying, L.: Fast discrete curvelet transforms. Multiscale Model. Simul. 5(3), 861–899 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  • Chan, T., Tang, W., Wan, W.: Wavelet sparse approximate inverse preconditioners. BIT Numer. Math. 37(3), 644–660 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • Chen, Y., Davis, T., Hager, W., Rajamanickam, S.: Algorithm 887: CHOLMOD, supernodal sparse Cholesky factorization and update/downdate. ACM Trans. Math. Softw. 35(3), 22 (2008)

    Article  MathSciNet  Google Scholar 

  • Cressie, N.: Statistics for Spatial Data. Wiley, New York (1993)

    Google Scholar 

  • Cressie, N., Johannesson, G.: Fixed rank kriging for large spatial datasets. J. R. Stat. Soc. B 70, 209–226 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • Culberson, J.: Iterated greedy graph coloring and the difficulty landscape. Technical report, University of Alberta (1992)

  • Davies, P.I., Higham, N.J.: Computing f(A)b for matrix functions f In: QCD and Numerical Analysis III, pp. 15–24. Springer, Berlin (2005)

    Chapter  Google Scholar 

  • Davis, T., Hager, W.: Modifying a sparse Cholesky factorization. SIAM J. Matrix Anal. Appl. 20(3), 606–627 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  • Driscoll, A.: The Schwarz Christoffel Toolbox (2009). Available at http://www.math.udel.edu/~driscoll/software/SC. Electronic

  • Eidsvik, J., Shaby, B.A., Reich, B.J., Wheeler, M., Niemi, J.: Estimation and prediction in spatial models with block composite likelihoods using parallel computing. Technical report, NTNU (2011)

  • Estrada, E., Hatano, N., Benzi, M.: The physics of communicability in complex networks (2011). Arxiv preprint arXiv:1109.2950

  • Frommer, A.: BiCGStab (l) for families of shifted linear systems. Computing 70(2), 87–109 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  • Fuentes, M.: Approximate likelihood for large irregularly spaced spatial data. J. Am. Stat. Assoc. 102(477), 321–331 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  • Gorjanc, G.: Graphical model representation of pedigree based mixed model. In: 32nd International Conference on Information Technology Interfaces (ITI), 2010, pp. 545–550. IEEE Press, New York (2010)

    Google Scholar 

  • Gröchenig, K.: Foundations of Time-Frequency Analysis. Birkhäuser, Basel (2001)

    Book  MATH  Google Scholar 

  • Hale, N., Higham, N.J., Trefethen, L.N.: Computing A α, log(A) and related matrix functions by contour integrals. SIAM J. Numer. Anal. 46, 2505–2523 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • Higham, N.J.: Functions of Matrices: Theory and Computation. SIAM, Philadelphia (2008)

    Book  Google Scholar 

  • Huckle, M., Grote, M.: Parallel preconditioning with sparse approximate inverses. SIAM J. Sci. Comput. 18(3), 838–853 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • Hutchinson, M.: A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Commun. Stat., Simul. Comput. 18, 1059–1076 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  • Jegerlehner, B.: Krylov space solvers for shifted linear systems (1996). arXiv:hep-lat/9612014v1

  • Karypis, G., Kumar, V.: A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM J. Sci. Comput. 20(1), 359 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  • Lindgren, F., Lindstrøm, J., Rue, H.: An explicit link between Gaussian fields and Gaussian Markov random fields: the SPDE approach. J. R. Stat. Soc. B 73, 423–498 (2011)

    Article  MATH  Google Scholar 

  • Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1998)

    MATH  Google Scholar 

  • MATLAB (2010). Version 7.11.0 (R2010b). The MathWorks Inc., Natick, Massachusetts

  • McPeters, R., Aeronautics, U.S.N., Scientific, S.A., Branch, T.I.: Nimbus-7 Total Ozone Mapping Spectrometer (TOMS) data products user’s guide. NASA, Scientific and Technical Information Branch (1996)

  • Rue, H., Held, L.: Gaussian Markov Random Fields. Chapman & Hall, London (2005)

    Book  MATH  Google Scholar 

  • Rue, H., Martino, S., Chopin, N.: Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. J. R. Stat. Soc., Ser. B, Stat. Methodol. 71(2), 319–392 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  • Rue, H., Tjelmeland, H.: Fitting Gaussian Markov random fields to Gaussian fields. Scand. J. Stat. 29, 31–49 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  • Saad, Y.: Iterative Methods for Sparse Linear Systems, 2nd edn. SIAM, Philadelphia (2003)

    Book  MATH  Google Scholar 

  • Simpson, D.: Krylov subspace methods for approximating functions of symmetric positive definite matrices with applications to applied statistics and anomalous diffusion. Ph.D. thesis, School of Mathematical Sciences, Queensland Univ. of Tech. (2008)

  • Simpson, D., Turner, I., Pettitt, A.: Fast sampling from a Gaussian Markov random field using Krylov subspace approaches. Technical report, School of Mathematical Sciences, Queensland Univ. of Tech. (2008)

  • Stein, M., Chen, J., Anitescu, M.: Stochastic approximation of score functions for Gaussian processes. Technical report (2012). Preprint ANL/MCSP-2091-0512

  • Tang, J., Saad, Y.: A probing method for computing the diagonal of the matrix inverse. Technical report, Minnesota Supercomputing Institute for Advanced Computational Research (2010)

  • van der Vorst, H., Melissen, J.: A Petrov-Galerkin type method for solving A×k=b, where A is symmetric complex. IEEE Trans. Magn. 26(2), 706–708 (1990)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Erlend Aune.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Aune, E., Simpson, D.P. & Eidsvik, J. Parameter estimation in high dimensional Gaussian distributions. Stat Comput 24, 247–263 (2014). https://doi.org/10.1007/s11222-012-9368-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-012-9368-y

Keywords

Navigation