[go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Univariate Bayesian nonparametric mixture modeling with unimodal kernels

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

Within the context of mixture modeling, the normal distribution is typically used as the components distribution. However, if a cluster is skewed or heavy tailed, then the normal distribution will be inefficient and many may be needed to model a single cluster. In this paper, we present an attempt to solve this problem. We define a cluster, in the absence of further information, to be a group of data which can be modeled by a unimodal density function.

Hence, our intention is to use a family of univariate distribution functions, to replace the normal, for which the only constraint is unimodality. With this aim, we devise a new family of nonparametric unimodal distributions, which has large support over the space of univariate unimodal distributions.

The difficult aspect of the Bayesian model is to construct a suitable MCMC algorithm to sample from the correct posterior distribution. The key will be the introduction of strategic latent variables and the use of the Product Space view of Reversible Jump methodology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Becker, M.: PearsonDS: Pearson Distribution System, R Package Version 0.93, http://CRAN.R-project.org/package=PearsonDS (2012)

  • Blackwell, D.: The discreteness of Ferguson selections. Ann. Stat. 1, 356–358 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  • Brunner, J.L., Lo, A.Y.: Bayes methods for symmetric unimodal density and its mode. Ann. Stat. 17, 1550–1566 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  • Byron, H.: LaplacesDemon: Software for Bayesian Inference, R Package Version 12.01.02, http://cran.r-project.org/web/packages/LaplacesDemon/index.html (2012)

  • Carlin, B.P., Chib, S.: Bayesian model choice via Markov chain Monte Carlo methods. J. R. Stat. Soc. B 57, 473–484 (1995)

    MATH  Google Scholar 

  • Damien, P., Walker, S.G.: Sampling truncated normal, beta and gamma densities. J. Comput. Graph. Stat. 10, 206–215 (2001)

    Article  MathSciNet  Google Scholar 

  • Devroye, L.: Random variate generation for multivariate unimodal densities. ACM Trans. Model. Comput. Simul. 7, 447–477 (1997)

    Article  MATH  Google Scholar 

  • Diebolt, J., Robert, C.: Estimation of finite mixture distributions through Bayesian sampling. J. R. Stat. Soc. B 56, 363–375 (1994)

    MATH  MathSciNet  Google Scholar 

  • Escobar, M.D.: Estimating the Means of Several Normal Populations by Nonparametric Estimation of the Distribution of the Means. Unpublished Ph.D. dissertation, Department of Statistics, Yale University (1988)

  • Escobar, M.D.: Estimating normal means with a Dirichlet process prior. J. Am. Stat. Assoc. 89, 268–277 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  • Escobar, M.D., West, M.: Bayesian density estimation and inference using mixtures. J. Am. Stat. Assoc. 90, 577–588 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  • Feller, W.: An Introduction to Probability Theory and Its Applications, pp. 157–158. Wiley, New York (1971)

    MATH  Google Scholar 

  • Ferguson, T.S.: A Bayesian analysis of some nonparametric problems. Ann. Stat. 1, 209–230 (1973)

    Article  MATH  MathSciNet  Google Scholar 

  • Ferguson, T.S.: Bayesian density estimation by mixtures of normal distributions. In: Chernoff, H., Rustagi, J.S., Rizvi, M.H., Siegmund, D. (eds.) Recent Advances in Statistics: Papers in Honor of Herman Chernoff on his Sixtieth Birthday, pp. 287–302. Academic Press, New York (1983)

    Google Scholar 

  • Fernandez, C., Steel, M.F.J.: On Bayesian modeling of fat tails and skewness. J. Am. Stat. Assoc. 93, 359–371 (1998)

    MATH  MathSciNet  Google Scholar 

  • Frühwirth-Schnatter, S.: Finite Mixture and Markov Switching Models. Springer Series in Statistics. Springer, New York (2006)

    MATH  Google Scholar 

  • Galassi, M., Davies, J., Theiler, J., Gough, B., Jungman, G., Alken, P., Booth, M., Rossi, F.: GNU Scientific Library Reference Manual, Network Theory Limited. http://www.gnu.org/software/gsl/ (2009)

  • Godsill, S.J.: On the relationship between Markov chain Monte Carlo methods for model uncertainty. J. Comput. Graph. Stat. 10, 230–248 (2001)

    Article  MathSciNet  Google Scholar 

  • Green, P.J.: Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82, 711–732 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  • Green, P.J.: Trans-dimensional Markov chain Monte Carlo. In: Green, P.J., Hjort, N.L., Richardson, S. (eds.) Highly Structured Stochastic Systems, pp. 179–198. Oxford University Press, Oxford (2003)

    Google Scholar 

  • Jasra, A., Holmes, C.C., Stephens, D.A.: Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modeling. Stat. Sci. 20, 50–67 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  • Kalli, M., Griffin, J.E., Walker, S.G.: Slice sampling mixture models. Stat. Comput. 21, 93–105 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  • Khinchin, A.Y.: On unimodal distributions. Trams. Res. Inst. Math. Mech. (University of Tomsk) 2, 1–7 (1938) (in Russian)

    Google Scholar 

  • Komárek, A.: A new R Package for Bayesian estimation of multivariate normal mixtures allowing for selection of number of components and interval-censored data. Comput. Stat. Data Anal. 53, 3932–3947 (2009)

    Article  MATH  Google Scholar 

  • Kottas, A., Gelfand, A.E.: Bayesian semiparametric median regression modeling. J. Am. Stat. Assoc. 96, 1456–1468 (2001)

    Article  MathSciNet  Google Scholar 

  • Lo, A.Y.: On a class of Bayesian nonparametric estimates: I. Density estimates. Ann. Stat. 12, 351–357 (1984)

    Article  MATH  Google Scholar 

  • Nobile, A., Fearnside, A.T.: Bayesian finite mixtures with an unknown number of components: the allocation sampler. Stat. Comput. 17, 147–162 (2007)

    Article  MathSciNet  Google Scholar 

  • Papaspiliopoulos, O., Roberts, G.O.: Retrospective Markov chain Monte Carlo methods for Dirichlet process hierarchical models. Biometrika 95, 169–186 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  • Papastamoulis, P., Iliopoulos, G.: An artificial allocations based solution to the label switching problem in Bayesian analysis of mixtures of distributions. J. Comput. Graph. Stat. 19, 313–331 (2010)

    Article  MathSciNet  Google Scholar 

  • Peng, R.D., Leeuw, J.: An Introduction to the .C Interface to R. UCLA: Academic Technology Services, Statistical Consulting Group, http://www.ats.ucla.edu/stat/r/library/interface.pdf (2002)

  • Postman, M., Huchra, J.P., Geller, M.J.: Probes of large-scale structures in the Corona Borealis region. Astrophys. J. 92, 1238–1247 (1986)

    Google Scholar 

  • Quintana, F.A., Steel, M.F.J., Ferreira, J.T.A.S.: Flexible univariate continuous distributions. Bayesian Anal. 4, 497–522 (2009)

    Article  MathSciNet  Google Scholar 

  • R Development Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing. http://www.R-project.org/ (2011)

  • Richardson, S., Green, P.J.: On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Stat. Soc. B 59, 731–792 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  • Rigby, R.A., Stasinopoulos, D.M.: Generalized additive models for location, scale and shape. J. R. Stat. Soc. C 54, 507–554 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  • Rodríguez, C.E., Walker, S.G.: Label switching in Bayesian mixture models: deterministic relabeling strategies (2012, submitted manuscript)

  • Rudin, W.: Principles of Mathematical Analysis, pp. 221–223. McGraw-Hill, New York (1976)

    MATH  Google Scholar 

  • Sethuraman, J.: A constructive definition of Dirichlet priors. Stat. Sin. 4, 639–650 (1994)

    MATH  MathSciNet  Google Scholar 

  • Sisson, S.A.: Transdimensional Markov chains: a decade of progress and future perspectives. J. Am. Stat. Assoc. 100, 1077–1089 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  • Stephens, M.: Bayesian analysis of mixtures with an unknown number of components an alternative to reversible jump methods. Ann. Stat. 28, 40–74 (2000a)

    Article  MATH  MathSciNet  Google Scholar 

  • Stephens, M.: Dealing with label switching in mixture models. J. R. Stat. Soc. B 62, 795–809 (2000b)

    Article  MATH  MathSciNet  Google Scholar 

  • Tao, D.: On multivariate unimodal distributions. University of British Columbia, MSc Thesis, https://circle.ubc.ca/handle/2429/27411 (1989)

  • Tierney, L.: Markov chains for exploring posterior distributions (with discussion). Ann. Stat. 22, 1701–1762 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  • Walker, S.G.: Sampling the Dirichlet mixture model with slices. Commun. Stat. 36, 45–54 (2007)

    MATH  Google Scholar 

  • Wiper, M., Insua, R.D., Ruggeri, F.: Mixtures of gamma distributions with applications. J. Comput. Graph. Stat. 10, 440–454 (2001)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, Statistics and Actuarial Science, University of Kent, Canterbury, Kent, UK

    Carlos E. Rodríguez & Stephen G. Walker

Authors
  1. Carlos E. Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen G. Walker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos E. Rodríguez.

Additional information

We would like to thank the editor and two anonymous referees for their constructive comments which helped to improve the manuscript. The first author is at the University of Kent with grant support from CONACYT, the Mexican National Council for Science and Technology.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(PDF 128 kB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rodríguez, C.E., Walker, S.G. Univariate Bayesian nonparametric mixture modeling with unimodal kernels. Stat Comput 24, 35–49 (2014). https://doi.org/10.1007/s11222-012-9351-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-012-9351-7

Keywords

Search

Navigation