Abstract
Finite mixtures of nonlinear mixed-effects models have emerged as a prominent tool for modeling and clustering longitudinal data following nonlinear growth patterns with heterogeneous behavior. This paper proposes an extended finite mixtures of nonlinear mixed-effects model in which the mixing proportions are related to some explanatory covariates. A logistic function is incorporated to describe the relationship between the prior classification probabilities and the covariates of interest. For parameter estimation, we develop an analytically simple expectation conditional maximization algorithm coupled with the first-order Taylor approximation to linearize the model with pseudo data. The calculation of the standard errors of estimators via a general information-based method and the empirical Bayes estimation of random effects are also discussed. The methodology is illustrated through several simulation experiments and an application to the AIDS Clinical Trials Group Protocol 315 study.
Similar content being viewed by others
References
Aitken AC (1926) On Bernoulli’s numerical solution of algebraic equations. Proc R Soc Edinb 46:289–305
Basso RM, Lachos VH, Cabral CRB, Ghosh P (2010) Robust mixture modeling based on scale mixtures of skew-normal distributions. Comput Stat Data Anal 54:2926–2941
Booth J, Casella G, Hobert J (2008) Clustering using objective functions and stochastic search. J R Stat Soc Ser B 70:119–139
Celeux G, Martin O, Lavergne C (2005) Mixture of linear mixed models for clustering gene expression profiles from repeated microarray experiments. Stat Model 5:243–267
Dang U, Punzo A, McNicholas P, Ingrassia S, Browne R (2017) Multivariate response and parsimony for Gaussian cluster-weighted models. J Classif 34:4–34
Dayton C, Macready G (1988) Concomitant-variable latent-class models. J Am Stat Assoc 83:173–178
De la Cruz-Mesía R, Quintana FA, Marshall G (2008) Model-based clustering for longitudinal data. Comput Stat Data Anal 52:1441–1457
Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J R Stat Soc Ser B 9:1–38
Fraley C, Raftery AE (1998) How many clusters? Which clustering method? Answers via model-based cluster analysis. Comput J 41:578–588
Fraley C, Raftery A (2002) Model-based clustering, discriminant analysis, and density estimation. J Am Stat Assoc 97:611–612
Frühwirth-Schnatter S (2006) Finite mixture and markov switching models. Springer, New York
Gaffney S, Smyth P (2003) Curve clustering with random effects regression mixtures. In: Bishop C, Frey B (Eds.) Proceedings of the ninth international workshop on artificial intelligence and statistics. Akademiai Kiado, KeyWest, FL
Gershenfeld N (1997) Nonlinear inference and cluster-weighted modeling. Ann NY Acad Sci 808:18–24
Goldfeld S, Quandt R (1973) A Markov model for switching regression. J Econom 1:3–15
Grün B, Leisch F (2008) Flexmix version 2: finite mixtures with concomitant variables and varying and constant parameters. J Stat Softw 28:1–35
Hammer S, Squires K, Hughes M, Grimes J, Demeter L, Currier J, Eron J, Feinberg J, Balfour H, Deyton L, Chodakewitz J, Fischl M (1997) A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of per cubic millimeter or less. N Engl J Med 337:725–733
Hartigan JA, Wong MA (1979) Algorithm AS 136: a \(K\)-means clustering algorithm. J R Stat Soc C 28:100–108
Hennig C (2000) Identifiability of models for clusterwise linear regression. J Classif 17:273–296
Huang Y, Dagne G (2011) A Bayesian approach to joint mixed effects models with a skew-normal distribution and measurement errors in covariates. Biometrics 67:260–269
Huang M, Yao W (2012) Mixture of regression models with varying mixing proportions: a semiparametric approach. J Am Stat Assoc 107:711–724
Hubert L, Arabie P (1985) Comparing partitions. J Classif 2:193–218
Hunter D, Lange K (2004) A tutorial on MM algorithms. Am Stat 58:30–37
Ingrassia S, Minotti S, Punzo A (2014) Model-based clustering via linear cluster-weighted models. Comput Stat Data Anal 71:159–182
Ingrassia S, Punzo A (2016) Decision boundaries for mixtures of regressions. J Korean Stat Soc 45:295–306
Ingrassia S, Punzo A (2020) Cluster validation for mixtures of regressions via the total sum of squares decomposition. J Classif 37:526–547
Ingrassia S, Punzo A, Vittadini G, Minotti S (2015) The generalized linear mixed cluster-weighted model. J Classif 32:85–113
Keribin C (2000) Consistent estimation of the order of mixture models. Sankhyõ A 62:49–66
Konrad S, Skinner S, Kazadi G, Gartner K, Lim H (2013) HIV disease progression to CD4 count\(<\)200 cells/\(\mu \)L and death in Saskatoon, Saskatchewan. Can J Infect Dis Med Microbiol 24:97–101
Krentz H, Auld M, Gill M (2004) The high cost of medical care for patients who present late (CD4\(<\)200 cells/\(\mu \)L) with HIV infection. HIV Med 5:93–98
Laird N, Ware J (1982) Random effects models for longitudinal data. Biometrics 38:963–974
Lamont A, Vermunt J, Horn MV (2016) Regression mixture models: Does modeling the covariance between independent variables and latent classes improve the results? Multivar Behav Res 51:35–52
Lavielle M, Aarons L (2016) What do we mean by identifiability in mixed effects models? J Pharmacokinet Pharmacodyn 43:111–122
Lederman M, Connick E, Landay A, Kuritzkes D, Spritzler J, Clair M, Kotzin B, Fox L, Chiozzi M, Leonard J, Rousseau F, Wade M, Roe J, Martinez A, Harold K (1998) Immunologic responses associated with 12 weeks of combination antiretroviral therapy consisting of zidovudine, lamivudine, and ritonavir: results of AIDS clinical trials group protocol 315. J Infect Dis 178:70–79
Lee W, Chen Y, Hsieh K (2003) Ultrasonic liver tissues classification by fractal feature vector based on M-band wavelet transform. IEEE Trans Med Imaging 22:382–392
Liang H, Wu H, Carroll R (2003) The relationship between virologic and immunologic responses in AIDS clinical research using mixed effects varying coefficient models with measurement error. Biostatistics 4:297–312
Lin TI, Lachos VH, Wang WL (2018) Multivariate longitudinal data analysis with censored and intermittent missing responses. Stat Med 37(19):2822–2835
Lin TI, Wang WL (2013) Multivariate skew-normal at linear mixed models for multi-outcome longitudinal data. Stat Model 13:199–221
Lin TI, Wang WL (2017) Multivariate-\(t\) nonlinear mixed models with application to censored multi-outcome AIDS studies. Biostatistics 18:666–681
Lin TI, Wang WL (2020) Multivariate-\(t\) linear mixed models with censored responses, intermittent missing values and heavy tails. Stat Meth Med Res 29(5):1288–1304
Lindstrom MJ, Bates DM (1990) Nonlinear mixed effects models for repeated measures data. Biometrics 46:673–687
Louis T (1982) Finding the observed information matrix when using the EM algorithm. J R Stat Soc Ser B 44:226–233
Matos LA, Lachos VH, Lin TI, Castro LM (2019) Heavy-tailed longitudinal regression models for censored data: a robust parametric approach. Test 28:844–878
McLachlan GJ, Krishnan T (2008) The EM algorithm and extensions, 2nd edn. John Wiley & Sons, New York
McLachlan G, Peel D (2000) Finite mixture models. Wiley, New York
Meng X, Rubin D (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80:267–278
Muñoz A, Carey V, Schouten JP, Segal M, Rosner B (1992) A parametric family of correlation structures for the analysis of longitudinal data. Biometrics 48:733–742
Ng S, McLachlan G (2014) Mixture models for clustering multilevel growth trajectories. Comput Stat Data Anal 71:43–51
Ng S, Mclachlan G, Wang K, Jones L, Ng S (2006) A mixture model with random-effects components for clustering correlated gene-expression profiles. Bioinformatrics 22:1745–1752
Perelson A, Essunger P, Cao Y, Vesanen M, Hurley A, Saksela K, Markowitz M, Ho D (1997) Decay characteristics of HIV-1-infected compartments during combination therapy. Nature 387:188–191
Pfeifer C (2004) Classification of longitudinal profiles based on semi-parametric regression with mixed effects. Stat Med 4:314–323
Pinheiro J, Bates D, Debroy S, Sarkar D, R Core Team (2016) nlme: linear and nonlinear mixed effects models. R package version 3.1-128. http://CRAN.R-project.org/package=nlme. Accessed from 8 Sep 2016
Punzo A (2014) Flexible mixture modeling with the polynomial Gaussian cluster-weighted model. Stat Model 14:257–291
Quandt R (1972) A new approach to estimating switching regressions. J Am Stat Assoc 67:306–310
Quandt R, Ramsey J (1978) Estimating mixtures of normal distributions and switching regressions. J Am Stat Assoc 73:730–738
R Core Team (2019) R: a Language and Environment for Statistical Computing. http://www.r-project.org/
Redner RA, Walker HF (1984) Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev 26:195–239
Regis M, Brini A, Nooraee N, Haakma R, van den Heuvel ER (2019) The \(t\) linear mixed model: model formulation, identifiability and estimation. Commun Stat Simul Comput. https://doi.org/10.1080/03610918.2019.1694153
Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J, Müller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinform 12(1):77
Sarbo W, Cron W (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5:249–282
Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6:461–464
Stephens M (2000) Dealing with label switching in mixture models. J R Stat Soc Ser B 62:795–809
Subedi S, Punzo A, Ingrassia S, McNicholas P (2015) Cluster-weighted \(t\)-factor analyzers for robust model-based clustering and dimension reduction. Stat Methods Appl 24:623–649
Trabzuni D, Thomson P, The United KingdomBrain Expression Consortium (UKBEC) (2014) Analysis of gene expression data using a linear mixed model/finite mixture model approach: application to regional differences in the human brain. Bioinformatics 30:1555–1561
Verbeke G, Lesaffre E (1996) A linear mixed-effects model with heterogeneity in the random-effects population. J Am Stat Assoc 91:217–221
Wang W (2013) Identifiability of linear mixed effects models. Electron J Stat 7:244–263
Wang W (2016) Identifiability of covariance parameters in linear mixed effects models. Linear Algebra Appl 506:603–613
Wang WL (2019) Mixture of multivariate \(t\) nonlinear mixed models for multiple longitudinal data with heterogeneity and missing values. TEST 28:196–222
Wang WL (2020) Bayesian analysis of multivariate linear mixed models with censored and intermittent missing responses. Stat Med 39(19):2518–2535
Wang WL, Lin TI (2014) Multivariate t nonlinear mixed-effects models for multi-outcome longitudinal data with missing values. Stat Med 33:3029–3046
Wang WL, Lin TI, Lachos VH (2018) Extending multivariate-\(t\) linear mixed models for multiple longitudinal data with censored responses and heavy tails. Stat Meth Med Res 27:48–64
Wedel M (2002) Concomitant variables in finite mixture models. Stat Neerl 56:362–375
Wedel M, Kamakura W (2000) Market segmentation: conceptual and methodological foundations, 2nd edn. Kluwer Academic Publishers, Boston
Wolfinger R, Lin X (1997) Two Taylor-series approximation methods for nonlinear mixed models. Comput Stat Data Anal 25:465–490
Yang YC, Lin TI, Luis MC, Wang WL (2020) Extending finite mixtures of \(t\) linear mixed-effects models with concomitant covariates. Comput Stat Data Anal 148:106961
Yau K, Lee A, Ng S (2002) Finite mixture regression model with random effects: application to neonatal hospital length of stay. Comput Stat Data Anal 41:359–366
Zeller CB, Cabral CRB, Lachos VH, Benites L (2019) Finite mixture of regression models for censored data based on scale mixtures of normal distributions. Adv Data Anal Classif 13:89–116
Acknowledgements
The authors gratefully acknowledge the Associate Editor and three anonymous referees for their insightful comments which helped to improve the quality of the paper. In addition, the authors are grateful to Mr. Yi-Cong Li for his skillful assistance of initial graphical outputs. This work was supported in part by the Ministry of Science and Technology of Taiwan under Grant Nos. 110-2118-M-006-006-MY3 and 109-2118-M-005-005-MY3.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Explicit expressions for the score vector and Hessian matrix
Appendix: Explicit expressions for the score vector and Hessian matrix
Taking the first partial derivative of (20) for the jth individual with respect to each entry of \({\varvec{\theta }}_i\) gives \({\varvec{s}}_{{\varvec{\theta }}_i}^{(j)}=({\varvec{s}}_{{\varvec{\beta }}_i}^{{(j)}^\top },s_{\sigma _i^2}^{(j)},{\varvec{s}}_{{\varvec{\alpha }}_i}^{{(j)}^\top })^\top \), where
and
with \(\dot{\tilde{{\varvec{\varLambda }}}}_{ ijl}=\partial \tilde{{\varvec{\varLambda }}}_{ij}/\partial ({\varvec{\alpha }}_i)_l=\tilde{{\varvec{Z}}}_{ij}\frac{\partial {\varvec{D}}_i}{\partial ({\varvec{\alpha }}_i)_l}\tilde{{\varvec{Z}}}_{ij}^{\top }\) if \(({\varvec{\alpha }}_i)_l=\text{ vech }({\varvec{D}}_i)\), and \(\dot{\tilde{{\varvec{\varLambda }}}}_{ ijl}=\frac{\partial {\varvec{C}}_{ij}({\varvec{\phi }}_i,{\varvec{t}}_j)}{\partial ({\varvec{\alpha }}_i)_l}\) if \(({\varvec{\alpha }}_i)_l={\varvec{\phi }}_i\), for \(l=1,\ldots ,q^*\) where \(q^*=q(q+1)/2+1\). The expressions for the entries of \({\varvec{H}}_{{\varvec{\theta }}_i{\varvec{\theta }}_i}^{(j)}\) derived from the minus second partial derivative of (20) for the jth individual are
and
for \(l,r=1,\ldots ,q^*\).
Rights and permissions
About this article
Cite this article
Wang, WL., Yang, YC. & Lin, TI. Extending finite mixtures of nonlinear mixed-effects models with covariate-dependent mixing weights. Adv Data Anal Classif 18, 271–307 (2024). https://doi.org/10.1007/s11634-022-00502-w
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11634-022-00502-w