Abstract
Sparse inverse covariance estimation, i.e., Graphical Lasso, can estimate the connections among a set of random variables basing on their observations. Recent research on Graphical Lasso has been extended to multi-task settings, where multiple graphs sharing the same set of variables are estimated collectively to reduce variances. However, different tasks usually involve different variables. For example, when we want to estimate gene networks w.r.t different diseases simultaneously, the related gene sets vary. In this paper, we study the problem of multi-task Graphical Lasso where the tasks may involve different variable sets. To share information across tasks, we consider the attributes of variables and assume that the structures of graphs are not only determined by observations, but influenced by attributes. We formulate the problem of learning multiple graphs jointly with observations and attributes, i.e., Multi-task Attributed Graphical Lasso (MAGL), and propose an effective algorithm to solve it. We rely on the LogDet divergence to explore latent relations between attributes of the variables and linkage structures among the variables. Multiple precision matrices and a projection matrix are optimized such that the \(\ell _1\)-penalized negative log-likelihood and the divergence are minimized.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Banerjee, O., Ghaoui, L.E., d’Aspremont, A.: Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. JMLR 9(Mar), 485–516 (2008)
Cai, T., Liu, W., Luo, X.: A constrained l1 minimization approach to sparse precision matrix estimation. JASA 106(494), 594–607 (2011)
Danaher, P., Wang, P., Witten, D.M.: The joint graphical lasso for inverse covariance estimation across multiple classes. J. R. Stat. Soc. Ser. B Stat. Methodol. 76(2), 373–397 (2014)
Davis, J.V., Dhillon, I.S.: Differential entropic clustering of multivariate gaussians. In: NeurIPS, pp. 337–344 (2007)
Duchi, J.C., Gould, S., Koller, D.: Projected subgradient methods for learning sparse Gaussians. In: UAI (2008)
Fan, J., Liao, Y., Liu, H.: An overview of the estimation of large covariance and precision matrices. ECONOMET J. 19(1), C1–C32 (2016)
Friedman, J., Hastie, T., Tibshirani, R.: Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9(3), 432–441 (2008)
Gentles, A.J., Plevritis, S.K., Majeti, R., Alizadeh, A.A.: Association of a leukemic stem cell gene expression signature with clinical outcomes in acute myeloid leukemia. JAMA 304(24), 2706–2715 (2010)
Grechkin, M., Fazel, M., Witten, D., Lee, S.: Pathway graphical lasso. In: AAAI, pp. 2617–2623 (2015)
Guo, J., Levina, E., Michailidis, G., Zhu, J.: Joint estimation of multiple graphical models. Biometrika 98(1), 1–15 (2011)
Haferlach, T., et al.: Clinical utility of microarray-based gene expression profiling in the diagnosis and subclassification of leukemia: report from the international microarray innovations in leukemia study group. Int. J. Clin. Oncol. 28(15), 2529–2537 (2010)
Hara, S., Washio, T.: Common substructure learning of multiple graphical Gaussian models. In: Gunopulos, D., Hofmann, T., Malerba, D., Vazirgiannis, M. (eds.) ECML PKDD 2011. LNCS (LNAI), vol. 6912, pp. 1–16. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23783-6_1
Hara, S., Washio, T.: Learning a common substructure of multiple graphical Gaussian models. Neural Netw. 38, 23–38 (2013)
Honorio, J., Samaras, D.: Multi-task learning of Gaussian graphical models. In: ICML, pp. 447–454 (2010)
Hsieh, C., Sustik, M.A., Dhillon, I.S., Ravikumar, P.D.: QUIC: quadratic approximation for sparse inverse covariance estimation. JMLR 15(1), 2911–2947 (2014)
Huang, S., et al.: Learning brain connectivity of Alzheimer’s disease by sparse inverse covariance estimation. NeuroImage 50(3), 935–949 (2010)
Kang, Z., Peng, C., Cheng, J., Cheng, Q.: Logdet rank minimization with application to subspace clustering. Comput. Intell. Neurosci. 2015, 68 (2015)
Kulis, B., Sustik, M., Dhillon, I.: Learning low-rank kernel matrices. In: ICML, pp. 505–512 (2006)
Lee, W., Liu, Y.: Joint estimation of multiple precision matrices with common structures. JMLR 16(1), 1035–1062 (2015)
Maaten, L.V.D., Hinton, G.: Visualizing data using T-SNE. JMLR 9(Nov), 2579–2605 (2008)
Mazumder, R., Hastie, T.: The graphical lasso: new insights and alternatives. EJS 6, 2125 (2012)
Mohan, K., London, P., Fazel, M., Witten, D., Lee, S.: Node-based learning of multiple Gaussian graphical models. JMLR 15(1), 445–488 (2014)
Sun, Y., Han, J., Gao, J., Yu, Y.: itopicmodel: information network-integrated topic modeling. In: ICDM, pp. 493–502 (2009)
Tao, Q., Huang, X., Wang, S., Xi, X., Li, L.: Multiple Gaussian graphical estimation with jointly sparse penalty. Sig. Process. 128, 88–97 (2016)
Whittaker, J.: Graphical Models in Applied Multivariate Statistics. Wiley, Hoboken (2009)
Witten, D.M., Friedman, J.H., Simon, N.: New insights and faster computations for the graphical lasso. J. Comput. Graph Stat. 20(4), 892–900 (2011)
Yang, S., Lu, Z., Shen, X., Wonka, P., Ye, J.: Fused multiple graphical lasso. SIOPT 25(2), 916–943 (2015)
Yin, H., Liu, X., Kong, X.: Coherent graphical lasso for brain network discovery. In: ICDM (2018)
Yuan, X.: Alternating direction methods for sparse covariance selection. Optimization Online (2009)
Zhang, Y., Xiong, Y., Liu, X., Kong, X., Zhu, Y.: Meta-path graphical lasso for learning heterogeneous connectivities. In: SDM, pp. 642–650 (2017)
Acknowledgement
This work is supported in part by the Shanghai Science and Technology Development Fund No. 19511121204, No. 19DZ1200802, and the National Natural Science Foundation of China Projects No. U1636207, No. U1936213.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Y., Xiong, Y., Kong, X., Liu, X., Zhu, Y. (2020). Multi-task Attributed Graphical Lasso. In: Wang, X., Zhang, R., Lee, YK., Sun, L., Moon, YS. (eds) Web and Big Data. APWeb-WAIM 2020. Lecture Notes in Computer Science(), vol 12317. Springer, Cham. https://doi.org/10.1007/978-3-030-60259-8_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-60259-8_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60258-1
Online ISBN: 978-3-030-60259-8
eBook Packages: Computer ScienceComputer Science (R0)