Abstract
Many longitudinal neuroimaging studies aim to improve the understanding of brain aging and diseases by studying the dynamic interactions between brain function and cognition. Doing so requires accurate encoding of their multidimensional relationship while accounting for individual variability over time. For this purpose, we propose an unsupervised learning model (called Contrastive Learning-based Graph Generalized Canonical Correlation Analysis (CoGraCa)) that encodes their relationship via Graph Attention Networks and generalized Canonical Correlational Analysis. To create brain-cognition fingerprints reflecting unique neural and cognitive phenotype of each person, the model also relies on individualized and multimodal contrastive learning. We apply CoGraCa to longitudinal dataset of healthy individuals consisting of resting-state functional MRI and cognitive measures acquired at multiple visits for each participant. The generated fingerprints effectively capture significant individual differences and outperform current single-modal and CCA-based multimodal models in identifying sex and age. More importantly, our encoding provides interpretable interactions between those two modalities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bachmann, D., et al.: Age-, sex-, and pathology-related variability in brain structure and cognition. Trans. psychiatry 13(1) (2023)
Benton, A., Khayrallah, H., Gujral, B., Reisinger, D.A., Zhang, S., Arora, R.: Deep generalized canonical correlation analysis. In: Proceedings of the 4th Workshop on Representation Learning for NLP, Florence, Italy, August 2, 2019, pp. 1–6. Association for Computational Linguistics (2019)
Bijsterbosch, J., Harrison, S., Duff, E., Alfaro-Almagro, F., Woolrich, M., Smith, S.: Investigations into within-and between-subject resting-state amplitude variations. Neuroimage 159, 57–69 (2017)
Cui, H., et al.: BrainGB: a benchmark for brain network analysis with graph neural networks. IEEE Trans. Med. Imaging 42(2), 493–506 (2022)
Finn, E.S., Todd Constable, R.: Individual variation in functional brain connectivity: implications for personalized approaches to psychiatric disease. Dialogues Clin. Neurosci. 18(3), 277–287 (2016)
Gao, M., et al.: Multimodal brain connectome-based prediction of suicide risk in people with late-life depression. Nature Mental Health 1(2), 100–113 (2023)
Honnorat, N., et al.: Alcohol use disorder and its comorbidity with HIV infection disrupts anterior cingulate cortex functional connectivity. Biol. Psychiatry: Cogn. Neurosci. Neuroimaging 7(11), 1127–1136 (2022)
Horst, P.: Generalized canonical correlations and their applications to experimental data. J. Clin. Psychol. 17, 331–47 (1961)
Hsieh, S., Yang, M.H., Yao, Z.F.: Age differences in the functional organization of the prefrontal cortex: analyses of competing hypotheses. Cereb. Cortex 33(7), 4040–4055 (2023)
Hu, J., Cao, L., Li, T., Dong, S., Li, P.: GAT-LI: a graph attention network based learning and interpreting method for functional brain network classification. BMC Bioinformatics 22(1), 1–20 (2021)
Ji, J., et al.: Mapping brain-behavior space relationships along the psychosis spectrum. Elife 10 (2021)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings (2017)
Lee, K., et al.: Human brain state dynamics reflect individual neuro-phenotypes. bioRxiv (2023)
Li, X., et al.: BrainGNN: interpretable brain graph neural network for fMRI analysis. Med. Image Anal. 74, 102233 (2021)
Li, Y., Wei, Q., Adeli, E., Pohl, K.M., Zhao, Q.: Joint graph convolution for analyzing brain structural and functional connectome. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2022 - 25th International Conference, Singapore, September 18-22, 2022, Proceedings, Part I. Lecture Notes in Computer Science, vol. 13431, pp. 231–240. Springer (2022)
Luo, L., et al.: Patterns of brain dynamic functional connectivity are linked with attention-deficit/hyperactivity disorder-related behavioral and cognitive dimensions. Psychol. Med. pp. 1–12 (2023)
Margolis, R.B., Scialfa, C.T.: Age differences in Wechsler memory scale performance. J. Clin. Psychol. 40(6), 1442–1449 (1984)
Moon, S., Hwang, J., Lee, H.: SDGCCA: supervised deep generalized canonical correlation analysis for multi-omics integration. J. Comput. Biol. 29(8), 892–907 (2022)
Morrison, M.W., Gregory, R.J., Paul, J.J.: Reliability of the finger tapping test and a note on sex differences. Percept. Mot. Skills 48(1), 139–142 (1979)
Nerrise, F., Zhao, Q., Poston, K.L., Pohl, K.M., Adeli, E.: An explainable geometric-weighted graph attention network for identifying functional networks associated with gait impairment. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2023 - 26th International Conference, Vancouver, BC, Canada, Proceedings, Part II. Lecture Notes in Computer Science, vol. 14221, pp. 723–733. Springer (2023)
Peng, L., Wang, N., Xu, J., Zhu, X., Li, X.: GATE: graph CCA for temporal self-supervised learning for label-efficient fMRI analysis. IEEE Trans. Med. Imaging 42(2), 391–402 (2022)
Radford, A., et al.: Learning transferable visual models from natural language supervision. In: Proceedings of the 38th International Conference on Machine Learning, 18-24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 8748–8763 (2021)
Ramanoël, S., York, E., Le Petit, M., Lagrené, K., Habas, C., Arleo, A.: Age-related differences in functional and structural connectivity in the spatial navigation brain network. Front. Neural Circuits 13, 69 (2019)
Statsenko, Y., et al.: Predicting age from behavioral test performance for screening early onset of cognitive decline. Front. Aging Neurosci. 13, 661514 (2021)
Sundararajan, M., Najmi, A.: The many Shapley values for model explanation. In: Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event. Proceedings of Machine Learning Research, vol. 119, pp. 9269–9278 (2020)
Thompson, B.: Canonical correlation analysis. Read. Underst. MORE Multivar. Stat. 285–316 (2000)
Tomasi, D., Volkow, N.D.: Measures of brain connectivity and cognition by sex in us children. JAMA Netw. Open 6(2), e230157–e230157 (2023)
Velickovic, P., et al.: Graph attention networks. stat. 1050(20), 10–48550 (2017)
Weis, S., Hodgetts, S., Hausmann, M.: Sex differences and menstrual cycle effects in cognitive and sensory resting state networks. Brain Cogn. 131, 66–73 (2019)
Weissenbacher, A., Kasess, C., Gerstl, F., Lanzenberger, R., Moser, E., Windischberger, C.: Correlations and anticorrelations in resting-state functional connectivity MRI: a quantitative comparison of preprocessing strategies. Neuroimage 47(4), 1408–1416 (2009)
Zhang, C., Dougherty, C.C., Baum, S.A., White, T., Michael, A.M.: Functional connectivity predicts gender: evidence for gender differences in resting brain connectivity. Hum. Brain Mapp. 39(4), 1765–1776 (2018)
Zhang, Y., et al.: Identification of psychiatric disorder subtypes from functional connectivity patterns in resting-state electroencephalography. Nature biomedical engineering 5(4), 309–323 (2021)
Zhu, X., Du, X., Kerich, M., Lohoff, F.W., Momenan, R.: Random forest based classification of alcohol dependence patients and healthy controls using resting state MRI. Neurosci. Lett. 676, 27–33 (2018)
Acknowledgments
The work was partly funded by the National Institute of Health (DA057567, AA05965, AA017347, AA010723, MH129694, MH130956, AG080425, AA028840), the DGIST Joint Research Project, the 2024 Stanford HAI Hoffman-Yee Grant, the Stanford HAI-Google Cloud Credits Award, BBRF Young Investigator Grant and the Lehigh University FIG (FIGAWD35) and CORE (001250) grants.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Y., Peng, W., Zhang, Y., Adeli, E., Zhao, Q., M. Pohl, K. (2025). Brain-Cognition Fingerprinting via Graph-GCCA with Contrastive Learning. In: Bathula, D.R., et al. Machine Learning in Clinical Neuroimaging. MLCN 2024. Lecture Notes in Computer Science, vol 15266. Springer, Cham. https://doi.org/10.1007/978-3-031-78761-4_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-78761-4_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-78760-7
Online ISBN: 978-3-031-78761-4
eBook Packages: Computer ScienceComputer Science (R0)
