ssrn.4635313

Multimodal integration of neuroimaging and genetic data for the diagnosis of mood disorders based on computer vision models.

URL:

https://doi.org/10.2139/ssrn.4635313

DOI

10.2139/ssrn.4635313

Authors and Affiliations:

Seungeun Lee Department of Mathematics, Korea University, Anamro 145, Seoungbuk-gu, Seoul, 02841, Republic of Korea.

Yongwon Cho Department of Computer Science and Engineering, Soonchunhyang University, South Korea, Republic of Korea.

Yuyoung Ji Division of Life Science, Korea University, Anamro 145, Seoungbuk-gu, Seoul, 02841, Republic of Korea.

Minhyek Jeon Division of Biotechnology, Korea University, Anamro 145, Seoungbuk-gu, Seoul, 02841, Republic of Korea; Computational Biology Department, Carnegie Mellon University, Pittsburgh, PA, 15213, United States.

Aram Kim Department of Biomedical Sciences, Korea University College of Medicine, Seoul, 02841, Republic of Korea.

Byung-Joo Ham Department of Psychiatry, Korea University Anam Hospital, 73, Goryeodae-ro, Seoungbuk-gu, Seoul, 02841, Republic of Korea. Electronic address: byungjoo.ham@gmail.com.

Yoonjung Yoonie Joo Department of Digital Health, Samsung Advanced Institute for Health Sciences & Technology (SAIHST), Sungkyunkwan University, Samsung Medical Center, 115 Irwon-Ro, Gangnam-Gu, Seoul, 06355, Republic of Korea. Electronic address: helloyjjoo@gmail.com.

Abstract:

Mood disorders, particularly major depressive disorder (MDD) and bipolar disorder (BD), are often underdiagnosed, leading to substantial morbidity. Harnessing the potential of emerging methodologies, we propose a novel multimodal fusion approach that integrates patient-oriented brain structural magnetic resonance imaging (sMRI) scans with DNA whole-exome sequencing (WES) data. Multimodal data fusion aims to improve the detection of mood disorders by employing established deep-learning architectures for computer vision and machine-learning strategies. We analyzed brain imaging genetic data of 321 East Asian individuals, including 147 patients with MDD, 78 patients with BD, and 96 healthy controls. We developed and evaluated six fusion models by leveraging common computer vision models in image classification: Vision Transformer (ViT), Inception-V3, and ResNet50, in conjunction with advanced machine-learning techniques (XGBoost and LightGBM) known for high-dimensional data analysis. Model validation was performed using a 10-fold cross-validation. Our ViT ⊕ XGBoost fusion model with MRI scans, genomic Single Nucleotide polymorphism (SNP) data, and unweighted polygenic risk score (PRS) outperformed baseline models, achieving an incremental area under the curve (AUC) of 0.2162 (32.03% increase) and 0.0675 (+8.19%) and incremental accuracy of 0.1455 (+25.14%) and 0.0849 (+13.28%) compared to SNP-only and image-only baseline models, respectively. Our findings highlight the opportunity to refine mood disorder diagnostics by demonstrating the transformative potential of integrating diverse, yet complementary, data modalities and methodologies.