Tags

Type your tag names separated by a space and hit enter

The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features.
Neuroimage 2018; 178:622-637N

Abstract

Individualized behavioral/cognitive prediction using machine learning (ML) regression approaches is becoming increasingly applied. The specific ML regression algorithm and sample size are two key factors that non-trivially influence prediction accuracies. However, the effects of the ML regression algorithm and sample size on individualized behavioral/cognitive prediction performance have not been comprehensively assessed. To address this issue, the present study included six commonly used ML regression algorithms: ordinary least squares (OLS) regression, least absolute shrinkage and selection operator (LASSO) regression, ridge regression, elastic-net regression, linear support vector regression (LSVR), and relevance vector regression (RVR), to perform specific behavioral/cognitive predictions based on different sample sizes. Specifically, the publicly available resting-state functional MRI (rs-fMRI) dataset from the Human Connectome Project (HCP) was used, and whole-brain resting-state functional connectivity (rsFC) or rsFC strength (rsFCS) were extracted as prediction features. Twenty-five sample sizes (ranged from 20 to 700) were studied by sub-sampling from the entire HCP cohort. The analyses showed that rsFC-based LASSO regression performed remarkably worse than the other algorithms, and rsFCS-based OLS regression performed markedly worse than the other algorithms. Regardless of the algorithm and feature type, both the prediction accuracy and its stability exponentially increased with increasing sample size. The specific patterns of the observed algorithm and sample size effects were well replicated in the prediction using re-testing fMRI data, data processed by different imaging preprocessing schemes, and different behavioral/cognitive scores, thus indicating excellent robustness/generalization of the effects. The current findings provide critical insight into how the selected ML regression algorithm and sample size influence individualized predictions of behavior/cognition and offer important guidance for choosing the ML regression algorithm or sample size in relevant investigations.

Authors+Show Affiliations

State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China; Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA.State Key Laboratory of Cognitive Neuroscience and Learning & IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, 100875, China; Beijing Key Laboratory of Brain Imaging and Connectomics, Beijing Normal University, Beijing, 100875, China. Electronic address: gaolang.gong@bnu.edu.cn.

Pub Type(s)

Journal Article
Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't

Language

eng

PubMed ID

29870817

Citation

Cui, Zaixu, and Gaolang Gong. "The Effect of Machine Learning Regression Algorithms and Sample Size On Individualized Behavioral Prediction With Functional Connectivity Features." NeuroImage, vol. 178, 2018, pp. 622-637.
Cui Z, Gong G. The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features. Neuroimage. 2018;178:622-637.
Cui, Z., & Gong, G. (2018). The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features. NeuroImage, 178, pp. 622-637. doi:10.1016/j.neuroimage.2018.06.001.
Cui Z, Gong G. The Effect of Machine Learning Regression Algorithms and Sample Size On Individualized Behavioral Prediction With Functional Connectivity Features. Neuroimage. 2018;178:622-637. PubMed PMID: 29870817.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - The effect of machine learning regression algorithms and sample size on individualized behavioral prediction with functional connectivity features. AU - Cui,Zaixu, AU - Gong,Gaolang, Y1 - 2018/06/02/ PY - 2018/02/28/received PY - 2018/05/31/revised PY - 2018/06/01/accepted PY - 2018/6/6/pubmed PY - 2019/3/21/medline PY - 2018/6/6/entrez KW - Functional magnetic resonance imaging (MRI) KW - Individualized prediction KW - Machine learning KW - Regression algorithm KW - Resting-state functional connectivity KW - Sample size SP - 622 EP - 637 JF - NeuroImage JO - Neuroimage VL - 178 N2 - Individualized behavioral/cognitive prediction using machine learning (ML) regression approaches is becoming increasingly applied. The specific ML regression algorithm and sample size are two key factors that non-trivially influence prediction accuracies. However, the effects of the ML regression algorithm and sample size on individualized behavioral/cognitive prediction performance have not been comprehensively assessed. To address this issue, the present study included six commonly used ML regression algorithms: ordinary least squares (OLS) regression, least absolute shrinkage and selection operator (LASSO) regression, ridge regression, elastic-net regression, linear support vector regression (LSVR), and relevance vector regression (RVR), to perform specific behavioral/cognitive predictions based on different sample sizes. Specifically, the publicly available resting-state functional MRI (rs-fMRI) dataset from the Human Connectome Project (HCP) was used, and whole-brain resting-state functional connectivity (rsFC) or rsFC strength (rsFCS) were extracted as prediction features. Twenty-five sample sizes (ranged from 20 to 700) were studied by sub-sampling from the entire HCP cohort. The analyses showed that rsFC-based LASSO regression performed remarkably worse than the other algorithms, and rsFCS-based OLS regression performed markedly worse than the other algorithms. Regardless of the algorithm and feature type, both the prediction accuracy and its stability exponentially increased with increasing sample size. The specific patterns of the observed algorithm and sample size effects were well replicated in the prediction using re-testing fMRI data, data processed by different imaging preprocessing schemes, and different behavioral/cognitive scores, thus indicating excellent robustness/generalization of the effects. The current findings provide critical insight into how the selected ML regression algorithm and sample size influence individualized predictions of behavior/cognition and offer important guidance for choosing the ML regression algorithm or sample size in relevant investigations. SN - 1095-9572 UR - https://www.unboundmedicine.com/medline/citation/29870817/The_effect_of_machine_learning_regression_algorithms_and_sample_size_on_individualized_behavioral_prediction_with_functional_connectivity_features_ L2 - https://linkinghub.elsevier.com/retrieve/pii/S1053-8119(18)30508-1 DB - PRIME DP - Unbound Medicine ER -