Evaluation and benchmarking of hybrid machine learning models for autism spectrum disorder diagnosis using a 2-tuple linguistic neutrosophic fuzzy sets-based decision-making model

• Published in: Neural computing & applications Vol. 36; no. 29; pp. 18161 - 18200
• Main Authors: Alqaysi, M. E., Albahri, A. S., Hamid, Rula A.
• Format: Journal Article
• Language: English
• Published: London Springer London 01.10.2024; Springer Nature B.V
• Subjects: Algorithms; Artificial Intelligence; Autism; Benchmarks; Classification; Computational Biology/Bioinformatics; Computational Science and Engineering; Computer Science; Data Mining and Knowledge Discovery; Decision making; Decision trees; Demographics; Diagnosis; Diagnostic tests; Feature selection; Fuzzy sets; Image Processing and Computer Vision; Linguistics; Machine learning; Medical tests; Methodology; Multiple criterion; Original Article; Performance evaluation; Performance measurement; Probability and Statistics in Computer Science; Recall; Regression analysis; Regression models; Sensitivity analysis; Sociodemographics; Testing time; Time; FWZIC; FDOSM; Feature selection; Machine learning; 2-Tuple linguistic neutrosophic fuzzy sets; Diagnosis; Autistic patients
• ISSN: 0941-0643; 1433-3058
• DOI: 10.1007/s00521-024-09905-6

Autism spectrum disorder (ASD) presents challenges for accurate diagnosis, prompting researchers to search for an optimal diagnostic process. Feature selection (FS) approaches and classification methods considering medical tests and socio-demographic characteristics are crucial for diagnosing autism. However, evaluating and benchmarking hybrid diagnosis machine learning (ML) models in the presence of multiple evaluation performance metrics, criteria trade-offs, and varying criteria importance present complex multi-criteria decision-making (MCDM) problems. This study proposes a three-phase methodology integrating FS, ML, and fuzzy MCDM to develop and evaluate diagnosis models. Firstly, an ASD dataset combining medical tests and socio-demographic characteristics is identified and preprocessed. Secondly, 72 hybrid diagnosis models are developed by combining eight FS techniques and nine ML algorithms using an intersection process. Thirdly, the following steps are performed: (i) A decision matrix is formulated based on nine evaluation metrics, including classification accuracy (CA), specificity, precision, F1 score, recall, test time, train time, log loss, and area under the curve (AUC); (ii) a new extension of fuzzy-weighted zero inconsistency is developed using 2-tuple linguistic neutrosophic fuzzy sets (2TLNFSs) to assign weights to the evaluation metrics criteria and address related issues; (iii) a new extension of the fuzzy decision-by-opinion score method is developed using 2TLNFSs as well to benchmark the 72 models. Results indicate that the selected FS techniques vary in the number of features chosen, with the sets ranging from 19 to 46 out of the 48 available features. Socio-demographic features were predominantly selected over medical tests. Regarding the evaluation and benchmarking results, the weights constructed by three experts suggest that CA holds high importance, precision and recall are assigned equal weights, and AUC and test time carry moderate weights. At the same time, F1 and log loss are considered less crucial in the decision-making process. Specificity and train time are assigned relatively lower weights, indicating their lower importance. The best-performing hybrid model identified was sequential feature selection/logistic regression (SFS/LR)-decision tree, with a score value of 4.3964. Decision trees and gradient boosting consistently achieved high rankings, demonstrating their effectiveness in diagnosing ASD, while SVM, random forest, and logistic regression showed mixed results across different hybrid models. The sensitivity analysis assessments were conducted to verify the efficiency of the proposed evaluation and benchmarking methodology. We benchmarked the proposed framework against three other benchmark studies and achieved a score of 100% across five key areas. The developed methodology can potentially advance and accelerate the selection of diagnostic tools for ASD therapy, benefiting individuals with ASD.