Effects of SVM parameter optimization on discrimination and calibration for post-procedural PCI mortality

• Published in: Journal of biomedical informatics Vol. 40; no. 6; pp. 688 - 697
• Main Authors: Matheny, Michael E., Resnic, Frederic S., Arora, Nipun, Ohno-Machado, Lucila
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.12.2007
• Subjects: Angioplasty, Balloon, Coronary - mortality; Artificial Intelligence; Boston - epidemiology; Calibration; Discriminant Analysis; Evolutionary computing; Genetic algorithms; Humans; Incidence; Logistic regression; Machine learning; Percutaneous coronary intervention; Prognosis; Proportional Hazards Models; Reproducibility of Results; Risk Assessment - methods; Risk Factors; Sensitivity and Specificity; Support vector machines; Survival Analysis; Survival Rate; Boston; Support vector machines; Logistic regression; Evolutionary computing; Percutaneous coronary intervention; Machine learning; Genetic algorithms
• ISSN: 1532-0464; 1532-0480; 1532-0480
• DOI: 10.1016/j.jbi.2007.05.008

Support vector machines (SVM) have become popular among machine learning researchers, but their applications in biomedicine have been somewhat limited. A number of methods, such as grid search and evolutionary algorithms, have been utilized to optimize model parameters of SVMs. The sensitivity of the results to changes in optimization methods has not been investigated in the context of medical applications. In this study, radial-basis kernel SVM and polynomial kernel SVM mortality prediction models for percutaneous coronary interventions were optimized using (a) mean-squared error, (b) mean cross-entropy error, (c) the area under the receiver operating characteristic, and (d) the Hosmer–Lemeshow goodness-of-fit test (HL χ 2). A threefold cross-validation inner and outer loop method was used to select the best models using the training data, and evaluations were based on previously unseen test data. The results were compared to those produced by logistic regression models optimized using the same indices. The choice of optimization parameters had a significant impact on performance in both SVM kernel types.