A holistic comparative analysis of diagnostic tests for urothelial carcinoma: a study of Cxbladder Detect, UroVysion® FISH, NMP22® and cytology based on imputation of multiple datasets

• Published in: BMC medical research methodology Vol. 15; no. 1; p. 45
• Main Authors: Breen, Vivienne, Kasabov, Nikola, Kamat, Ashish M., Jacobson, Elsie, Suttie, James M., O’Sullivan, Paul J., Kavalieris, Laimonis, Darling, David G.
• Format: Journal Article
• Language: English
• Published: London BioMed Central 12.05.2015; BioMed Central Ltd
• Subjects: Algorithms; Analysis; Carcinoma, Transitional Cell - diagnosis; Carcinoma, Transitional Cell - genetics; Comparative analysis; Cytodiagnosis; Data analysis; Databases, Factual - statistics & numerical data; Diagnostic Tests, Routine - methods; Diagnostic Tests, Routine - standards; Diagnostic Tests, Routine - statistics & numerical data; Fluorescence; Health Sciences; Humans; In Situ Hybridization, Fluorescence; Medicine; Medicine & Public Health; Methods; Neural networks; Rankings; Reproducibility of Results; Research Article; Sensitivity and Specificity; Statistical Theory and Methods; statistics and modelling; Statistics for Life Sciences; Theory of Medicine/Bioethics; Urinary Bladder Neoplasms - diagnosis; Urinary Bladder Neoplasms - genetics; Multiple data integration; Data imputation; Cxbladder detect; FISH; Cancer diagnostic tests ranking; Diagnostic test accuracy; Urothelial carcinoma; Urine cytology; NMP22
• ISSN: 1471-2288; 1471-2288
• DOI: 10.1186/s12874-015-0036-8

Background Comparing the relative utility of diagnostic tests is challenging when available datasets are small, partial or incomplete. The analytical leverage associated with a large sample size can be gained by integrating several small datasets to enable effective and accurate across-dataset comparisons. Accordingly, we propose a methodology for a holistic comparative analysis and ranking of cancer diagnostic tests through dataset integration and imputation of missing values, using urothelial carcinoma (UC) as a case study. Methods Five datasets comprising samples from 939 subjects, including 89 with UC, where up to four diagnostic tests (cytology, NMP22®, UroVysion® Fluorescence In-Situ Hybridization (FISH) and Cxbladder Detect) were integrated into a single dataset containing all measured records and missing values. The tests were firstly ranked using three criteria: sensitivity, specificity and a standard variable (feature) ranking method popularly known as signal-to-noise ratio (SNR) index derived from the mean values for all subjects clinically known to have UC versus healthy subjects. Secondly, step-wise unsupervised and supervised imputation (the latter accounting for the ‘clinical truth’ as determined by cystoscopy) was performed using personalized modelling, k -nearest-neighbour methods, multiple logistic regression and multilayer perceptron neural networks. All imputation models were cross-validated by comparing their post-imputation predictive accuracy for UC with their pre-imputation accuracy. Finally, the post-imputation tests were re-ranked using the same three criteria. Results In both measured and imputed data sets, Cxbladder Detect ranked higher for sensitivity, and urine cytology a higher specificity, when compared with other UC tests. Cxbladder Detect consistently ranked higher than FISH and all other tests when SNR analyses were performed on measured, unsupervised and supervised imputed datasets. Supervised imputation resulted in a smaller cross-validation error. Cxbladder Detect was robust to imputation showing a 2 % difference in its predictive versus clinical accuracy, outperforming FISH, NMP22 and cytology. Conclusion All data analysed, pre- and post-imputation showed that Cxbladder Detect had higher SNR and outperformed all other comparator tests, including FISH. The methodology developed and validated for comparative ranking of the diagnostic tests for detecting UC, may be further applied to other cancer diagnostic datasets across population groups and multiple datasets.