Development and application of a novel metric to assess effectiveness of biomedical data

• Published in: BMJ open Vol. 3; no. 8; p. e003220
• Main Authors: Bloom, Gregory C, Eschrich, Steven, Hang, Gang, Schabath, Matthew B, Bhansali, Neera, Hoerter, Andrew M, Morgan, Scott, Fenstermacher, David A
• Format: Journal Article
• Language: English
• Published: England BMJ Publishing Group LTD 23.08.2013; BMJ Publishing Group
• Subjects: American Recovery & Reinvestment Act 2009-US; Cancer therapies; Cost control; Health care; Lung cancer; Medical records; Patient Protection & Affordable Care Act 2010-US; Patients; Quality of life; Research Methods; Oncology; Epidemiology
• ISSN: 2044-6055; 2044-6055
• DOI: 10.1136/bmjopen-2013-003220

Objective Design a metric to assess the comparative effectiveness of biomedical data elements within a study that incorporates their statistical relatedness to a given outcome variable as well as a measurement of the quality of their underlying data. Materials and methods The cohort consisted of 874 patients with adenocarcinoma of the lung, each with 47 clinical data elements. The p value for each element was calculated using the Cox proportional hazard univariable regression model with overall survival as the endpoint. An attribute or A-score was calculated by quantification of an element's four quality attributes; Completeness, Comprehensiveness, Consistency and Overall-cost. An effectiveness or E-score was obtained by calculating the conditional probabilities of the p-value and A-score within the given data set with their product equaling the effectiveness score (E-score). Results The E-score metric provided information about the utility of an element beyond an outcome-related p value ranking. E-scores for elements age-at-diagnosis, gender and tobacco-use showed utility above what their respective p values alone would indicate due to their relative ease of acquisition, that is, higher A-scores. Conversely, elements surgery-site, histologic-type and pathological-TNM stage were down-ranked in comparison to their p values based on lower A-scores caused by significantly higher acquisition costs. Conclusions A novel metric termed E-score was developed which incorporates standard statistics with data quality metrics and was tested on elements from a large lung cohort. Results show that an element's underlying data quality is an important consideration in addition to p value correlation to outcome when determining the element's clinical or research utility in a study.