Propensity score balance measures in pharmacoepidemiology: a simulation study

• Published in: Pharmacoepidemiology and drug safety Vol. 23; no. 8; pp. 802 - 811
• Main Authors: Ali, M. Sanni, Groenwold, Rolf H. H., Pestman, Wiebe R., Belitser, Svetlana V., Roes, Kit C. B., Hoes, Arno W., de Boer, Anthonius, Klungel, Olaf H.
• Format: Journal Article
• Language: English
• Published: Chichester Blackwell Publishing Ltd 01.08.2014; Wiley; Wiley Subscription Services, Inc
• Subjects: balance measure; Bias; Biological and medical sciences; Clinical trial. Drug monitoring; Computer Simulation; confounding; Epidemiology; General pharmacology; Humans; Medical sciences; model selection; Models, Statistical; Monte Carlo Method; pharmacoepidemiology; Pharmacoepidemiology - methods; Pharmacology; Pharmacology. Drug treatments; Propensity Score; Sample Size; Simulation; Statistics, Nonparametric; Drug; confounding; Treatment efficiency; Selection; balance measure; model selection; Pharmacovigilance; Confounding factor; Epidemiology; pharmacoepidemiology; Propensity score; Simulation; Models; propensity score
• ISSN: 1053-8569; 1099-1557; 1099-1557
• DOI: 10.1002/pds.3574

ABSTRACT Background Conditional on the propensity score (PS), treated and untreated subjects have similar distribution of observed baseline characteristics when the PS model is appropriately specified. The performance of several PS balance measures in assessing the balance of covariates achieved by a specific PS model and selecting the optimal PS model was evaluated in simulation studies. However, these studies involved only normally distributed covariates. Comparisons in binary or mixed covariate distributions with rare outcomes, typical of pharmacoepidemiologic settings, are scarce. Methods Monte Carlo simulations were performed to examine the performance of different balance measures in terms of selecting an optimal PS model, thus reduction in bias. The balance of covariates between treatment groups was assessed using the absolute standardized difference, the Kolmogorov–Smirnov distance, the Lévy distance, and the overlapping coefficient. Spearman's correlation coefficient (r) between each of these balance measures and bias were calculated. Results In large sample sizes (n ≥ 1000), all balance measures were similarly correlated with bias (r ranging between 0.50 and 0.68) irrespective of the treatment effect's strength and frequency of the outcome. In smaller sample sizes with mixed binary and continuous covariate distributions, these correlations were low for all balance measures (r ranging between 0.11 and 0.43), except for the absolute standardized difference (r = 0.51). Conclusions The absolute standardized difference, which is an easy‐to‐calculate balance measure, displayed consistently better performance across different simulation scenarios. Therefore, it should be the balance measure of choice for measuring and reporting the amount of balance reached, and for selecting the final PS model. Copyright © 2014 John Wiley & Sons, Ltd.