Statistical power estimation in non-invasive brain stimulation studies and its clinical implications: An exploratory study of the meta-analyses

• Published in: Asian journal of psychiatry Vol. 44; pp. 29 - 34
• Main Authors: Mitra, Sayantanava, Mehta, Urvakhsh Meherwan, Binukumar, Bhaskarapillai, Venkatasubramanian, Ganesan, Thirthalli, Jagadisha
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01.08.2019
• Subjects: Data Interpretation, Statistical; ECT; Electroconvulsive Therapy - statistics & numerical data; Humans; Mental Disorders - therapy; Meta-Analysis as Topic; Non-Invasive brain stimulation; Outcome Assessment, Health Care - statistics & numerical data; rTMS; Sample size; Statistical power; tDCS; Transcranial Direct Current Stimulation - statistics & numerical data; Transcranial Magnetic Stimulation - statistics & numerical data; Sample size; tDCS; rTMS; Statistical power; Non-Invasive brain stimulation; ECT
• ISSN: 1876-2018; 1876-2026; 1876-2026
• DOI: 10.1016/j.ajp.2019.07.006

•We calculated achieved power of individual studies on Non-invasive brain stimulation interventions in neuro-psychiatric conditions to detect the effect indicated by their corresponding meta-analysis.•We found mean and median powers to be 0.50, with a mode at 0.83; with median reported effect size of 0.61. The mean total sample size in these studies was 22.2 ± 24.9 subjects.•According to our findings, these studies missed around 50% of true positive results.•It appears that most researchers in this field chase statistical significance with small sample sizes, thus compromising the quality of their conclusions. Non-invasive brain stimulation (NIBS) techniques have emerged as a promising tool for understanding and treating psychiatric disorders, necessitating a caution in terms of interpreting research results. This study aimed at systematically evaluating a representative sample of research conducted using NIBS interventions in neuro-psychiatric conditions, and assessing the power these studies achieved, given their sample sizes. A database search was conducted with defined keyword combinations. Using reported summary effects of the meta-analyses as estimate of the true effects, we calculated achieved power of each individual study to detect the effect indicated by the corresponding meta-analysis. Findings suggest that mean and median powers in the field of NIBS were 0.50, with a mode at 0.83 (range 0.05–1.00). When analysed separately, the median powers were 0.27 for tDCS, 0.70 for TMS and 0.97 for ECT. These studies had a mean total sample size of 22.2 ± 24.9 subjects and the median reported effect size across all studies was 0.61. According to our findings, studies conducted in NIBS miss around 50% of true positive results. Further, it appears that most of the researchers in this field chase statistical significance with small sample sizes, thus compromising the quality of their conclusions.