The Relationship Between Visual Depictions of Rate of Improvement and Quantitative Effect Sizes in Academic Single-Case Experimental Design Studies

• Published in: Journal of behavioral education Vol. 33; no. 3; pp. 667 - 688
• Main Authors: Van Norman, Ethan R., Boorse, Jaclin, Klingbeil, David A.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01.09.2024; Springer; Springer Nature B.V
• Subjects: Behavioral Science and Psychology; Child and School Psychology; Educational research; Educational tests & measurements; Effect Size; Electronic data processing; Fluency; Graphs; Learning and Instruction; Least Squares Statistics; Methods; Oral reading; Original Paper; Outcomes of Education; Psychological research; Psychology; Reading Fluency; Single subject research; United States; Writing; Reading; Math; Visual analysis; Effect sizes
• ISSN: 1053-0819; 1573-3513
• DOI: 10.1007/s10864-022-09500-6

Despite the increased number of quantitative effect sizes developed for single-case experimental designs (SCEDs), visual analysis remains the gold standard for evaluating methodological rigor of SCEDs and determining whether a functional relation between the treatment and the outcome exists. The physical length and range of values plotted on x and y -axes can influence the visual display of data and subsequent interpretations of treatment effects. We explored the extent to which geometric slope (the angle of inclination) of treatment phase data corresponded to three within-case effect sizes (percent exceeding baseline trend, Tau-U, and generalized least squares) in a dataset of published multiple-baseline designs that targeted fluency in academic outcomes (oral reading fluency, written production, and math computation). A secondary purpose of the study was to explore the distributional properties of said effect sizes across a specific class of outcomes. Results suggest that there is very little correspondence between visual depictions of treatment trend and the effect sizes. Noticeable ceiling effects were observed for overlap-based effect sizes and extremely broad ranges were observed for the regression-based effect sizes. The need to standardize graphing procedures for SCEDs using ordinal axes is offered.