A Video-Based Assessment Tool to Measure Intraoperative Laparoscopic Suturing Using a Modified Script Concordance Methodology

• Published in: Journal of Surgical Education Vol. 80; no. 7; pp. 1005 - 1011
• Main Authors: Enani, Ghada, Vassiliou, Melina, Kaneva, Pepa, Watanabe, Yusuke, Munshi, Amani
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01.07.2023; Elsevier BV
• Subjects: assessment; Clinical Competence; clinical reasoning; Educational Measurement; Educational Measurement - methods; Humans; Intraoperative assessment; Laparoscopic suturing; Laparoscopy; Laparoscopy - education; Neurosurgical Procedures; Reproducibility of Results; script concordance test; Surgeons; Surgeons - education; assessment; Patient Care; script concordance test; clinical reasoning; Medical Knowledge; Practice-Based Learning and Improvement; Laparoscopic suturing; Intraoperative assessment
• ISSN: 1931-7204; 1878-7452; 1878-7452
• DOI: 10.1016/j.jsurg.2023.04.015

•Laparoscopic suturing (LS) is a complex skill that is not modeled in current simulation and assessment platforms. We applied the script concordance test (SCT) methodology to assess the cognitive aspects of LS.•We performed an online test to assess the cognitive component of the LS skill using videos of Nissen Fundoplications and Paraesophageal hernia repairs and based on the SCT methodology. Four domains were assessed: needle handling (NH), tissue handling (TH), knot tying techniques (KT) and operative ergonomics (OE).•The test contained 26 questions in the following domains: 8 NH, 2 TH, 11 KT and 5 OE with a test reliability level (Cronbach a) is 0.80.•The mean score was 72.3% ± 9 and 62.5% ± 15 (p = 0.02) for experts and novices, respectively.•This novel LS assessment tool may capture the clinical difference and surgeon preferences in performing LS intraoperatively and help assess intraoperative decision-making skills. Laparoscopic suturing (LS) is a challenging laparoscopic skill to teach. Its complexity and nuances are not modeled or measured in current simulation and assessment platforms.The script concordance test (SCT) is used to assess clinical reasoning.The purpose of this study is to provide evidence for validity of this novel SCT based online assessment for LS skills. We designed a video-based online SCT for LS using a cognitive task analysis and expert panelists.The CTA yielded 4 LS domains: needle handling (NH), tissue handling (TH), knot tying techniques (KT) and operative ergonomics (OE). Five-point scales with anchoring descriptors from -2 to +2 were used. Scoring was based on a modified SCT methodology. The test was administrated to 37 subjects (18 experts and 19 novices). There was no time limit given. A different expert group from the minimal invasive surgery (MIS) panelist were recruited. Experts were defined as surgeons and fellows with LS experience of >25 cases annually. Validity was assessed by comparing SCT scores of experienced and inexperienced surgeons. Cronbach's alpha was used to assess the internal consistency of the test. The survey started off with 47 questions in each of the following domains: 13 NH, 4 TH, 20 KT and 10 OE. Thirty-seven surgeons (18 experts and 19 inexperienced surgeons). Questions that demonstrated a large discrepancy among experts and panelists with a weighted score difference more than 40 were discarded (n = 20). One question was discarded because it received a 100% score from all participants. This yielded 26 remaining questions in the following domains: 8 NH, 2 TH, 11 KT and 5 OE. The test reliability level (Cronbach a) was 0.80. The mean score was 72 ± 9% and 63 ± 15% (p = 0.02) for experts and inexperienced surgeons, respectively. The mean time to complete the test was 21 minutes. This study provides validity evidence for a novel intraoperative LS assessment. The variability of responses between experts and panelists suggests that SCT may capture the clinical differences/surgeon preferences in performing LS intraoperatively.