The following is a summary of “Assessing operative competence in core surgical training: A reliability analysis,” published in the NOVEMBER 2023 issue of Surgery by Toale, et al.
For a study, researchers sought to address the imperative task of quantifying the requisite number of observations essential for a dependable assessment of operative competence among Core Surgical Trainees (CSTs) in Ireland. The assessment tool employed is the Supervised Structured Assessment of Operative Performance (SSAOP).
Examining SSAOPs conducted from April 2016 to February 2021, encompassing an array of undifferentiated procedures and three frequently performed general surgery procedures (appendicectomy, abdominal wall hernia repair, and skin/subcutaneous lesion excision) by CSTs. Employing Generalizability and Decision studies, the research aimed to ascertain the minimum number of observations necessary to achieve dependability indices equal to or exceeding 0.8, a critical threshold for high-stakes assessments.
Analysis of 2,294 SSAOPs revealed that employing four assessors, each observing 10 cases, can generate sufficiently reliable scores for application in high-stakes assessments. The study recommended a focused approach on a subset of core procedures to enhance reliability indices.
The study concluded by emphasizing the importance of repeated assessments concentrating on smaller procedures to bolster reliability. Additionally, it suggested diversifying the assessor cohort within programs to attain the requisite dependability indices suitable for high-stakes assessment scenarios.
Source: americanjournalofsurgery.com/article/S0002-9610(23)00273-8/fulltext
