Researchers developed and validated a method for evaluating a candidate’s suturing skills, including criteria for evaluating their mastery of all important subskills. Cognitive task analysis was used to dissect robotic suturing into its component technical skill domains and subskill descriptions by a team of 5 seasoned surgeons and an educational psychologist. Then, using the Delphi technique, a group of 16 surgical educators from different institutions examined each component of the cognitive task analysis and approved it for inclusion in the final product when the content validity score reached more than or equal to 0.80.
Afterward, 3 blinded reviewers independently scored 8 training videos and 39 vesicourethral anastomosis using EASE (End-to-End Assessment of Suturing Expertise); additionally, 10 vesicourethral anastomosis were scored using RACE (Robotic Anastomosis Competency Evaluation), a previously validated but simplified suturing assessment tool. Intraclass correlation was used to assess inter-rater reliability for normally distributed data, whereas prevalence-adjusted bias-adjusted Kappa was used to evaluate reliability for skewed distributions. Finally, with the help of a generalized linear mixed model, they compared the EASE ratings of experts (with at least ≥100 prior robotic instances) and trainees (with at least <100 cases) on the non-training cases.
Experts on the panel reached a consensus after 2 rounds of the Delphi procedure on a set of domains, a set of subskills, and 57 thorough descriptions of the subskills themselves, all of which had a content validity index of more than or equal to 0.80. The level of agreement between raters was fairly high (intra-class correlation median: 0.69, range: 0.51-0.97; prevalence-adjusted bias-adjusted Kappa: 0.77, 0.62-0.97). Surgeon experience could be differentiated using a variety of EASE subskill scores. In general, there was a 0.635 (P =.003) Spearman’s rho association between the EASE and RACE scores. The cognitive task analysis and Delphi process led to the creation of EASE, whose suturing subskills are able to differentiate between surgeon experience levels while still preserving rater dependability.
Source: auajournals.org/doi/10.1097/UPJ.0000000000000344
