In:
Annals of Surgery, Ovid Technologies (Wolters Kluwer Health), Vol. 277, No. 4 ( 2023-04), p. 704-711
Abstract:
To gather validity evidence supporting the use and interpretation of scores from the American College of Surgeons Entering Resident Readiness Assessment (ACS ERRA) Program. Summary and Background Data: ACS ERRA is an online formative assessment program developed to assess entering surgery residents’ ability to make critical clinical decisions, and includes 12 clinical areas and 20 topics identified by a national panel of surgeon educators and residency program directors. Methods: Data from 3 national testing administrations of ACS ERRA (2018–2020) were used to gather validity evidence regarding content, response process, internal structure (reliability), relations to other variables, and consequences. Results: Over the 3 administrations, 1975 surgery residents participated from 125 distinct residency programs. Overall scores [Mean = 64% (SD = 7%)] remained consistent across the 3 years ( P = 0.670). There were no significant differences among resident characteristics (gender, age, international medical graduate status). The mean case discrimination index was 0.54 [SD = 0.15]. Kappa inter-rater reliability for scoring was 0.87; the overall test score reliability (G-coefficient) was 0.86 (Ф-coefficient = 0.83). Residents who completed residency readiness programs had higher ACS ERRA scores (66% versus 63%, Cohen’s d = 0.23, P 〈 0.001). On average, 15% of decisions made (21/140 per test) involved potentially harmful actions. Variability in scores from graduating medical schools (7%) carried over twice as much weight than from matched residency programs (3%). Conclusions: ACS ERRA scores provide valuable information to entering surgery residents and surgery program directors to aid in development of individual and group learning plans.
Type of Medium:
Online Resource
ISSN:
0003-4932
DOI:
10.1097/SLA.0000000000005358
Language:
English
Publisher:
Ovid Technologies (Wolters Kluwer Health)
Publication Date:
2023
detail.hit.zdb_id:
2002200-1
Bookmarklink