Back to Annual Meeting Posters
Clinical and Surgical Approach to Upper Extremity Pathology: Development of an Outcome-Based Accreditation Model for Orthopaedic Surgical Residents
Jonathan Dattilo, BS1; Johnathan Bernard, MD, MPH1; Uma Srikumaran, MD1; Bashir Zikria, MD1; Dawn M. Laporte, MD2
1Johns Hopkins University, Baltimore, MD 2Johns Hopkins Hospital, Baltimore, MD
Introduction: Motor skills and technical competency are important components of assessing resident training progression; however, assessment methods of motor skills in orthopaedic surgery are underreported. The Objective Structured Assessment of Technical Skills (OSATS) and Global Rating Scale (GRS) have been used to assess surgical competency amongst general surgical residents. While they have been used to assess hand surgical procedures, they have not been validated for open upper extremity approaches in orthopaedic surgery. The purpose of this study is to determine the reliability and validity of three methods of assessing orthopaedic surgical competency for open approaches to the upper extremity (OSATS, GRS, and surgical approach pass/fail assessment), determine differences between training levels, and to compare time to surgical completion.
Methods: Twenty-five residents representing five years of surgical training were tested for their knowledge base and technical skill pertaining to upper extremity surgical approaches. Residents were evaluated independently by two orthopaedic specialists trained in sports and upper extremity surgery in three surgical approaches to the shoulder. Residents were scored using OSATS detailed checklist score, GRS score, and pass/fail assessment. Time to completion was recorded. A comparison between each assessment tool and resident training year was performed.
Results: Performance assessed via both GRS scores and OSATS varied significantly by training year. Subjective pass/fail assessment and time to completion also significantly varied by level of training. Validity between OSATS was confirmed with Cronbach's alpha (0.928) but was not upheld for subjective pass/fail assessment (0.683). Reliability calculated with Spearman's Rho between GRS score and OSATS checklists was significant for deltopectoral (0.904, p<0.05), lateral (0.872, p<0.05), and posterior (0.818, p<0.05) approaches. Poor reliability was demonstrated between OSATS checklists and subjective pass/fail assessment. Inter-rater reliability was strong (>0.8) for OSATS checklists but not for subjective pass/fail assessment.
Conclusions: With the increased focus on competencies at the ACGME level there is a consensus among orthopaedic residency program directors that surgical skills laboratories are crucial for orthopaedic training. In a climate of patient safety and emphasis on reduction of surgical errors, training and evaluation of residents with appropriate competency and motor skill level will continue to draw the attention of accreditation organizations. Evaluating motor skills and technical competency is an important component of assessing resident progression. The present study demonstrates that both GRS scores and OSATS are valid and reliable tools to evaluate upper extremity surgical competency and are superior to subjective assessment.
Back to Annual Meeting Posters