Tags

Type your tag names separated by a space and hit enter

Assessment of surgery residents' operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study.
Surgery. 2014 Nov; 156(5):1078-88.S

Abstract

BACKGROUND

With the implementation of competency-based curricula, Objective Structured Assessment of Technical Skills (OSATS) increasingly is being used for the assessment of operative skills. Although evidence for its usefulness has been demonstrated in experimental study designs, data supporting OSATS application in the operating room are limited. This study evaluates the validity and reliability of the OSATS instrument to assess the operative skills of surgery residents in the operating theater.

METHODS

Twenty-four residents were recruited from seven hospitals within a general surgical training region and classified equally into three groups according to postgraduate training year (PGY). Each resident had to perform five different types of operations. Surgical performance was measured using a modified OSATS consisting of three scales: Global Rating Scale, Overall Performance Scale, and Alphabetic Summary Scale. Validity and reliability metrics included construct validity (Kruskal-Wallis test) and internal consistency reliability (Cronbach's α coefficient). Spearman's correlation coefficients were calculated to determine correlations between the different scales.

RESULTS

Eighteen residents (PGY 1-2 [n = 7]; PGY 3-4 [n = 8]; PGY 5-6 [n = 3]) performed 249 operations. Comparisons of the performance scores revealed that evidence for construct validity depended on the difficulty level of the selected procedures. For individual operations, internal consistency reliability of the Global Rating Scale ranged from 0.93 to 0.95. Scores on the different scales correlated strongly (r = 0.62-0.76, P < .001).

CONCLUSION

Assessment of operative skills in the operating theater using this modified OSATS instrument has the potential to establish learning curves, allowing adequate monitoring of residents' progress in achieving operative competence. The Alphabetic Summary Scale seems to be of additional value. Use of the Overall Performance Scale should be reconsidered.

Links

Publisher Full Text (DOI)

Authors+Show Affiliations

Hopmans CJ
Department of Surgery, Erasmus MC University Medical Center, Rotterdam, The Netherlands. Electronic address: c.hopmans@erasmusmc.nl.
den Hoed PT
Department of Surgery, Ikazia Hospital, Rotterdam, The Netherlands.
van der Laan L
Department of Surgery, Amphia Hospital, Breda, The Netherlands.
van der Harst E
Department of Surgery, Maasstad Hospital, Rotterdam, The Netherlands.
van der Elst M
Department of Surgery, Reinier de Graaf Groep, Delft, The Netherlands.
Mannaerts GH
Department of Surgery, Sint Franciscus Gasthuis, Rotterdam, The Netherlands.
Dawson I
Department of Surgery, IJsselland Hospital, Capelle aan den IJssel, The Netherlands.
Timman R
Department of Surgery, Erasmus MC University Medical Center, Rotterdam, The Netherlands.
Wijnhoven BP
Department of Surgery, Erasmus MC University Medical Center, Rotterdam, The Netherlands.
IJzermans JN
Department of Surgery, Erasmus MC University Medical Center, Rotterdam, The Netherlands.

MeSH

Breast NeoplasmsCholecystectomy, LaparoscopicClinical CompetenceEducational MeasurementGeneral SurgeryHerniorrhaphyHip FracturesHumansInternship and ResidencyProspective Studies

Pub Type(s)

Evaluation Study
Journal Article
Multicenter Study
Observational Study

Language

eng

PubMed ID

25231747

Citation

Hopmans, Cornelis J., et al. "Assessment of Surgery Residents' Operative Skills in the Operating Theater Using a Modified Objective Structured Assessment of Technical Skills (OSATS): a Prospective Multicenter Study." Surgery, vol. 156, no. 5, 2014, pp. 1078-88.
Hopmans CJ, den Hoed PT, van der Laan L, et al. Assessment of surgery residents' operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study. Surgery. 2014;156(5):1078-88.
Hopmans, C. J., den Hoed, P. T., van der Laan, L., van der Harst, E., van der Elst, M., Mannaerts, G. H., Dawson, I., Timman, R., Wijnhoven, B. P., & IJzermans, J. N. (2014). Assessment of surgery residents' operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study. Surgery, 156(5), 1078-88. https://doi.org/10.1016/j.surg.2014.04.052
Hopmans CJ, et al. Assessment of Surgery Residents' Operative Skills in the Operating Theater Using a Modified Objective Structured Assessment of Technical Skills (OSATS): a Prospective Multicenter Study. Surgery. 2014;156(5):1078-88. PubMed PMID: 25231747.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - Assessment of surgery residents' operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study. AU - Hopmans,Cornelis J, AU - den Hoed,Pieter T, AU - van der Laan,Lijckle, AU - van der Harst,Erwin, AU - van der Elst,Maarten, AU - Mannaerts,Guido H H, AU - Dawson,Imro, AU - Timman,Reinier, AU - Wijnhoven,Bas P L, AU - IJzermans,Jan N M, Y1 - 2014/09/16/ PY - 2013/12/20/received PY - 2014/04/24/accepted PY - 2014/9/19/entrez PY - 2014/9/19/pubmed PY - 2015/4/14/medline SP - 1078 EP - 88 JF - Surgery JO - Surgery VL - 156 IS - 5 N2 - BACKGROUND: With the implementation of competency-based curricula, Objective Structured Assessment of Technical Skills (OSATS) increasingly is being used for the assessment of operative skills. Although evidence for its usefulness has been demonstrated in experimental study designs, data supporting OSATS application in the operating room are limited. This study evaluates the validity and reliability of the OSATS instrument to assess the operative skills of surgery residents in the operating theater. METHODS: Twenty-four residents were recruited from seven hospitals within a general surgical training region and classified equally into three groups according to postgraduate training year (PGY). Each resident had to perform five different types of operations. Surgical performance was measured using a modified OSATS consisting of three scales: Global Rating Scale, Overall Performance Scale, and Alphabetic Summary Scale. Validity and reliability metrics included construct validity (Kruskal-Wallis test) and internal consistency reliability (Cronbach's α coefficient). Spearman's correlation coefficients were calculated to determine correlations between the different scales. RESULTS: Eighteen residents (PGY 1-2 [n = 7]; PGY 3-4 [n = 8]; PGY 5-6 [n = 3]) performed 249 operations. Comparisons of the performance scores revealed that evidence for construct validity depended on the difficulty level of the selected procedures. For individual operations, internal consistency reliability of the Global Rating Scale ranged from 0.93 to 0.95. Scores on the different scales correlated strongly (r = 0.62-0.76, P < .001). CONCLUSION: Assessment of operative skills in the operating theater using this modified OSATS instrument has the potential to establish learning curves, allowing adequate monitoring of residents' progress in achieving operative competence. The Alphabetic Summary Scale seems to be of additional value. Use of the Overall Performance Scale should be reconsidered. SN - 1532-7361 UR - https://www.unboundmedicine.com/medline/citation/25231747/Assessment_of_surgery_residents'_operative_skills_in_the_operating_theater_using_a_modified_Objective_Structured_Assessment_of_Technical_Skills__OSATS_:_a_prospective_multicenter_study_ DB - PRIME DP - Unbound Medicine ER -