Tags

Type your tag names separated by a space and hit enter

Item analysis to improve reliability for an internal medicine undergraduate OSCE.
Adv Health Sci Educ Theory Pract. 2005; 10(2):105-13.AH

Abstract

Utilization of objective structured clinical examinations (OSCEs) for final assessment of medical students in Internal Medicine requires a representative sample of OSCE stations. The reliability and generalizability of OSCE scores provides validity evidence for OSCE scores and supports its contribution to the final clinical grade of medical students. The objective of this study was to perform item analysis using OSCE stations as the unit of analysis and evaluate the extent to which OSCE score reliability can be improved using item analysis data. OSCE scores from eight cohorts of fourth-year medical students (n = 435) in a 6-year undergraduate program were analyzed. Generalizability (G) coefficients of OSCE scores were computed for each cohort. Item analysis was performed by considering each OSCE station as an item and computing the corrected item-total correlation. OSCE stations which negatively impacted the reliability were deleted and the G-coefficient was recalculated. The G-coefficients of OSCE scores from the eight cohorts ranged from 0.48 to 0.80 (median 0.62). The median number of OSCE stations that negatively impacted the G-coefficient was 3.5 (out of a median of 25 total stations). When the ''problem stations'' were deleted, the median G-coefficient across eight cohorts increased to 0.62--0.72. In conclusion, item analysis of OSCE stations is useful and should be performed to improve the reliability of total OSCE scores. Problem stations can then be identified and improved.

Authors+Show Affiliations

Auewarakul C
Department of Medicine, Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand. chirayuaue@yahoo.com
Downing SM
No affiliation info available
Praditsuwan R
No affiliation info available
Jaturatamrong U
No affiliation info available

MeSH

Clinical CompetenceEducation, Medical, UndergraduateEducational MeasurementInternal Medicine

Pub Type(s)

Journal Article
Research Support, Non-U.S. Gov't
Validation Study

Language

eng

PubMed ID

16078095

Citation

Auewarakul, Chirayu, et al. "Item Analysis to Improve Reliability for an Internal Medicine Undergraduate OSCE." Advances in Health Sciences Education : Theory and Practice, vol. 10, no. 2, 2005, pp. 105-13.
Auewarakul C, Downing SM, Praditsuwan R, et al. Item analysis to improve reliability for an internal medicine undergraduate OSCE. Adv Health Sci Educ Theory Pract. 2005;10(2):105-13.
Auewarakul, C., Downing, S. M., Praditsuwan, R., & Jaturatamrong, U. (2005). Item analysis to improve reliability for an internal medicine undergraduate OSCE. Advances in Health Sciences Education : Theory and Practice, 10(2), 105-13.
Auewarakul C, et al. Item Analysis to Improve Reliability for an Internal Medicine Undergraduate OSCE. Adv Health Sci Educ Theory Pract. 2005;10(2):105-13. PubMed PMID: 16078095.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - Item analysis to improve reliability for an internal medicine undergraduate OSCE. AU - Auewarakul,Chirayu, AU - Downing,Steven M, AU - Praditsuwan,Rungnirand, AU - Jaturatamrong,Uapong, PY - 2004/01/13/received PY - 2005/02/16/accepted PY - 2005/8/4/pubmed PY - 2005/11/16/medline PY - 2005/8/4/entrez SP - 105 EP - 13 JF - Advances in health sciences education : theory and practice JO - Adv Health Sci Educ Theory Pract VL - 10 IS - 2 N2 - Utilization of objective structured clinical examinations (OSCEs) for final assessment of medical students in Internal Medicine requires a representative sample of OSCE stations. The reliability and generalizability of OSCE scores provides validity evidence for OSCE scores and supports its contribution to the final clinical grade of medical students. The objective of this study was to perform item analysis using OSCE stations as the unit of analysis and evaluate the extent to which OSCE score reliability can be improved using item analysis data. OSCE scores from eight cohorts of fourth-year medical students (n = 435) in a 6-year undergraduate program were analyzed. Generalizability (G) coefficients of OSCE scores were computed for each cohort. Item analysis was performed by considering each OSCE station as an item and computing the corrected item-total correlation. OSCE stations which negatively impacted the reliability were deleted and the G-coefficient was recalculated. The G-coefficients of OSCE scores from the eight cohorts ranged from 0.48 to 0.80 (median 0.62). The median number of OSCE stations that negatively impacted the G-coefficient was 3.5 (out of a median of 25 total stations). When the ''problem stations'' were deleted, the median G-coefficient across eight cohorts increased to 0.62--0.72. In conclusion, item analysis of OSCE stations is useful and should be performed to improve the reliability of total OSCE scores. Problem stations can then be identified and improved. SN - 1382-4996 UR - https://www.unboundmedicine.com/medline/citation/16078095/Item_analysis_to_improve_reliability_for_an_internal_medicine_undergraduate_OSCE_ DB - PRIME DP - Unbound Medicine ER -