Tags

Type your tag names separated by a space and hit enter

Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment.
Med Educ. 2004 Aug; 38(8):825-31.ME

Abstract

BACKGROUND

No method of standard setting for objective structured clinical examinations (OSCEs) is perfect. Using scores aggregated across stations risks allowing students who are incompetent in some core skills to pass an examination, which may not be acceptable for high stakes assessments.

AIM

To assess the feasibility of using a factor analysis of station scores in a high stakes OSCE to derive measures of underlying competencies.

METHODS

A 12-station OSCE was administered to all 192 students in the penultimate undergraduate year at the University of Aberdeen Medical School. Analysis of the correlation table of station scores was used to exclude stations performing unreliably. Factor analysis of the remaining station scores was carried out to characterise the underlying competencies being assessed. Factor scores were used to derive pass/fail cut-off scores for the examination.

RESULTS

Four stations were identified as having unpredicted variations in station scores. Analysis of the content of these stations allowed the underlying problems with the station designs to be isolated. Factor analysis of the remaining 8 stations revealed 3 main underlying factors, accounting for 53% of the total variance in scores. These were labelled "examination skills", "communication skills" and "history taking skills".

CONCLUSION

Factor analysis is a useful tool for characterising and quantifying the skills that are assessed in an OSCE. Standard setting procedures can be used to calculate cut-off scores for each underlying factor.

Authors+Show Affiliations

University of Aberdeen, Aberdeen, Scotland, UK. alistair.chesser@bartsandthelondon.nhs.ukNo affiliation info availableNo affiliation info availableNo affiliation info availableNo affiliation info available

Pub Type(s)

Journal Article

Language

eng

PubMed ID

15271042

Citation

Chesser, Alistair M S., et al. "Factor Analysis Can Be a Useful Standard Setting Tool in a High Stakes OSCE Assessment." Medical Education, vol. 38, no. 8, 2004, pp. 825-31.
Chesser AM, Laing MR, Miedzybrodzka ZH, et al. Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment. Med Educ. 2004;38(8):825-31.
Chesser, A. M., Laing, M. R., Miedzybrodzka, Z. H., Brittenden, J., & Heys, S. D. (2004). Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment. Medical Education, 38(8), 825-31.
Chesser AM, et al. Factor Analysis Can Be a Useful Standard Setting Tool in a High Stakes OSCE Assessment. Med Educ. 2004;38(8):825-31. PubMed PMID: 15271042.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - Factor analysis can be a useful standard setting tool in a high stakes OSCE assessment. AU - Chesser,Alistair M S, AU - Laing,Malcolm R, AU - Miedzybrodzka,Zosia H, AU - Brittenden,Julie, AU - Heys,Steven D, PY - 2004/7/24/pubmed PY - 2004/9/17/medline PY - 2004/7/24/entrez SP - 825 EP - 31 JF - Medical education JO - Med Educ VL - 38 IS - 8 N2 - BACKGROUND: No method of standard setting for objective structured clinical examinations (OSCEs) is perfect. Using scores aggregated across stations risks allowing students who are incompetent in some core skills to pass an examination, which may not be acceptable for high stakes assessments. AIM: To assess the feasibility of using a factor analysis of station scores in a high stakes OSCE to derive measures of underlying competencies. METHODS: A 12-station OSCE was administered to all 192 students in the penultimate undergraduate year at the University of Aberdeen Medical School. Analysis of the correlation table of station scores was used to exclude stations performing unreliably. Factor analysis of the remaining station scores was carried out to characterise the underlying competencies being assessed. Factor scores were used to derive pass/fail cut-off scores for the examination. RESULTS: Four stations were identified as having unpredicted variations in station scores. Analysis of the content of these stations allowed the underlying problems with the station designs to be isolated. Factor analysis of the remaining 8 stations revealed 3 main underlying factors, accounting for 53% of the total variance in scores. These were labelled "examination skills", "communication skills" and "history taking skills". CONCLUSION: Factor analysis is a useful tool for characterising and quantifying the skills that are assessed in an OSCE. Standard setting procedures can be used to calculate cut-off scores for each underlying factor. SN - 0308-0110 UR - https://www.unboundmedicine.com/medline/citation/15271042/Factor_analysis_can_be_a_useful_standard_setting_tool_in_a_high_stakes_OSCE_assessment_ DB - PRIME DP - Unbound Medicine ER -