Tags

Type your tag names separated by a space and hit enter

Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education.
BMC Med Educ. 2005 Mar 07; 5(1):9.BM

Abstract

BACKGROUND

As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge.

METHODS

We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options.

RESULTS

Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified.

CONCLUSION

Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards.

Links

PMC Free PDF

Authors+Show Affiliations

Bhakta B
Academic Unit of Musculoskeletal and Rehabilitation Medicine, University of Leeds, UK. B.Bhakta@leeds.ac.uk
Tennant A
No affiliation info available
Horton M
No affiliation info available
Lawton G
No affiliation info available
Andrich D
No affiliation info available

MeSH

AdultAlgorithmsCalibrationClinical CompetenceData Interpretation, StatisticalEducation, MedicalEducation, Medical, UndergraduateEducational MeasurementHumansMedicineModels, StatisticalProbabilityPsychometricsSchools, MedicalSpecializationSpecialties, SurgicalWestern Australia

Pub Type(s)

Journal Article
Research Support, Non-U.S. Gov't
Validation Study

Language

eng

PubMed ID

15752421

Citation

Bhakta, Bipin, et al. "Using Item Response Theory to Explore the Psychometric Properties of Extended Matching Questions Examination in Undergraduate Medical Education." BMC Medical Education, vol. 5, no. 1, 2005, p. 9.
Bhakta B, Tennant A, Horton M, et al. Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education. BMC Med Educ. 2005;5(1):9.
Bhakta, B., Tennant, A., Horton, M., Lawton, G., & Andrich, D. (2005). Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education. BMC Medical Education, 5(1), 9.
Bhakta B, et al. Using Item Response Theory to Explore the Psychometric Properties of Extended Matching Questions Examination in Undergraduate Medical Education. BMC Med Educ. 2005 Mar 7;5(1):9. PubMed PMID: 15752421.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education. AU - Bhakta,Bipin, AU - Tennant,Alan, AU - Horton,Mike, AU - Lawton,Gemma, AU - Andrich,David, Y1 - 2005/03/07/ PY - 2004/03/16/received PY - 2005/03/07/accepted PY - 2005/3/9/pubmed PY - 2005/12/13/medline PY - 2005/3/9/entrez SP - 9 EP - 9 JF - BMC medical education JO - BMC Med Educ VL - 5 IS - 1 N2 - BACKGROUND: As assessment has been shown to direct learning, it is critical that the examinations developed to test clinical competence in medical undergraduates are valid and reliable. The use of extended matching questions (EMQ) has been advocated to overcome some of the criticisms of using multiple-choice questions to test factual and applied knowledge. METHODS: We analysed the results from the Extended Matching Questions Examination taken by 4th year undergraduate medical students in the academic year 2001 to 2002. Rasch analysis was used to examine whether the set of questions used in the examination mapped on to a unidimensional scale, the degree of difficulty of questions within and between the various medical and surgical specialties and the pattern of responses within individual questions to assess the impact of the distractor options. RESULTS: Analysis of a subset of items and of the full examination demonstrated internal construct validity and the absence of bias on the majority of questions. Three main patterns of response selection were identified. CONCLUSION: Modern psychometric methods based upon the work of Rasch provide a useful approach to the calibration and analysis of EMQ undergraduate medical assessments. The approach allows for a formal test of the unidimensionality of the questions and thus the validity of the summed score. Given the metric calibration which follows fit to the model, it also allows for the establishment of items banks to facilitate continuity and equity in exam standards. SN - 1472-6920 UR - https://www.unboundmedicine.com/medline/citation/15752421/Using_item_response_theory_to_explore_the_psychometric_properties_of_extended_matching_questions_examination_in_undergraduate_medical_education_ DB - PRIME DP - Unbound Medicine ER -