Tags

Type your tag names separated by a space and hit enter

Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial.
BMC Med Educ. 2006 Oct 10; 6:50.BM

Abstract

BACKGROUND

Long-menu questions (LMQs) are viewed as an alternative method for answering open-ended questions (OEQs) in computerized assessment. So far this question type and its influence on examination scores have not been studied sufficiently. However, the increasing use of computerized assessments will also lead to an increasing use of this question type. Using a summative online key feature (KF) examination we evaluated whether LMQs can be compared with OEQs in regard to the level of difficulty, performance and response times. We also evaluated the content for its suitability for LMQs.

METHODS

We randomized 146 fourth year medical students into two groups. For the purpose of this study we created 7 peer-reviewed KF-cases with a total of 25 questions. All questions had the same content in both groups, but nine questions had a different answer type. Group A answered 9 questions with an LM type, group B with an OE type. In addition to the LM answer, group A could give an OE answer if the appropriate answer was not included in the list.

RESULTS

The average number of correct answers for LMQs and OEQs showed no significant difference (p = 0.93). Among all 630 LM answers only one correct term (0.32%) was not included in the list of answers. The response time for LMQs did not significantly differ from that of OEQs (p = 0.65).

CONCLUSION

LMQs and OEQs do not differ significantly. Compared to standard multiple-choice questions (MCQs), the response time for LMQs and OEQs is longer. This is probably due to the fact that they require active problem solving skills and more practice. LMQs correspond more suitable to Short answer questions (SAQ) then to OEQ and should only be used when the answers can be clearly phrased, using only a few, precise synonyms.LMQs can decrease cueing effects and significantly simplify the scoring in computerized assessment.

Links

PMC Free PDF
PMC Free Full Text
FREE Publisher Full Text

Authors+Show Affiliations

Rotthoff T
Department of Endocrinology, Diabetes and Rheumatology University Hospital, Duesseldorf, Germany. rotthoff@med.uni-duesseldorf.de
Baehring T
No affiliation info available
Dicken HD
No affiliation info available
Fahron U
No affiliation info available
Richter B
No affiliation info available
Fischer MR
No affiliation info available
Scherbaum WA
No affiliation info available

MeSH

AdultClinical CompetenceDecision MakingEducation, Medical, UndergraduateEducational MeasurementFemaleGermanyHumansInternal MedicineMaleProblem SolvingProgram EvaluationStudents, MedicalTimeUser-Computer Interface

Pub Type(s)

Comparative Study
Journal Article
Randomized Controlled Trial
Research Support, Non-U.S. Gov't

Language

eng

PubMed ID

17032439

Citation

Rotthoff, Thomas, et al. "Comparison Between Long-Menu and Open-Ended Questions in Computerized Medical Assessments. a Randomized Controlled Trial." BMC Medical Education, vol. 6, 2006, p. 50.
Rotthoff T, Baehring T, Dicken HD, et al. Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial. BMC Med Educ. 2006;6:50.
Rotthoff, T., Baehring, T., Dicken, H. D., Fahron, U., Richter, B., Fischer, M. R., & Scherbaum, W. A. (2006). Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial. BMC Medical Education, 6, 50.
Rotthoff T, et al. Comparison Between Long-Menu and Open-Ended Questions in Computerized Medical Assessments. a Randomized Controlled Trial. BMC Med Educ. 2006 Oct 10;6:50. PubMed PMID: 17032439.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - Comparison between Long-Menu and Open-Ended Questions in computerized medical assessments. A randomized controlled trial. AU - Rotthoff,Thomas, AU - Baehring,Thomas, AU - Dicken,Hans-Dieter, AU - Fahron,Urte, AU - Richter,Bernd, AU - Fischer,Martin R, AU - Scherbaum,Werner A, Y1 - 2006/10/10/ PY - 2006/06/02/received PY - 2006/10/10/accepted PY - 2006/10/13/pubmed PY - 2006/10/28/medline PY - 2006/10/13/entrez SP - 50 EP - 50 JF - BMC medical education JO - BMC Med Educ VL - 6 N2 - BACKGROUND: Long-menu questions (LMQs) are viewed as an alternative method for answering open-ended questions (OEQs) in computerized assessment. So far this question type and its influence on examination scores have not been studied sufficiently. However, the increasing use of computerized assessments will also lead to an increasing use of this question type. Using a summative online key feature (KF) examination we evaluated whether LMQs can be compared with OEQs in regard to the level of difficulty, performance and response times. We also evaluated the content for its suitability for LMQs. METHODS: We randomized 146 fourth year medical students into two groups. For the purpose of this study we created 7 peer-reviewed KF-cases with a total of 25 questions. All questions had the same content in both groups, but nine questions had a different answer type. Group A answered 9 questions with an LM type, group B with an OE type. In addition to the LM answer, group A could give an OE answer if the appropriate answer was not included in the list. RESULTS: The average number of correct answers for LMQs and OEQs showed no significant difference (p = 0.93). Among all 630 LM answers only one correct term (0.32%) was not included in the list of answers. The response time for LMQs did not significantly differ from that of OEQs (p = 0.65). CONCLUSION: LMQs and OEQs do not differ significantly. Compared to standard multiple-choice questions (MCQs), the response time for LMQs and OEQs is longer. This is probably due to the fact that they require active problem solving skills and more practice. LMQs correspond more suitable to Short answer questions (SAQ) then to OEQ and should only be used when the answers can be clearly phrased, using only a few, precise synonyms.LMQs can decrease cueing effects and significantly simplify the scoring in computerized assessment. SN - 1472-6920 UR - https://www.unboundmedicine.com/medline/citation/17032439/Comparison_between_Long_Menu_and_Open_Ended_Questions_in_computerized_medical_assessments__A_randomized_controlled_trial_ L2 - https://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-6-50 DB - PRIME DP - Unbound Medicine ER -