Tags

Type your tag names separated by a space and hit enter

The one minute mentor: a pilot study assessing medical students' and residents' professional behaviours through recordings of clinical preceptors' immediate feedback.
Educ Health (Abingdon). 2009 May; 22(1):189.EH

Abstract

INTRODUCTION

The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners.

GOAL

We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors' brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach.

METHODS

Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar's clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner's performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process.

RESULTS

Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach alpha around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming.

DISCUSSION

This method of assessing learners' professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed.

Authors+Show Affiliations

Topps D
Northern Ontario School of Medicine, ON, Canada. topps@ucalgary.ca
Evans RJ
No affiliation info available
Thistlethwaite JE
No affiliation info available
Nan Tie R
No affiliation info available
Ellaway RH
No affiliation info available

MeSH

AustraliaComputers, HandheldFeasibility StudiesFeedbackHumansInternship and ResidencyMentorsPilot ProjectsPreceptorshipProfessional CompetenceReproducibility of ResultsStudents, MedicalTape RecordingUser-Computer Interface

Pub Type(s)

Journal Article

Language

eng

PubMed ID

19953438

Citation

Topps, David, et al. "The One Minute Mentor: a Pilot Study Assessing Medical Students' and Residents' Professional Behaviours Through Recordings of Clinical Preceptors' Immediate Feedback." Education for Health (Abingdon, England), vol. 22, no. 1, 2009, p. 189.
Topps D, Evans RJ, Thistlethwaite JE, et al. The one minute mentor: a pilot study assessing medical students' and residents' professional behaviours through recordings of clinical preceptors' immediate feedback. Educ Health (Abingdon). 2009;22(1):189.
Topps, D., Evans, R. J., Thistlethwaite, J. E., Nan Tie, R., & Ellaway, R. H. (2009). The one minute mentor: a pilot study assessing medical students' and residents' professional behaviours through recordings of clinical preceptors' immediate feedback. Education for Health (Abingdon, England), 22(1), 189.
Topps D, et al. The One Minute Mentor: a Pilot Study Assessing Medical Students' and Residents' Professional Behaviours Through Recordings of Clinical Preceptors' Immediate Feedback. Educ Health (Abingdon). 2009;22(1):189. PubMed PMID: 19953438.
* Article titles in AMA citation format should be in sentence-case
TY - JOUR T1 - The one minute mentor: a pilot study assessing medical students' and residents' professional behaviours through recordings of clinical preceptors' immediate feedback. AU - Topps,David, AU - Evans,Rebecca J, AU - Thistlethwaite,Jill E, AU - Nan Tie,Rodney, AU - Ellaway,Rachel H, Y1 - 2009/05/08/ PY - 2009/12/3/entrez PY - 2009/12/3/pubmed PY - 2010/2/24/medline SP - 189 EP - 189 JF - Education for health (Abingdon, England) JO - Educ Health (Abingdon) VL - 22 IS - 1 N2 - INTRODUCTION: The assessment of professional development and behaviour is an important issue in the training of medical students and physicians. Several methods have been developed for doing so. What is still needed is a method that combines assessment of actual behaviour in the workplace with timely feedback to learners. GOAL: We describe the development, piloting and evaluation of a method for assessing professional behaviour using digital audio recordings of clinical supervisors' brief feedback. We evaluate the inter-rater reliability, acceptability and feasibility of this approach. METHODS: Six medical students in Year 5 and three GP registrars (residents) took part in this pilot project. Each had a personal digital assistant (PDA) and approached their clinical supervisors to give approximately one minute of verbal feedback on professionalism-related behaviours they had observed in the registrar's clinical encounters. The comments, both in transcribed text format and audio, were scored by five evaluators for competence (the learner's performance) and confidence (how confident the evaluator was that the comment clearly described an observed behaviour or attribute that was relevant). Students and evaluators were surveyed for feedback on the process. RESULTS: Study evaluators rated 29 comments from supervisors in text and audio format. There was good inter-rater reliability (Cronbach alpha around 0.8) on competence scores. There was good agreement (paired t-test) between scores across supervisors for assessments of comments in both written and audio formats. Students found the method helpful in providing feedback on professionalism. Evaluators liked having a relatively objective approach for judging behaviours and attributes but found scoring audio comments to be time-consuming. DISCUSSION: This method of assessing learners' professional behaviour shows potential for providing both formative and summative assessment in a way that is feasible and acceptable to students and evaluators. Initial data shows good reliability but to be valid, training of clinical supervisors is necessary to help them provide useful comments based on defined behaviours and attributes of students. In addition, the validity of the scoring method remains to be confirmed. SN - 1469-5804 UR - https://www.unboundmedicine.com/medline/citation/19953438/The_one_minute_mentor:_a_pilot_study_assessing_medical_students'_and_residents'_professional_behaviours_through_recordings_of_clinical_preceptors'_immediate_feedback_ DB - PRIME DP - Unbound Medicine ER -