|
 
 |
| BRIEF COMMUNICATION |
|
| Year : 2013 | Volume
: 26
| Issue : 3 | Page : 188-191 |
|
Brief structured observation of medical student hospital visits
J Rush Pierce, Leonard Noronha, N Perryman Collins, Edward Fancovic
Department of Internal Medicine, University of New Mexico School of Medicine, Albuquerque, New Mexico, USA
| Date of Web Publication | 28-Jan-2014 |
Correspondence Address:
J Rush Pierce
Department of Internal Medicine, MSC 10 5550, 1 University of New Mexico, Albuquerque, NM, USA, 87131 0001
USA
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/1357-6283.126003
Introduction: Students' clinical, communication, and professionalism skills are best assessed when faculty directly observe clinical encounters with patients. Prior to 2009, third-year medical students at our institution had one observed clinical encounter by clinic-based faculty during a required internal medicine clerkship. These observations averaged 45 minutes, feedback was not standardized, and student and faculty satisfaction was low. Methods: Two hospital-based faculty members redesigned a shorter, standardized exercise during which a faculty member observed the student making rounds on a hospitalized patient that they were actively following. On a checklist, faculty recorded observations about communication (8 items), physical examination (5 items), and professionalism (4 items). Faculty provided immediate feedback. Results: Faculty's direct observation of medical students prerounding on hospitalized internal medicine patients averaged 27 minutes including the feedback to students. In one year, 67/71 (94%) students completed the exercise; records were available for 66 (99%) of these encounters. Time of observation averaged 13.5 minutes (range 3-26 minutes). Feedback averaged 13.4 minutes (range 8-25 minutes). Faculty provided feedback in the following areas (proportion of students): Communication (66/66, 100%); examination skills (63/66, 95%); and professionalism (65/66, 98%). Forty-three students (64%) completed an anonymous satisfaction survey. Thirty-nine of these (91%) found the exercise useful or very useful (average 5-point Likert score = 4.30) and 38 (88%) found it easy or very easy to schedule (average 5-point Likert score = 4.30). Discussion: Students found this exercise useful and easy to schedule. Faculty consistently provided feedback to students in areas of communication, physical examination, and professionalism. Keywords: Direct observation, mini-CEX, medical student assessment, medical education, work-based assessment instruments
How to cite this article:
Pierce J R, Noronha L, Collins N P, Fancovic E. Brief structured observation of medical student hospital visits. Educ Health 2013;26:188-91 |
| Introduction | |  |
Medical educators have written that clinical, communication, and professionalism skills of medical students are best taught and assessed by faculty directly observing clinical encounters between students and patients. [1],[2] Nonetheless, the medical literature suggests that these direct observations of clinical encounters occur infrequently; in one study direct observations occurred in less than one-third of internal medicine clerkships in the United States. [3] In Taiwan and the United States, a significant proportion of medical school graduates report that they were never observed performing a history or physical examination on a patient by a faculty member. [4],[5] Lack of faculty time is often cited as a barrier to direct observation exercises. [2],[6] Previously reported observations in internal medicine clerkships have involved interviewing, examining and presenting new patients, and averaged 45 minutes to complete and provide feedback to the student. [7]
The objective of this study was to learn if faculty observation during medical student prerounding on their hospital patients would be a useful learning exercise for medical students and to assess the faculty time required.
| Methods | | |
All third-year medical students at our institution, a public medical school in the Southwest US, complete a required eight-week internal medicine rotation during which they rotate through a 435-bed tertiary care academic hospital as well as a government (Veterans Affairs) hospital. Before 2009, third-year medical students at our institution were required to have one observation by a faculty of a clinical encounter during this rotation. This was an unstructured, ungraded formative assessment performed by ambulatory clinic-based faculty on a hospitalized patient, averaging about 45 minutes/observation. The exercise was limited to days the clinic-based faculty members were available to be in the hospital. Feedback to students was not standardized or recorded. Previous student and faculty surveys had found that students found the exercise difficult to schedule and somewhat intimidating, and that faculty satisfaction with the exercise was low because of time constraints of the ambulatory faculty whose clinics were located off-site.
In 2009, two hospital-based faculty members redesigned the exercise to be more standardized and shorter. These faculty developed and tested a 17-item checklist [Figure 1] to make and record specific observations in three categories: Communication (8 items), physical examination (5 items), and professionalism (4 items). In order to standardize the exercise, after a two-month trial period the observing faculty members reviewed each other's written feedback and then observed each other's verbal feedback to students. The restructured observation exercise was fully implemented in 2010. Since then, three hospital-based faculty members have performed all of these observation exercises. | Figure 1: Observation checklist. Faculty made written comments directly on this form and gave a copy to the student immediately after completing feedback
Click here to view |
Students were required to schedule a direct observation exercise during the rotation. Students received weekly email reminders from the clerkship administrator asking them to select a patient on their hospital service that they were actively following and then arrange an observation exercise by a faculty member during the student's usual morning hospital visit to that patient. This usually occurred between 7 and 8 a.m. Hospital-based faculty members were generally available for these observations 7 days a week. Faculty used the 17-item observation checklist to observe one hospital visit per student and then provide standardized feedback to the student. Faculty observers were instructed to provide immediate feedback that was honest, specific, and covered areas of competency as well as suggestions for improvement. Standardized and personalized comments were added in a blank field. Students were given a copy of the checklist with comments. The exercise was formative and not graded. Faculty also timed and recorded the duration of the observation and feedback in minutes. At the end of each eight-week rotation, students were asked to complete a voluntary anonymous 5-item survey about their participation and satisfaction with the exercise, its ease of scheduling, whether they recalled the faculty using a checklist, and if they estimated that the duration of the combined observation and feedback was more than 30 minutes. The survey also invited narrative comments about the exercise.
| Evaluation | | |
The Human Research Protections Office of the University of New Mexico Health Sciences Center approved this study of the restructured exercise during the first full academic year after its implementation (2010-2011). In one academic year (2010-2011), 67 of 71 (94%) third-year medical students at our institution completed the exercise. Records were available for 66 (99%) of these encounters. Time of observation of the encounter averaged 13.5 minutes (range 3-26 minutes). Time of feedback averaged 13.4 minutes (range 8-25 minutes). Faculty provided feedback to students at the following frequencies in the three areas: Communication (66/66 students, 100%); examination skills (63/66, 95%); and professionalism (65/66, 98%). Forty-three of 67 (64%) students completed the voluntary anonymous satisfaction survey. Thirty-nine of these 43 (91%) students found the exercise to be useful or very useful (average 5-point Likert score = 4.30) and 38 (88%) found it easy or very easy to schedule (average 5-point Likert score = 4.30). Forty one of 43 (95%) students recalled the faculty using a checklist and 20 of 43 (47%) estimated that the total duration of the exercise (observation and feedback) was greater than 30 minutes. Review of narrative comments on the checklists and semi-structured interviews with the three faculty members who performed all the observations revealed 12 themes that occurred in at least 10% of feedback sessions with students [Table 1].
| Discussion | | |
This restructured observation exercise proved to be a brief experience that medical students found useful and easy to schedule. Hospital-based faculty members' direct observations of clinical encounters of medical students prerounding on hospitalized internal medicine patients averaged 27 minutes, including feedback to the student. Interestingly, nearly half of the students estimated that the exercise duration was greater than 30 minutes. Feedback was consistently provided in all three planned domains of observation (communication skills, examination skills, and professionalism). The exercise was shorter than most of those described in the literature, but still provided detailed observation and feedback on students' communication, physical examination, and professionalism skills.
A number of oft-discussed themes emerged during the feedback sessions [Table 1]. Identification of these themes may provide insight into educational needs of third-year medical students. Many students commented on the survey that the one-on-one observation with immediate feedback was very helpful in improving some clinical skills while validating areas in which they were competent. We believe that students were more relaxed and more open to formative feedback because the exercise was ungraded and that seeing patients with whom they were already familiar made the exercise less intimidating. Several students indicated on the survey that this was the only feedback they received from faculty members on encounters with real patients during the core clerkships. We also believe that using a checklist resulted in a more standardized observation, made the exercise easier for faculty and helped to make the experience less stressful for the students.
Before 2009, the course administrator had scheduled the exercise. Coordinating the student and faculty schedules was very time-consuming for the administrator and often difficult for both the student and faculty. Requiring the students to schedule their own observation with faculty has proven to be much easier for both faculty and student. A high proportion of our students felt that the exercise was easy to schedule. Faculty observers reported that scheduling the observation via direct email communication with the students was much easier than scheduling the observation through the course coordinator. That hospital-based faculty often saw the students at hospital-based conferences or informally while on the wards also contributed to the ease of scheduling.
A number of tools for direct observation exercises have been published. [2],[3],[7],[8],[9],[10] Like many of the previously described exercises, our exercise is formative and provides for open-ended feedback. Our exercise is different in that it is shorter and utilizes patients with whom the student is already familiar. Our tool [Figure 1] is different than most of those previously described [10] in that it simply records observations and does not use a quantitative scale.
This evaluation of our exercise is subject to several limitations. It is the experience of a single institution. Only three faculty members have conducted these observations here, and though we believe that it would be easy to train additional faculty evaluators, we have not yet done so. Our satisfaction survey has not been validated, and the return rate of 64% is relatively low. Opinions of students who did not participate in the survey may differ from those who did, and different conclusions might emerge due to nonresponse bias. Additionally, educational outcomes were not assessed.
We have continued to require this exercise of students during their internal medicine rotation and the three faculty members have subsequently completed nearly 300 observations. Students and faculty continue to report high degrees of satisfaction with the exercise. The exercise has been an opportunity to compliment students with natural skills and reassure others that efficiency and confidence will improve with deliberate practice. Many students report that they desired a similar observation exercise during other rotations. Several students later established mentoring relationships with supervising faculty.
| Conclusions | | |
Hospital-based faculty members used a checklist to perform a structured observation of medical students during hospital prerounds. Averaging less than half an hour, faculty members completed the observation and consistently provided feedback in areas of communication, physical examination, and professionalism. Students found the exercise highly satisfactory and easy to schedule.
| References | | |
| 1. | Farrell SE. Evaluation of student performance: Clinical and professional performance. Acad Emerg Med 2005;12:302e6-10. 
[PUBMED] |
| 2. | Fromme HB, Karani R, Downing SM. Direct observation in medical education: A review of the literature and evidence for validity. Mt Sinai J Med 2009;76:365-71.
[PUBMED] |
| 3. | Kogan JR, Hauer KE. Brief report: Use of the mini-clinical evaluation exercise in internal medicine core clerkships. J Gen Intern Med 2006;21:501-2.
[PUBMED] |
| 4. | Holmboe ES. Faculty and the observation of trainees' clinical skills: Problems and opportunities. Acad Med 2004;79:16-22.
[PUBMED] |
| 5. | Chen W, Liao SC, Tsai CH, Huang CC, Lin CC, Tsai CH. Clinical skills in final-year medical students: The relationship between self-reported confidence and direct observation by faculty or residents. Ann Acad Med Singapore 2008;37:3-8.
[PUBMED] |
| 6. | Hauer KE, Holmboe ES, Kogan JR. Twelve tips for implementing tools for direct observation of medical trainees' clinical skills during patient encounters. Med Teach 2011;33:27-33.
[PUBMED] |
| 7. | Hauer KE. Enhancing feedback to students using the mini-CEX (Clinical Evaluation Exercise). Acad Med 2000;75:524.
[PUBMED] |
| 8. | Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 2003;78 (10 Suppl):S33-5.
|
| 9. | Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. JAMA 2009;302:1316-26.
[PUBMED] |
| 10. | Pelgrim EA, Kramer AW, Mokkink HG, van den Elsen L, Grol RP, van der Vleuten CP. In-training assessment using direct observation of single-patient encounters: A literature review. Adv Health Sci Educ Theory Pract 2011;16:131-42.
[PUBMED] |
[Figure 1]
[Table 1]
| This article has been cited by | | 1 |
Role of Communication, Professionalism, and Clinical Care Skills of Postgraduate Students on Patients Recall Visits in Dental School—An Observational Study |
|
| Mohan Kumar Pasupuleti,Gautami Subhadra Penmetsa,Meghana Gangolu,Santosh Venkata Ramesh Konathala,Sruthima Naga Venkata Satya | | Journal of Patient Experience. 2020; 7(6): 1563 | | [Pubmed] | [DOI] | | | 2 |
Comparison of Medical Student Communication Skills Measured by Standardized Patients During an OSCE and by Faculty During an In-Hospital Encounter |
|
| Mary Lacy,Leonard Noronha,Yuridia Leyva,John Rush Pierce | | Southern Medical Journal. 2019; 112(2): 70 | | [Pubmed] | [DOI] | | | 3 |
Medical Student H&Ps: Do You Have to Observe Them All At Once? |
|
| David M. Mills,Michele K. Friesinger,Sherron M. Jackson,Patricia G. McBurney | | Southern Medical Journal. 2018; 111(12): 727 | | [Pubmed] | [DOI] | |
|
 |
|
|
|
Search Pubmed for