|
 
 |
| BRIEF COMMUNICATION |
|
| Year : 2017 | Volume
: 30
| Issue : 3 | Page : 215-222 |
|
Near peer-assisted learning to improve confidence for medical students' situational judgment tests
Naomi Gostelow1, Germander Soothill2, Seema Vawda3, David Annan4
1 Department of General Medicine, North Middlesex University Hospital; Department of Clinical and Professional Practice, University College London Medical School, London, United Kingdom
2 Department of General Medicine, North Middlesex University Hospital; Department of General Medicine, Royal Free Hospital, London, United Kingdom
3 Department of General Medicine, North Middlesex University Hospital, London; Department of General Medicine, Princess Alexandra Hospital, Harlow, United Kingdom
4 Department of General Medicine, North Middlesex University Hospital, London, United Kingdom
| Date of Web Publication | 18-Apr-2018 |
Correspondence Address:
Naomi Gostelow
Rm 138 UCL Medical School, Rockefeller Building, University Street, London, WC1E 6BT, England
United Kingdom
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/efh.EfH_21_17
Background: The situational judgment test (SJT) was introduced for all graduating United Kingdom medical students in 2013. Students have anxiety over time pressures and heavy weighting of a single examination. Aims: This study aimed to examine formal SJT preparation available, perceptions of a near peer-delivered course, and to measure improvement in students' confidence. Innovation: Foundation doctors ( first 2 years of postgraduate training) produced a “Situational Judgment Test Preparation Course” in November 2015. Methods: Feedback was collected via Likert scores rating teaching, a mock examination, and pre- and post-course confidence along with free-text responses. Delayed feedback was collected via an online survey. Results: Forty-four students completed the feedback. Seventy percent reported <2 h of university SJT preparation. There were significant post-course improvements in familiarity with structure, scoring system, knowledge and content, and overall SJT confidence (P < 0.05). Delayed feedback showed sustained improvement in familiarity with knowledge and content (P < 0.05). Qualitative analysis revealed themes of improved confidence, approachable tutors, and identifying question strategies. Discussion: Students perceived a lack of formal SJT preparation which was reflected in low pre-course confidence. Improvements in confidence may reflect a unique insight into how to approach the examination from those having recently undertaken it. Keywords: Education, educational measurement/methods, feedback, medical, medical, peer group, program development, program evaluation, psychological, questionnaires, students, surveys, teaching/methods, undergraduate/methods
How to cite this article:
Gostelow N, Soothill G, Vawda S, Annan D. Near peer-assisted learning to improve confidence for medical students' situational judgment tests. Educ Health 2017;30:215-22 |
| Background | |  |
The situational judgment test (SJT) was introduced for all exiting medical graduates in the United Kingdom in 2013.[1] The examination replaced written questions, which were felt to be more open to coaching [2] and contributes 50% to the candidates' overall score for application to postgraduate training, the remainder derived from academic performance.
SJTs measure nonacademic attributes such as professionalism,[1] are good predictors of future work performance, relatively resistant to coaching, and cost- and time-efficient.[3] Students have anxiety over time pressures during the examination and its heavy weighting on their application.[4] Benefits of near peer-assisted learning (NPAL) have been attributed to tutors' recent experience and understanding students' anxiety. Furthermore, there is a social benefit of interacting with those who have recently overcome similar challenges.[5] It seems likely that graduates with first-hand experience are well placed to help students prepare for this examination.
This brief communication evaluated a near peer teaching innovation aimed at the final year students preparing for the SJT aiming to identify the level of formal training available, students' perceptions of the course, and any improvement in confidence.
Innovation
The “Situational Judgment Test Preparation Course” was designed and developed by four recent graduates. Teaching experience ranged from informal ward-based sessions to postgraduate qualifications. The course consisted of lecture-style presentations and a mock examination. The course was free to attend and of 3 h duration, held in November 2015. Students were invited via E-mail up to a capacity of 50. Attendees were from two London medical schools.
| Methods | | |
Pre- and post-course questionnaires collected non-identifiable demographics, prior SJT preparation, 5-point Likert scale evaluations of the course, presenters, and pre- and post-course confidence [Appendix 1]. Students were also asked to volunteer their academic score (educational performance measure)[1] contributing 50% of their application to postgraduate training. Questionnaires included free-text answers. Delayed feedback using the same questionnaire was collected via an online survey in March 2016 after SJT result release [Appendix 2].
Data analysis
An inductive thematic analysis was performed by two independent researchers using a hybrid model of emergent and template coding.[6] Statistical data were presented where appropriate as percentage values of student responses and median values of Likert scores with interquartile range (IQR). The Wilcoxon matched pairs test was used for comparison of paired pre- and postcourse data. The Mann–Whitney U-test was used for comparison of nonpaired pre- and post-course data with delayed feedback. Statistical analyses were performed using the “StatPlus” package (AnalystSoft Inc, Walnut, California United States). P < 0.05 was used to define statistical significance.
Ethical considerations
As an evaluation of a teaching initiative, NHS Health Research Authority for England deemed the study exempt from ethical approval. Questionnaires indicated responses would be used for evaluation and research.
| Results | | |
Prior SJT preparation
Forty-six students completed pre- and postcourse feedback. Nearly 74% (n = 31) estimated <2 h of university SJT preparation time and 26% (n = 12) attended other preparation courses, nine of whom paid a fee.
The mean disclosed Educational Performance Measure was 42.1 out of 50 (n = 36). The 2015 average values for the two attendee's medical schools were 40.0 and 38.5.[7]
Post-course feedback
Post-course surveys evaluating main sessions entitled “Introduction to the SJT” and “Content of the SJT” were fully completed by 44 students. Results are shown in [Table 1]. | Table 1: Postcourse feedback presented as median Likert scores and interquartile range
Click here to view |
Almost 100% of participants (n = 44) agreed or strongly agreed that practising questions under examination conditions was useful. Nearly 95% (n = 42) agreed or strongly agreed that the course was improved by the mock examination.
Qualitative feedback
Students identified improved confidence and approachable tutors as strong points. The course was a “good opportunity to ask questions” and “going through questions together” improved their confidence. The teaching style, handouts, and strategies for approaching questions were also useful.
In terms of improvement, students showed preferences for small group teaching and requested “more scenarios” and “more (practice) questions.” Comments regarding pace requested that more time be spent on explanations. Themes and quotes are shown in [Table 2]. | Table 2: Themes identified by qualitative analysis when asked “what were the strong point of the course?” and “what could have been improved about the course?”
Click here to view |
Pre- and post-course confidence scoring
Students were asked to rate their familiarity and confidence for the SJT before and after the course. [Figure 1] shows median Likert-scale scores pre-course (Pr) and post-course (Po). Results showed increased overall confidence for the upcoming examination (Pr = 3, IQR: 2–3; Po = 4, IQR: 3–4). Additionally, there was increase in familiarity with knowledge and content (Pr = 3, IQR: 3–4; Po = 5, IQR: 4–5), scoring system (Pr = 4, IQR: 4–4; Po = 5, IQR: 5–5), and structure (Pr = 4, IQR: 4–4; Po = 5, IQR: 4–5). Comparison of pre- and post-course scores was statistically significant (P < 0.05). | Figure 1: Comparison of Pre- (Pr), Post- (Po) and Delayed (De) responses. Comparison of Pr and Po responses showed a statistical improvement in all domains (P < 0.05). Comparison of Po and De for knowledge and content showed no significant reduction (P = 0.15) and also showed a significant improvement in comparison to Pre (P < 0.05) denoted by the asterix. Familiarity with scoring system and structure both showed a significant reduction comparing Po and De, P = 0.40 and 0.39 respectively). Overall confidence showed improvement retrospectively
Click here to view |
Delayed evaluation
Almost 48% (n = 21) completed the delayed survey (De) in March 2016. There was a small but significant reduction in familiarity with scoring system (Po = 5, IQR: 5-5; De = 4, IQR: 4–4.25; P < 0.05) and structure (Po = 5, IQR: 4–5; De = 4, IQR: 4–4.25; P < 0.05). Furthermore, pre-course and delayed responses showed no significant difference, hence no sustained improvement (scoring system: P = 0.40, structure: P = 0.39). However, responses relating to content and knowledge showed no statistical reduction when comparing post-course to delayed feedback (Po = 5, IQR: 4–5; De = 4, IQR: 4–5; P = 0.15) and sustained improvement compared to pre-course scores (P ≤ 0.05). Overall confidence (Po = 4, IQR: 3–4; De = 4, IQR: 4–4.25) was not analyzed for significance, as the questions were not directly comparable.
Examination results
Students could volunteer their final SJT result within the delayed survey. Mean score was 39.3 out of 50, n = 19 (national mean = 38.5).[7]
| Discussion | | |
As far as the authors are aware, this is the first publication of a near peer-led SJT preparation innovation. Students perceived that they had little formal preparation which is reflected in low pre-course confidence.
The course was well received with content and presentation scoring highly. Although the SJT is resistant to coaching,[3] an introduction to content and structure may improve confidence and reduce student anxiety in particular by practising under timed conditions, an aspect shown to contribute to student stress.[4] This was supported by qualitative data where students valued discussing questions and felt the course “improved (their) answering skills.” Post-course confidence scores were significantly improved in all areas assessed. Similar effects have been shown in near peer-delivered mock examinations, most commonly objective structured clinical examinations. Although not correlated with examination performance, studies show that near peer-led revision courses positively impact on learning.[8]
Statistical improvement in confidence was not maintained in delayed feedback for familiarity with structure and scoring system. Rashid et al.[8] suggested that feedback could be positively biased by immediate collection but this could also reflect the availability of official information concerning structure and scoring elsewhere. Familiarity with knowledge and content remained statistically improved from pre-course questioning and may indicate a unique insight into approaching the examination offered by those having recently undertaken it. This supports the educational theory behind NPAL which offers the combination of recent examination with on-the-job experience with benefits over both peer-assisted and faculty-led teaching.[9] Further benefits, supported by our data, include approachable tutors and cognitive congruence: breaking down explanations at appropriate levels.[5] This study supports the use of NPAL at all stages of training, including preparation for postgraduate examinations, where the interaction between postgraduate doctors at different stages may also be educationally beneficial. Delayed evaluation allows students to reflect on course content having used it within the examination but commonly has low response rates [10] which were observed in this study.
The course also benefits junior doctor tutors who require teaching experience and increasingly face SJT-based recruitment methods within the United Kingdom specialty training.
Limitations
This was a small-scale study, and although the group's average SJT score was higher than national averages, we cannot comment if this is generalizable or a direct result of the course. These data were collected from 19 self-selected attendees who may naturally perform better than average supported by a higher than average educational performance measure. The authors acknowledge that they may have worked or studied with some attendees which could have led to positive bias in the results, as described by Rashid et al.[8]
Pre- and post-confidence scores cannot be correlated to examination performance and are not surrogate outcomes for ability. However, as recent graduates, we believe that this forum for practise and discussion could improve student experience alleviating on-the-day examination nerves and the educational rationale behind NPAL supports its use in this context. Authors were a self-selected group interested in medical education, and teaching experience was not a prerequisite. If introduced on a wider scale, institutions should consider tutor training to ensure standardization of sessions.
| Conclusions | | |
Final-year students perceived a lack of formal SJT preparation. A NPAL-delivered course was well received with statistical improvements in confidence showing that such a course is not only feasible, but could also offer a unique insight into how to approach the SJT. Further in-depth studies are needed to ascertain whether formal preparation correlates with positive student experience and reduced anxiety.
Acknowledgment
We would like to thank the Academic Centre at North Middlesex University Hospital for allowing use of their facilities during the implementation of this innovation. We would also like to thank Wesleyan for their support and providing refreshments at our event.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | |
| 2. | |
| 3. | Patterson F, Ashworth V, Zibarras L, Coan P, Kerrin M, O'Neill P, et al. Evaluations of situational judgement tests to assess non-academic attributes in selection. Med Educ 2012;46:850-68.  |
| 4. | Sharma N. Medical students' perceptions of the situational judgement test: A mixed methods study. Br J Hosp Med (Lond) 2015;76:234-8. [ PUBMED] |
| 5. | Lockspeiser TM, O'Sullivan P, Teherani A, Muller J. Understanding the experience of being taught by peers: The value of social and cognitive congruence. Adv Health Sci Educ Theory Pract 2008;13:361-72. |
| 6. | Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006;3:77-101. |
| 7. | |
| 8. | Rashid MS, Sobowale O, Gore D. A near-peer teaching program designed, developed and delivered exclusively by recent medical graduates for final year medical students sitting the final objective structured clinical examination (OSCE). BMC Med Educ 2011;11:11. |
| 9. | Rodrigues J, Sengupta A, Mitchell A, Kane C, Kane C, Maxwell S, et al. The Southeast Scotland foundation doctor teaching programme – Is “near-peer” teaching feasible, efficacious and sustainable on a regional scale? Med Teach 2009;31:e51-7. |
| 10. | Gibson C. Mindless evaluations by students and teachers. Med Educ 2015;49:856-8. |
[Figure 1]
[Table 1], [Table 2]
| This article has been cited by | | 1 |
The CASPER preparation program innovation: increasing self-perceived competence and confidence of underrepresented applicants on the novel CASPER Snapshot and CanMEDS roles |
|
| Farhan Mahmood, Julianah O. Oguntala, Claudine Henoud, Libny Lahelle Pierre-Louis, Asli Fuad, Ike Okafor | | BMC Medical Education. 2023; 23(1) | | [Pubmed] | [DOI] | | | 2 |
Características e impactos del aprendizaje entre pares en estudios universitarios en ciencias de la salud: una revisión sistemática |
|
| J. Hernández Coliñir,L. Molina Gallardo,D. González Morales,C. Ibáñez Sanhueza,O. Jerez Yañez | | Revista Clínica Española. 2021; | | [Pubmed] | [DOI] | | | 3 |
Characteristics and impacts of peer assisted learning in university studies in health science: A systematic review |
|
| J. Hernández Coliñir,L. Molina Gallardo,D. González Morales,C. Ibáñez Sanhueza,O. Jerez Yañez | | Revista Clínica Española (English Edition). 2021; | | [Pubmed] | [DOI] | |
|
 |
|
|
|
Search Pubmed for