Education for Health

: 2019  |  Volume : 32  |  Issue : 3  |  Page : 127--130

Medical students' critical thinking assessment with collaborative concept maps in a blended educational Strategy

Oscarina da Silva Ezequiel1, Sandra Helena Cerrato Tibirica1, Ivana Lucia Damasio Moutinho1, Alessandra Lamas Granero Lucchetti1, Giancarlo Lucchetti1, Suely Grosseman2, Paulo Marcondes-Carvalho-Jr3,  
1 Department of Medical Education, Medical School, Federal University of Juiz de Fora, Juiz de Fora, Brazil
2 Department of Medical Education, Medical School, Federal University of Santa Catarina, Florianópolis, Brazil
3 Department of Medical Education, Medical School, Faculdade de Medicina de Itajubá, Itajubá, Brazil

Correspondence Address:
Oscarina da Silva Ezequiel
Eugênio do Nascimento Avenue - Dom Bosco, Zip Code: 36038-330


Background: Concept maps (CMs) are tools used to represent how new knowledge is integrated into the cognitive structure. In this study, we investigated the role of collaborative CMs in improving medical students' critical thinking and knowledge acquisition. Methods: A pre-post interventional study was conducted. In the 1st week of the clerkship rotation, a group of 10–14 students were asked by a faculty member to make a CM (CM1). After this first exposure (weeks 2/3), students learned the content through online forums. In the final week (week 4), students discussed what they had learned and made a final CM (CM2). Results: A total of 104 students participated in the study, making twenty CM1 and twenty CM2. There was a statistically significant difference between CM1 and CM2 for overall scores, proposition units, and hierarchy units (P < 0.001). Discussion: Collaborative CMs may be useful tools to help teachers better understand their students' critical thinking changes during a blended strategy.

How to cite this article:
Silva Ezequiel Od, Cerrato Tibirica SH, Damasio Moutinho IL, Granero Lucchetti AL, Lucchetti G, Grosseman S, Marcondes-Carvalho-Jr P. Medical students' critical thinking assessment with collaborative concept maps in a blended educational Strategy.Educ Health 2019;32:127-130

How to cite this URL:
Silva Ezequiel Od, Cerrato Tibirica SH, Damasio Moutinho IL, Granero Lucchetti AL, Lucchetti G, Grosseman S, Marcondes-Carvalho-Jr P. Medical students' critical thinking assessment with collaborative concept maps in a blended educational Strategy. Educ Health [serial online] 2019 [cited 2023 Jan 29 ];32:127-130
Available from:

Full Text


In the last undergraduate years, critical thinking and clinical problem-solving, as opposed to memorization of facts and procedural algorithms, are of critical importance in medical education. However, assessing the improvement of medical students' critical thinking and medical knowledge acquisition during a clerkship is no easy task for teachers, particularly when they are working outside the medical school campus.[1],[2] In order to achieve this goal, several assessment strategies were created, including the use of concept maps (CMs).

CMs were first proposed in the 1970s as a tool to represent how new knowledge is integrated into the cognitive structure, and are considered a potential strategy for the development of critical thinking.[3] In hierarchical structuring, the concepts are presented both in terms of progressive differentiation and integrative reconciliation, which can lead to assessment of critical thinking.[3],[4]

The use and impact of CMs for teaching have been increasingly studied over the last few decades. However, on the basis of a recent literature review on CMs in medical education,[2] only 19 studies were published, most of which were nonexperimental, involving individual CMs and conducted in the preclerkship curriculum.[2]

Taking into account gaps in the current scientific literature, the present study investigated the role of collaborative CMs in improving medical students' critical thinking and knowledge acquisition during a primary health-care clerkship.


We designed a pre-post interventional study, including all 5th-year medical students participating in a primary care clerkship rotation (PCCR) in a Brazilian federal university. The research project was approved by the Institutional Review Board of the Federal University of Juiz de Fora, Brazil, and students signed a consent form.

During the primary care rotation, students performed their daily clinical activities (i.e., health promotion, disease prevention, treatment of common health conditions, and rehabilitation) at different “primary health-care units” where they shadowed a staff member (1–3 students/unit). However, one of the biggest challenges in this rotation was to evaluate how these students were acquiring knowledge and critical thinking.

In view of this challenge, a blended learning strategy was implemented. In the 1st week of the rotation, there was a 4-h-tutorial meeting between the faculty and all students in the rotation (10–14 students). This was followed by students presenting clinical cases (the most challenging ones) which had happened in their rotation activities. The group then chose and discussed one clinical case and identified their knowledge gaps using a CM (CM1). Students were oriented to the CM as follows: (1) brainstorm important topics related to the medical case chosen; (2) choose the most important concepts; and (3) integrate all these concepts using cross-links. After this first exposure (weeks 2/3), students learned the content through online interactive forums. In the final week (week 4), students discussed what they had learned and composed a final CM2.

CMs were then analyzed based on the structural scoring method proposed by Novak and Gowin[3] and Mintzes et al.[5] that takes into account the subcategories of proposition units, hierarchy level, and cross-links. The proposition unit is the smallest unit of knowledge, which is expressed in statements about an object or event. The propositions are represented in a hierarchical way in the CMs so that the more general propositions are at the top and the more specific at the bottom. The cross-links are relationships/links between the propositions (long connections between concepts in different domains of the CM), which facilitate the understanding of the diverse relations between the concepts.[6]

A weighted scale was used to score the three subcategories (based on the depth of thought or understanding required): proposition unit (2 points); hierarchy level (5 points); and cross-link (10 points). Invalid concepts or invalid links were scored 0.[3],[7] Scores, then, for each category were added together to produce a total structural score for each map. The minimum score was calculated by the sum of “proposition units” number (multiplied by two), the number of “hierarchies” (multiplied by five), and the number of “cross-links” (multiplied by ten). Higher scores denoted less knowledge gaps.

Two researchers were trained to perform the CM analysis. To calibrate their ratings, they initially analyzed ten CMs individually, followed by comparing their analysis and discussing and clarifying any doubts or divergences between the rates.

Each evaluator analyzed CM1 and CM2 independently, giving a specific score for each category (i.e., proposition units, hierarchy, and cross-links). These specific scores were based on the sum of points given by the two evaluators. After that, an overall score was calculated based on the sum of each category assessed by the two evaluators. These overall and specific scores were then assessed and compared in the initial (moment 1) and final (moment 3) maps, in order to investigate the changes in medical students' critical thinking before and after the intervention (blended learning).

To compare subcategory scores and the overall score differences between CM1 and CM2, we used the nonparametric-paired Wilcoxon's test, with P < 0.05 as statistically significant. We also calculated the intraclass correlations between CM evaluators.


A total of 104 clerkship medical students participating in the PCCR were included in the study. Student participation in the activities was good: 99.6% of the students attended the first and second face-to-face meetings and 96.8% participated in the online discussion forums. During this period, 42 CMs were made. However, two of the CMs had a very inconsistent structure and were removed from the final analysis. Therefore, our final sample consisted of forty CMs, twenty from the first meeting (CM1) and twenty from the second meeting (CM2).

The comparison between the collaborative CMs before and after the online forums showed significant differences for overall score, proposition units, and hierarchy levels. Cross-links were marginally nonsignificant [Table 1]. The intraclass correlation for the overall score was 0.92: for proposition units, 0.99; for hierarchy levels, 0.77; and for cross-links, 0.382.{Table 1}

[Figure 1] and [Figure 2] are examples of the CM1 and CM2, respectively, produced by a group of students.{Figure 1}{Figure 2}


In the present study, we found significant differences comparing pre- and post-CMs for the overall score, the proposition units, and the hierarchy levels. These findings provide further evidence of the value of collaborative CMs in critical thinking assessment in medical education and are consistent with previous studies.

A previous meta-analysis evaluated the use of CMs by students at levels ranging from Grade 4 to postsecondary in several domains (science, psychology, statistics, and nursing, among others).[8] They found that the use of CMs was associated with increased knowledge retention for both individual and cooperative maps. In another study, Ho et al.[9] showed that the use of CM for junior medical students improved their learning of concepts in pathology.

We also found a high intraclass correlation, indicating reliability between raters assessing the same CM, which is in accordance with a past study.[7] Making the assessment of CM reliable is important in the sense that different teachers can evaluate students in similar ways.

Overall, our study shows that CM seems to be an adequate strategy for monitoring changes in students' group conceptual structure and critical thinking after a blended learning intervention, involving face-to-face encounters and computer-mediated instructions. This strategy could be more appropriate in the clerkship, using real cases from their own clinical practice.

Our study has a number of strengths including prospective design, blended educational strategy, evaluation of students during distant clerkship rotations, and high interrater reliability between evaluators in all components, except cross-links. We also demonstrated the value of evaluating collaborative CMs to better understand group critical thinking. However, there are also potential limitations. The sample was small, we did not have a control group, and the reasons for the low differences between the cross-links as well as the low interrater reliability on these components need to be better understood through qualitative assessment.[10]

We conclude that collaborative CMs are useful in understanding students' critical thinking changes in blended learning during PCCRs. However, there is a need for further studies on collaborative CMs using larger samples being conducted in different medical training settings to increase the generalizability of findings and consequently, encourage the use of collaborative CMs as a formative tool in medical education.


The authors would like to acknowledge the faculty development program of the Foundation for Advancement of International Medical Education and Research, FAIMER Regional Institute—Brazil for helping in the development of this study.

Financial support and sponsorship


Conflicts of interest

There are no conflicts of interest.


1West DC, Pomeroy JR, Park JK, Gerstenberger EA, Sandoval J. Critical thinking in graduate medical education: A role for concept mapping assessment? JAMA 2000;284:1105-10.
2Daley BJ, Torre DM. Concept maps in medical education: An analytical literature review. Med Educ 2010;44:440-8.
3Novak JD, Gowin DB. Learning How to Learn. New York, NY, USA: Cambridge University Press; 1984.
4Sánchez-Quevedo MD, Cubero MA, Alaminos M, Vicente Crespo P, Campos A. The concept map. A versatile educational tool for Health sciences. Application in histology. Educ Méd 2006;9:51-8.
5Mintzes JJ, Wandersee JH, Novak JD. Teaching Science for Understanding: A Human Constructivist View. Burlington, MA, USA: Elsevier; 2005.
6Novak JD. Learning, Creating, and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations. New York, NY, USA: Taylor & Francis; 2010.
7West DC, Park JK, Pomeroy JR, Sandoval J. Concept mapping assessment in medical education: A comparison of two scoring systems. Med Educ 2002;36:820-6.
8Nesbit JC, Adesope OO. Learning with concept and knowledge maps: A meta-analysis. Rev Educ Res 2006;76:413-48.
9Ho V, Kumar RK, Velan G. Online testable concept maps: Benefits for learning about the pathogenesis of disease. Med Educ 2014;48:687-97.
10Srinivasan M, McElvany M, Shay JM, Shavelson RJ, West DC. Measuring knowledge structure: Reliability of concept mapping assessment in medical education. Acad Med 2008;83:1196-203.