EDITORIAL COMMENTARY Year : 2023  |  Volume : 69  |  Issue : 1  |  Page : 7-8

The McNamara fallacy in medical education: Spot it, stop it

MA Hirkani
Department of Physiology, Seth G. S. Medical College and K. E. M. Hospital, Mumbai, Maharashtra, India

Date of Submission27-Sep-2022Date of Decision14-Oct-2022Date of Acceptance17-Oct-2022Date of Web Publication10-Jan-2023

Correspondence Address:
Dr. M A Hirkani
Department of Physiology, Seth G. S. Medical College and K. E. M. Hospital, Mumbai, Maharashtra
India

Source of Support: None, Conflict of Interest: None

Check

DOI: 10.4103/jpgm.jpgm_765_22

How to cite this article:
Hirkani M A. The McNamara fallacy in medical education: Spot it, stop it. J Postgrad Med 2023;69:7-8

A good program of assessment has the power to assure quality medical training.[1] Do the learners possess the requisite knowledge, skills, attitudes, and values to function effectively as a physician of first contact of the community? Are they “fit for the purpose? What are the gaps in their performance? How can they improve?” These are the questions to be answered to record the attainment of learning outcomes in competency-based medical education.

Effective delivery of health care requires the graduate to be adept in the various domains of competence like communication skills, interpersonal skills, teamwork professionalism, leadership, lifelong learning apart from the core clinical knowledge and skills. In this issue of the journal, Singh and Shah have discussed in depth the fallacies of the prevailing assessment system and ways to escape the trap.[2]

The authors have elucidated that we assess only those competencies that are amenable to measurement resulting in appraising only a limited area of student learning. The fallout of this error is that students assume that aspects of clinical competence that were not assessed are less valued and meant to be ignored.[3] The authors analyzed the historical McNamara fallacy and brought forth the crucial perspective of relying too heavily on metrics and numbers to make decisions.

The authors also reflect on the fact that even though psychometric rigor makes assessment fair, standardized, and comparable, not all domains of competence can be assessed by them. Akin to carrying out mixed method research, the complete picture of the students' learning can be captured using a combination of both quantitative and qualitative assessment methods. Both these methods complement each other and when used together with narratives, will capture elements of performance that cannot be quantified but measure very crucial learning outcomes. They will also provide feedback to the learner for improvement.[4] The resulting alignment of the test items with the competencies will make the assessment more valid.

To redeem ourselves from falling into the trap of “The McNamara Fallacy,” the authors have suggested five very practical and vital points to consider when planning the program of assessment. One suggestion is for medical educationists to build rigor in qualitative assessment methods by use of rubrics and shifting from marks to grades. Unfortunately, grades do not capture progress and competence among learners in the complex tasks and roles required to practice medicine. Narrative assessment, especially in formative and internal assessments, can be used as an alternative to grades and numerical rating scales, and it will also provide rich data for further learning and improvement.[5] Such narratives will also be useful for making entrustment decisions during clerkship.

Assessor training will be the key to carrying out good qualitative assessments as highlighted by the authors. Assessors will also need to be trained in writing high-quality narrative comments describing the performance with a focus on attitudes, knowledge, skills; provide a balanced message between positive elements and elements needing improvement; recommend to the learners ways to improve their performance; compare the observed performance to an expected standard; justify the mark/score given; use language that is clear and easily understood and always be non-judgmental.[6]

The way forward to manage and review the large amount of data generated from qualitative assessments is to use learning analytics that will help predict students' performance, personalize instruction, build adaptive systems, and warn of potential underachievement in real-time.[7]

The challenge of introducing qualitative assessment in India for graduate medical education is the time, effort, and expertise it demands from the teachers and learners. Moreover, strategies to ensure their trustworthiness and authenticity have to be implemented. Further research is needed to explore the experiences of various tools for qualitative assessments on undergraduate medical students' learning. By the interesting parallels drawn between lessons from the McNamara fallacy during the Vietnam war and assessment in medical education, the authors drive home the point that the successful implementation of competency-based medical education hinges on a robust and aligned competency-based assessment.

 

 :: References 
1.Shumway JM, Harden RM, Association for Medical Education in Europe. AMEE Guide No. 25: The assessment of learning outcomes for the competent and reflective physician. Med Teach 2003;25:569-84.  
    2.Singh T, Shah N. Competency-based medical education and the McNamara fallacy: Assessing the important or making the assessed important? J Postgrad Med 2023;69:35-40.  
[PUBMED]  [Full text]  3.Singh T. Student assessment: Moving over to programmatic assessment. Int J Appl Basic Med Res 2016;6:149-50.  
    4.Humphrey-Murto S, Wood TJ, Ross S, Tavares W, Kvern B, Sidhu R, et al. Assessment pearls for competency-based medical education. J Grad Med Educ 2017;9:688-91.  
    5.Hanson JL, Rosenberg AA, Lane JL. Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United States. Front Psychol 2013;4:668.  
    6.Chakroun M, Dion VR, Ouellet K, Graillon A, Désilets V, Xhignesse M, et al. Narrative assessments in higher education: A scoping review to identify evidence-based quality indicators. Acad Med 2022;97:1699-706.  
    7.Saqr M. Learning analytics and medical education. Int J Health Sci 2015;9:V-VI.