1104: Teaching Trainees How to Think: A Quality Improvement Project to Increase Recognition of Cognitive Errors in Rheumatology Fellowship

Megan Milne¹, Rebecca Sadun², Lena Eder³, Sonali Bracken⁴, Poorva Apte², Catherine Sims², Nathaniel Harris⁴, Amanda Eudy⁵ and david leverenz⁶, ¹Duke University Medical Center Division of Rheumatology and Immunology, Durham, NC, ²Duke University, Durham, NC, ³Duke University Hospital, Durham, NC, ⁴Duke University Medical Center, Durham, NC, ⁵Duke University, Raleigh, NC, ⁶Duke University School of Medicine, Durham, NC

Background/Purpose: Faulty synthesis of information due to Cognitive Errors (CEs) is the most common cause of incorrect medical decision making. Medical education often focuses on improving trainee knowledge and data gathering but fails to improve trainee information synthesis. To address this gap in medical training, a quality improvement (QI) curriculum was implemented in a rheumatology fellowship training program on CE recognition and mitigation strategies. The project assessed whether formal training in CEs would improve trainee recognition of bias and patient care.

Methods: Six rheumatology fellows created and participated in this project. Fellows completed surveys weekly from 11/2021 - 5/2022. Survey questions assessed time spent considering/discussing CEs over the preceding week, types of CEs considered, and frequency of negative or positive impact of metacognition (not at all, very little, somewhat, or a great deal). Examples were required if "a great deal" of positive impact was selected.

Data were collected and analyzed at baseline (11/6/21 - 11/26/21) and over three Plan-Do-Study-Act (PDSA) cycles as follows: PDSA1 (11/27/21 - 1/21/22), in which fellows received a lecture on CEs and metacognition; PDSA2 (1/22/22 - 2/18/22), in which fellows reflected on initial survey data and created a case-based teaching session about CE for fellows at other institutions; and PDSA3 (2/19/22 - 5/6/22), in which fellows practiced identifying infrequently reported biases in real-life cases as a group, learned de-biasing strategies, and created their own mnemonics to remember common types of CE.

Our primary aim was to increase the percentage of responses indicating that consideration of CE positively impacted patient care "somewhat" or "a great deal" to greater than 50%. We used Fisher's exact test to compare outcomes from before versus after achieving our aim.

Results: We received 80 of 137 (58.4%) potential weekly responses and met our primary aim during PDSA2 (76.5%) and PDSA3 (72.2%), increased from baseline (36.4%) and PDSA1 (42.1%) (Figure 1). Fellows were more likely to report that CE metacognition positively impacted patient care and were more likely to perform CE metacognition on more than half of weekdays during PDSA cycles with active engagement (PDSA2/3) compared with passive engagement (baseline/PDSA 1) (p = 0.004 and p – 0.017 respectively). There was a significant increase in fellow recognition of cognitive errors due to personal attributes (44% versus 13%, p = 0.006) (Table 2). Fellows reported numerous examples of positive patient care impacts resulting from the consideration of CEs (Table 3).

Conclusion: A fellow-driven curriculum increased fellow recognition of CEs and identification of positive patient outcomes resulting from use of debiasing strategies. This curriculum achieved the highest level of Miller's pyramid, as evidenced by fellows' examples of how their actions improved patient outcomes. The most benefit was seen during periods of active fellow engagement versus passive learning. This is the first study in rheumatology to demonstrate a positive impact in patient care from a curriculum on CE.
Figure 1. Survey Results Demonstrating Fellow-Reported Active CE Metacognition Across All PDSA Cycles

Table 1. Fellow-Reported Cognitive Error Metacognition and Perceived Negative Impacts in Baseline and PDSA1 time periods versus PDSA2 and PDSA3 time periods.

Table 2. Fellow-Reported Instances of Cognitive Error Discussions Impacted Patient Care “A Great Deal.”
Disclosures: M. Milne, None; R. Sadun, None; L. Eder, None; S. Bracken, None; P. Apte, None; C. Sims, None; N. Harris, None; A. Eudy, GlaxoSmithKlein(GSK), Pfizer, Exagen; d. leverenz, Pfizer, Rheumatology Research Fund.