MedEd Studio on Clinical Assessment

What’s around the next corner in clinical assessment? And what needs yet to be developed? Dr Eric Holmboe shares a global perspective on clinical assessment: the latest developments and how technology will impact on assessment in the future

MedEd Studio had the privilege to interview Dr Eric Holmboe, chief of evaluation and milestone development in the US Accreditation Council (ACGME). Dr Holmboe is a globally leading scholar in clinical assessment. In this episode of MedEd Studio he shares his thoughts on the current state of the art globally in clinical assessment.

Below you will find article that covers the areas discussed in the video.

Clinical assessment - now and in the future

There has been a push toward more work-based assessment with the move to outcomes-based medical education over the last 25+ years. In the U.S., this has created a tension between the balance of high-stakes exams and workplace-based assessments (WBA). The strength, but limitation, of current WBAs is the heavy reliance on rater observations/questioning and judgment.

New Technology

As we become awash in more clinical data, and other ways to capture “clinical performance,” there will be a shift to use this information more directly (in essence removing part of the faculty “filter”), but will likely be a slow process. For example, artificial intelligence and machine learning will make it easier to pull data from medical records and help “make sense” of the quality of care delivered, and the contribution of the health professional(s) involved in that care. This is happening with the use of clinical quality and patient safety indicators (measures) as an early phase that requires humans to pull and analyze data, but uptake has been very uneven.

While an “old” technology, video recordings of clinical care, especially procedural care, will likely grow. High-powered computing will be able to assess in granular detail movements and interactions during the procedure and correlate findings with patient outcomes. The same can occur in capturing recordings of non-procedural care and using NLP (natural language processing) along with machine learning to assess the quality of encounters. Unfortunately, medical education suffers from substantial inertia and “immunity to change.”

Programmatic Assessment

Entrustment in the workplace is central to this shift of focus where safe and unsupervised practice are central concerns. All of this requires a new approach to assessment (programmatic assessment) where different methods are systematically combined in a system to enhance validity, reliability, and feedback for learners

There is also a new emphasis on assessment for learning where the progressive development of competencies, such as clinical skills, is the core philosophy of the assessment system. Assessment frameworks that incorporate developmental, narrative descriptors, often called milestones, can provide guidance for how a specific competence looks at different stages of development for the learner. [1]

What is yet to explore

There is some important research that show how patient safety is strengthened by a rigorous assessment system has positive impact on patient safety, particularly regarding interprofessional teams, team collaboration and communication.[2]

Professionalism has become a special focus in many curricula around the world and there is evidence indicating a statistically significant relationship between regulatory concerns for malpractice or medical licensing issues later in the career of a health professional and performance at university. Appropriate and rigorous assessment of professionalism is of central importance.[3]

Eric Holmboe, Chief, Research, Milestones Development and Evaluation Officer, ACGME.

Dr Eric Holmboe

Dr. Holmboe is Chief, Research, Milestones Development and Evaluation Officer at the Accreditation Council for Graduate Medical Education (ACGME). He is Adjunct Professor of Medicine at the Yale University School of Medicine and the Uniformed Services University of the Health Sciences. He is Adjunct Professor of Medical Education at the Feinberg School of Medicine at Northwestern University.

He previously served as the Associate Program Director, Yale Primary Care Internal Medicine Residency Program, Director of Student Clinical Assessment, Yale School of Medicine and Assistant Director of the Yale Robert Wood Johnson Clinical Scholars program. Before joining Yale in 2000, he served as Division Chief of General Internal Medicine at the National Naval Medical Center. Dr. Holmboe retired from the US Naval Reserves in 2005.

His research interests include interventions to improve quality of care and methods in the assessment of clinical competence. His professional memberships include the American College of Physicians, where he is a Master of the College, Society of General Internal Medicine and Association of Medical Education in Europe. He is an honorary Fellow of the Royal College of Physicians in London, honorary Fellow of the Royal College of Physicians and Surgeons of Canada, and honorary fellow of the Academy of Medical Educators.

Dr. Holmboe is a graduate of Franklin and Marshall College and the University of Rochester School of Medicine. He completed his residency and chief residency at Yale-New Haven Hospital, and was a Robert Wood Johnson Clinical Scholar at Yale University.

References

Edgar L, McLean S, Hogan SO, Hamstra S, Holmboe ES. (2018). The Milestones Guidebook. Ver. 2020. Accessed September 24 at Competency-Based Training (acgme.org); Edgar L, Roberts S, Holmboe E. Milestones 2.0: A Step Forward. J Grad Med Educ. 2018 Jun;10(3):367-369.
Lagoo, J, Berry, WR. Miller et al. (2019) Multisource Evaluation of Surgeon Behavior Is Associated With Malpractice Claims, Annals of Surgery: July 2019 - Volume 270 - Issue 1 - p 84-90
Cooper WO, Spain DA, Guillamondegui O, et al. Association of Coworker Reports About Unprofessional Behavior by Surgeons With Surgical Complications in Their Patients. JAMA Surg. 2019;154(9):828–834.

Lipner RS, Young A, Chaudhry HJ, Duhigg LM, Papadakis MA. Specialty Certification Status, Performance Ratings, and Disciplinary Actions of Internal Medicine Residents. Acad Med. 2016 Mar;91(3):376-81.
Papadakis MA, Teherani A, Banach MA, Knettler TR, Rattner SL, Stern DT, Veloski JJ, Hodgson CS. (2005). Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med. 2005 Dec 22;353(25):2673-82.