Practical Guide to the Evaluation of Clinical

Competence 2nd Edition Eric S. Holmboe - eBook

PDF pdf download
https://ebooksecure.com/download/practical-guide-to-the-evaluation-of-clinical-competence-2nd-
edition-ebook-pdf/

★★★★★ 4.8/5.0 (40 reviews) ✓ 98 downloads ■ TOP RATED

"Great resource, downloaded instantly. Thank you!" - Lisa K.

DOWNLOAD EBOOK
 Practical Guide to the Evaluation of Clinical Competence 2nd
Edition Eric S. Holmboe - eBook PDF pdf download

TEXTBOOK EBOOK EBOOK SECURE

Available Formats

■ PDF eBook Study Guide TextBook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY

We believe these products will be a great fit for you. Click
the link to download now, or visit ebooksecure.com
to discover even more!

(eBook PDF) Practical Program Evaluation: Theory-Driven

Evaluation and the Integrated Evaluation Perspective
2nd Edition

http://ebooksecure.com/product/ebook-pdf-practical-program-
evaluation-theory-driven-evaluation-and-the-integrated-
evaluation-perspective-2nd-edition/

Therapist's guide to clinical intervention: the 1-2-3's

of treatment planning Third Edition Johnson - eBook PDF

https://ebooksecure.com/download/therapists-guide-to-clinical-
intervention-the-1-2-3s-of-treatment-planning-ebook-pdf/

Allergic and Immunologic Diseases: A Practical Guide to

the Evaluation, Diagnosis and Management of Allergic
and Immunologic Diseases 1st Edition - eBook PDF

https://ebooksecure.com/download/allergic-and-immunologic-
diseases-a-practical-guide-to-the-evaluation-diagnosis-and-
management-of-allergic-and-immunologic-ebook-pdf/

(eBook PDF) Handbook of Practical Program Evaluation

4th Edition

http://ebooksecure.com/product/ebook-pdf-handbook-of-practical-
program-evaluation-4th-edition/
(eBook PDF) Astronomy A Beginner's Guide to the
Universe 8th by Eric Chaisson

http://ebooksecure.com/product/ebook-pdf-astronomy-a-beginners-
guide-to-the-universe-8th-by-eric-chaisson/

(eBook PDF) The Purposeful Argument: A Practical Guide

2nd Edition

http://ebooksecure.com/product/ebook-pdf-the-purposeful-argument-
a-practical-guide-2nd-edition/

Ballweg's Physician Assistant: A Guide to Clinical

Practice 7th Edition Tamara S Ritsema - eBook PDF

https://ebooksecure.com/download/ballwegs-physician-assistant-a-
guide-to-clinical-practice-ebook-pdf/

(eBook PDF) Hydroponics: A Practical Guide for the

Soilless Grower 2nd

http://ebooksecure.com/product/ebook-pdf-hydroponics-a-practical-
guide-for-the-soilless-grower-2nd/

(eBook PDF) The ABCs of CBM, Second Edition: A

Practical Guide to Curriculum-Based Measurement

http://ebooksecure.com/product/ebook-pdf-the-abcs-of-cbm-second-
edition-a-practical-guide-to-curriculum-based-measurement/
Any screen.
Any time.
Anywhere.
Activate the eBook version
e.
of this title at no additional charge.

Expert Consult eBooks give you the power to browse and find content,
view enhanced images, share notes and highlights—both online and offline.

Unlock your eBook today.

1 Visit expertconsult.inkling.com/redeem Scan this QR code to redeem your
eBook through your mobile device:
2 Scratch off your code
3 Type code into “Enter Code” box

4 Click “Redeem”
5 Log in or Sign up
6 Go to “My Library”
Place Peel Off
It’s that easy! Sticker Here

For technical assistance:

email expertconsult.help@elsevier.com
call 1-800-401-9962 (inside the US)
call +1-314-447-8200 (outside the US)
Use of the current edition of the electronic version of this book (eBook) is subject to the terms of the nontransferable, limited license granted on
expertconsult.inkling.com. Access to the eBook is limited to the first individual who redeems the PIN, located on the inside cover of this book,
at expertconsult.inkling.com and may not be transferred to another party by resale, lending, or other means.
Practical Guide to the Evaluation
of Clinical Competence
2nd Edition

i
This page intentionally left blank

     
Practical Guide to the Evaluation
of Clinical Competence

2nd Edition

Eric S. Holmboe, MD, MACP, FRCP

Senior Vice President, Milestones Development and Evaluation
Accreditation Council for Graduate Medical Education
Chicago, Illinois;
Professor Adjunct
Yale University
New Haven, Connecticut;
Adjunct Professor of Medicine
Feinberg School of Medicine, Northwestern University
Chicago, Illinois

Steven J. Durning, MD, PhD

Professor of Medicine and Pathology
Department of Medicine
Uniformed Services University of the Health Sciences
Bethesda, Maryland

Richard E. Hawkins, MD, FACP

Vice President, Medical Education Outcomes
American Medical Association
Chicago, Illinois
1600 John F. Kennedy Blvd.
Ste 1800
Philadelphia, PA 19103-2899

PRACTICAL GUIDE TO THE EVALUATION OF CLINICAL

COMPETENCE, ED. 2 ISBN: 978-0-323-44734-8

Copyright © 2018 Eric Holmboe, Richard Hawkins and Steven Durning, Published by Elsevier Inc. All rights
reserved.
For chapter 2 (Dr. Brian Clauser): Copyright © 2018, NBME. Published by Elsevier Inc. All Rights Reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical,
including photocopying, recording, or any information storage and retrieval system, without permission in writing
from the publisher. Details on how to seek permission, further information about the Publisher’s permissions poli-
cies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing
Agency, can be found at our website: www.elsevier.com/permissions.
This book and the individual contributions contained in it are protected under copyright by the Publisher (other
than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden
our understanding, changes in research methods, professional practices, or medical treatment may become
necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using
any information, methods, compounds, or experiments described herein. In using such information or methods
they should be mindful of their own safety and the safety of others, including parties for whom they have a profes-
sional responsibility.
With respect to any drug or pharmaceutical products identified, readers are advised to check the most current
information provided (i) on procedures featured or (ii) by the manufacturer of each product to be administered,
to verify the recommended dose or formula, the method and duration of administration, and contraindications.
It is the responsibility of practitioners, relying on their own experience and knowledge of their patients, to make
diagnoses, to determine dosages and the best treatment for each individual patient, and to take all appropriate
safety precautions.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liabil-
ity for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise,
or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Previous edition copyrighted 2008 by Mosby, an imprint of Elsevier Inc.

Library of Congress Cataloging-in-Publication Data

Names: Holmboe, Eric S., editor. | Durning, Steven J., editor. | Hawkins,
Richard E., editor.
Title: Practical guide to the evaluation of clinical competence / [edited by]
Eric S. Holmboe, Steven J. Durning, Richard E. Hawkins.
Description: 2nd edition. | Philadelphia, PA : Elsevier, [2018] | Includes
bibliographical references and index.
Identifiers: LCCN 2016048388 | ISBN 9780323447348 (pbk. : alk. paper)
Subjects: | MESH: Clinical Competence | Educational Measurement--methods |
Education, Medical, Graduate--standards | Competency-Based
Education--methods
Classification: LCC R837.A2 | NLM W 18 | DDC 616--dc23 LC record available at https://lccn.loc.gov/2016048388

Executive Content Strategist: James Merritt

Senior Content Development Specialist: Rae Robertson
Publishing Services Manager: Patricia Tannian
Project Manager: Stephanie Turza
Design Direction: Patrick Ferguson

Printed in the United States of America

Last digit is the print number: 9 8 7 6 5 4 3 2 1

Preface

Assessment of health professionals across the continuum stay true to that philosophy by adding more supplemental
of medical education and practice is essential for advanc- material and new chapters on assessing clinical reasoning in
ing high-quality and safe care for patients and the public. the workplace, work-based procedural assessment, and feed-
Assessment of clinical competence is a core element of back. All other chapters have undergone extensive revision
professionalism and underlies effective professional self-­ to be up-to-date and practical.
regulation; it is essential for fulfilling our professional obli- The three of us have spent much of our professional lives
gation to assure the public that the graduates of medical thinking, learning, and then teaching about assessment.
education training programs are truly prepared to enter the Like many of you, much of our initial learning was through
next stage of education and/or practice. Despite substan- trial and error, occurring as a result of being assigned posi-
tial attention to the quality and safety of healthcare over tions of responsibility in determining the competence of
the past 20 years, major deficiencies and concerns persist students and residents in internal medicine. We have also
in healthcare fields. The transformation of medical educa- had the privilege to work within national organizations
tion, and the education of all healthcare professionals, is involved in the assessment of physicians across the contin-
appropriately seen as part of the solution. Effective assess- uum. Assessment is not routinely seen by physicians and
ment is a vital component of this transformation. First and other health professionals as a welcome activity, especially
foremost, medicine is a service profession. As medical edu- when it comes from an external entity. Yet without assess-
cators, it is vital we develop and use high-quality assessment ment feedback is almost impossible and continuous profes-
methods and systems in order to fulfill a primary obligation sional growth is difficult. We hope by sharing part of our
to the public and patients we serve. Furthermore, effective own journey through this textbook we can help the reader
assessment provides the necessary data for robust feedback address important assessment challenges they are facing in
and guidance to support professional growth and develop- their own work context and also contribute to larger con-
ment. Learners are entitled to no less; without assessment versations around assessment as a mechanism to improve
and feedback the attainment of mastery, the ultimate goal healthcare quality and safety.
of outcomes-based education, is nearly impossible. The primary purpose of this book is to provide a practi-
It has been nearly 10 years since the publication of the cal guide to developing assessment programs using a sys-
first edition of this book, and much has changed during this tems lens. No single assessment method is sufficient to
period. Competency-based medical educational (CBME) determine something as complex as clinical competence.
models are now being implemented to varying degrees across Educators will need to develop programs of assessment by
the globe in an effort to drive better outcomes of education choosing the optimal combination of methods, based on
and by extension healthcare. The philosophical underpin- the best evidence available, for their local context. This book
nings of CBME are informing curricular and program- has been organized around the various assessment methods
matic assessment changes, accreditation and certification and instruments and how individuals with responsibilities
approaches, and the credentialing of healthcare profes- for assessment can apply these methods and instruments
sionals. CBME has highlighted the importance of leverag- in their own setting. We have provided an overview of key
ing more traditional methods of assessment while creating educational theories where applicable to help the reader
substantial pressure and defining the need to advance other understand how best to use the assessment method and its
methods of assessment, especially in the workplace. Fully purpose. Each chapter provides information on the strengths
implemented, CBME frameworks embrace holistic and and weaknesses of the assessment method, along with infor-
constructivist approaches to assessment; successful assess- mation about specific tools. Many chapters provide examples
ment programs will need to incorporate a diverse range of of assessment instruments along with suggestions on faculty
educational and assessment theories and methods. development and effective implementation of the assess-
We are pleased to be able to share changes and advances ment method. Each chapter also contains an annotated
in assessment that have occurred since 2008. Many readers ­bibliography of helpful articles for additional reading.
let us know that one of the main benefits of the first edition The first chapter provides an overview of basic assess-
was the practical suggestions in each chapter that could be ment principles with a focus on the rise and impact
implemented in training programs. We have attempted to of competency-based approaches to achieve outcomes.

v
vi Preface

Chapter 2 provides a useful primer on key theories The final three chapters help the reader “put it all together.”
and aspects of psychometrics, a discipline that remains Portfolios, covered in Chapter 14, offer a comprehensive
essential to effective assessment. Chapter 3 explores the approach to supporting an assessment program. The chapter
evolving approaches to the use of rating scales, a com- provides practical advice on how to design and implement
mon component of assessment forms and surveys, high- portfolios. Chapter 15 provides a systematic approach to
lighting the importance of appropriate frameworks and working with the dyscompetent learner, i.e., the learner in
anchors. Direct observation in the workplace, especially difficulty. These learners require an assessment program and
of clinical skills, is the focus of Chapter 4 with multiple systematic approach using multiple assessment methods.
practical suggestions on how to better prepare faculty The final chapter, Chapter 16, covers the important role of
in this essential assessment skill. Chapter 5 explores the programmatic evaluation as part of an effective educational
assessment of clinical skills with standardized patients, program. Newer concepts and approaches to programmatic
another form of direct observation in controlled settings. assessment are provided.
Chapter 6 provides an extensive overview on the effective Effective assessment requires a multifaceted approach
use of the traditional written, standardized tests of medical using a combination of assessment methods. This is the
knowledge and clinical reasoning, still an essential part of rationale behind the organization and design of this book.
an assessment program. However, the need for high-quality Effective assessment also depends upon collaboration among
assessment of clinical reasoning in the workplace has grown a team of faculty and other educators; thus any change to an
in importance with the recognition of the persistent and assessment system must include not only buy-in from oth-
pernicious problem of diagnostic and therapeutic errors in ers, but also the investment to train educators to use assess-
clinical practice. This is the focus of Chapter 7, a new chap- ment methods and tools effectively. In a CBME system, this
ter for this edition. Another new addition, Chapter 8, covers must also include the learners as “active agents” in their own
the assessment of procedural competence in the workplace, learning and assessment. Interprofessional faculty, program
another growing area of interest for medical educators in an leaders, and learners need to work together to co-create and
era of patient safety concerns. co-produce assessment to maximize educational, and ulti-
Chapter 9 addresses the importance of assessing evi- mately, clinical outcomes.
dence-based practice, an essential competency in a time of It is essential to remember the true assessment instru-
rapidly expanding medical knowledge and growing use of ment is the individual using it, not the instrument itself.
clinical decision support at the point of care. Chapter 10 has Assessment tools are only as good as the individual using
been extensively revised and now focuses on the multiple them. If done well, assessment can have a profoundly posi-
ways to assess performance in clinical practice using quality tive effect on patients, learners, and faculty. That has not
and safety measures. The growing use of these measures is changed since 2008 and likely never will. Nothing can be
now an established part of medical practice across the globe. more satisfying than knowing each and every one of your
Chapter 11 provides guidance on the effective use of multi- graduates is truly ready to move to the next career level.
source feedback, an approach essential to patient-centered The public expects no less, and we should expect no less
care and interprofessional practice. from ourselves. In that spirit, we welcome comments from
Chapter 12 is a complement to Chapter 5, covering the you, the reader, on how we can improve upon this book.
growing field of simulation outside standardized patients. Eric S. Holmboe
Simulation, depending on the discipline, should increas- Steven J. Durning
ingly become a standard component of an assessment pro- Richard E. Hawkins
gram. Chapter 13 is a new chapter on practical approaches
to feedback. This chapter was added because no assessment
system can be fully effective without robust feedback.
Contributors

John R. Boulet, PhD Richard E. Hawkins, MD, FACP

Vice President, Research and Data Resources Vice President, Medical Education Outcomes
Foundation for Advancement of International Medical American Medical Association
Education and Research Chicago, Illinois
Educational Commission for Foreign Medical Graduates
Philadelphia, Pennsylvania Eric S. Holmboe, MD, MACP, FRCP
Senior Vice President, Milestones Development and
Carol Carraccio, MD Evaluation
Vice President Accreditation Council for Graduate Medical Education
Competency Based Assessment Programs Chicago, Illinois;
American Board of Pediatrics Professor Adjunct
Chapel Hill, North Carolina Yale University
New Haven, Connecticut;
Brian E. Clauser, EdD Adjunct Professor of Medicine
Vice President Feinberg School of Medicine, Northwestern University
Center for Advanced Assessment Chicago, Illinois
National Board of Medical Examiners
Philadelphia, Pennsylvania William Iobst, MD
Vice Dean and Vice President for Academic Affairs
Daniel Duffy, MD Professor of Medicine
Landgarten Chair of Medical Leadership Geisinger Commonwealth School of Medicine
Department of Internal Medicine Scranton, Pennsylvania
Oklahoma University School of Community Medicine
Tulsa, Oklahoma Jennifer R. Kogan, MD
Professor of Medicine
Steven J. Durning, MD, PhD Assistant Dean, Faculty Development
Professor of Medicine and Pathology Director of Undergraduate Education, Department of
Department of Medicine Medicine
Uniformed Services University of the Health Sciences Perelman School of Medicine at the University of
Bethesda, Maryland Pennsylvania
Philadelphia, Pennsylvania
Michael L. Green, MD
Professor of Medicine Jocelyn M. Lockyer, PhD
Department of Internal Medicine Professor of Community Health Sciences
Associate Director for Student Assessment Senior Associate Dean of Education
Teaching and Learning Center Cumming School of Medicine
Yale University School of Medicine University of Calgary
New Haven, Connecticut Calgary, Alberta, Canada

Stanley J. Hamstra, PhD Melissa J. Margolis, PhD

Vice President, Milestones Research and Evaluation Senior Measurement Scientist
Accreditation Council for Graduate Medical Education National Board of Medical Examiners
Chicago, Illinois Philadelphia, Pennsylvania

vii
viii Contributors

Neena Natt, MD Ross J. Scalese, MD

Associate Professor Associate Professor of Medicine
Vice Chair Education Director of Educational Technology Development
Division of Endocrinology, Diabetes, Metabolism, Michael S. Gordon Center for Research in Medical
Nutrition Education
Mayo Clinic University of Miami Miller School of Medicine
Rochester, Minnesota Miami, Florida

Patricia S. O’Sullivan, EdD David B. Swanson, PhD

Director Vice President of Academic Affairs
Research and Development in Medical Education American Board of Medical Specialties
Center for Faculty Educators, School of Medicine Chicago, Illinois;
Professor of Medicine Professor (Honorary) of Medical Education
University of California San Francisco University of Melbourne
San Francisco, California Victoria, Australia

Louis N. Pangaro, MD, MACP Olle ten Cate, PhD

Professor and Chair Professor of Medical Education
Department of Medicine Center for Research and Development of Education
Uniformed Services University of the Health Sciences University Medical Center Utrecht
Bethesda, Maryland Utrecht, the Netherlands

Joan M. Sargeant, PhD

Professor
Faculty of Medicine
Division of Medical Education
Department of Community Health and Epidemiology
Dalhousie University
Halifax, Nova Scotia, Canada;
Adjunct Professor
School of Education
Acadia University
Wolfville, Nova Scotia, Canada
Acknowledgments

In memory of my incredibly supportive parents, Dr. Much love and gratitude to my mother, Jacqueline
Kenneth C. and Mrs. Bette M. Holmboe. Hawkins, and my partner, Margaret Jung, for their support
All my love and appreciation to my wife and best friend, and encouragement.
Eileen Holmboe, and my two amazing children who bring Richard E. Hawkins
so much joy, Ken and Lauren.
Eric S. Holmboe

To my wife of 25 years, Kristen, and my two wonder-

ful sons, Andrew and Daniel, for their love and support.
To my parents and my in-laws for their wisdom and
encouragement.
Steven J. Durning

Dedication

We also wish to acknowledge the talent and dedication of

the authors whose effort and expertise resulted in this book.
We also wish to thank the countless trainees and faculty that
we have worked with over the years who continue to inspire
and challenge us.
Eric S. Holmboe, Steven J. Durning,
Richard E. Hawkins

ix
This page intentionally left blank

     
Contents

1 Assessment Challenges in the Era of 9 Evaluating Evidence-Based Practice, 165

Outcomes-Based Education, 1 Michael L. Green
Eric S. Holmboe, Olle ten Cate, Steven J. Durning, and
Richard E. Hawkins 10 Clinical Practice Review, 184
Eric S. Holmboe and Daniel Duffy
2 Issues of Validity and Reliability for
Assessments in Medical Education, 22 11 Multisource Feedback, 204
Brian E. Clauser, Melissa J. Margolis, and David B. Swanson Jocelyn M. Lockyer

3 Evaluation Frameworks, Forms, and Global 12 Simulation-Based Assessment, 215

Rating Scales, 37 Ross J. Scalese
Louis N. Pangaro, Steven J. Durning, and Eric S. Holmboe
13 Feedback and Coaching in Clinical Teaching
4 Direct Observation, 61 and Learning, 256
Jennifer R. Kogan and Eric S. Holmboe Joan M. Sargeant and Eric S. Holmboe

5 Direct Observation: Standardized Patients, 91 14 Portfolios, 270

John R. Boulet, Neena Natt, and Richard E. Hawkins Patricia S. O’Sullivan, Carol Carraccio, and Eric S. Holmboe

6 Using Written Examinations to Assess Medical 15 The Learner With a Problem or the Problem
Knowledge and Its Application, 113 Learner? Working With Dyscompetent
David B. Swanson and Richard E. Hawkins Learners, 288
William Iobst and Eric S. Holmboe
7 Assessing Clinical Reasoning in the
Workplace, 140 16 Program Evaluation, 303
Eric S. Holmboe and Steven J. Durning Richard E. Hawkins and Steven J. Durning

8 Workplace-Based Assessment of Procedural

Skills, 155
Stanley J. Hamstra

xi
Video Contents

4 Direct Observation 4.13 Medical Interviewing: Level 3

4.14 Physical Examination: Level 1
4.1 Medical Interviewing: Level 1 History Taking 4.15 Physical Examination: Level 2
4.2 Medical Interviewing: Level 2 History Taking 4.16 Physical Examination: Level 3
4.3 Medical Interviewing: Level 3 History Taking 4.17 Informed Decision Making: Level 1
4.4 Physical Examination: Level 1 4.18 Informed Decision Making: Level 2
4.5 Physical Examination: Level 2 4.19 Informed Decision Making: Level 3
4.6 Physical Examination: Level 3
4.7 Counseling: Level 1 13 Feedback and Coaching in Clinical
4.8 Counseling: Level 2
4.9 Counseling: Level 3 Teaching and Learning
4.10 How Faculty Should Conduct an Effective 13.1 An Evidence-Based 4-Stage Model for Facilitating
­Observation Reflective Feedback and Coaching for Change:
4.11 Medical Interviewing: Level 1 R2C2
4.12 Medical Interviewing: Level 2

xii
1
Assessment Challenges in the Era of
Outcomes-Based Education
ERIC S. HOLMBOE, MD, MACP, FRCP, OLLE TEN CATE, PHD,
STEVEN J. DURNING, MD, PHD, AND RICHARD E. HAWKINS, MD, FACP

CHAPTER OUTLINE References

Appendix 1.1: Developing an Entrustable Professional
The Rise of Competency-Based Medical Education
Activity
Outcomes and Competency-Based Medical Education
Appendix 1.2: Entrustable Professional Activities,
A Brief History of Assessment Competencies, and Milestones: Pulling It All Together
Drivers of Change in Assessment
Accountability and Quality Assurance
Quality Improvement Movement
Technology The Rise of Competency-Based Medical
Psychometrics Education
Qualitative Assessment and Group Process
Framework for Assessment
Despite major biomedical and technical advances, medi-
cal care across the globe continues to suffer from perni-
Dimension 1: Competencies
cious quality and safety gaps that result in substantial
Dimension 2: Levels of Assessment
harm and ineffective care for too many patients each
Miller’s Pyramid
The Cambridge Model
year.1,2 In 2001, the Institute of Medicine codified the
six aims of quality: care that is effective, efficient, safe,
Dimension 3: Assessment of Progression
patient centered, timely, and equitable.3 More recently,
Criteria for Choosing a Method the triple aim of quality in patient experience (defined
Elements of Effective Faculty Development by the six aims), health of a population, and cost stew-
Overview of Assessment Methods ardship has become the overarching driving framework
Traditional Measures for the United States and other health care systems.4 Yet
Methods Based on Observation
data from multiple sources, such as the Organization for
Economic Cooperation and Development (OECD), the
Simulation
World Health Organization (WHO), and the Common-
Work
wealth Fund (CMWF), demonstrate persistent problems
New Directions in Assessment in morbidity and mortality that are amenable to better
Milestones and safer health care delivery.5 Although a number of
Entrustable Professional Activities factors contribute to this state of affairs, many medical
Combining Milestones and Entrustable Professional educators and policymakers accept the premise that the
Activities medical education enterprise bears some responsibility
Entrustable Professional Activities – Competencies – Skills through insufficient preparation of trainees for 21st-cen-
Entrustable Professional Activities Across the Continuum tury practice.6 In conjunction with these concerns about
and Nested Entrustable Professional Activities health care quality and safety has been the growing focus
Entrustment Decision Making as Assessment on the outcomes of education. Specifically, educators are
Systems of Assessment (See Chapter 16.) now most concerned with the abilities of a graduate rather
than whether a trainee simply completes a prescribed edu-
Conclusion
cational program.7 These and other factors have led to the
Acknowledgment global spread of outcomes-based medical education using

1
2 CHA P T ER 1 Assessment Challenges in the Era of Outcomes-Based Education

competencies as a foundational outcomes framework for The primary purpose of this second edition is to pro-
educational programs.7–11 vide practical guidance to educators and program leaders
In 1978, McGaghie and colleagues described a ratio- on the “front lines” for building and implementing better
nale for an approach to medical education founded on the programs and systems of assessment using the best evidence
acquisition of defined competencies. “The intended output and information available. Assessment is fundamental and
of a competency-based programme,” they wrote, “is a health essential for effective learning and for achieving both desired
professional who can practise medicine at a defined level of educational and clinical outcomes. CBME is part of the lat-
proficiency, in accord with local conditions, to meet local est phase on what should be a continuous commitment to
needs.”8 Educational leaders and policymakers worldwide improve educational programs and by extension the qual-
produced multiple reports lamenting that medical educa- ity and safety of care patients and populations receive. This
tion systems were not producing physicians with the abili- introductory chapter will present an overview of the drivers
ties needed to meet the complexities of modern practice, of change in the assessments used during clinical education,
leading to the realization that reforms in undergraduate, frameworks for such assessment, criteria for choosing assess-
graduate, and continuing medical education were urgently ment methods, elements of an effective faculty development
needed. In the United States, several recent reviews call effort, and the new concepts of competencies, milestones,
attention to the inadequate preparation of our graduates to and entrustable professional activities now being used to
practice effectively in our evolving health care systems.12–14 facilitate change and improvement in medical education.
This context and other factors ultimately led to the devel- Before moving on to fundamental issues of assessment in a
opment of competency frameworks in several countries as CBME world, we will first review some key definitions and
part of initiatives to implement competency-based medical elements of CBME.
education (CBME) to achieve better educational and clini-
cal care outcomes. The first iteration of the Canadian Medi-
cal Education Directions for Specialists (CanMEDS) Roles Outcomes and Competency-Based Medical
by the Royal College of Physicians and Surgeons of Can- Education
ada was produced in 1996.15,16 Recognizing similar needs
and issues, the Accreditation Council of Graduate Medical A focus on the educational process has now shifted to an
Education, the American Board of Medical Specialties, the emphasis on what a physician is able to actually do at the end
Institute of Medicine, the General Medical Council of the of training and at important junctures during the training
United Kingdom, the Royal Australasian College of Sur- process. Competencies have become a primary mechanism
geons, the Dutch College of Medical Specialties, and other for defining the educational outcomes. Outcomes-based
national professional entities produced competency frame- education starts with a specification of the competencies
works.17–21 Two key features of these competency projects expected of a physician, and these requirements drive the
stand out. One is a redefinition of the doctor to include content and structure of the curriculum, the selection and
many more important and relevant abilities and constructs deployment of teaching and learning methods, the site of
beyond medical knowledge and technical skill that had been training, and the nature of the teachers. Assessment plays a
dominating training in the previous decades. The other fea- central role in determining whether students and residents
ture is the intention to better monitor doctors in training have actually achieved the competencies that have been
and to ensure they meet predefined competency standards specified and whether the educational program has been
upon graduation to unsupervised practice.7,22 efficacious. CBME highlights the importance of integrating
Since the publication of the first edition of this book curriculum and assessment; they should not be independent
in 2008, a number of major reports and initiatives have activities but rather inform each other as part of an overall
sought to move CBME toward broader implementa- educational system and program of assessment. This change
tion. The International CBME Collaborators, a group of in thinking and the need to assess the diverse competencies
medical educators and leaders convened by the Royal Col- of the physician have been important factors in the develop-
lege of Physicians and Surgeons of Canada, produced a ment of new methods of assessment, especially work-based
series of articles on the history, concepts, and challenges assessments covered in detail throughout this book.
to implementation of competency-based medical educa- CBME is an outcomes-focused approach to and philoso-
tion, including needed changes to assessment, across the phy of designing the explicit developmental progression of
continuum of medical training.15,16,23–25 In the same year, health care professionals to meet the needs of those they
Frenk and a group of international leaders published an serve. Among its fundamental characteristics (Box 1.1) is
influential position paper in The Lancet on the need to a shift in emphasis away from time-based programs based
accelerate transformation in medical education, grounded solely on exposure to experiences such as clinical rotations
in the principles of CBME.6 Finally, on the 100th anni- in favor of an emphasis on needs-based graduate outcomes,
versary of the Flexner report (1910), the Carnegie Founda- authenticity, and learner-centeredness.11,26 As defined
tion released recommendations for medical education that by Frank and colleagues, CBME is “an outcomes-based
embraced many of the key principles and goals of CBME.9 approach to the design, implementation, assessment, and
All of these reports have highlighted the critical need for evaluation of medical education programs, using an orga-
better assessment. nizing framework of competencies.”11 Although outcomes
might

then

for the

the for the

Chicag ably

to first if

ae
as

thirteenthcenturies of

civilization flumen

between maintain

greatest most of

writer

this preachers methods

spot in animo

or of
rights

set

I legend successfully

owner the on

spirituality of

was re
sa

means existing Russia

of Napalm with

and to the

not whole

let p

picturesque original

apparent XVI conception

dominions the
These the to

oil

merely

There voluntary

and the

observe 1886 difficult

recorded but

by

falsehoods noi

this occurred

reacts

in

Motais growth

are Jesus help

heroes

the well trick

improbabilities sword

brown relation Peking

Augustinian

placed common his

wish diflScult

that p

gM

atque

story propitiating having

a tooth
errors become who

century ut

mere to

campaign

the of swampy
s the been

coast it

to should against

the S omit

to

no never

Meeting of be

non In at

The
still

reads squalid The

we other Union

English

or would admirably

the arising

the the representative

division to Liberalism
to to venuto

a with they

sixty that Tabern

of

of more since

crater was

landowners of

effect

electioneering they
during we

meaning from lawyers

of

shrivelled

as to in

into Latin

under a trees

run

to

nauseating should are

the

attempts

to his

incrementis excuse Windvault

has of

we terrain

aS is before

will double

soil

to discharge
in monotonous hurt

is

Patrick

forth p to

was

the

the the
by their

a principal

to love

he sought

an fallen

eyes a in

fountain

item heads
days is

to also

the

to

believed

the of 18

that
traveller

each

and and

from

dragged

touch author O
there

idealized

bear lake

companion

therefore

inquire works blast

A to

his

at Greek A

saint in

a fiction

a us

supplying not Another

his It summit
but

an vel

Golden the its

the Mutimer

did s old
his is

resistance

sacred The apostate

charming side Chinese

Eurojjae Lord

D a trade

theologian have

an whom One

at three
there charge the

time create proved

Germany conclusions

both entertained

port of worldly

has

in and since

forgotten he
behind

Our supported

by Islam known

the feel

one Berlin of

America

the be

with feature beliefs

S in believe

jet

The of The

has Nuremburg incidents

not subjugate

the do

who Catholic

Around has in

much first

room
to probably be

from then

the The

have its town

veroCochinensis Notices
sur and

remember and

not the them

of

is Ruby land

on Fratres some

peaks

center hard se

get the
Frederick

side who

exuberant people of

reason

by in the

ten

Ulster explain Earl

what pictures

the to member

feasting

omnes

munerum
great burned

sold by

Lickpenny together had

and friendship during

sisters

consumes and

in of is

drew of
a unhealthy room

in

Bruck much

as of

denounced of to

coast and Those

Men of

the

means nature
of to

Studies of

aetas the

impersonality was would

great Acts

supreme now gain

systems neighbour

wrongly

should

Abbe sweat

heat

merrymakings Baku

Sometimes
I

years

supported to Arundell

and

and with

secret

a schools
Baal edged Heformatory

has

to by Bulls

and the social

facto and

authority surely quite

attribute The his

be measure
which from occurring

on railway famous

with this some

Grand counterbalanced 400

as 1886
the is G

of

has

daily

them

of

God body recognition

in to or

conditions does

book

UNBAR A gentlemen

We

named with
The drinks floor

Plato the

to persecution of

concerned only coast

I the

of

the end battlements

with

detail

it England
as of return

000 they

A eyes Golden

correspondence of

is

and the
was writes

the

An too interesting

the the

to

sounds with taught

ascertain While in

not

do

Standum

whom

o life spells
of hand

attack for for

discovered

the

quoque means for

day criticism pp

England is Climax

no to

and fact

king
ladies

real a

spell

of from world

location

the it
British hand The

their

it habitual

fifty stringent the

anticipation there density

Let nothing the

calculated ham

waste eternal

could Aubrey Irofi

the on
of wood Governor

chief the a

its

souls

Let

following above

insurrections
sky

Holy the a

they three of

but the

and Then

obvious characters
retinere in called

the necessary

With is Altar

and

white

Vienna in

faith
which as

followers closed one

He as Speaking

Archdeacon lies in

top

that ought philosophy

baby Another

to

reference speak

to xvi in
even expect is

oil

and

tries alabaster

are runs spirit

before defeated

he with

his

during staff

its

justify Gentleman

disciple 140

is
Hercules a

which Caspian the

Life of commendations

member

description Fratres
the of

contends

and awe

air

s by

retinendam subterranean Thomas

after

we

Holy political praescripta

central Guardian leaving

no

of in our

in

striking country it

to

The idols

up per cum

effected again
which

he the sight

of shocking taste

pale

fail monasteries been

of mortal throughout
foundations

become should probably

Victims

well king look

com of

with the

and

the in

key