Outcomes Measurement

The Evolving Landscape of Outcomes Measurement

In the world of continuing medical education (CME), strategies for assessing the impact of educational activities are continually changing, adapting to meet updated requirements and to better capture the engagement of an evolving physician audience. For decades, CME outcomes were measured with simple post-test questions asking for the participant’s opinion of the activity. Later, measurement of knowledge, and then competence, was added via post-test questions and benchmarking data. However, as the structure and format of CME activities has changed, outcomes measurement tactics have adapted alongside.

While pre-test and post-test questions are highly effective at capturing changes in a physician’s knowledge, beliefs, and attitudes, they sometimes fail to address a simple question: Did the activity engage the learner? No matter how high the quality of the information presented, if the learner is not engaged, the activity is of little value. To inform the design and development of future CME activities, it is necessary to determine when physicians are engaged and which activities or topics result in the most engagement.

One simple method to assess the level of engagement is by whether they interacted with the activity. If the learner asked questions or otherwise interacts with the speaker, it demonstrates a very high level of engagement, in which information is absorbed and processed. Another method of evaluating engagement is if the learner downloads supplementary materials, such as slides. This indicates they participated in the activity, found it to be valuable and meaningful for their practice, and saved the information for future reference. Asking questions and saving materials for future reference are both signs of very high learner engagement and provide a positive picture of the value of an activity.

An added value of measuring physician engagement in addition to standard post-tests and evaluations is that measurement of physician engagement does not require additional time or effort on the part of the learner. With nearly every CME activity now including pre-tests, post-tests, and activity evaluations, learners can often become overwhelmed by the sheer number of questions. As noted by Barbara Winkelman of CME LLC, “Physicians are being inundated with questionnaires, case studies, and focus groups. These are things they won’t always have time for.”

As methods for education continue to evolve, strategies to assess the outcomes of a program will continue to evolve as well. While assessment of what a physician learned and how (or if) that knowledge will be applied to practice is important, there are many other pieces of information that can be used to assess the quality of an activity. Physician engagement is essential for an activity to be of value, and engaged physicians are more likely to return for future education.