Outcomes Studies have become an increasingly important part of assessing the effectiveness of CME in recent years. But
with increased popularity comes a danger: physician overload. Many states require clinicians to earn as many as 50 CME credits
per year, at one to four credits per program. If providers conduct outcomes measurement studies for each program they offer,
as some do, clinicians could be asked to complete ten or more outcomes surveys in just one year's time.
And clinicians are already feeling inconvenienced. In late 2005, for example, CME LLC surveyed 676 primary care physicians
and 327 psychiatrists to better understand their opinions regarding outcomes studies. Seventy percent of the primary care
physicians said they already felt inconvenienced by outcomes measurement, and more than 50 percent of psychiatrists agreed.
Other findings from the survey:
- There were big differences between primary care physicians (PCPs) and psychiatrists. Roughly two-thirds of psychiatrists
said outcomes studies should be an integral part of CME activities. Two-thirds of PCPs said they shouldn't.
- The numbers also showed more PCPs felt outcomes studies were a burden. Forty-six percent of the 676 primary care physicians
felt they are beneficial, but inconvenient, and 27 percent of the 327 psychiatrists felt the same.
The findings show that, although outcomes measurement studies are a practical tool to ascertain behavior changes, they should
only be used sparingly. Overuse could dilute their effectiveness and if clinicians become less eager to contribute vital information,
the overuse could become a detriment to future medical education. Even when they are used in selective situations, there are
ways to maximize response rates, such as keeping follow-up surveys short and working around the doctor's time. In most cases,
simple pre- and post-test questions can be implemented as viable alternatives to gauge educational outcomes.
Know When the Time Is Right
Good outcomes measurement starts with good needs assessment, so first companies have to know what they're looking for (see
"The CME Cycle"). Not all outcomes studies require the same amount of measurement. If an educational program only seeks to
update knowledge, then using pre- and post-tests are sufficient. But if behavior change needs to be measured, a more comprehensive
approach, that includes a follow-up assessment, is necessary. And when is this the case? When the education seeks to change
behavior in such instances as disease diagnosis, significant new side effects, or new treatment guidelines.
The CME Cycle
But follow-up surveys, which go out 30-90 days after the program's completion, should be used judiciously because you don't
want to overwhelm a clinician. If the goal of the education program is to discover how much knowledge was transferred, then
these surveys are superfluous. There are quicker and easier ways to get the outcomes measurements. Just ask questions that
get an overview of what clinicians have learned.
For example, think about the way a professor might conduct an academic exam. Although a professor asks his students to read
an entire chapter, he will only ask a random sample of questions to assess if his students understood the material. To assess
knowledge transfer in outcomes measurement studies, CME providers use the same method. Clinicians answer a random sample of
questions, and the responses will determine if clinicians understood and retained the information. These questions also identify
residual education gaps that need to be addressed.
The following are some cases when CME providers should only look for a transfer of knowledge:
- When the program seeks to educate clinicians about a new dosage form of an existing drug; since there are no problems with
misdiagnosis, the education does not seek to change behavior.
- When a new FDA-approved indication is released for an existing medication or class of drugs
- When a new treatment option becomes available for an existing disease or class of drugs