Don't Wear it Out

CME providers say outcomes studies are the best gauge of program effectiveness. But clinicians call surveys a hassle. You can use them better (and less).
Sep 01, 2006
By Pharmaceutical Executive Editors

Outcomes Studies have become an increasingly important part of assessing the effectiveness of CME in recent years. But with increased popularity comes a danger: physician overload. Many states require clinicians to earn as many as 50 CME credits per year, at one to four credits per program. If providers conduct outcomes measurement studies for each program they offer, as some do, clinicians could be asked to complete ten or more outcomes surveys in just one year's time.

And clinicians are already feeling inconvenienced. In late 2005, for example, CME LLC surveyed 676 primary care physicians and 327 psychiatrists to better understand their opinions regarding outcomes studies. Seventy percent of the primary care physicians said they already felt inconvenienced by outcomes measurement, and more than 50 percent of psychiatrists agreed. Other findings from the survey:

  • There were big differences between primary care physicians (PCPs) and psychiatrists. Roughly two-thirds of psychiatrists said outcomes studies should be an integral part of CME activities. Two-thirds of PCPs said they shouldn't.
  • The numbers also showed more PCPs felt outcomes studies were a burden. Forty-six percent of the 676 primary care physicians felt they are beneficial, but inconvenient, and 27 percent of the 327 psychiatrists felt the same.

The findings show that, although outcomes measurement studies are a practical tool to ascertain behavior changes, they should only be used sparingly. Overuse could dilute their effectiveness and if clinicians become less eager to contribute vital information, the overuse could become a detriment to future medical education. Even when they are used in selective situations, there are ways to maximize response rates, such as keeping follow-up surveys short and working around the doctor's time. In most cases, simple pre- and post-test questions can be implemented as viable alternatives to gauge educational outcomes.

Know When the Time Is Right

The CME Cycle
Good outcomes measurement starts with good needs assessment, so first companies have to know what they're looking for (see "The CME Cycle"). Not all outcomes studies require the same amount of measurement. If an educational program only seeks to update knowledge, then using pre- and post-tests are sufficient. But if behavior change needs to be measured, a more comprehensive approach, that includes a follow-up assessment, is necessary. And when is this the case? When the education seeks to change behavior in such instances as disease diagnosis, significant new side effects, or new treatment guidelines.

But follow-up surveys, which go out 30-90 days after the program's completion, should be used judiciously because you don't want to overwhelm a clinician. If the goal of the education program is to discover how much knowledge was transferred, then these surveys are superfluous. There are quicker and easier ways to get the outcomes measurements. Just ask questions that get an overview of what clinicians have learned.

For example, think about the way a professor might conduct an academic exam. Although a professor asks his students to read an entire chapter, he will only ask a random sample of questions to assess if his students understood the material. To assess knowledge transfer in outcomes measurement studies, CME providers use the same method. Clinicians answer a random sample of questions, and the responses will determine if clinicians understood and retained the information. These questions also identify residual education gaps that need to be addressed.

The following are some cases when CME providers should only look for a transfer of knowledge:

  • When the program seeks to educate clinicians about a new dosage form of an existing drug; since there are no problems with misdiagnosis, the education does not seek to change behavior.
  • When a new FDA-approved indication is released for an existing medication or class of drugs
  • When a new treatment option becomes available for an existing disease or class of drugs

lorem ipsum