For example, the Dannemiller Memorial Education Foundation (DMEF) includes an outcomes study for all of its activities. This
study is directly tied into DMEF's certificate system, and is an e-mail and Internet-based process. Approximately 30-45 days
after a clinician has completed a CME activity, he receives a survey designed to assess the participant's incorporation of
the CME-activity information into his daily practice. As of January 2005, DMEF had sent more than 13,000 surveys, with a return
rate of over 22 percent. An analysis of these surveys indicated:
86 percent of respondents had learned something new from completing the activity
- 71 percent of participants noted that they incorporated the new knowledge into their practices
- 86 percent reported having learned everything they expected to from the activity.
Medscape, a CME provider and leading distributor of online professional education, has developed Medscape CME Metrics. This
is a quantitative evaluation designed to determine the effectiveness of educational material presented online.
The assessment measures retention of course content, adherence to clinical practice patterns, attitudes toward therapeutic
management, and barriers to optimal clinical management. The survey instrument that is utilized consists of a series of case
studies, which are then followed by questions designed to assess the clinician's motivation, confidence, knowledge, and barriers
No More Easy Money
Grantors are taking steps to ensure that effectiveness measurement is an integral part of the CME learning process. Companies
that fund CME endeavors are looking to include language in their requests for proposals and letters of agreement that clearly
delineate the inclusion of effectiveness outcomes. For example, the Novartis Office of Grants and Education, in its grant-request
submission process, states that one of the documents that need to be included for consideration for funding is an outcomes
Medical-education communication companies are also starting to include effectiveness and outcome measures into their programs.
One very basic approach that is being utilized is the check survey for measuring readership of enduring materials. This process
consists of the inclusion of a monetary check (for a token amount that can be donated to a charitable organization) in the
body of a publication; this check is randomly inserted in a subsample of the publications mailed. SynerMed has conducted more
than 40 check surveys (encompassing 95,000 clinicians) assessing a variety of physician/health provider specialties. The median
response rate is approximately 20 percent (with a range of seven percent to 37 percent); this compares with a two-to-three
percent response rate for other direct mail pieces.
Although steps are being taken to improve outcomes measurement in CME activities, a clear definition of what specific measures
need to be assessed—as well as guidance on the means and techniques for achieving those measures—is still lacking.
Much of medicine today is influenced by guidelines. Maybe it is time for those organizations most concerned with the provision
of CME services and programs to take leadership roles in the development of the CME Program Outcome Assessment Guidelines.
The biggest challenge in gauging the effectiveness of CME activities is identifying and developing the means that will allow
for the efficient incorporation of these measurement tools into the educational process. Demonstrating the effectiveness of
one's services is necessary to ensure that the educational needs of healthcare clinicians are met, thereby facilitating their
provision of services to their patients. In the end, it is the total healthcare system that will be the winner.
Robert Schneider is VP of professional relations at SynerMed. He can be reached at firstname.lastname@example.org