Medical Education: Gauging Success

Pharma is investing in CME. Is the money being well spent?
Oct 01, 2006


Robert Schneider
With all the money being spent on continuing medical education (CME), measuring the effectiveness of the programs is becoming a priority. Stakeholders want to know if their time and money is being spent appropriately and that the educational needs of the healthcare clinicians they are trying to reach are being met.

Get What You Paid For

It has been estimated that grants from the pharmaceutical industry fund more than half of all CME activities. To put this in perspective, the Accreditation Council for Continuing Medical Education (ACCME), the regulatory body that oversees physician continuing medical education, said that the "CME Enterprise" for 2004 included 129,000 activities, encompassing 1.2 million hours of CME credits that involved 9.1 million physicians. In fact, recent data released by the Senate Finance Committee reported that twenty-three pharmaceutical companies provided nearly $1.47 billion in educational grants for CME activities in 2004—nearly a 20-percent increase over 2003, according to The New York Times.

Yet, the unanswered question is: How effective are these CME activities—in terms of incorporating new knowledge to improve quality medical care?




Today, there is a growing trend among ACCME accredited sponsors, medical education communication companies, and industry grantors to identify means to answer this question. These efforts are designed to assess both the short-term, and more importantly, long-term effects of CME activities on the quality of healthcare provided by the physicians engaged in these endeavors.

Satisfaction Guaranteed?

In the not-too-distant past, CME effectiveness was often measured by the number of attendees at a CME event or the number of clinicians returning a CME program evaluation to obtain credit. Or, it was measured by an evaluation or post-test completed at the conclusion of a CME activity. Although these assessments could measure clinician satisfaction with the activity or assess how much the clinician learned from the program, they could not measure the long-term impact of the activity. While ACCME has always included "activity evaluation" as one of the core elements necessary for accreditation, it is clear that more needs to be done in this area. ACCME has developed a new standard for assessing this criterion, with its highest level of designation, "exemplary compliance," for those providers whose "educational activities are evaluated consistently for effectiveness in meeting identified educational needs, as measured by practice application and/or health status improvement."

The Alliance for Continuing Medical Education, comprised of CME providers, medical-education communication companies, industry grantors, and other CME professionals, has identified "performance measurement" as one of the competency areas for CME professionals. Specifically, these individuals should be able to "use measurement data to assess results of the learning interventions as a basis for determining future learning needs and for determining the application of the educational knowledge and skills."

The Game Plan

What specifically are providers, communication companies, and grantors doing to ensure that the effectiveness of the CME activities they are involved with is being measured? First, there are some CME providers that now require effectiveness measurement to be included in their programs.