Get What You Paid For
It has been estimated that grants from the pharmaceutical industry fund more than half of all CME activities. To put this in perspective, the Accreditation Council for Continuing Medical Education (ACCME), the regulatory body that oversees physician continuing medical education, said that the "CME Enterprise" for 2004 included 129,000 activities, encompassing 1.2 million hours of CME credits that involved 9.1 million physicians. In fact, recent data released by the Senate Finance Committee reported that twenty-three pharmaceutical companies provided nearly $1.47 billion in educational grants for CME activities in 2004—nearly a 20-percent increase over 2003, according to The New York Times.Yet, the unanswered question is: How effective are these CME activities—in terms of incorporating new knowledge to improve quality medical care?
In the not-too-distant past, CME effectiveness was often measured by the number of attendees at a CME event or the number of clinicians returning a CME program evaluation to obtain credit. Or, it was measured by an evaluation or post-test completed at the conclusion of a CME activity. Although these assessments could measure clinician satisfaction with the activity or assess how much the clinician learned from the program, they could not measure the long-term impact of the activity. While ACCME has always included "activity evaluation" as one of the core elements necessary for accreditation, it is clear that more needs to be done in this area. ACCME has developed a new standard for assessing this criterion, with its highest level of designation, "exemplary compliance," for those providers whose "educational activities are evaluated consistently for effectiveness in meeting identified educational needs, as measured by practice application and/or health status improvement."
The Alliance for Continuing Medical Education, comprised of CME providers, medical-education communication companies, industry grantors, and other CME professionals, has identified "performance measurement" as one of the competency areas for CME professionals. Specifically, these individuals should be able to "use measurement data to assess results of the learning interventions as a basis for determining future learning needs and for determining the application of the educational knowledge and skills."
The Game Plan
What specifically are providers, communication companies, and grantors doing to ensure that the effectiveness of the CME activities they are involved with is being measured? First, there are some CME providers that now require effectiveness measurement to be included in their programs.