With all the money being spent on continuing medical education (CME), measuring the effectiveness of the programs is becoming
a priority. Stakeholders want to know if their time and money is being spent appropriately and that the educational needs
of the healthcare clinicians they are trying to reach are being met.
Get What You Paid For
It has been estimated that grants from the pharmaceutical industry fund more than half of all CME activities. To put this
in perspective, the Accreditation Council for Continuing Medical Education (ACCME), the regulatory body that oversees physician
continuing medical education, said that the "CME Enterprise" for 2004 included 129,000 activities, encompassing 1.2 million
hours of CME credits that involved 9.1 million physicians. In fact, recent data released by the Senate Finance Committee reported
that twenty-three pharmaceutical companies provided nearly $1.47 billion in educational grants for CME activities in 2004—nearly
a 20-percent increase over 2003, according to The New York Times.
Yet, the unanswered question is: How effective are these CME activities—in terms of incorporating new knowledge to improve
quality medical care?
Today, there is a growing trend among ACCME accredited sponsors, medical education communication companies, and industry grantors
to identify means to answer this question. These efforts are designed to assess both the short-term, and more importantly,
long-term effects of CME activities on the quality of healthcare provided by the physicians engaged in these endeavors.
In the not-too-distant past, CME effectiveness was often measured by the number of attendees at a CME event or the number
of clinicians returning a CME program evaluation to obtain credit. Or, it was measured by an evaluation or post-test completed
at the conclusion of a CME activity. Although these assessments could measure clinician satisfaction with the activity or
assess how much the clinician learned from the program, they could not measure the long-term impact of the activity. While
ACCME has always included "activity evaluation" as one of the core elements necessary for accreditation, it is clear that
more needs to be done in this area. ACCME has developed a new standard for assessing this criterion, with its highest level
of designation, "exemplary compliance," for those providers whose "educational activities are evaluated consistently for effectiveness
in meeting identified educational needs, as measured by practice application and/or health status improvement."
The Alliance for Continuing Medical Education, comprised of CME providers, medical-education communication companies, industry
grantors, and other CME professionals, has identified "performance measurement" as one of the competency areas for CME professionals.
Specifically, these individuals should be able to "use measurement data to assess results of the learning interventions as
a basis for determining future learning needs and for determining the application of the educational knowledge and skills."
The Game Plan
What specifically are providers, communication companies, and grantors doing to ensure that the effectiveness of the CME activities
they are involved with is being measured? First, there are some CME providers that now require effectiveness measurement to
be included in their programs.