It is not possible to correct downstream errors by upstream interventions based on how some patients and physicians behave.
It would represent most unwise policy to let the FDA make any decision to allow sale of a drug based on the competence of
individual patients in the potential user pool. When it makes its upstream decision, any ban that it imposes, like the summer
rain, falls on the competent and the incompetent alike.
Decisions on treatment choice are so intensely individual that they must be made downstream, not only for drug usage but also
for any and all aspects of health care.
Heterogeneity of the overall population emerges as the critical issue. In their recent study on FDA approval policies, Anup
Malani and Feifang Hu note that the FDA "employs a simple rule when deciding whether to approve a new drug for use by physicians:
the average treatment effect of the new drug must be superior to the average effect of a placebo." They then criticize this
model on the powerful ground that it does not lead to decisions that maximize the expected utility of drug usage.
By placing its focus on the average use, the FDA ignores the variation in individual responses. Even when the FDA or companies
stratify patients into various cohorts, the problem is not eliminated. There could easily be wide variations even within the
smaller classes. It could well be that on average a particular drug does not perform as well as a placebo. But so what? That
only shows that most people should not take the drug, not that it should be banned.
The FDA has been criticized on this point from the vantage point of those who want tougher conditions for entry. Thus Arnold
Relman and Marcia Angell, fierce critics of the drug industry, urge the FDA to ratchet up its policy of looking at averages
yet another notch. "Unfortunately, the FDA will approve a me-too drug on the basis of clinical trials comparing it not with
an older drug of the same type, but with a placebo or a drug of another type."
Looked at in its most favorable aspect, their proposal follows the flawed FDA methodology, with a heightened baseline, equal
to the average performance of an approved drug. The effect is therefore to keep still more drugs off the market and thus to
entrench the monopoly position of the initial entrant.
Moreover, the proposal ignores that drugs of equal efficacy may have pronounced variations because of allergy, intolerance,
or other factors that make some better and some worse in individual cases. Worse, it forces the new entrant to meet a moving
target. Its initial task is to match the performance of the first entrant. But that estimation will vary over time, making
it difficult to know what level of performance will justify FDA approval.
The desired reforms on this issue move in the opposite direction from the ill-considered Relman-Angell proposal. The correct
procedure treats this variation in individual response as critical. It first asks whether there is a significant fraction
of cases in which the drug under review outperforms the placebo.
The answer to that question is likely to be negative if the mean response is well below that of the placebo and the variance
in responses across individuals is small. But as the variance in individual response increases, the FDA's procedure is ever
more likely to lead to incorrect results. It is even possible for the mean of the placebo to lie above the mean for the drug, even though some substantial fraction of the population is better off with the drug than without it.
An example might help illustrate the point. Suppose that we rate patient response to treatment on a scale of 0 to 100, where
the current drug has an average response of 50 but a variation in responses from 25 to 75. Now put a new drug on the market
that has an average response of 45, with a variance of 20 to 70. The question is whether the second drug should be allowed
on the market, when each relevant parameter is 5 points below that of the original drug.
If all individuals have the identical rank order of response, then, sure, keep it off. Given that assumption, any person who
scores X with the current drug will turn X minus 5 with the new one. Individual choice could only compound error.
But heterogeneity totally undermines that assumption. Now even though the whole curve has shifted to the left for the new
drug, some fraction of individuals will score better with the new one than the old one. Since we don't know who these individuals
are, we pay a high price in letting the entire patient population have only one choice instead of two.