As it applies to commercial decisions, Test & Learn means trying an idea (such as a new marketing, sales force, or pricing
program) with a small group of markets, patients, physicians, or reps and comparing the change in behavior of that group with
the behavior of a carefully-matched control group that has not been exposed to the idea. When effectively applied, Test &
Learn answers three key questions about any proposed business idea, before taking on the risk and expense of rollout:
» Will the program work?
» With which patients, reps, physicians, or markets will the program work better?
» How can we fine-tune the idea so that we are only moving forward with the most effective components?
These questions are addressed by a variety of large and small companies across the business frontier: from large companies
such as Walmart and Target, Walgreens and CVS, Hilton and Holiday Inn, Kraft and Anheuser-Busch InBev, McDonald's and Starbucks,
to smaller companies such as Guitar Center and Banfield the Pet Hospital. Some of these companies' internal estimates of the
value of Test & Learn eclipse $100 million per year in added profits.
As pharmaceutical companies move forward with Test & Learn they are learning two things. First, analytically it is very different
from patient-level testing. A test of a new sales force targeting program, which can be applied at the market or prescribing
physician level, and the analytical challenges of driving reliable results from such a test are quite different than the challenges
involved with the traditional double-blind patient-level test. Different analytical approaches are needed for test design,
test and control group selection, and test evaluation. Ongoing developments in big data, coupled with extraordinary advances
in algorithm development, have allowed analysts to overcome issues of business volatility, competitive initiatives, the confounding
influence of payers and patients, as well as continuous macro-economic changes, to allow organizations to isolate the incremental
impact of new ideas with high confidence.
Second, and more importantly, companies in our industry are learning that Test & Learn is much more precise and actionable
than the more conventional conjoint, correlation, or regression analyses that evaluate historical data in an effort to inform
future alternatives. Test & Learn provides direct insight into the specific impact of the new idea, as well as opportunities
to modify it for maximum effectiveness and target its rollout for maximum returns.
Leading pharmaceutical companies can drive millions of dollars in shareholder value by applying Test & Learn tools to measure,
refine, and target national and local media campaigns, reimbursement strategies (including couponing), speaker program investments,
electronic marketing programs, sales strategies and incentives, and patient communication. Early results of Test & Learn being
applied at pharmaceutical companies are compelling and it is believed those companies that adopt this methodology soon will
have a long-standing competitive advantage over those companies conducting business as usual.
As Tom Davenport argues in an HBR article titled, "How to Design Smart Business Experiments," "testing, and learning from
testing, should become central to any organization's decision making. The principles of the scientific method work as well
in business as in any sector of life. It's time to replace 'I'll bet' with 'I know.'"
Ben Bonifant is President of Bonifant Insights Group, LLC. He can be reached at
Scott Setrakianis Managing Director at Applied Predictive Technologies.He can be reached at