Test and Learn: Making Confident Decisions on the Commercial Side - Pharmaceutical Executive


Test and Learn: Making Confident Decisions on the Commercial Side

Pharmaceutical Executive

As it applies to commercial decisions, Test & Learn means trying an idea (such as a new marketing, sales force, or pricing program) with a small group of markets, patients, physicians, or reps and comparing the change in behavior of that group with the behavior of a carefully-matched control group that has not been exposed to the idea. When effectively applied, Test & Learn answers three key questions about any proposed business idea, before taking on the risk and expense of rollout:

Will the program work?

With which patients, reps, physicians, or markets will the program work better?

How can we fine-tune the idea so that we are only moving forward with the most effective components?

These questions are addressed by a variety of large and small companies across the business frontier: from large companies such as Walmart and Target, Walgreens and CVS, Hilton and Holiday Inn, Kraft and Anheuser-Busch InBev, McDonald's and Starbucks, to smaller companies such as Guitar Center and Banfield the Pet Hospital. Some of these companies' internal estimates of the value of Test & Learn eclipse $100 million per year in added profits.

As pharmaceutical companies move forward with Test & Learn they are learning two things. First, analytically it is very different from patient-level testing. A test of a new sales force targeting program, which can be applied at the market or prescribing physician level, and the analytical challenges of driving reliable results from such a test are quite different than the challenges involved with the traditional double-blind patient-level test. Different analytical approaches are needed for test design, test and control group selection, and test evaluation. Ongoing developments in big data, coupled with extraordinary advances in algorithm development, have allowed analysts to overcome issues of business volatility, competitive initiatives, the confounding influence of payers and patients, as well as continuous macro-economic changes, to allow organizations to isolate the incremental impact of new ideas with high confidence.

Second, and more importantly, companies in our industry are learning that Test & Learn is much more precise and actionable than the more conventional conjoint, correlation, or regression analyses that evaluate historical data in an effort to inform future alternatives. Test & Learn provides direct insight into the specific impact of the new idea, as well as opportunities to modify it for maximum effectiveness and target its rollout for maximum returns.

Leading pharmaceutical companies can drive millions of dollars in shareholder value by applying Test & Learn tools to measure, refine, and target national and local media campaigns, reimbursement strategies (including couponing), speaker program investments, electronic marketing programs, sales strategies and incentives, and patient communication. Early results of Test & Learn being applied at pharmaceutical companies are compelling and it is believed those companies that adopt this methodology soon will have a long-standing competitive advantage over those companies conducting business as usual.

As Tom Davenport argues in an HBR article titled, "How to Design Smart Business Experiments," "testing, and learning from testing, should become central to any organization's decision making. The principles of the scientific method work as well in business as in any sector of life. It's time to replace 'I'll bet' with 'I know.'"

Ben Bonifant is President of Bonifant Insights Group, LLC. He can be reached at

Scott Setrakianis Managing Director at Applied Predictive Technologies.He can be reached at


blog comments powered by Disqus

Source: Pharmaceutical Executive,
Click here