Test and Learn: Making Confident Decisions on the Commercial Side

Mar 01, 2013

Because rigorous application of the scientific method is at the foundation of the pharmaceutical development cycle, readers of Pharm Exec may be surprised to learn that the industry has much to learn from companies in other sectors where scientific testing on the commercial side of the business is rapidly becoming the norm.

The power of insights drawn from in-market testing has increased dramatically because of advances in information technology, large-scale data management, and computation speed. Catching this wave, a growing number of leading retailers, banks, insurance companies, restaurants, and hotel chains have adopted the scientific method to accelerate innovation while mitigating risk. These organizations have termed the process "Test & Learn," and they are applying it to understand the impact of digital marketing programs, new advertising content and targeting, promotion effectiveness, sales force management tactics, and customer communications. Test & Learn enables organizations to learn which ideas work and which don't before taking the risk and expense of rolling them out. Though most companies have tried to measure commercial tests ad-hoc, the best practitioners of Test & Learn are institutionalizing this capability and prioritize, design, and measure tests using dedicated software, ultimately enabling a consistent decision-making process.

A few leading pharmaceutical companies have recently begun applying Test & Learn. Given the industry's depth of experience with lab-based testing, it may seem ironic that market-based experimentation is just now catching on in Big Pharma. But interest in the Test & Learn approach has been sparked by ever tightening commercial budgets and the industry's long-term mission to reach the right customers with the right products more successfully, quickly, and efficiently.

In one recent example, a leading pharmaceutical company operating across multiple therapeutic units took a sophisticated Test & Learn approach to building and evaluating a DTC advertising campaign supporting a blockbuster drug. The brand team launched a print advertising campaign in select markets, targeting the most attractive demographic and customers. The team was having a difficult time evaluating the results to determine the cause-and-effect relationship between the ad campaign and new scripts—if any.

The company used advanced Test & Learn software to get an accurate and useful understanding of ROI. Analysts used the Test & Learn software to create optimal control groups for the test markets, in order to remove the inherent volatility in prescription data that occur by-location, by-physician, and by-patient. Changes in prescriptions in the test markets (receiving the campaign) were then compared to changes in scrips of the custom control group (not receiving the campaign).

Test & Learn analysis showed that the advertising was driving a significant increase in new scripts overall. This sales lift was generating profits well in excess of the cost of the advertising—a sign that the ads were working well.

Moreover, by de-averaging the results, the brand team learned that the program was most successful in driving scripts from primary care physicians with a history of prescribing the product, and in markets with higher median income. Based on these findings, the team doubled its advertising investment, targeting the rollout to specific markets predicted to cause a significant increase in overall scripts and profitability.

Test & Learn's advantages are increasingly acclaimed by academic and business leaders. Jim Collins wrote in "Great by Choice" that, "In the face of instability uncertainty and rapid change...analytic skills still matter, but empirical validation matters much more." And a Harvard Business Review article entitled "A Step-by-Step Guide to Smart Business Experiments," argues that, "It's easier to draw the right conclusions using data generated through experiments than by studying historical transactions."

lorem ipsum