This week Harry quizzes Ulo Palm, the senior vice president of digital sciences at Allergan, about the long and problematic reign of the p-value in statistical analysis, and why it may be time for the biopharma industry to look to more nuanced measures of whether a drug trial succeeded.
Though the p-value “determines everything we do in drug development or medical research,” says Dr. Ulo Palm, it may be one of the most misunderstood and misused quantities in experimental science—drug discovery included. At its core, the p-value shows the probability that an observed effect was due to random chance. In other words, if a drug seems to outperform a placebo with an associated p-value of 0.05, there’s only a 5 percent chance that the study was wrong and that the drug is, in fact, no better than the placebo. A p-value of 0.05 is the accepted threshold for validity in most scientific research, even though it’s an arbitrary standard set nearly a century ago by statistician Sir Ronald Fisher. “People don’t often realize that this p-value of 5 percent was pulled out of thin air,” Dr. Palm says. “If Sir Ronald Fisher had had six fingers, we would all be using a p-value of 6 percent.”
The issue, Palm says, is that an arbitrary dividing line of 0.05 leads journal publishers (and paper authors themselves) to reject or ignore real effects that don’t happen to meet the threshold. If a drug trial yields a p-value of more than 0.05, “You should never ever say it is not working,” he tells Harry. “You can only say we were not able to make a determination. That’s it.” By examining the spread of a data set, confidence intervals, data from individuals, and other measures, Palm says, today’s researchers can get a more realistic picture of the promise of new compounds as medicines.
Please rate and review MoneyBall Medicine on Apple Podcasts! Here’s how to do that from an iPhone, iPad, or iPod touch:
• Launch the “Podcasts” app on your device. If you can’t find this app, swipe all the way to the left on your home screen until you’re on the Search page. Tap the search field at the top and type in “Podcasts.” Apple’s Podcasts app should show up in the search results.
• Tap the Podcasts app icon, and after it opens, tap the Search field at the top, or the little magnifying glass icon in the lower right corner.
• Type MoneyBall Medicine into the search field and press the Search button.
• In the search results, click on the MoneyBall Medicine logo.
• On the next page, scroll down until you see the Ratings & Reviews section. Below that, you’ll see five purple stars.
• Tap the stars to rate the show.
• Scroll down a little farther. You’ll see a purple link saying “Write a Review.”
• On the next screen, you’ll see the stars again. You can tap them to leave a rating if you haven’t already.
• In the Title field, type a summary for your review.
• In the Review field, type your review.
• When you’re finished, click Send.
• That’s it, you’re done. Thanks!