I'm actually surprised it isn't going up over time. That is naively what you would expect as the low-hanging fruit is plucked. So the fact that it's been stable is probably a sign that scientific advances are roughly keeping pace with the (presumably) increasing challenge of finding ever more targets for drugs.
I suspect there's a ceiling on failure rates people are willing to operate under. A diversified investor might still be willing to take high risk/reward bets. But workers can go their whole career producing nothing of value. It becomes hard to tell the difference between smart, hard working people who were unlucky and lazy grifters. I've done 2 series-A startups and I don't think I can do another one.
He points out that it basically has gone up, since the 80s and 90s brought some new therapeutic targets and approaches that by now have been mined out.
> I’m fond of saying that the most important single statistic about the drug industry is the clinical failure rate, which is (by any reasonable standard) appallingly high.
He starts off on the wrong note, there is nothing at all wrong with a high rate of clinical failures. If anything, the reasonable argument standard might be that this rate is too low. It implies researchers are trying things that they expect to have a 10% chance of working out. That means we're missing out on all the cures and techniques that have a 1% chance of working out but but nonetheless do work.
Failed attempts cost society nearly nothing and successes will have compounding benifits for, y'know, lets optimistically say the human race survives for centuries. 1% or 0.1% success rates sound completely reasonable with that sort of lopsided risk profile. There isn't much of a reason not to try anything and everything that has the faintest chance of helping and see what happens.
Rather notably, companies are profit-driven, not good-for-society driven. If 99% of your attempts which cost millions to do fail, the company either has to raise the prices of the drug insanely high or not work as a company. In addition, most drugs aren't humanity saving or crucial for humans to live. You could solve all forms of cancer tomorrow, which would be a great thing, but you aren't "saving humanity". You add less than a decade or so, maximum, to the average human lifespan. This isn't critical to humanity's existence, so this "try or die" mentality doesn't work.
If this try or die mentality does work, why don't we try every possible amino acid chain in every configuration possible? It's an infinitesimally small chance but try or die "outweighs". This is evidently absurd.
Except that governing bodies approve many of these drugs which, further down the line, consumers may experience high failure rates or even toxicity, which in other industries such results would be considered outright fraud subject to legal action. if sold to consumers.
As someone who works in research, this isn’t surprising at all. It’s hard to find something that treats (well most often reduces symptoms) of a particular disease. It’s even more difficult to find something that is also safe at the dose level it takes to treat said disease. Are there faster ways to do this? Probably not. AI is only going to help out so much, just like automated drug screening only helped so much since its introduction in the 90s.
But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground
Is an utterly irrelevant comparison. For physical thing we have engineering, and the practical application of the trades and craft, trial and error.
For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
Indeed, I suspect the failure rate of, say, new jet engine designs is rather high as well -- those failures just never get reported in a federal repository, unlike RCTs, since they never make it out of the simulator or the prototyping lab. And we have, comparatively, much better computational models of how airplanes fly than how cancer cells mutate. FWIW this clinical stat is far better than Edison's supposed lightbulb-idea failure rate!
Jet engine design is iterative, and the basic principles are well-understood.
Drug design seems a lot more binary. You can find a new pathway, but drugs themselves are fairly simply molecules, and you can't iteratively 'fix bugs' the way you can in an engine or a piece of software.
The engine is a given, it's almost astronomically complicated, you know a lot less about how it works than you'd like to, and you're trying to change how it works while it's running without breaking anything, using tiny rigid parts that have to snap into place correctly and can't be bent to fit.
Targeted gene therapy, cancer survival rates, trauma care, the advancements in hip and knee replacement, all sorts of surgery, HIV is now a non-issue with the right care. The list goes on.
Drug development is a hard problem because the solution space is poorly constrained: biochemistry is complex and messy, expecting one chemical substance to have narrow positive effects is probably hopeless.
I’ve been hearing for 10 years now on how “AI” will change this clinical rate. It’s now like the “This is the year of Linux” prediction meme. And like Derek has mentioned multiple times you can’t statiscialise biology. I don’t think many people get it so we have tech bros like us throw AI at data.
There's an alternative that is slowly emerging. Many clinical trials "fail" but the drug candidate in question works really well for identifiable subsets of the participants. Right now pharma companies won't bother pursuing those drugs because they can't market it broadly. But it's still possible these drugs could help people in the future.
Not really an alternative if you can't predict who those candidates are prior to spending a hundred million dollars putting the drug into a broader group of people and then retroactively saying "oh wait now we can 'identify' who it works in."
If they could do that today, they would. Trial criteria are already incredibly narrow specifically to try to encode as much of this knowledge as the company has prior to starting the trial. But it empirically turns out they don't have nearly enough to matter.
It's really not a problem that pre-filtering to that responders group produces too small a market. There are plenty of drug programs going after way-too-small markets because of all these wacky perverse incentives created by insurers and regulators to incentivize the creation of billion-dollar++ drugs for diseases virtually no one has.
If you can think of a way that you can have a much higher chance than 9% of success in getting a useful treatment to market, you can make a boatload of money. Go ahead, reform it.
It's not for lack of trying. People are definitely trying to find better ways of finding stronger candidates earlier. People are trying to gain a more precise understanding of disease mechanisms.
There are breakthroughs that I could have never imagined such as mRNA vaccines.
How? He’s saying that he can’t think of any good ones, and in my experience that’s true of most informed people. This is one of those hard problems that needs solutions, and so far the present system is the best anyone has managed.
Lower numbers don’t mean we’re doing better, it means we’re trying less.
ugh, I don't know
He starts off on the wrong note, there is nothing at all wrong with a high rate of clinical failures. If anything, the reasonable argument standard might be that this rate is too low. It implies researchers are trying things that they expect to have a 10% chance of working out. That means we're missing out on all the cures and techniques that have a 1% chance of working out but but nonetheless do work.
Failed attempts cost society nearly nothing and successes will have compounding benifits for, y'know, lets optimistically say the human race survives for centuries. 1% or 0.1% success rates sound completely reasonable with that sort of lopsided risk profile. There isn't much of a reason not to try anything and everything that has the faintest chance of helping and see what happens.
If this try or die mentality does work, why don't we try every possible amino acid chain in every configuration possible? It's an infinitesimally small chance but try or die "outweighs". This is evidently absurd.
But let’s think about that 91% failure rate for a moment. When I bring this up in presentations, I invite the audience to consider what the auto industry would look like of 91% of new car designs proved unable to roll out of the factory, or if 91% of new airliner models were unable to leave the ground
Is an utterly irrelevant comparison. For physical thing we have engineering, and the practical application of the trades and craft, trial and error.
For modern medicine, we're only just starting to come out of the wild west era. Or perhaps slightly further along than that.
Drug design seems a lot more binary. You can find a new pathway, but drugs themselves are fairly simply molecules, and you can't iteratively 'fix bugs' the way you can in an engine or a piece of software.
The engine is a given, it's almost astronomically complicated, you know a lot less about how it works than you'd like to, and you're trying to change how it works while it's running without breaking anything, using tiny rigid parts that have to snap into place correctly and can't be bent to fit.
High failure rates aren't surprising.
Why do you say that? What's the evidence? We continue to have virtually no clue how to make drugs, per TFA.
Drug development is a hard problem because the solution space is poorly constrained: biochemistry is complex and messy, expecting one chemical substance to have narrow positive effects is probably hopeless.
If they could do that today, they would. Trial criteria are already incredibly narrow specifically to try to encode as much of this knowledge as the company has prior to starting the trial. But it empirically turns out they don't have nearly enough to matter.
It's really not a problem that pre-filtering to that responders group produces too small a market. There are plenty of drug programs going after way-too-small markets because of all these wacky perverse incentives created by insurers and regulators to incentivize the creation of billion-dollar++ drugs for diseases virtually no one has.
There are breakthroughs that I could have never imagined such as mRNA vaccines.