際際滷

際際滷Share a Scribd company logo

R&D Progression Bias and Rational Optimism

Download as PPTX, PDF
Dennis Lendrem
Dennis Lendrem

Opportunity costs in drug development. This presentation demonstrates how to estimate the opportunity costs of false positives and false negatives in pharmaceutical R&D decision making. While the industry fears false negatives, false positives are demonstrated to be much more costly to the organization.

1 of 34
Download to read offline
RATIONAL OPTIMISM IN RESEARCH & DEVELOPMENT Dennis Lendrem
RATIONAL OPTIMISM IN RESEARCH & DEVELOPMENT Dennis Lendrem
Lendrem D.W., Senn S.J., Lendrem B.C., Isaacs J.D. (2015), R&D productivity rides again?, Pharmaceut. Statist., 14, 13, doi: 10.1002/pst.1653
Lendrem D., Senn S.J., Lendrem B.C., and Isaacs J.D. (2015), R&D productivity rides again?, Pharmaceut. Statist., 14, 13, doi: 10.1002/pst.1653
Years Cycle Time = 4 yrs 1 2 3 4 5 6 Cycle Time = 6 yrs 1 2 3 4 5 6 Before After
Years Cycle Time = 4 yrs 1 2 3 4 5 6 Cycle Time = 6 yrs 1 2 3 4 5 6 Before After
Rational Optimism By placing tasks in parallel to reduce cycle time we inadvertently: Increased R&D costs Increased late-stage attrition rates Increased expected time to market
Rational Optimism By placing tasks in parallel to reduce cycle time we inadvertently: Increased R&D costs Increased late-stage attrition rates Increased expected time to market Lendrem DW, Lendrem BC Torching the Haystack: Modelling fast-fail strategies in drug development. Drug Discov Today. 2013 Apr;18(7-8):331-6. http://dx.doi.org/10.1016/j.drudis.2012.11.011 Lendrem DW, Lendrem BC The Development Speed Paradox: Can increasing development speed reduce R&D productivity? Drug Discov Today. 2014 Mar;9(3):209-214 http://dx.doi.org/10.1016/j.drudis.2013.09.002
Rational Optimism Looking for a needle in a haystack? Torch the haystack.
Rational Optimism It is more important to do the right science than to do the wrong science quickly. Build opportunities to kill projects earlier in the development cycle. Focus on tests that prevent projects from entering the development process. Focus on tests that allow elimination of projects early in the development process.
Rational Optimism
Rational Optimism Progression Bias terminations are viewed as losses and project teams are loss- averse sunk-costs fallacy terminating a project risks losing investment that has already been lost marginal cost associated with a decision to continue development may seem trivial compared to the losses associated with terminating a potential NTD organizational and market incentives to load the pipeline with promising candidates Progression bias leads to continued development even in the light of strong evidence supporting termination.
Rational Optimism Rational Optimism Continued pursuit of a line of research in the knowledge that it is likely to be successful.
Rational Optimism Rational Optimism Continued pursuit of a line of research in the knowledge that it is likely to be successful. Mindless Optimism Continued pursuit of a line of research in the hope that it will be successful.
R&D Progression Bias and Rational Optimism
Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
Rational Optimism Opportunity Costs: Loss due to false negatives = B NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
Rational Optimism Opportunity Costs: Loss due to false negatives = B Loss due to false positives = C NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
Rational Optimism Opportunity Costs: Loss due to false negatives = B Loss due to false positives = CxA NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Positives when 留 = 67% 硫 =10% mindless optimism rational optimismvs
Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =15%Cost of False Positives when 留 = 67% mindless optimism rational optimismvs
Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I mindless optimism rational optimism 硫 =20% Cost of False Positives when 留 = 67% vs
Rational Optimism So? Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Positives when 留 = 67% Cost of False Negatives when 硫 =10% mindless optimism rational optimism vs Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I mindless optimism rational optimism Cost of False Negatives when 硫 =20% Cost of False Positives when 留 = 67% vs The false negative rate has to be rather high before it outweighs the cost of false positives!
Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% Cost of False Positives when 留 = 67% mindless optimism rational optimismvs
Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% mindless optimism rational optimismvs
Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% mindless optimism rational optimismvs
Rational Optimism So? Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Negatives when 硫 =20% Cost of False Positives when 留 = 67% mindless optimism rational optimism vs Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Negatives when 硫 =20% mindless optimism rational optimism vs Early Development Late Development
Rational Optimism So? We fear false negatives, but the opportunity costs of false positives are huge. Terminating projects frees up R&D resources to pursue other prospects. Thinking about opportunity costs allows project teams to reframe termination decisions as prospective future gains.
Rational Optimism Lendrem D.W., Lendrem B.C., Peck R.W., Senn S.J., Day, S., Isaacs J.D. (2015), Progression Bias and Rational Optimism in Research & Development, Nat. Rev. Drug Discov., In press NTD p 1-硫1 1-硫2 1-硫3 1-硫4 1-硫5 1-硫6 1-硫7 1-硫8 硫8 硫7 硫6 硫5 硫4 硫3 硫2 硫1 1-p 留1 留2 留3 留4 留5 留6 留7 留8 1-留8 1-留7 1-留6 1-留5 1-留4 1-留3 1-留2 1-留1
Rational Optimism Richard Peck (Roche) Stephen Senn (CRP Sant辿) Simon Day (MHRA) John Isaacs (MRG) John Loughlin (MRG) NIHR Newcastle BRC
Rational Optimism Richard Peck (Roche) Stephen Senn (CRP Sant辿) Simon Day (MHRA) John Isaacs (MRG) John Loughlin (MRG) Clare Lendrem (AHG) NIHR Newcastle BRC
fin

More Related Content

R&D Progression Bias and Rational Optimism

  • 1. RATIONAL OPTIMISM IN RESEARCH & DEVELOPMENT Dennis Lendrem
  • 2. RATIONAL OPTIMISM IN RESEARCH & DEVELOPMENT Dennis Lendrem
  • 3. Lendrem D.W., Senn S.J., Lendrem B.C., Isaacs J.D. (2015), R&D productivity rides again?, Pharmaceut. Statist., 14, 13, doi: 10.1002/pst.1653
  • 4. Lendrem D., Senn S.J., Lendrem B.C., and Isaacs J.D. (2015), R&D productivity rides again?, Pharmaceut. Statist., 14, 13, doi: 10.1002/pst.1653
  • 5. Years Cycle Time = 4 yrs 1 2 3 4 5 6 Cycle Time = 6 yrs 1 2 3 4 5 6 Before After
  • 6. Years Cycle Time = 4 yrs 1 2 3 4 5 6 Cycle Time = 6 yrs 1 2 3 4 5 6 Before After
  • 7. Rational Optimism By placing tasks in parallel to reduce cycle time we inadvertently: Increased R&D costs Increased late-stage attrition rates Increased expected time to market
  • 8. Rational Optimism By placing tasks in parallel to reduce cycle time we inadvertently: Increased R&D costs Increased late-stage attrition rates Increased expected time to market Lendrem DW, Lendrem BC Torching the Haystack: Modelling fast-fail strategies in drug development. Drug Discov Today. 2013 Apr;18(7-8):331-6. http://dx.doi.org/10.1016/j.drudis.2012.11.011 Lendrem DW, Lendrem BC The Development Speed Paradox: Can increasing development speed reduce R&D productivity? Drug Discov Today. 2014 Mar;9(3):209-214 http://dx.doi.org/10.1016/j.drudis.2013.09.002
  • 9. Rational Optimism Looking for a needle in a haystack? Torch the haystack.
  • 10. Rational Optimism It is more important to do the right science than to do the wrong science quickly. Build opportunities to kill projects earlier in the development cycle. Focus on tests that prevent projects from entering the development process. Focus on tests that allow elimination of projects early in the development process.
  • 12. Rational Optimism Progression Bias terminations are viewed as losses and project teams are loss- averse sunk-costs fallacy terminating a project risks losing investment that has already been lost marginal cost associated with a decision to continue development may seem trivial compared to the losses associated with terminating a potential NTD organizational and market incentives to load the pipeline with promising candidates Progression bias leads to continued development even in the light of strong evidence supporting termination.
  • 13. Rational Optimism Rational Optimism Continued pursuit of a line of research in the knowledge that it is likely to be successful.
  • 14. Rational Optimism Rational Optimism Continued pursuit of a line of research in the knowledge that it is likely to be successful. Mindless Optimism Continued pursuit of a line of research in the hope that it will be successful.
  • 16. Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
  • 17. Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
  • 18. Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
  • 19. Rational Optimism Opportunity Costs: Loss due to false negatives = B NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
  • 20. Rational Optimism Opportunity Costs: Loss due to false negatives = B Loss due to false positives = C NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
  • 21. Rational Optimism Opportunity Costs: Loss due to false negatives = B Loss due to false positives = CxA NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
  • 22. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Positives when 留 = 67% 硫 =10% mindless optimism rational optimismvs
  • 23. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =15%Cost of False Positives when 留 = 67% mindless optimism rational optimismvs
  • 24. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I mindless optimism rational optimism 硫 =20% Cost of False Positives when 留 = 67% vs
  • 25. Rational Optimism So? Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Positives when 留 = 67% Cost of False Negatives when 硫 =10% mindless optimism rational optimism vs Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I mindless optimism rational optimism Cost of False Negatives when 硫 =20% Cost of False Positives when 留 = 67% vs The false negative rate has to be rather high before it outweighs the cost of false positives!
  • 26. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% Cost of False Positives when 留 = 67% mindless optimism rational optimismvs
  • 27. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% mindless optimism rational optimismvs
  • 28. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% mindless optimism rational optimismvs
  • 29. Rational Optimism So? Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Negatives when 硫 =20% Cost of False Positives when 留 = 67% mindless optimism rational optimism vs Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Negatives when 硫 =20% mindless optimism rational optimism vs Early Development Late Development
  • 30. Rational Optimism So? We fear false negatives, but the opportunity costs of false positives are huge. Terminating projects frees up R&D resources to pursue other prospects. Thinking about opportunity costs allows project teams to reframe termination decisions as prospective future gains.
  • 31. Rational Optimism Lendrem D.W., Lendrem B.C., Peck R.W., Senn S.J., Day, S., Isaacs J.D. (2015), Progression Bias and Rational Optimism in Research & Development, Nat. Rev. Drug Discov., In press NTD p 1-硫1 1-硫2 1-硫3 1-硫4 1-硫5 1-硫6 1-硫7 1-硫8 硫8 硫7 硫6 硫5 硫4 硫3 硫2 硫1 1-p 留1 留2 留3 留4 留5 留6 留7 留8 1-留8 1-留7 1-留6 1-留5 1-留4 1-留3 1-留2 1-留1
  • 32. Rational Optimism Richard Peck (Roche) Stephen Senn (CRP Sant辿) Simon Day (MHRA) John Isaacs (MRG) John Loughlin (MRG) NIHR Newcastle BRC
  • 33. Rational Optimism Richard Peck (Roche) Stephen Senn (CRP Sant辿) Simon Day (MHRA) John Isaacs (MRG) John Loughlin (MRG) Clare Lendrem (AHG) NIHR Newcastle BRC
  • 34. fin

Editor's Notes

  • #8: As a result, the industry became really slick at delivering late-stage failures to the market place, precipitating the current R&D productivity crisis.
  • #9: As a result, the industry became really slick at delivering late-stage failures to the market place, precipitating the current R&D productivity crisis.
  • #11: 10