R&D Progression Bias and Rational Optimism
Download as pptx, pdf1 like1,510 views
The document discusses the concept of rational optimism in research and development, highlighting its impact on R&D productivity and costs. It emphasizes that placing tasks in parallel to reduce cycle time can inadvertently lead to increased costs and higher attrition rates. The analysis also warns against progression bias and rational versus mindless optimism, advocating for early project termination to optimize resource allocation.
R&D Progression Bias and Rational Optimism
- 1. RATIONAL OPTIMISM IN RESEARCH & DEVELOPMENT Dennis Lendrem
- 2. RATIONAL OPTIMISM IN RESEARCH & DEVELOPMENT Dennis Lendrem
- 3. Lendrem D.W., Senn S.J., Lendrem B.C., Isaacs J.D. (2015), R&D productivity rides again?, Pharmaceut. Statist., 14, 13, doi: 10.1002/pst.1653
- 4. Lendrem D., Senn S.J., Lendrem B.C., and Isaacs J.D. (2015), R&D productivity rides again?, Pharmaceut. Statist., 14, 13, doi: 10.1002/pst.1653
- 5. Years Cycle Time = 4 yrs 1 2 3 4 5 6 Cycle Time = 6 yrs 1 2 3 4 5 6 Before After
- 6. Years Cycle Time = 4 yrs 1 2 3 4 5 6 Cycle Time = 6 yrs 1 2 3 4 5 6 Before After
- 7. Rational Optimism By placing tasks in parallel to reduce cycle time we inadvertently: Increased R&D costs Increased late-stage attrition rates Increased expected time to market
- 8. Rational Optimism By placing tasks in parallel to reduce cycle time we inadvertently: Increased R&D costs Increased late-stage attrition rates Increased expected time to market Lendrem DW, Lendrem BC Torching the Haystack: Modelling fast-fail strategies in drug development. Drug Discov Today. 2013 Apr;18(7-8):331-6. http://dx.doi.org/10.1016/j.drudis.2012.11.011 Lendrem DW, Lendrem BC The Development Speed Paradox: Can increasing development speed reduce R&D productivity? Drug Discov Today. 2014 Mar;9(3):209-214 http://dx.doi.org/10.1016/j.drudis.2013.09.002
- 9. Rational Optimism Looking for a needle in a haystack? Torch the haystack.
- 10. Rational Optimism It is more important to do the right science than to do the wrong science quickly. Build opportunities to kill projects earlier in the development cycle. Focus on tests that prevent projects from entering the development process. Focus on tests that allow elimination of projects early in the development process.
- 12. Rational Optimism Progression Bias terminations are viewed as losses and project teams are loss- averse sunk-costs fallacy terminating a project risks losing investment that has already been lost marginal cost associated with a decision to continue development may seem trivial compared to the losses associated with terminating a potential NTD organizational and market incentives to load the pipeline with promising candidates Progression bias leads to continued development even in the light of strong evidence supporting termination.
- 13. Rational Optimism Rational Optimism Continued pursuit of a line of research in the knowledge that it is likely to be successful.
- 14. Rational Optimism Rational Optimism Continued pursuit of a line of research in the knowledge that it is likely to be successful. Mindless Optimism Continued pursuit of a line of research in the hope that it will be successful.
- 16. Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
- 17. Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
- 18. Rational Optimism NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
- 19. Rational Optimism Opportunity Costs: Loss due to false negatives = B NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
- 20. Rational Optimism Opportunity Costs: Loss due to false negatives = B Loss due to false positives = C NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
- 21. Rational Optimism Opportunity Costs: Loss due to false negatives = B Loss due to false positives = CxA NTD p 1-硫 A 硫 B 1-p 留 C 1-留 D Success Miss False Discovery Success
- 22. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Positives when 留 = 67% 硫 =10% mindless optimism rational optimismvs
- 23. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =15%Cost of False Positives when 留 = 67% mindless optimism rational optimismvs
- 24. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I mindless optimism rational optimism 硫 =20% Cost of False Positives when 留 = 67% vs
- 25. Rational Optimism So? Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Positives when 留 = 67% Cost of False Negatives when 硫 =10% mindless optimism rational optimism vs Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I mindless optimism rational optimism Cost of False Negatives when 硫 =20% Cost of False Positives when 留 = 67% vs The false negative rate has to be rather high before it outweighs the cost of false positives!
- 26. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% Cost of False Positives when 留 = 67% mindless optimism rational optimismvs
- 27. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% mindless optimism rational optimismvs
- 28. Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I 硫 =20% mindless optimism rational optimismvs
- 29. Rational Optimism So? Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Negatives when 硫 =20% Cost of False Positives when 留 = 67% mindless optimism rational optimism vs Opportunity Costs (%) Probability (%) 20 15 10 5 0 25 - - - - - - 0 20 40 60 80 100 I I I I I I Cost of False Negatives when 硫 =20% mindless optimism rational optimism vs Early Development Late Development
- 30. Rational Optimism So? We fear false negatives, but the opportunity costs of false positives are huge. Terminating projects frees up R&D resources to pursue other prospects. Thinking about opportunity costs allows project teams to reframe termination decisions as prospective future gains.
- 31. Rational Optimism Lendrem D.W., Lendrem B.C., Peck R.W., Senn S.J., Day, S., Isaacs J.D. (2015), Progression Bias and Rational Optimism in Research & Development, Nat. Rev. Drug Discov., In press NTD p 1-硫1 1-硫2 1-硫3 1-硫4 1-硫5 1-硫6 1-硫7 1-硫8 硫8 硫7 硫6 硫5 硫4 硫3 硫2 硫1 1-p 留1 留2 留3 留4 留5 留6 留7 留8 1-留8 1-留7 1-留6 1-留5 1-留4 1-留3 1-留2 1-留1
- 32. Rational Optimism Richard Peck (Roche) Stephen Senn (CRP Sant辿) Simon Day (MHRA) John Isaacs (MRG) John Loughlin (MRG) NIHR Newcastle BRC
- 33. Rational Optimism Richard Peck (Roche) Stephen Senn (CRP Sant辿) Simon Day (MHRA) John Isaacs (MRG) John Loughlin (MRG) Clare Lendrem (AHG) NIHR Newcastle BRC
- 34. fin
Editor's Notes
- #8: As a result, the industry became really slick at delivering late-stage failures to the market place, precipitating the current R&D productivity crisis.
- #9: As a result, the industry became really slick at delivering late-stage failures to the market place, precipitating the current R&D productivity crisis.
- #11: 10