This document summarizes a presentation given to the Product Tank in Kansas City on unconscious choice and product management. It discusses how human decision making relies on mental shortcuts or heuristics that can lead to biases. It outlines several cognitive biases and heuristics, like the planning fallacy and halo effect, that can negatively impact product decisions if not accounted for. The presentation advocates using slow, deliberate thinking to overcome biases by tactics like considering alternative hypotheses, seeking outside perspectives, and separating ideas to avoid halo effects of order. The goal is to move from being a victim of cognitive biases to intentionally using critical thinking techniques to improve product decisions.
3. Most people do not
take the trouble to
think through [a]
problem.
Intelligence is not
only the ability to
reason; it is also the
ability to find
relevant material in
memory and to
deploy attention
when needed.
4. Heuristic #1: PRIMING
Heuristic #2: COGNITIVE EASE
Heuristic #3: COHERENT STORIES (ASSOCIATIVE COHERENCE)
Heuristic #4: CONFIRMATION BIAS
Heuristic #5: THE HALO EFFECT
Heuristic #6: JUDGEMENT
Heuristic #7: SUBSTITUTION
Heuristic #8: AFFECT
Heuristic #9: THE LAW OF SMALL NUMBERS
Heuristic #10: CONFIDENCE OVER DOUBT
Heuristic #11: THE ANCHORING EFFECT
Heuristic #12: THE AVAILABILITY HEURISTIC
Heuristic #13: AVAILABILITY CASCADES
Heuristic #14: REPRESENTATIVENESS
Heuristic #15: THE CONJUNCTION FALLACY
Heuristic #16: OVERLOOKING STATISTICS
Heuristic #17: OVERLOOKING LUCK
Heuristic #18: INTUITIVE PREDICTIONS
19. Heuristic #1: PRIMING
Heuristic #2: COGNITIVE EASE
Heuristic #3: COHERENT STORIES
Heuristic #4: CONFIRMATION BIAS
Heuristic #5: THE HALO EFFECT
Heuristic #6: JUDGEMENT
Heuristic #7: SUBSTITUTION
Heuristic #8: AFFECT
Heuristic #9: THE LAW OF SMALL NUMBERS
Heuristic #10: CONFIDENCE OVER DOUBT
Heuristic #11: THE ANCHORING EFFECT
Heuristic #12: THE AVAILABILITY HEURISTIC
Heuristic #13: AVAILABILITY CASCADES
Heuristic #14: REPRESENTATIVENESS
Heuristic #15: THE CONJUNCTION FALLACY
Heuristic #16: OVERLOOKING STATISTICS
Heuristic #17: OVERLOOKING LUCK
Heuristic #18: INTUITIVE PREDICTIONS
Heuristic #19: THE NARRATIVE FALLACY
Heuristic #20: THE HINDSIGHT ILLUSION
Heuristic #21: THE ILLUSION OF VALIDITY
Heuristic #23: TRUSTING EXPERT INTUITION
Heuristic #24: THE PLANNING FALACY
Heuristic #25: THE OPTIMISTIC BIAS
Heuristic #26: OMITTING SUBJECTIVITY
Heuristic #27: THEORY-INDUCED BLINDNESS
Heuristic #28: LOSS AVERSION
Heuristic #29: THE ENDOWMENT EFFECT
Heuristic #30: LOSS AVERSION
Heuristic #31: THE POSSIBILITY EFFECT
Heuristic #32: THE CERTAINTY EFFECT
Heuristic #33: THE EXPECTATION PRINCIPLE
Heuristic #34: OVERESTIMATING THE LIKELIH
Heuristic #35: THINKING NARROWLY
Heuristic #36: THE DISPOSITION EFFFECT
Heuristic #37: THE SUNK COST FALLACY
Heuristic #38: FEAR OF REGRET
Heuristic #39: IGNORING JOINT EVALUATION
Heuristic #40: IGNORING FRAMES
Heuristic #41: IGNORING OUR TWO SELVES
Heuristic #42: THE PEAK END RULE
Heuristic #43: DURATION NEGLECT
Laws
20. Going to court.
Unconscious and conscious product
managementVictim of the thinking laws?
(Thinking Fast Errors)
Thinking above the law?
(Thinking Slow Master)
22. The Planning Fallacy Law
Research shows we are overly optimistic in forecasts
and have an illusion of control
Studies:
- Railroad Projects: For a span of 40 years, 90% railroads projects
overestimated usage by 106% and underestimated cost by 45%
- Kitchen Remodels: estimate $20K more expensive than estimated
- CFO Estimates on S&P Performance: 80% confidence, slightly above 0%
correlations with actual performance and estimates
The story that breeds overconfidence
23. Many fails: I know the stats and we
just take more shots, faster, to get to
success
Ignore the stats of failure: We will be
different than the 90% of ideas that
fail
The Planning Fallacy LawProduct Management Implications
Victim of the Law Above the Law
Overly optimistic in forecasts and have an illusion of control
% Intent: Plan to spend X% designing
the new or in customer development &
discovery... then Fridayrepeat
Believe forecast of success on hope:
The market is $500m, if we can get
just 1%, and hockey stick users
Pad your agile roadmap: Lets add an
extra month in there, just in case
Intentional %: execute product thinking
goals like shipping code
Equal transparency: Publish
possibilities of success and failure
equally on future plans (roadmaps,
profit, estimates, & adoption)
personally and with team
24. Many fails/experiments: I know the stats and
we just take more shots, faster to get to
success
The Planning Fallacy LawTactics for Mastery
Above the Law
Equal transparency: Publish possibilities of
success and failure equally on future plans
(roadmaps, profit, adoption)
Dual Idea Perspective: Add a competing hypothesis at the beginning
Find an Outside View: Baseline with external forecasts/studies and customize to
avoid recycling internal trash data
Pre-mortems: Team writes a story of that product team being a disaster
(especially leadership or clients)
25. The Halo Effect Law
Sequence matters and impression is lazy
Studies:
- Solomon Asch sequencing
- Teacher essay grading
exaggerated emotional coherence
26. Solomon Asch Research
What do you think of Alan and Ben?
Alan is intelligent, industrious, impulsive, critical,
stubborn, and envious.
Ben is envious, stubborn, critical, impulsive,
industrious, and intelligent.
27. Solomon Asch Research
What do you think of Alan and Ben?
Alan is intelligent, industrious, impulsive, critical,
stubborn, and envious.
Ben is envious, stubborn, critical, impulsive,
industrious, and intelligent.
28. The Halo Effect Law
Sequence matters and impression is lazy
Studies:
- Solomon Asch sequencing
- Teacher essay grading
exaggerated emotional coherence
29. Strategic facilitator: pre-plan meetings
with influencers to build over/under
confidence
Scrub input: intake of ideas separate
from each other (decorrelate)
Victim of meeting pirates: allows meetings
to get hijacked by first to speak (feedback,
strategy pitches, scrum)
The Halo Effect LawProduct Management Implications
Victim of the Law Above the Law
Sequence matters and impression is
Writing off sources due to history: loud
customers, lame coworkers, old methods
Trojan horse interviewees: like agreement
and like you, so buzz from the beginning
Blink pro: intentionally owning and
omitting first impression bias
30. Solo-solutions: intake of ideas separate from
each other (decorrelate)
The Halo Effect LawTactics for Mastery
Above the Law
Strategic facilitator: pre-plan meetings with
influencers to build over/under confidence
Wisdom of Crowds Method1: for prioritization, value forecasting, etc. intake all
of the data/feedback decorrelated, and average them (omit outliers) for estimate
Independent Thought First: before an issue is discussed, all members of the
committee are asked to write a brief summary of their position (approvals,
prioritization, brainwriting ideation)
1Wisdom of Crowds, James Surowiecki, 2004
31. Unconscious Product Management
Which one do you find yourself being?
Ignore the possibility of failure
Hope > data
Only using internal data
Padding roadmap
Victim of halo effect meetings
Labeling your areas of input
Working with others like you
Thinking Fast Errors Intentionally Thinking
Slow
33. End of Year Review Activity
Think of a challenge for yourself on using thinking
slow tactics to beat bias, overconfidence, or error in
choice in 2017
Write it down a short note to your future self
Mailed to you at end of Q1
Product Thinking Velocity: % intent v. intentional %, personal sprint cycle
34. Combine overconfidence with critical thinking.
Increase clear thinking by increasing doubt radar.
Be aware and be intentional.
#2: The Pm court
Judge danielkahneman
Accused of doing x, when said this
Guilty of article #34 this of that
Letter with a stamp, a card with the 4.
Resolution
Mail it to you at the end of 1Q
You are on trial
#3: Background
Kahnemans research explores cognitive science, uncertainty, illogical thinking, behavioral economics
Resolution: humans assume certain things automatically without having thought thru them carefully
#4: Resolution: humans assume certain things automatically without having thought thru them carefully
Most people do not take the trouble to think through [a] problem, (page 45). Intelligence is not only the ability to reason; it is also the ability to find relevant material in memory and to deploy attention when needed,
#5: Broken into 38 different heuristics, these are the heuristics lead to a mudded thinking
Its a huge book, and 10 different studies for each of these.
#6: System 1: automatically, involuntary
Drive, read a facial expression, take a shower
System 2: deliberating, solving, reasoning, focus, considering other data weighing the options
Math problem, complicated form, where to invest money
These two systems often conflict with one another. System 1 operates on heuristics that may not be accurate. System 2 requires effort evaluating those heuristics and is prone error.
#12: System 2: deliberating, solving, reasoning, focus, considering other data, weighing the options
Math problem, complicated form, where to invest money
#14: These two systems often conflict with one another.
Think Fast: operates on heuristics that may not be accurate. Snap judgement. Incorrect read.
Think Slow: Requires effort that you may not have energy for, or choose to use.
Go with our gut.. Error in judgement.
#16: These two systems often conflict with one another.
Think Fast: operates on heuristics that may not be accurate.
Think Slow: Requires effort evaluating those heuristics
Go with our gut.. Error in judgement.
#19: Honorable Judge Daniel Kahneman presiding
You are on the stand. Breaking or violating
3 Examples
#20: Heuristics are the laws of human psychology. As a product manager you can break these laws or be over them.
Break a law: jump to a conclusion, only consider on reference or go based on your gut
Above the law: you know psychology and are intentional about how you approach it
#21: Heuristics are the laws of human psychology. As a product manager you can break these laws or be over them.
Break a law: jump to a conclusion, only consider on reference or go based on your gut
Above the law: you know psychology and are intentional about how you approach it
#22: Honorable Judge Daniel Kahneman presiding
You are on the stand. Breaking or violating
3 Examples
#23: Duke
Plans are unrealistically close to best case scenarios
Desire to get the plan approved (best case) and the sunk cost falicy that projects are not abandoned
Extra:
Causes to accept that a statement is true
When information is scarce, system 1 is designed to jump to conclusions from little evidence. but it does not know how far it jump is
When presented with information, we assume everything is there to make a decision (What-you-see-is-all-there-is)
#24: Takeaway: Avoid delusional optimism and use rational weighting of gains, losses, and probabilities
Product Managers: Breaking the Law
Believe your own external marketing from the pitch deck for future forecasting
Grow 10% of $100m market, or hockey sticking new customers
Use buffer or padding for roadmap or agile velocity (self-fulling prophecy)
Feature driven planning/decisions with no
Believe that you are special and will not fail like the others
Above:
Many fails: know the stakes and maximize fail ratio
Publish possibilities of success and failure equally on future plans (roadmaps, financial projects, tech)
#25: Takeaway: Avoid delusional optimism and use rational weighting of gains, losses, and probabilities
Product Managers: Breaking the Law
Believe your own external marketing from the pitch deck for future forecasting
Grow 10% of $100m market, or hockey sticking new customers
Use buffer or padding for roadmap or agile velocity (self-fulling prophecy)
Feature driven planning/decisions with no
Believe that you are special and will not fail like the others
Above:
Many fails: know the stakes and maximize fail ratio
Publish possibilities of success and failure equally on future plans (roadmaps, financial projects, tech)
#26: Result: The warm emotion we feel toward a person, place, or thing predisposes us to like everything about that person, place, or thing.
Story: If you are told that you meet Joan at a party and she is very likable. When asked, is she charitable? Respondents say yes.
Other examples: teachers grading essays or the worst yet executives in meetings all day
#29: Result: The warm emotion we feel toward a person, place, or thing predisposes us to like everything about that person, place, or thing.
Story: If you are told that you meet Joan at a party and she is very likable. When asked, is she charitable? Respondents say yes.
Other examples: teachers grading essays or the worst yet executives in meetings all day
#30: Executives well versed in all day meetings know that open discussion give weight to early, assertive opinions (line up behind)
Examples : product pitches, roadmap meetings, feature tradeoff discussions
Writing Off Sources: discredit ideas of based on historical impression of the source
Drowning out or anti-behavior against sources of first impression of low value (grading papers example) : loud customers drown out, lame coworker brainstorms, old methods
Suede Interviewing: think you are more likely to like team members who agreed, furthermore use product buzz words around best practices
We just lean startup, that guerrilla design thinking process to a user-centric MVP
#31: Sequence matters, be agnostic of impressions and nothing at the end is treated the same as the beginning (new customers, employees, ideas)
Weight of first impressions, renders post items irrelevant
#33: These two systems often conflict with one another.
Think Fast: operates on heuristics that may not be accurate. Snap judgement. Incorrect read.
Think Slow: Requires effort that you may not have energy for, or choose to use.
Go with our gut.. Error in judgement.
#35: Combat over confidence by basing our beliefs not on subjective feelings but critical thinking.
Increase clear thinking by giving doubt and ambiguity its day in court.
Be aware and be intentional.
#38: Broad Thinking: as in financial market trading, find bets where there is a small win and then maximize the amount of bets (1000 bets)
Broadly is non-intuitive. Its a System 2 task that takes work.
We therefore are wired by System 1 to think irrationally economically (saying no to easy money).
Of 25 divisions, offered $100 loss or $200 gain. They said No. The CEO said yes to all, because there were 25 chances.
#39: We therefore are wired by System 1 to think irrationally economically
#40: Broad Thinking: as in financial market trading, find bets where there is a small win and then maximize the amount of bets (1000 bets)
Broadly is non-intuitive. Its a System 2 task that takes work.
We therefore are wired by System 1 to think irrationally economically
Of 25 divisions, offered $100 loss or $200 gain. They said No. The CEO said yes to all, because there were 25 chances.
#41: Saying no to risks until they are close to a sure thing
Using only internal data for building a business case
Looking at technology investments as only benefiting their product or the whole companynothing in between
Tactical suggestion:
Create risk policies for the company that give all leaders a framework for making decisions and avoid loss aversion
Never buying warranties, knowing the risk; Always
Pool risks with other product leaders to give an aggregate risk balance
Of 25 divisions, offered $100 loss or $200 gain. They said No. The CEO said yes to all, because there were 25 chances.