MModern Wisdom
← All episodes
Annie Duke17 October 2020

How To Make Better Decisions - Annie Duke - #233

5Frameworks
12Insights

Listen

Frameworks in this episode

Insights & moments

The myth-busts, hot takes, explainers, and tools worth keeping.

Myth Buster· 1

Myth Buster31:30

Resulting: why a good outcome doesn't mean a good decision

The biggest thing people get wrong is assuming a good decision yields a good result and a bad decision a bad result. Duke calls this 'resulting': over-indexing on the outcome to judge the underlying process. Running a red light and arriving safely doesn't make you a good decision-maker; a sound investment can still lose to volatility. Good decisions only reliably pay off in aggregate, not in the short run.

  • Resulting judges decision quality by the single outcome observed
  • This confuses being right with being accurate
  • Good decisions win in aggregate but can lose in the short run
  • Because we learn only from experience, resulting badly corrupts our lessons

the difference between being right and being accurate

Annie Duke · 31:30

essentially we over index on the outcome this is just resulting

Annie Duke · 32:30
#decision-making#cognitive-bias#outcomes#luck

Hot Take· 1

Hot Take29:00

The biggest danger is a strategy that's already winning

Duke argues the most dangerous position is having a strategy that wins, because winning removes your incentive to explore whether you could win more. Someone making 4% actively trading feels successful and never asks whether indexing would beat them. Beating zero is not the bar; the opportunity cost of a merely-fine strategy is invisible.

  • Winning discourages exploration of better alternatives
  • The worst outcome can be a small win that keeps you from a bigger one
  • Active investors making 4% may be losing to a simple index fund
  • Examine winning strategies, not just losing ones

the big danger is you have a strategy that's winning that's like the biggest danger that you can ever be in

Annie Duke · 29:00
#strategy#investing#explore-exploit#opportunity-cost

Explainer· 5

Explainer04:00

A decision is choosing which futures stay available to you

Duke defines a decision as a choice about which path to take, where the future branches into many possibilities. Choosing one road makes certain futures available and eliminates others. Critically, staying on your current path is also a decision, so the difference between commission and omission is just whether you actively enact change.

  • The future branches; a decision picks which branch you make available
  • Choosing one option rejects the futures the other options would have opened
  • Not changing anything is still a decision (omission vs commission)
  • Every choice carries the cost of the paths it forecloses

staying on the path that you're already in is also a decision

Annie Duke · 05:30
#decision-making#mental-models#optionality
Explainer27:00

The reversed-handlebar bike: why you must keep checking your gut

Duke uses learning to ride a bike as an analogy for how deliberate practice becomes automatic gut behavior. But if someone hands you a bike whose handlebars steer backwards, your gut makes you fall. Environments and markets change the same way, so even a well-honed gut needs periodic re-examination, and sometimes a better method exists even when nothing has changed.

  • Deliberate skill eventually becomes effortless gut behavior, like cycling
  • When the world changes the rules, gut habits suddenly fail
  • Markets and environments shift, so gut calls need re-checking
  • A better strategy may exist even when the world hasn't changed

what happens when someone hands you a bike where now when you turn the handles it goes to the right it goes left

Annie Duke · 27:30
#intuition#learning#adaptation#mental-models
Explainer42:30

We misremember what we knew at the time we decided

Beyond over-indexing on outcomes, Duke notes people rewrite their own past knowledge, insisting 'I knew that would happen.' She recounts pitching the Clinton analysis to a newspaper editor who refused it because he 'already knew,' despite never having published it. This corrupted memory of prior beliefs makes it nearly impossible to build fidelity into a decision-review process.

  • Hindsight makes people believe they foresaw outcomes they didn't
  • An editor claimed prior knowledge yet had published nothing
  • You can't review a decision honestly if you can't recall what you actually believed
  • This is why recording forecasts at decision time matters

literally they're misremembering what their state of knowledge is

Annie Duke · 43:30
#hindsight-bias#memory#decision-review
Explainer48:00

When to move fast: options expire and information is the reward

Duke explains that going slow has two hidden costs. Options expire: waiting to ramp up PPE production, or to invest before a funding round closes, can remove a good solution entirely. And moving fast on many small, reversible actions generates information about how the world and you respond, building better models for the genuinely big decisions. The core rule: if the downside isn't that bad, go fast.

  • Slowness can let a good option disappear entirely, as with early COVID PPE
  • Some preferences can only be learned by trying, not by researching
  • Fast, cheap actions build better models of the world for big decisions
  • The governing question is simply: how bad is the bad?

if the bad isn't that bad it doesn't really matter you should probably just like move fast and break things

Annie Duke · 52:00
#speed#opportunity-cost#optionality#decision-making
Explainer74:00

Why 'a real possibility' is a useless, unaccountable phrase

Duke attacks vague probability language. A Mauboussin survey found people hearing 'a real possibility' interpret it as anywhere from 20% to 80%. Such words fake agreement and let people weasel out of accountability later, since the target was never pinned down. She insists on numeric probabilities with lower and upper bounds, and on defining loaded words like 'spectacular' before treating them as shared.

  • 'A real possibility' means 20% to 80% to different listeners
  • Vague terms create the illusion of agreement that isn't there
  • Weasel words let people dodge accountability for their forecasts
  • Demand numeric probabilities and bounded ranges instead

somewhere between 20 and 80 of the time ... it's like they're totally useless words

Annie Duke · 74:30
#communication#probability#forecasting#precision

Story· 2

Story00:30

How a poker sabbatical turned Annie Duke into a decision scientist

Annie Duke was an ABD cognitive-science PhD student at UPenn heading for a professorship when illness forced a year off. She started playing poker to support herself, and the 'meantime' became an 18-year career. Eight years in she realized poker's high-stakes, high-luck, imperfect-information environment was a live laboratory for how people learn to decide.

  • Studied cognitive science at Penn, went ABD before her thesis defense
  • Fell into poker during a forced year off and stayed 18 years
  • Retired from poker in 2012 to consult, speak, and write on decision strategy
  • Poker mirrors nearly every real decision: lots of luck and hidden information

the mean time turned into like 18 years as it does

Annie Duke · 01:30
#biography#poker#cognitive-science#decision-making
Story35:30

The Hillary Clinton rust-belt myth as resulting in the wild

Duke uses the 2016 election to show resulting at scale. The consensus that Clinton 'bungled' Michigan, Wisconsin, and Pennsylvania rests on a loss margin of roughly 80,000 votes, yet the first critical articles appeared only on November 9th, the day after. If it had truly been an obvious error, given how many pundits weigh in, people would have said so beforehand. Decisions must be judged on knowledge available at the time.

  • Clinton lost three states by a combined ~80,000 votes
  • Critiques of her rust-belt strategy appeared only after the result
  • If it were an obvious mistake, commentators would have flagged it in advance
  • Judge decisions on information available at the moment they were made

the first one of them appears on november 9th which is the day after the election

Annie Duke · 39:30
#resulting#hindsight-bias#politics#decision-making

Q&A· 1

Q&A15:00

Why you can't rely on your gut as a decision tool

Chris asks why anyone needs a decision process rather than just their gut. Duke says the gut can produce good outcomes the way a coin flip sometimes wins, but it isn't a tool: it's not repeatable, examinable, or teachable, and it's where cognitive bias and noise live. Without those properties you can't judge or improve your decisions.

  • Good gut outcomes are often just luck, like winning a coin flip
  • A real decision tool is repeatable, examinable, teachable, and accountable
  • The gut hides all the bias and noise that frustrate decisions
  • You can still go with your gut on low-risk calls once you know what good looks like

your gut is where all the cognitive bias lives your gut is where all the noise lives

Annie Duke · 17:00
#decision-making#intuition#cognitive-bias

Tool· 1

Tool88:30

The Alliance for Decision Education: teaching decision skills in K-12

Duke closes by highlighting the Alliance for Decision Education, a nonprofit she co-founded to bring decision science, probabilistic thinking, and better-habit formation into K-12 curricula. The premise is that a large body of work exists for helping adults decide well, but it is absent from school, and the motto is that better decisions lead to better lives and a better society.

  • Nonprofit co-founded by Annie Duke
  • Aims to pull decision science into K-12 curricula
  • Draws on work from Kahneman, Tetlock, James Clear, and Morgan Housel
  • Motto: better decisions lead to better lives and a better society

our motto is better decisions lead to better lives in a better society

Annie Duke · 89:30
#education#decision-making#nonprofit

Takeaway· 1

Takeaway81:00

Disagreement is a gift: you don't have to end up agreeing

Duke reframes disagreement as the most valuable thing in a conversation. Much of our stress comes from believing the goal is to agree by the end; it isn't. Exposing a difference in belief either moderates both views toward accuracy, or reveals you're flatly wrong and lets you reverse a belief that would have poisoned every future decision. Even being right benefits you, because articulating your rationale reveals how shakily you actually hold it.

  • The goal of a disagreement is to become better informed, not to agree
  • Three outcomes: both moderate, you're wrong and correct course, or you're right and learn why
  • Reversing a wrong belief improves every future decision it would have touched
  • Explaining a belief (like why the earth is round) exposes how little you truly understand it

there is no reason that you need to agree at the end of the interchange

Annie Duke · 83:00
#communication#disagreement#beliefs#learning