Monadic Testing (The Need to Experiment)
Rerun the study in your own category with split groups instead of trusting what customers claim
- Difficulty
- Moderate
- Time to result
- ~weeks to results
- Steps
- 4
- Confidence
- 80%
Because behavioural findings shift with context — an effect proven in a bar may not hold on a website — and because many rest on a single study, you can't assume a tactic will work for you. The fix is to rerun the study in your own category. Crucially, don't ask customers directly: quizzed about whether a super-premium tier would change their choice, people insist they're rational and it wouldn't — stated preferences lie. Instead use a monadic test: randomise a representative sample into groups, show each a single offering (e.g. basic-vs-premium to one group, basic-premium-super-premium to another), hold everything constant except one variable, and attribute any performance difference to that change. Since these studies are public, the methodology is easy to copy. The asymmetry is favourable — winners run forever, losers get switched off within a week.
Origin
Extracted from Richard Shotton's discussion on Modern Wisdom, drawing on standard behavioural-science experimental method (monadic, between-subjects testing) applied to marketing.
Core principles
- 01Findings vary by context, so rerun studies in your own setting
- 02Never rely on stated preferences — wait for revealed behaviour
- 03Split audiences into groups; change one variable at a time
- 04Winning tests run forever (big upside); losing tests stop in a week (small downside)
How to run it
- 1
Choose a study to rerun
Select a published behavioural finding relevant to your product; the methodology is usually public.
Watch out Don't assume a result proven in one context (a bar) transfers to another (a website).
- 2
Randomise into groups
Split a representative sample into groups so no one compares options side by side.
- 3
Change one variable
Give each group a nearly identical experience, varying only the single element you're testing.
Pro tip 'Monadic' just means people aren't comparing side by side — don't overcomplicate it.
Watch out Never ask customers directly what motivates them; they'll deny being influenced.
- 4
Attribute the difference
Assign any gap in performance between groups to the one variable you changed.
Pro tip Run winners indefinitely; kill losers within a week — the asymmetry favours testing.
In the wild
To check whether adding a super-premium tier lifts mid-tier sales, don't ask customers — they'll swear it wouldn't sway them. Instead show one randomised group basic-vs-premium and another basic-premium-super-premium, keep everything else identical, and read the difference in behaviour as the effect of the extra tier.
→ Reveals the true behavioural effect that stated preferences would hide.
Common mistakes
Asking customers directly
People deny being influenced and insist they weigh price against benefit rationally, so stated preferences mislead — measure revealed behaviour instead.
Changing more than one variable
Vary several things at once and you can't attribute any result to a specific change.
Is it for you?
Best for
Any brand wanting to validate a behavioural tactic before rolling it out.
Not ideal for
Decisions too small or too fast to justify a controlled experiment.
From the transcript
“You cannot ask people directly... half the audience has seen one of the offerings... you change one variable between the two groups, and any difference…”
“If they work, you run them forever, massive upside. If they don't work, you run them for a week, turn them off, limited downside.”
From the episode
8 Psychology Hacks Behind The World’s Biggest Businesses - Richard Shotton - #626
Richard Shotton