Research Consultant & Former Head of Research, Airbnb
RESEARCH STRATEGY
The Problem
User-Centered Performance vs. Real Impact
"Every time a PM comes to a researcher at the end of a product process and says, 'Can you just run a quick user study just to validate our assumptions,' that's user-centered performance. It's too late to matter. What they want is to check the box."
Most research is symbolic — checking a box rather than changing decisions
Research rarely drives real product impact at scale
Researchers want to be right, not to be wrong
The system is broken, not individual researchers
Research Framework
Three Levels of Research (And Why Middle Range Fails)
3
research tiers
70%
spent in middle
10%
of impact
Macro: Big picture, competitive landscape, concept car projects, aligned to annual planning
Middle: "How do users feel about X?" — interesting but unfocused, not tied to business OKRs
Micro: Specific, measurable, 48-hour turnaround, highest ROI per hour spent
The multimillion dollar buttonResearch revealed the CTA text scared users away from completing purchases. Changing seven characters added 1% conversion. That's micro research done overnight.
The Integration Problem
Research as Reactive Service vs. Strategic Partner
The vicious cycle: Researchers hired without being integrated → do reactive, low-impact work → executives conclude they're not valuable → layoffs → cycle repeats
The fix: Researchers in the room from day one, shaping questions, participating in decisions, driving the framing
The result: A great researcher embedded consistently drives product metrics, growth, and business impact
Most research teams are service functions, not strategic partners
Research's true role
Researchers are the repository of insights needed for growth. But they're rarely in the room where decisions are made.
The PM-researcher relationship
When PMs and researchers are deeply integrated, the researcher shapes the right question before work even starts.
The Five Tools
What Great Researchers Actually Have
1. Formative research — ethnographic, generative, open-ended, look ahead
2. Evaluative research — usability testing, usability research
3. Survey design — rigorous, scaled, gets you answers fast
4. Applied statistics — you can't AB test without basic stats
5. Technical skills — SQL, or now prompt engineering to work with your data
The evolution of researchQualitative research alone is no longer enough. Great researchers are multi-method, combining qual + stats + technical chops.
Tropes (Myths PMs Believe)
Research Misconceptions That Kill Impact
✗Research just slows us downINSTEAD →✓ Great research goes fast (48 hours to 1 week). Broken research at the end of a process is what slows you down.
✗I can do my own researchINSTEAD →✓ Talking to one user isn't research. It's idiosyncratic. Researchers turn garbage into signal and avoid bias.
✗AB test everything, numbers don't lieINSTEAD →✓ AB tests show what changed, not why. Research + data scientists = causal claims + deep understanding.
✗"We knew that already, that was obvious"INSTEAD →✓ Hindsight bias. Duncan Watts: "Everything is Obvious If You Already Know the Answer." We selectively remember and self-gaslight.