KnowledgeSketches|← All EpisodesLenny’s PodcastResearch papers
Based on Lenny's Podcast data
Lenny's Knowledge Sketch

The 100-Person Lab That Became
AI's Secret Weapon

Edwin Chen
Founder & CEO, Surge AI; teaches AI right from wrong
DEC 7 2025
The Thesis

Human Feedback IS
the AI Moat

HUMAN JUDGMENTRLHFALIGNED AI
  • RLHF quality determines model quality more than architecture choices
  • The labelers are not a commodity, domain expertise + good judgment = rare combination
  • Surge built the expert network that taught AI what's good and bad
  • The invisible layer: most people don't know this work exists, but it determines what AI does
Framework

How RLHF Actually Works

RAW MODELHUMAN FEEDBACKALIGNED MODEL
$1B+
Surge revenue last year
<100
people on the team
10×
better product vs competitors
  • Not just "is this good?", expert raters understand nuance, context, and edge cases
  • Domain expertise matters: physicists probe their own research, teachers build lesson plans, coders stress-test the model at work
  • Scale + quality: Surge's bet is expert-quality feedback at production scale
  • The eval loop: better feedback → better model → harder evals → need better feedback
The hidden variableThe most important people in AI development are often not the researchers or the engineers, they're the expert labelers.
The RLHF Industry

What Most People Don't Know

  • Quality: Bad RLHF produces confidently wrong, helpful-sounding AI
  • Specialization: Code RLHF, reasoning RLHF, safety RLHF require different experts
  • The problem: Most AI companies underinvest in RLHF quality, and it shows
Why Claude feels different

Claude's helpfulness and safety balance comes from deliberate, expert human feedback, not just prompting.

The eval arms race

As models get better, the humans rating them need to be smarter. The bar keeps rising.

Playbook

Think About AI Quality

  • Ask: where does the AI's "judgment" come from? Trace it to the training data.
  • Build eval datasets with real domain experts, not just crowdsourced workers
  • The best product insight: what does the AI confidently get wrong in your domain?
  • Invest in evals before you invest in prompts, you can't improve what you can't measure
Edwin's missionSurge exists because good AI requires good humans. The quality of human feedback is the quality of the AI.
Contrarian

AI Quality Myths

✗Bigger models need less human feedbackINSTEAD →✓ Bigger models need better human feedback. The bar rises with model capability.
✗Crowdsourced labeling is fineINSTEAD →✓ Crowdsourced labeling is fine for simple tasks. Expert labeling is required for nuanced judgment.
✗RLHF is just annotationINSTEAD →✓ RLHF is judgment at scale. Edwin calls it closer to raising a child than data labeling, and his best raters are experts at the top of their fields.
✗Models will soon self-improve without humansINSTEAD →✓ Models self-improve with AI feedback loops. But the root signal is still human values.
Based on Edwin Chen's episode on Lenny's Podcast. All ideas on this page are from the episode.Watch on YouTubeFollow @echen on X
GO DEEPER IN THE EPISODE
04:48 AI’s role in business efficiency07:08 Building a contrarian company08:55 An explanation of what Surge AI does09:36 The importance of high-quality data13:31 How Claude Code has stayed ahead17:37 Edwin’s skepticism toward benchmarks21:54 AGI timelines and industry trends28:33 The Silicon Valley machine33:07 Reinforcement learning and future AI training39:37 Understanding model trajectories41:11 How models have advanced and will continue to advance42:55 Adapting to industry needs44:39 Surge’s research approach48:07 Predictions for the next few years in AI50:43 What’s underhyped and overhyped in AI52:55 The story of founding Surge AI01:02:18 Lightning round and final thoughts
Every sketch here began as a transcript.

Send us yours and we’ll draw it. $1 a sketch, or $49 for all 11 formats.

Sketch mine →
𝕏︎ X / Twitterin LinkedIn📸 Instagram🔗 Copy link
0:00