030: Yes, founders trick themselves.
Unless they measure reliability.
đ Hi, Iâm Mike and Iâm obsessed with startups with no customers. This is the summary newsletter đŹ for Episode 030 of the Nascent podcast đ§ (Spotify, Apple, YouTube).
Yes, founders trick themselves when they analyze interviews, even though interviews are the most effective way to evaluate a startup idea. The catch is that founders struggle to analyze interviews reliably. In fact, until now, founders have never measured the reliability of their interview analysis so we have every reason to believe that the analyses are no better than randomly rolling dice. This drives foundersâ most common outcome: a year wasted on a bad startup idea. The entire goal of Nascent is to provide founders with a toolkit that dramatically reduces that time down to days. Measurement of both interviews AND reliability is the key to better outcomes.
Judgments based on vibes result in lots of wasted effort. Founders today are kind of like medieval bakers sticking their hands in an oven to judge the heat. Sometimes the bread is burnt. Sometimes itâs raw, uncooked dough. Either way, the result was often a mess, at least until bakers began measuring temperature. Measurement unlocks improvement.
To that end, Iâm excited to share two concepts that will do for startups what Celsius and Fahrenheit did for baking: content analysis and Krippendorffâs Alpha. Most founders have never heard of these tools but they desperately need to.
Fundamentally, content analysis frames foundersâ process of analyzing interviews as a data transformation challenge. Founders go from raw, unstructured interview transcripts to specific, defined categories. Founders typically use interviews to decide whether the interviewee is a potential customer (Category A) or not a customer (Category B), but this approach is ad hoc. Klaus Krippendorff developed content analysis as âa research technique for making replicable and valid inferences from texts.â What this means for founders is that we can structure and measure our analysis of discovery interviews. Specifically, Krippendorffâs Alpha measures how reliably founders execute the data transformation (âthe inter-rater reliabilityâ) from completely random (K-Alpha = 0) to perfect reliability (K-Alpha = 1). Krippendorffâs Alpha measures how much founders are tricking themselves, when the interview analysis feels meaningful but is no better than rolling dice. đČ
Analyzing interviews is not an inherent ability, but a learnable skill.
As far as I can tell, I am the first person to apply content-analysis techniques to customer discovery interviews to evaluate startup ideas. Specifically, last year I measured reliability of 79 âhuman ratersâ analyzing discovery interviews. Before training, their analysis was essentially random â their K-Alpha was 0.1. They didnât really know what they were searching for, even though I told them to look for emotional pain (i.e., interviewees who might care about the startup idea). After I trained them, their K-Alpha jumped dramatically to 0.8, meaning their analysis was much more reliable.
When someone tells a baker that the oven is hot, they donât accept it at face value. The baker asks for the temperature. In that same way, as a founder, if someone tells you that an interview means something, I hope youâll ask them: How did you measure it? And if they didnât use content analysis or something similar, itâs likely that theyâre tricking themselves.
Over the coming posts, Iâll expand on the big ideas in this episode: analyzing interviews as data transformation, content analysis techniques and the details of how I measured the 0.1 â 0.8 jump in Krippendorffâs Alpha.
Many thanks to Paul Tepper for letting me know about content analysis and providing feedback on how Iâm applying it to Nascent.
Mike supports founders of pivoting startups who want to avoid wasting another year on a bad idea. I offer personalized workshops that train your team to reliably analyze discovery interviews. Book a call with me at NascentIdea.com.
For the past 10 years, Iâve been building Nascent as the strategy for startups with no customers. As of 2026, Iâm publishing Nascent, a few ideas at a time, in regular newsletter posts and podcast episodes. This newsletter summarizes podcast Ep030. For a deeper dive, check out the podcast on Spotify / Apple Podcasts / YouTube.