In 2021, I made a terrible business decision based on three customer conversations. All three people had told me something positive about an idea. I was excited. I started building.
Two months later, I went back to review my notes. The “positive signals” were:
- “That’s an interesting concept.” (Politeness, not interest.)
- “I could see someone using that.” (They wouldn’t use it themselves.)
- “Yeah, the market is definitely moving that way.” (Commentary on trends, not personal need.)
Had I written these quotes down immediately with honest annotations, I would have seen the weakness in the signal. Instead, my memory edited the conversations to match what I wanted to hear. My brain turned mediocre feedback into strong validation because that’s what brains do — they confirm existing beliefs.
This is why I now keep a validation journal. Not a diary. Not a notebook of scattered thoughts. A structured tracking system that forces honest interpretation of evidence and prevents memory from rewriting history.
Why Memory Is Your Worst Validation Tool
Human memory is reconstructive, not reproductive. We don’t record experiences like a camera. We rebuild them each time we recall them, and the rebuilding process is influenced by our current beliefs, desires, and emotional state.
In the context of validation, this means:
- Positive signals get amplified in memory (“she was really excited” when she was actually polite)
- Negative signals get minimized (“he was probably just having a bad day”)
- Ambiguous signals get categorized as positive (because you want them to be)
- The timing and context of signals get distorted (“I think that was the third person who said that” when it was actually the only person)
I’ve tested this with founders in the accelerator. I ask them to describe their customer interview findings from memory, then play back the recording. The gap between their recollection and reality is consistently startling. Founders recall conversations as more positive than they were. Every time.
The validation journal solves this by capturing data at the point of collection, not at the point of decision. When you write down exactly what someone said, in their words, within an hour of the conversation, the data is clean. When you try to recall it three weeks later, the data is corrupted by three weeks of wishful thinking.
The Journal Structure
Here’s the exact template I use. It’s a spreadsheet with one row per data point and the following columns:
Date: When the data was collected.
Source: Who said it or where it came from. (Customer interview, community post, sales call, support ticket, etc.)
Raw data: The exact quote or observation. No interpretation. No paraphrasing. Exactly what was said or observed.
Category: Problem signal, solution signal, pricing signal, competitive signal, or objection.
Strength: Strong (behavioral evidence or money), moderate (specific stories or emotions), or weak (opinions or generalities).
My interpretation: What I think this means. (This column is deliberately separate from the raw data.)
Counter-interpretation: What this could mean if I’m wrong. (This is the most important column.)
The counter-interpretation column is what makes the journal work. For every piece of evidence, you’re forced to consider the alternative explanation. “She said she’d buy this” — counter-interpretation: “She was being polite because I was clearly excited.” “Three people mentioned this problem” — counter-interpretation: “I was asking leading questions that prompted the topic.”
It feels excessive. It takes discipline. And it has saved me from at least four bad decisions that I know of, probably more that I’ll never count.
How to Use the Journal for Go/No-Go Decisions
After two to three weeks of validation work, your journal should have 30-50 entries. Here’s how I analyze them.
Step 1: Filter by strength.
Remove all “weak” entries from your analysis. Opinions and generalities don’t count. Only look at “strong” (behavioral evidence, money) and “moderate” (specific stories, emotions) entries.
Step 2: Count the signals.
How many strong positive signals do you have? How many strong negative signals? What’s the ratio?
In my experience:
- 5+ strong positive signals with 0-1 strong negatives: green light, start building
- 3-4 strong positives with 1-2 strong negatives: yellow light, investigate the negatives further
- Fewer than 3 strong positives, or 3+ strong negatives: red light, pivot or kill
Step 3: Read the counter-interpretations.
Go through every counter-interpretation you wrote. Are any of them more plausible than your primary interpretation? If so, your evidence is weaker than it appears.
This is the hardest step because it requires intellectual honesty. I sometimes ask a friend or advisor to read the counter-interpretations and tell me which ones they find convincing. An outside perspective catches self-deception that I can’t catch myself.
Step 4: Make the decision.
Based on the filtered, counted, and counter-checked evidence, decide: build, pivot, or kill. Write the decision and the reasoning in the journal. This creates accountability — you can review the decision later and learn whether your evidence-reading skills are improving.
The whole analysis takes about an hour. One hour of structured thinking to prevent months of building the wrong thing. That’s a trade I’ll make every time.
Pattern Recognition Across Multiple Ideas
The validation journal has a secondary benefit that becomes valuable over time: pattern recognition.
After journaling through several business ideas, you start seeing patterns in your own validation behavior. Maybe you consistently over-weight positive signals from people in your network. Maybe you consistently ignore pricing objections. Maybe you tend to stop interviewing too early, before negative signals have a chance to emerge.
I reviewed my journals from four different validation rounds and discovered that I had a consistent blind spot: I was treating “expressions of frustration with the status quo” as strong buying signals when they were actually just venting. People who are frustrated enough to complain are not necessarily frustrated enough to switch. I was conflating “will complain about it at dinner” with “will pay to fix it.”
Once I identified this pattern, I adjusted my signal-strength calibration. Frustration alone became a “moderate” signal. Frustration plus demonstrated switching behavior (they’ve already tried alternative solutions) became “strong.” This one adjustment dramatically improved the accuracy of my go/no-go decisions.
You can only see these patterns if you have records. If you validate by gut feel and memory, every round is a fresh start. If you validate with a journal, every round builds on the last.
The founders I’ve worked with who adopt this system consistently make better pivot/persevere decisions because they have actual data to reference instead of remembered impressions.
Digital vs. Physical: What Works
I’ve tried both digital and physical validation journals. Here’s what I’ve found.
Digital (spreadsheet or Notion database):
- Better for analysis and filtering
- Better for long-term storage and pattern recognition
- Easier to share with advisors or co-founders
- Worse for in-the-moment capture during conversations
Physical (notebook):
- Better for capturing notes during or immediately after conversations
- Better for counter-interpretation brainstorming (something about handwriting activates different thinking)
- Worse for analysis and pattern recognition
- Easy to lose or damage
My current system: I take physical notes during or immediately after conversations, then transcribe the key data points into a digital spreadsheet within 24 hours. The physical capture preserves accuracy. The digital format enables analysis.
The specific tool doesn’t matter. I’ve used Google Sheets, Notion, Airtable, and a plain notebook. What matters is that you do it consistently and that your structure includes the counter-interpretation column. Without that column, you’re just keeping organized notes. With it, you’re building an intellectual honesty practice.
When Not to Journal
The journal is a validation tool, not a life management system. Here’s when I don’t use it:
After the go/no-go decision is made. Once you’ve decided to build, stop journaling validation data and start tracking revenue and growth metrics instead. The journal served its purpose.
When you’re procrastinating. If journaling becomes a way to feel productive without actually talking to customers or testing ideas, it’s become part of the preparation trap. The journal records field data. If you’re not in the field, there’s nothing to record.
When the evidence is already overwhelming. If five people have already pre-paid for your product, you don’t need a journal entry to tell you there’s demand. Sometimes the signal is so clear that structured analysis adds no value.
The journal is for ambiguous situations — when you have some evidence but aren’t sure what it means. When things are clearly working or clearly not working, trust the obvious and act.
A Real Journal Example
Let me share an anonymized excerpt from an actual validation journal I kept for a digital product idea in 2023.
| Date | Source | Raw Data | Category | Strength | Interpretation | Counter |
|---|---|---|---|---|---|---|
| Mar 3 | Interview #4, SME owner | ”I spend every Sunday night doing this manually and I hate it” | Problem | Strong | Acute recurring pain | Could be venting, may not prioritize fixing it |
| Mar 5 | Reddit r/smallbusiness | 12 upvotes on a post asking for exactly this solution | Problem | Moderate | Others have the same problem | Reddit upvotes don’t indicate willingness to pay |
| Mar 7 | Interview #7, freelancer | ”I use a spreadsheet for this, it takes me 2 hours a week” | Problem | Strong | Existing workaround indicates real problem | May be comfortable with their spreadsheet |
| Mar 8 | Interview #8, agency owner | ”We already have a tool for this, it’s fine” | Competitive | Strong negative | Market may be served | Their tool might be tolerated, not loved |
| Mar 10 | Pre-sell attempt | 3 out of 40 purchased at €29 | Pricing | Strong | 7.5% conversion from targeted list | Small sample, could be friends/supporters |
After the analysis: 3 strong positives, 1 strong negative, 1 strong revenue signal. My decision: proceed with a narrow focus on SME owners and freelancers (who had the problem acutely) and avoid agencies (who had existing solutions). The journal directly shaped the niche targeting decision.
Key Takeaways
- Memory rewrites validation data to confirm your beliefs. A journal captures evidence at the point of collection, before wishful thinking corrupts it.
- The counter-interpretation column is the most important element. For every piece of evidence, write down what it means if you’re wrong.
- Use a consistent structure: date, source, raw data, category, strength, interpretation, counter-interpretation.
- Analyze with the three-step process: filter by strength, count signals, read counter-interpretations. Make go/no-go decisions based on the filtered evidence.
- The journal builds pattern recognition over time. After several rounds, you’ll start seeing your own validation blind spots and can correct for them.
