When evidence grades add value
Use them when
- The thesis depends on private, incomplete, or conflicting information.
- Several sources may repeat the same original claim.
- The committee needs to see what is fact, inference, and open risk.
Do not misuse them
- As a numerical veneer over weak judgment.
- As a substitute for reading the underlying sources.
- To average away a contradiction that should remain visible.
A claim-level grading framework
1. Break the thesis into falsifiable claims
A thesis such as "this company has a durable data moat" is too broad to grade. Decompose it: exclusive data is generated through normal use; the data improves the product; the improvement affects customer choice; competitors cannot reproduce the loop economically; and the advantage persists as the market changes. Each link should be capable of being wrong.
2. Assess the source, not only the statement
Consider proximity, independence, directness, recency, incentives, method, and reproducibility. An audited filing may be strong for recognized revenue but weak for future demand. A customer interview is close to behavior but may represent one segment. Three articles repeating the same company announcement are one source, not three. Record the original source wherever possible.
3. Assign a confidence class
Established means multiple reliable, independent lines of evidence support the claim and material contradiction is absent. Probable means the balance of evidence supports it, with a meaningful gap or inferential step. Contested means credible evidence supports more than one interpretation. Unknown means available evidence cannot carry a responsible conclusion. These classes describe the current evidence, not permanent truth.
4. Preserve competing hypotheses
For important observations, write at least one alternative explanation. Rising gross margin might reflect scale, a favorable mix shift, deferred spending, or classification choices. Faster growth might indicate product advantage, unsustainable incentives, or a temporary channel effect. Identify the observation that would discriminate between explanations and search for it deliberately.
5. Connect evidence to priors and decision impact
Start with relevant base rates before company-specific adjustment. Then ask how strongly each piece of evidence should move the view. A credible claim with low decision impact may not deserve more work. A contested, load-bearing assumption deserves a decisive test, a valuation sensitivity, or protective terms. Evidence quality and decision materiality belong in the same register.
6. Pre-register updates and exits
State what would upgrade or downgrade each load-bearing claim and by when it should be observable. Separate monitoring signposts from kill criteria. A signpost changes conviction; a kill criterion changes the action. Recording them before the outcome reduces the temptation to reinterpret every result as consistent with the thesis.
Grading a customer-retention claim
A company claims "enterprise customers rarely leave." Management presents 96% gross revenue retention, two reference customers praise the product, and sales material calls switching expensive. The claim initially looks established.
The evidence register shows that both references were selected by management, the retention calculation excludes customers still in implementation, and one large contract dominates the cohort. Raw invoice data independently supports strong retention among mature customers, but renewal concentration and onboarding exclusions remain. The claim is graded probable, not established. An independent sample of churned customers and a cohort reconstruction are assigned high information value because they can change the downside case.
A practical evidence register
| Field | Question to record | Why it matters |
|---|---|---|
| Claim | What exactly must be true? | Prevents broad narratives from hiding weak links |
| Source and origin | Who observed what, when, and by what method? | Separates independent evidence from repetition |
| Confidence class | Established, probable, contested, or unknown? | Makes uncertainty visible to the reader |
| Alternative explanation | What else could produce this observation? | Protects against confirmation bias |
| Decision impact | What changes if the claim is wrong? | Prioritizes work by materiality |
| Update rule | What evidence upgrades, downgrades, or kills it? | Creates a monitorable thesis |
Common failure modes
Source-count inflation: repeated reporting is treated as independent corroboration. Authority substitution: a prestigious source is accepted outside its actual knowledge. Numeric theater: subjective scores are averaged to two decimal places. Missing disconfirmation: the memo contains only evidence management could have supplied. Confidence by prose: decisive language makes a weak claim feel strong. Stale evidence: an old operating reality is treated as current. No decision link: open questions accumulate without showing which one can change the action.
Memo evidence checklist
- The core thesis is decomposed into causal, falsifiable claims.
- Every material claim links to its originating source and date.
- Independent corroboration is distinguished from repetition.
- Management-selected evidence and incentives are disclosed.
- Competing explanations remain visible until evidence retires them.
- Confidence and decision impact are recorded separately.
- Open questions are ranked by the value of resolving them.
- Signposts, review dates, and kill criteria are pre-registered.
Limitations
Evidence grades require judgment and can create false comfort if the criteria are applied mechanically. Sources can be honest and still wrong; independent sources can share a hidden dependency; and unavailable evidence may remain unavailable. The framework does not remove uncertainty or produce a recommendation. It makes uncertainty auditable, focuses further research, and helps future reviewers distinguish a poor outcome from a poor decision process.
Related services and cases
Make the evidence carry its own weight
We can structure the thesis, evidence register, and review rules for a live decision.