Skip to main content

Chapter 5: Lumen Evidence Quality: Reading the Grades

Chapter 5: Lumen Evidence Quality: Reading the Grades

Every recommendation in a Lumen report carries an evidence quality grade. This chapter explains what the grades actually mean, how to improve them over time, and what to do when you need to act on a LOW-evidence recommendation.

What Gets Graded

Each agent grades its own output independently. The Orchestrator then applies one rule: the overall report grade is the lowest individual agent grade.

If SignalMonitor produces a HIGH score and DiscoveryOS produces MEDIUM, the overall report is MEDIUM. One weak link lowers the chain.

Agents that were skipped (because an MCP was not connected) are marked UNAVAILABLE and excluded from the aggregation. A skipped agent does not count as LOW. It just means that section of the report is blank.

What HIGH, MEDIUM, and LOW Actually Mean

The grading criteria differ by agent, but the underlying logic is the same: sample size, recency, source quality, and completeness.

HIGH means all four criteria are met. For SignalMonitor, that looks like:

  • PostHog connected with at least 90 days of event data
  • Survey n > 40 in the relevant segment
  • Data is less than 30 days old
  • No major missing events in the validated schema

MEDIUM means one criterion is missing. The most common causes:

  • PostHog is connected but cohort data is only 30 days old (too short for retention curves)
  • Interview sample is between 20–40 (acceptable but not at saturation)
  • A secondary MCP like HubSpot is absent (churn signals are inferred rather than measured)

LOW means two or more criteria are missing. Common causes:

  • PostHog is not connected (SignalMonitor skips PMF scoring entirely and marks sections UNAVAILABLE, not LOW)
  • Interview n < 20
  • All behavioural data is older than 90 days
  • The event schema has gaps in critical events, like activation or retention triggers

How the Grade Appears in Reports

Each section header carries its grade:

## PMF Scores by Segment

Evidence: HIGH · PostHog 90-day cohort · n=2,400 MAU · Last updated: 2026-03-10

Team plan:   58/100 — PMF CONFIRMED

Solo plan:   29/100 — PMF WEAK

Org plan:    41/100 — PMF CONFIRMED (borderline)

## Opportunity Tree

Evidence: MEDIUM · 14 interviews · Saturation not reached · Bias: power-user skew detected

Top opportunities by frequency and urgency:

1. Roadmap-to-sprint handoff (11/14 participants cited)

2. Cross-team visibility on blockers (9/14 participants cited)

3. AI-assisted prioritization (6/14 participants, mostly Team plan)

The Limitations section at the bottom of every report lists exactly what would improve the grade:

## Limitations

- DiscoveryOS: 14 interviews is below the 20-interview saturation threshold.

  Run 6 more interviews focused on Org plan users to reach MEDIUM → HIGH.

- SignalMonitor: HubSpot not connected.

  Churn signals are derived from PostHog behavioural patterns only.

  Connect HubSpot to add CRM-based exit signals.

- Overall: MEDIUM — upgrading DiscoveryOS to HIGH would raise overall to HIGH.

This is the most actionable section of every report. It tells you exactly where to invest to improve the quality of your next run.

Acting on a LOW-Evidence Recommendation

A LOW recommendation does not mean the recommendation is wrong. It means you have less data than you need to act with high confidence.

The right response to a LOW recommendation is not to ignore it. It is to treat it as a hypothesis and close the gap before committing.

For example, if the W3 strategy produces a LOW-evidence roadmap because PostHog is not connected, you have two options. Run a targeted data collection sprint to close the gap (connect PostHog, gather 30 days of data, re-run). Or treat the roadmap as a draft and add an explicit validation milestone before the committed work begins.

Either way, Lumen has done something useful: it surfaced the gap before you committed resources, not after.

How to Improve Evidence Quality Over Time

Three investments pay the biggest dividends.

Connect PostHog. It is the single biggest lever for W1. Without it, SignalMonitor cannot run at all for PMF scoring. With it, you get quantitative PMF scores per segment backed by real behavioral data.

Accumulate interview data. DiscoveryOS synthesizes interview transcripts that you provide. The more you have — up to about 20 per segment for saturation — the better the opportunity tree. Make customer interviews a standing habit, not a pre-workflow sprint.

Connect HubSpot and Stripe. These two MCPs add CRM churn signals and billing-based NRR data to W2 and W6. Without them, churn analysis is entirely behavioral (PostHog patterns). With them, you get both behavioral and financial signals, which is when recommendations become truly defensible.

The UNAVAILABLE State

When an agent is skipped because a required MCP is absent, the corresponding report section shows UNAVAILABLE instead of a grade.

## PMF Scores by Segment

Status: UNAVAILABLE — PostHog not connected.

W1 is running in PARTIAL mode. Connect PostHog and re-run SignalMonitor

to complete the PMF Discovery workflow.

UNAVAILABLE sections do not affect the overall grade. They do show up in the Limitations section with a specific connection instruction.