Methodology

How We Compute the Counterfactual Behind Ghost MRR

7 min read

Ghost MRR is the recurring revenue you give away by discounting trial users who would have paid full price anyway. It's a simple idea with one genuinely hard number inside it: for any discounted conversion, would this user have converted without the discount? That question is a counterfactual, and counterfactuals can't be observed — you can't both give and withhold the same offer from the same person. This post is about how we estimate it, and just as importantly, how we keep ourselves honest about it.

Why the naive version is wrong

The tempting shortcut is to call every discounted conversion "saved revenue" and every non-converter "lost." Both are wrong. Some of the users who took your discount were always going to buy — discounting them cost you margin and taught them to wait for a coupon. Some of the users who churned were never reachable at any price. The only cohort that matters for intervention is the one in between: the fence-sitters whose decision actually changes when you act.

Ghost MRR isolates the discounts that changed nothing. It is not churn, and it is not your total discount spend — it is the specific slice of discounting that was unnecessary because the buyer was already decided.

The formula, and where the hard part hides

The definition is arithmetic once you have the counterfactual probability:

Ghost MRR = Σ ( P(would_pay_full_price) × discount_amount )

Every term except the first is bookkeeping. The whole problem collapses into estimating P(would_pay_full_price) for each user — the probability they'd have converted at full price with no nudge and no discount. Get that right and Ghost MRR is trustworthy. Hand-wave it and the number is theater.

How KaQuill estimates it

We treat each trial as a sequence of behavioral events — signups, feature usage, activation moments, plan views — and learn a policy over that sequence rather than scoring users against hand-written rules. Three pieces do the work:

  • Behavioral embeddings. Raw events are noisy and product-specific, so we represent each trial's trajectory as a learned embedding — a compressed signature of "how this user is behaving" that generalizes across the specific features they happen to touch.
  • A learning policy (PPO + DPO), not an if/then engine. The decision of whether to intervene is a reinforcement-learning problem: the reward is a real conversion, the cost is margin given away. The policy learns which states are already converging on payment (leave them alone) and which are genuinely on the fence (worth a nudge).
  • Confidence, surfaced not hidden. The model reports how sure it is. Below a confidence threshold it does nothing rather than guess — a wrong nudge has a real cost, and silence is a valid action.

How we keep it honest: randomized holdouts

A model that estimates a counterfactual can quietly drift into telling you what you want to hear. The only defense is to measure the thing the model is predicting. So we hold out a random slice of otherwise-eligible trials and don't intervene on them. Comparing the holdout's conversion rate to the treated group gives a population-level, actually observed measure of lift — and that ground truth is what calibrates the per-user estimates.

You can't measure the counterfactual for one person. You can measure it for a population, and use that to keep the individual estimates anchored to reality. When the model claims a cohort would have converted anyway, the holdout is what proves it — or corrects it.

What this means for you

Practically: KaQuill stops firing discounts and nudges at the trials who'd have paid regardless, and concentrates them on the fence-sitters where intervention measurably moves the outcome. Ghost MRR is the scoreboard for that — the margin you were giving away, now visible and shrinkable. You can estimate yours in 30 seconds with the free calculator (no signup), or read how KaQuill compares to rules-based tools like Userpilot.

The counterfactual will never be perfect — it's an estimate, and we say so. But an estimate you validate against holdouts and act on carefully beats the status quo, which is to discount everyone equally and never measure the waste at all.

Frequently asked

Can a counterfactual ever be measured, not just modeled?+

Not for an individual user directly — you can't both give and withhold the same discount from the same person. But across a population you can measure it with a randomized holdout: withhold the intervention from a random slice and compare conversion. That population-level truth is what calibrates the per-user model.

Isn't a conversion-probability score the same thing?+

No. A score ranks who is likely to convert. The counterfactual behind Ghost MRR asks a narrower question — who would convert without help — which is what tells you when a discount or nudge is wasted. They are different quantities and KaQuill computes the second.

How much data before the estimate is trustworthy?+

Directional insight appears after roughly 200 trial sessions; calibrated live decisions typically take 2–3 weeks as the model accumulates enough outcomes to validate against holdouts. KaQuill surfaces its confidence so you never act on a number it isn't sure of.

See what you're leaving on the table.

Free forever to see your Ghost MRR. Upgrade only when you're ready to act on it.