A/B Tests Predictions
How Apphud predicts future revenue per A/B test variation — predicted metrics and the payback iteration methodology.
Overview
A/B test predictions use the same approach as the main LTV Predictions. They let you compare variations on projected revenue before the test has actually run for that long.
Available predicted metrics:
| Metric | Window | Predicts |
|---|---|---|
| pARPU 1M / 3M / 6M / 1Y | 1 month / 3 months / 6 months / 1 year | ARPU at that horizon |
| pARPPU 1M / 1Y | 1 month / 1 year | ARPPU at that horizon |
| pARPAS 1M / 1Y | 1 month / 1 year | ARPAS at that horizon |
| pProceeds 1M / 3M / 6M / 1Y | 1 month / 3 months / 6 months / 1 year | Proceeds at that horizon |
Analyzing predicted metrics is identical to analyzing their factual counterparts (ARPU, ARPPU, ARPAS, Proceeds) — see Analyzing Experiments.
Enable predictions
Experiment predictions require LTV Predictions to be enabled for your app.
- Enable LTV Predictions per app via App settings → LTV predictions banner at the top of the App settings page.
- Once enabled, predicted metrics become available in the experiment column picker (marked Only admin). Toggle them on via Edit columns.
For the LTV Predictions setup process and feature scope, see Predictions.
Payback period calculation
Subscriptions are contractual — purchases happen at fixed intervals (1 week, 1 month, 3 months, etc.). That creates several reasonable ways to define a "payback period". The example below shows why.
We compare two subscriptions over a 1-year payback period:
- Product 1 — 3-month duration, no trial, $20 price.
- Product 2 — 3-month duration, 7-day trial, $20 price.
Predicted rebill rates:
| Product | Iteration | Predicted rebill rate | S(t) |
|---|---|---|---|
| product_1 | 0 | 1.000 | |
| product_1 | 1 | 1.526 | 52.63% |
| product_1 | 2 | 1.889 | 36.30% |
| product_1 | 3 | 2.169 | 27.92% |
| product_1 | 4 | 2.396 | 22.79% |
| product_2 | 0 | 1.000 | |
| product_2 | 1 | 1.600 | 60.00% |
| product_2 | 2 | 2.000 | 40.00% |
| product_2 | 3 | 2.286 | 28.57% |
| product_2 | 4 | 2.500 | 21.43% |
Assume 100 subscriptions of product_1. The prediction plays out over days like this:
| Date | Event | Iteration | Amount |
|---|---|---|---|
| 2023-01-01 | install | - | 1000 |
| 2023-01-01 | subscription_started (current iteration) | 0 | 100 |
| 2023-04-01 | subscription_renewal (predict) | 1 | 52.6 |
| 2023-06-30 | subscription_renewal (predict) | 2 | 36.3 |
| 2023-09-28 | subscription_renewal (predict) | 3 | 28 |
| 2023-12-27 | subscription_renewal (predict) | 4 | 22.7 |
| 2024-01-01 | 1-year payback (from first paid transaction) | (4) |
For product_2:
| Date | Event | Iteration | Amount |
|---|---|---|---|
| 2022-12-25 | install | - | 1000 |
| 2023-01-01 | trial_started | - | 400 |
| 2023-01-08 | subscription_renewal | 0 | 100 |
| 2023-04-08 – 2023-04-22 | billing issue + grace period | ||
| 2023-04-22 | subscription_renewal (current iteration) | 1 | 60 |
| 2023-07-21 | subscription_renewal (predict) | 2 | 40 |
| 2023-10-19 | subscription_renewal (predict) | 3 | 28.6 |
| 2023-12-25 | 1-year payback (from install) | (3) | |
| 2024-01-01 | 1-year payback (from trial) | (3) | |
| 2024-01-08 | 1-year payback (from first paid transaction) | (4) | |
| 2024-01-17 | subscription_renewal (predict) | 4 | 21.4 |
The prediction method differs based on which "payback point" you use.
In LTV Predictions the calculation would look like:
- Product 1:
pARPPU = (fact revenue (0) + predicted revenue (1, 2, 3, 4)) / paid subs = (100 * $20 + (52.6 + 36.3 + 28) * $20) / 100 = $47.92 - Product 2:
pARPPU = (fact revenue (0, 1) + predicted revenue (2, 3)) / paid subs = ((100 + 60) * $20 + (40 + 28.6) * $20) / 100 = $45.72
This counts how many renewals are predicted within 365 days from the install date — but that produces a different number of iterations for the two products because of the trial delay. Comparing them isn't fair.
To fix this, A/B test predictions ignore the install date and 365-day window and instead use the maximum number of iterations that could happen within the payback period regardless of trial. A 3-month subscription can produce up to 5 transactions (iterations 0–4) in a year, so both products predict to iteration 4:
- Product 1:
pARPPU = (fact (0) + predicted (1, 2, 3, 4)) / paid subs = (100 * $20 + (52.6 + 36.3 + 28) * $20) / 100 = $47.92 - Product 2:
pARPPU = (fact (0, 1) + predicted (2, 3, 4)) / paid subs = ((100 + 60) * $20 + (40 + 28.6 + 21.4) * $20) / 100 = $50.00
Now they're comparable, and product_2 performs better.
Iteration boundaries by duration and payback window:
| 1M | 3M | 6M | 1Y | |
|---|---|---|---|---|
| 1 week | 4 | 12 | 25 | 52 |
| 1 month | 1 | 2 | 6 | 12 |
| 2 month | 0 | 1 | 3 | 6 |
| 3 month | 0 | 1 | 2 | 4 |
| 6 month | 0 | 0 | 1 | 2 |
| 1 year | 0 | 0 | 0 | 1 |
Notes
-
Trials vs no-trials variations. If one variation has only trial subscriptions and the other has only non-trial subscriptions, pARPAS isn't a valid comparison metric — the p-value isn't calculated for pARPAS in that case.
-
ARPPU alone can mislead. Optimizing for ARPPU doesn't always optimize total revenue. Example with a 50/50 split:
- Variation A — 1000 installs, 100 paid subscriptions, $40 pARPPU → $4000 total.
- Variation B — 1000 installs, 200 paid subscriptions, $30 pARPPU → $6000 total.
Variation A looks better on pARPPU, but Variation B brings more revenue. Check pARPU alongside pARPPU when overall revenue matters.