After the experiment is started, it will accumulate users' data and all related metrics.

Cohort and Non-Cohort Metrics

Experiment analytics allows to display both cohort and non-cohort metrics.

Cohort metrics are those metrics, for which date picker is applied to a date when users joined the experiment.

Non-cohort metrics are those, for which date picker is applied to an metrics's event date.

Cohort metrics are:

• ARPPU
• ARPU
• ARPAS
• View to Action
• View to Trial
• View to Purchase
• Trial Conversion

Non-Cohort Metrics are:

• Sales
• Proceeds
• Refunds
• Trials
• Unique Purchases
• Unique Views
• Views
• Users

📘

Do not compare cohort and non-cohort metrics

This is analytics error to compare cohort metrics with non-cohort. For example, View to Trial metric will not match with the result of dividing Trials by Unique Views, since View to Trial metric allows trials to be after the selected dates.

Target Metrics

View to Purchase

Understand which variation is better in terms of View to Purchase metric.

View to Trial

Understand which variation is better in terms of View to Trial metric.

View to Action

Understand which variation is better in terms of View to Action metric.

ARPU

Understand which variation is better in terms of ARPU metric.

ARPPU

Understand which variation is better in terms of ARPPU metric.

ARPAS

Understand which variation is better in terms of ARPAS metric.

Effect

The "Effect" metric quantifies the relative change between a selected metric in Variation B compared to Variation A. This measurement is expressed as a percentage, indicating how much one variation outperforms or underperforms compared to the other.

Example: If the purchase conversion rate in Variation A is 5% and in Variation B it is 10%, the Effect is +100%. This means Variation B's performance is double that of Variation A, or a 100% improvement.

P-value

The P-value is a statistical metric ranging from 0 to 1, utilized to test hypotheses in experiments. It helps determine whether the observed results are due to chance or are statistically significant.

In our experiments, we set a significance threshold of 5% (P-value = 0.05). A result is considered statistically significant, and thus allows for the rejection of the null hypothesis, if its P-value is less than or equal to this threshold (P-value ≤ 0.05).

We will notify you when the test results are confirmed to be significant based on the P-value criterion.