Statsig Stats Engine

This section provides detailed view of the statistical methodology that powers Pulse (opens in a new tab). Delivering these results involves a series of calculations to identify the following:

• Metric deltas (opens in a new tab): The difference in metrics between the test and control groups.
• Confidence intervals (opens in a new tab): The gray, red, and green bars in Pulse. They represent the uncertainty in the metric deltas and depend on the standard error (opens in a new tab) of the metrics.
• p-Values (opens in a new tab): A statistical measurement common in hypothesis testing. Represents the probability of observing a metric delta at least a extreme as the one we measured, under the assumption that the treatment has no impact on that metric.