Statistics

Eppo's statistics engine is designed to provide a best-in-class analysis of experiment results. Our approach is built upon four core tenets: flexibility, power, interpretability, and trustworthiness. This section of the Eppo documentation delves into every aspect of our statistical engine, offering a comprehensive understanding of its capabilities.

Flexible

We understand that different statistical methods have their strengths and limitations. To cater to your specific use case, Eppo offers the flexibility to choose the statistical regime that best suits your needs. Our platform supports various methodologies, including classical frequentist tests (such as t-tests), sequential analysis (always valid), and Bayesian statistics. This flexibility empowers you to employ the statistical approach that aligns with your experiment design and goals.

Powerful

Maximizing the value of your data is crucial, regardless of the scale of your experimentation. Eppo is committed to providing you with powerful statistical tools that drive both effective decision-making and the credibility of your experimentation program. Our implementation of CUPED++ stands out as a prime example, leveraging historical data to deliver more accurate and efficient results. With our powerful statistical capabilities, you can unlock deeper insights and accelerate your business outcomes.

Interpretable

We believe that statistical results should be accessible to both experts and individuals with limited statistical background. Eppo strives to make statistical analysis interpretable by offering sensible defaults that ensure trustworthy results out of the box. While our platform caters to a wide range of users, advanced users also have the ability to customize their experience and delve deeper into the statistical nuances for more specialized analyses.

Trustworthy

When critical decisions rely on experimental results, accuracy becomes paramount. At Eppo, we rigorously test our methodologies to ensure that the results we provide adhere to the promised statistical guarantees. We prioritize the trustworthiness of our statistical analyses, taking into account factors such as detecting and notifying you when key assumptions are violated, such as a traffic imbalance. With Eppo, you can have confidence in the accuracy and reliability of the statistical insights that drive your decision-making process.

By combining flexibility, power, interpretability, and trustworthiness, Eppo's statistics engine equips you with the tools and confidence to derive meaningful insights from your experiments and make data-driven decisions with conviction.