A/B test results analysis
Advanced Data AnalystYou're a data analyst evaluating the results of an A/B test on a new checkout flow. Traffic was split evenly between the original design (Control A) and a redesigned version (Treatment B) over a two-week experiment.
Your goal: determine whether the redesign improved conversion performance, and whether the observed difference is statistically significant at the 95% confidence level using a two-sided z-test.
Fill in the eight calculated fields in the spreadsheet. The Notes column shows mathematical notation for the statistical formulas. For the Z-score, use the pooled standard error from the Results section, where pooled SE is based on pooled conversion rate across both variants. Then return a final Significant? decision based on the 95% two-sided threshold.
Need some help?
Hint 1
Start with conversion rate and standard error for each variant: use conversions/visitors for p, then apply √(p × (1−p) / n) with SQRT().
Hint 2
For pooled standard error, first compute pooled conversion rate from total conversions and total visitors across both variants, then apply √(p_pool × (1−p_pool) × (1/n_A + 1/n_B)); compute Z as (p_B − p_A) / pooled SE.
Hint 3
For the final decision, compare ABS(Z) to 1.96 for a 95% two-sided test and return "Yes" when it exceeds the threshold, otherwise "No".
Related function(s)
A/B test results analysis
Advanced Data AnalystYou're a data analyst evaluating the results of an A/B test on a new checkout flow. Traffic was split evenly between the original design (Control A) and a redesigned version (Treatment B) over a two-week experiment.
Your goal: determine whether the redesign improved conversion performance, and whether the observed difference is statistically significant at the 95% confidence level using a two-sided z-test.
Fill in the eight calculated fields in the spreadsheet. The Notes column shows mathematical notation for the statistical formulas. For the Z-score, use the pooled standard error from the Results section, where pooled SE is based on pooled conversion rate across both variants. Then return a final Significant? decision based on the 95% two-sided threshold.
Need some help?
Hint 1
Start with conversion rate and standard error for each variant: use conversions/visitors for p, then apply √(p × (1−p) / n) with SQRT().
Hint 2
For pooled standard error, first compute pooled conversion rate from total conversions and total visitors across both variants, then apply √(p_pool × (1−p_pool) × (1/n_A + 1/n_B)); compute Z as (p_B − p_A) / pooled SE.
Hint 3
For the final decision, compare ABS(Z) to 1.96 for a 95% two-sided test and return "Yes" when it exceeds the threshold, otherwise "No".