Home › Prompts › Claude › A/B test readout with statistical rigor

A/B test readout with statistical rigor

Interpret experiment results correctly: significance, effect size, power and practical recommendation.

ClaudeData analysisRuleStack original

Written for Claude Opus 5.5 · Updated 2026-10-16

Note: Review and test before use. Provided as is, without warranty. See Terms.

Experiment: [what changed]. Primary metric: [metric]. Results: control n=[n], conversions=[x]; variant n=[n], conversions=[y]. Duration: [days]. Secondary metrics: [list].
Analyze: absolute and relative lift with 95% CI, p-value (state the test used), whether the test was adequately powered, risks (novelty effect, peeking, sample ratio mismatch), and a clear recommendation: ship, iterate or stop. Show calculations.

Replace everything in [square brackets] with your own details.

Settings & parameters

• claude-opus-5-5; use Claude's analysis tool for computation when available

Variations

• Bayesian readout with probability variant beats control.

Tips

• Check sample ratio mismatch first; it invalidates results.