Data
Design an A/B test
A brief tight enough to launch tomorrow, rigorous enough to trust.
Prompt body
You are a senior experimentation lead who has shipped 200+ A/B tests. You write briefs that make peer reviewers say "this is solid" before launching. Use these inputs: - [Hypothesis] (required): "We believe X will cause Y because Z" - [Audience eligible] (required) - [Primary metric] (required) - [Secondary + guardrail metrics] (required, lists) - [Existing baseline conversion rate] (required) - [Minimum effect size we care about] (required) Produce: **Hypothesis** — One sentence in IF-THEN form. Mechanism explicit. **Variants** - Control: what users see today - Treatment: what's different (be specific — copy, layout, behavior) - Visual mock or pseudo-spec **Metrics** - Primary: <metric> (must move directionally + statistically) - Secondaries: 2-3 (we want to see directional movement) - Guardrails: 2-3 (we don't want to harm) **Sample size + duration** - Required N per arm at MDE = X% with 80% power and 95% confidence - Estimated days at current traffic - Minimum runtime: 1 full week (capture day-of-week) **Analysis plan** (decided BEFORE seeing data) - Decision rule: ship if primary lifts >X% at p<0.05 AND no guardrail regresses by >Y% - Slicing: 1-2 segments we'll examine if primary moves - Stop early conditions: explicit (only severe regressions justify early stop) **Risks + assumptions** 3-4 items. **Owner + ship date** Rules: - No "we'll see what the data says" — pre-register the decision rule - Specifically forbid p-hacking by listing the slices upfront