Event contract
Fields the query expects
| Field | Type | Why it exists |
|---|---|---|
| timestamp_utc | Timestamp | Assignment or outcome event time in UTC. |
| experiment_name | Utf8 | Stable experiment identifier. |
| variant | Utf8 | control or a named treatment. |
| user_id | Utf8 | Stable participant identifier. |
| event_name | Utf8 | experiment_assigned or purchase_completed. |
| revenue_usd | Float64 | Normalized revenue on a completed purchase. |
Copy the query
WITH participant_outcomes AS (
SELECT
experiment_name,
variant,
user_id,
MAX(CASE WHEN event_name = 'experiment_assigned' THEN 1 ELSE 0 END)
AS was_assigned,
MAX(CASE WHEN event_name = 'purchase_completed' THEN 1 ELSE 0 END)
AS converted,
SUM(CASE
WHEN event_name = 'purchase_completed' THEN revenue_usd
ELSE 0.0
END) AS revenue_usd
FROM experiment_events
WHERE timestamp_utc >= now() - INTERVAL '60 days'
GROUP BY experiment_name, variant, user_id
),
variant_summary AS (
SELECT
experiment_name,
variant,
COUNT(*) AS participants,
SUM(converted) AS converted_participants,
100.0 * SUM(converted) / NULLIF(COUNT(*), 0) AS conversion_rate_pct,
SUM(revenue_usd) / NULLIF(COUNT(*), 0) AS revenue_per_participant_usd
FROM participant_outcomes
WHERE was_assigned = 1
GROUP BY experiment_name, variant
),
control AS (
SELECT
experiment_name,
conversion_rate_pct AS control_conversion_rate_pct
FROM variant_summary
WHERE variant = 'control'
)
SELECT
summary.experiment_name,
summary.variant,
summary.participants,
summary.converted_participants,
summary.conversion_rate_pct,
summary.revenue_per_participant_usd,
100.0 * (
summary.conversion_rate_pct - control.control_conversion_rate_pct
) / NULLIF(control.control_conversion_rate_pct, 0) AS relative_lift_pct
FROM variant_summary AS summary
JOIN control
ON control.experiment_name = summary.experiment_name
ORDER BY summary.experiment_name, summary.variant;This read-only query is planned and executed against an empty typed table with Apache DataFusion 45.2.0. The deterministic sample output is synthetic and reviewed separately; validate field types, thresholds, and business definitions against your own data. Read the testing methodology.
Query result
Conversion lift by experiment variant
The treatment has higher observed conversion and revenue per participant; statistical uncertainty still needs review before rollout.
| experiment_name | variant | participants | converted_participants | conversion_rate_pct | revenue_per_participant_usd | relative_lift_pct |
|---|---|---|---|---|---|---|
| annual_plan_checkout | control | 2,140 | 278 | 12.99 | 18.42 | 0 |
| annual_plan_checkout | proof_near_cta | 2,187 | 326 | 14.91 | 21.76 | 14.78 |
Synthetic example output. Run the query against your own event schema and thresholds before using it for operational decisions.
How the SQL works
- 1The first CTE creates one outcome record per assigned participant, preventing repeated purchase or tracking events from inflating the denominator.
- 2Variant summaries preserve participant volume, conversion, and revenue so a higher conversion rate is not evaluated without commercial value.
- 3The final join compares every variant with the control rate and reports relative rather than percentage-point lift.
Edge cases to decide
- Exclude participants exposed to multiple variants or assign them according to a documented intent-to-treat rule.
- Apply a mature outcome window so recently assigned participants are not treated as non-converters.
- Calculate confidence intervals or a pre-agreed statistical test before declaring a winner.
Recommended dashboard
- Bars: conversion_rate_pct by variant
- Table: participants, revenue per participant, and relative lift
- Trend: mature cohort conversion by assignment week
Alert guidance
Do not page on normal experiment variation; alert only when assignment balance, event delivery, or a guardrail metric breaks.
Read alert setupPut the recipe to work
Related instrumentation and guides
Continue the analysis
Build a Signup-to-Activation Funnel in SQL
Calculate unique-user conversion through signup, onboarding, integration, and first-value milestones.
Open recipeCalculate Weekly Cohort Retention
Group users by first activity week and measure the percentage returning in later weeks.
Open recipeFind Features Used Before Upgrade
Join feature events to upgrade events and rank behaviors that occur before paid conversion.
Open recipeRun it on real events
Create a table, adapt the fields, and save the result
Start free, send structured events, and use the query result as a chart, shared dashboard widget, or alert input.