Composition · Full experiments cockpit

Mufflermen A/B runtime — three live experiments

Composes all 14 primitives end-to-end against three real workshop experiments. Primary metric is quote-accept rate; secondary is revenue per visitor. AU + NZ split. Visual reference only.

Production answer

Mufflermen A/B runtime — three live experiments is a reusable Oak Flats Muffler Men UI primitive with documented states, accessibility expectations, theme behavior, and implementation evidence.

Primary CTAReview Mufflermen A/B runtime — three live experiments states
Generative search brief

Mufflermen A/B runtime — three live experiments: Composes all 14 primitives end-to-end against three real workshop experiments. Primary metric is quote-accept rate; secondary is revenue per visitor. AU + NZ split. Visual reference only.

Visual reference only — no real backend wired

Experiment 01 · Quote PDF redesignEXP-OF-0421 · Running · day 14 of 28

RunningAU+NZ

Quote PDF redesign

Brand-led header + structured line-items on quote PDFs lifts quote-accept rate vs the legacy mono-block layout.

  1. Legacy PDF· Control18.40%50%
  2. ★Brand header v2· Treatment20.60%50%
Quote-accept rate
20.6%
+11.6%★★★
p = 0.009n = 14,215 / arm
Revenue per visitor
$184.20
+4.8%★★★
p = 0.041AUD, 14d
Daily lift · 95% CI band

Quote PDF redesign — daily lift over legacy

Observed lift95% CI
Quote PDF redesign — daily lift over legacyQuote PDF redesign — daily lift over legacy. Series spans 14 days, final observed lift 11.6%.-5.4%+0.5%+6.4%+12.3%+18.2%May 12May 19May 25
Stop rules · Futility & Superiority

Stopping rules

Superiority · Posterior P(treatment beats control) ≥ 0.97Currently 0.94 · 6 days to projected fire
Armed
Futility · Posterior P(lift < +1%) ≥ 0.90Posterior 0.04 — no futility signal
Off
Guardrail · Quote PDF download rate drop > 5 ppPDF downloads -1.6 pp
Off
Max duration · Max 28 daysDay 14 of 28
Off

Funnel impact

Land on quote pagequote_page_view
Control100.0%
Treatment100.0%
0.0%
Open quote PDFquote_pdf_open
Control64.1%
Treatment71.4%
+11.4%
Scroll past line itemsquote_pdf_scroll_50
Control42.3%
Treatment51.9%
+22.7%
Tap accept CTAquote_accept_tap
Control24.8%
Treatment28.0%
+12.9%
Quote acceptedquote_accept_confirm
Control18.4%
Treatment20.6%
+12.0%

Segment breakdown

Mobile+14.2% · p<0.01n 9.4k
Desktop+8.4% · p<0.05n 4.1k
AU+12.6% · p<0.001n 11.4k
NZ-2.1% · n.s.n 2.1k
RecommendationQuote PDF redesign · 14d

Ship Brand header v2

Quote PDF redesign · 14d
Rationale
  • Primary metric: +11.6% lift, p = 0.009, ★★
  • Posterior P(treatment beats control) = 0.94 — superiority armed
  • No guardrail breaches above threshold this week
  • Holdout NZ wholesale appears unaffected (excluded by design)
Confidence92%
Impact · +$4.2k / wk RPVOwner · @daniel

Wired feature flags

quote_pdf_brand_header_v2Quote PDF — brand header v2Renders the brand-led header + structured line items on quote PDFs.
Ramping50%
PRD
Holdout audience

NZ wholesale holdout

Hold NZ wholesale account traffic out of the Quote PDF redesign — pricing and tax lines differ for those accounts.

10%
Holdout
Exclude rules
countryis
account_typein
Est. 3.1k subjectsEffective until 2026-08-01

Experiment 02 · Suburb landing CTAEXP-OF-0422 · Ramping · day 6 of 21

RampingAU

Suburb landing CTA

Suburb-pinned CTA copy (e.g. 'Book Wollongong workshop') lifts suburb-page → quote click-through vs the generic CTA.

  1. Generic CTA· Control9.80%70%
  2. ★Suburb-pinned· Treatment10.40%30%
Variant editor · Control vs Treatment

Suburb landing CTA — Wollongong copy

ControlGeneric
TreatmentSuburb-pinned
Diff
Daily lift · 95% CI band

Suburb landing CTA — daily lift

Observed lift95% CI
Suburb landing CTA — daily liftSuburb landing CTA — daily lift. Series spans 7 days, final observed lift 6.4%.-4.7%-0.7%+3.3%+7.2%+11.2%May 20May 23May 26
Allocation · Live

Variant traffic

70%
30%
Total allocation100% · OK
RecommendationSuburb landing CTA · 6d

Iterate

Suburb landing CTA · 6d
Rationale
  • Lift +6.2% on Wollongong + Shellharbour — significant
  • Flat on Bega + Eden — copy reads as out of place
  • Iterate to use state-level fallback when suburb < 1k visitors / mo
Confidence62%
Impact · +$1.1k / wk RPVOwner · @growth-team

Experiment 03 · Mobile dock vs sidebarEXP-OF-0419 · Shipped · 28-day run

ShippedAU+NZ · mobile

Mobile dock vs sidebar

Bottom-dock nav (mobile only) reduces quote-flow drop-off vs the legacy hamburger sidebar.

  1. Hamburger sidebar· Control7.60%50%
  2. ★Bottom dock· Treatment8.70%50%
Mobile quote completion
8.7%
+14.8%★★★
p < 0.001n = 30,602 / arm
Revenue per mobile visitor
$112.40
+9.6%★★★
p = 0.012AUD, mobile only
Stat-sig · Two-proportion z-test

Significance calculator

P-value< 0.001 ***p < 0.001
Z-score4.97
Lift (rel)+14.5%+1.10 pp abs
Power100%

Recent experiment historyLast 4 decisions

2026-03-04 → 2026-04-01Mobile dock vs sidebarBottom dock lifted mobile quote completion by 14.8% on iOS + Android. Shipped to 100% mobile.
+14.8%Shipped28d
2026-02-10 → 2026-03-08Workshop bay availability tileLive bay tile moved nothing on same-day bookings. Underpowered at observed effect; not worth re-running.
+1.4%Inconclusive26d
2026-01-08 → 2026-02-04Parts AR overlay vs 3D viewerAR overlay tanked add-to-quote on Android. Killed at day 22 after guardrail breach.
-3.8%Killed27d
2025-12-01 → 2025-12-21Quote line-item iconsPer-line icons lifted quote-accept 4.6% with no guardrail moves. Shipped to all AU.
+4.6%Shipped20d