How to A/B test iOS onboarding

Test onboarding, not just the paywall

A paywall test changes who buys among the users who reach it. An onboarding test changes how many users reach it at all — and arrive warmer. Because onboarding sits at the top of the funnel, a win there compounds through every screen below it, including the paywall. If you only A/B test the last screen, you’re optimizing the smallest lever.

What to test first

Start with the variables that move the most users, not the cosmetic ones. In rough order of leverage:

• Length — does cutting two screens lift completion without hurting paywall conversion?
• The order — value before the ask, or the ask before value?
• The first screen — a question that pulls the user in vs a static welcome.
• Personalization — does a quiz that visibly shapes the plan raise intent?
• Permission timing — priming before the system prompt vs asking cold.
• Paywall placement — straight after onboarding, or one screen later.

Change one variable at a time

If variant B reorders screens and rewrites the headline and drops the price, a win tells you nothing about why. Isolate one change per test. It feels slower, but it’s the only way the result is usable — a clean read on one variable beats a muddy read on three.

Sticky bucketing is non-negotiable

A user must stay in the variant they were first assigned, for the life of the test. If bucketing isn’t sticky — if reweighting or a relaunch can move someone from A to B — your conversion numbers mix both experiences and the test becomes a noise generator wearing a chart. Steplark buckets per device and keeps it sticky, so exposure and conversion line up.

Don't call it early

A 3-screen onboarding with a few thousand starts swings a lot day-to-day. Decide the metric before you start (completion rate, or better, paywall conversion of the cohort that started onboarding), let it run until the difference is stable, and resist ending the test the first morning B looks good. Early calls are how teams ship losers that looked like winners for 36 hours.

Read the whole funnel, not one number

A variant can lift completion and still lose money if the extra finishers convert worse on the paywall. Judge a test on the metric that pays — conversion and revenue of the started cohort — and use per-screen analytics to see where each variant gained or leaked. The screen-by-screen view tells you why B won, which is what makes the next test smarter.

Ship the winner without a release

The reason most teams under-test onboarding is that every iteration means an App Store build and a review wait. Take that out of the loop and you can run a test a week instead of a test a quarter. With Steplark, onboarding is served over the air: publish the winner and it reaches devices on the next launch. Start from a template and your first test can be live today.