Shopify Rollouts: how to A/B test your checkout without Plus

You paid for the click. The shopper filled a cart and made it to your checkout. Then something on that last page made them hesitate, and for years Shopify gave you no built-in way to find out what. So you shipped your best guess and hoped. Shopify Rollouts changed that in June 2026. It lives in your admin and can A/B test a new checkout against the one you have now. You don't need Shopify Plus for it. Grow or higher will do.

On November 4, we started one for a client, a mobile-first store that isn't on Plus. Half its visitors get the current checkout. The other half get a copy with updated brand colours. Two days in, the new version was ahead. Shopify's own report said not to trust that yet, and I agree with it. The report is further down, and reading it right is half the job.

What Shopify Rollouts is, and which plan you need

Rollouts lives in your admin under Markets > Rollouts. Shopify describes a rollout as a scheduled set of changes to your theme, checkout and accounts pages, discounts or catalogs. There are three types:

• A Launch makes your changes permanent.
• An Event publishes changes for a set period, then rolls them back.
• An Experiment is the A/B test. Some visitors see the control, your store as it is now. The rest see the treatment, your change.

Rollouts started as a theme tool. Checkout and customer accounts configurations arrived on June 5, 2026. Before that, a Shopify staff member said in December 2024 there was no built-in way to A/B test the checkout. The old Plus-only way to restyle it, checkout.liquid, stopped working on the Information, Shipping and Payment steps in August 2024. So this is the first time a store below Plus can A/B test its checkout at all.

Grow or higher, not Plus

This is where most of the bad advice lives. Shopify's requirements page says "Rollouts are available on the Basic plan or higher." Then: "Experiments are available to stores on the Grow plan or higher." Plus isn't mentioned anywhere on that page. If a guide told you a checkout test needs Plus or Advanced, it's out of date.

The second checkout isn't Plus-only either. Basic, Grow and Advanced stores can keep up to 20 draft checkout configurations. Plus stores get 99.

What a checkout test can change below Plus

On Basic, Grow and Advanced, the checkout editor covers your logo, colours, buttons and fonts. Apps on the Information, Shipping and Payment steps are Plus-only, and so is the Checkout Branding API. Below Plus, then, a checkout test is mostly a branding test. That sounds small. I think branding does more work at the checkout than most owners give it credit for.

Why the checkout is the page to test

Baymard Institute puts average cart abandonment at 70.22%, across 50 studies. Set aside the 42% who were just browsing. Of the shoppers who abandoned for any other reason, 19% didn't trust the site with their credit card information. That's from Baymard's 2025 survey of 1,026 US adults.

Baymard also found that a shopper's sense of security comes down to gut feeling, and that gut feeling is largely set by how visually secure the page looks. Your shoppers aren't auditing your encryption. They're reacting to what they see. That makes trust a design problem, and design is the part every plan can change. On Grow and up, you can test it too.

How to set up your first checkout A/B test

First check that you're on Grow or higher, your theme isn't a vintage theme, and your staff account can view and edit checkout and customer accounts. Headless and custom storefront checkouts can't be tested. Then:

1. Duplicate your checkout. Go to Settings > Checkout. In the Configurations section, click the menu next to your active configuration, then click Duplicate. That makes a draft. Your live checkout doesn't change.
2. Make one change. Open the draft in the checkout editor and change one thing, like the button colour or the logo. Change three things and win, and you won't know which one did the work.
3. Create the Experiment. Go to Markets > Rollouts, click Create rollout, name it, and choose Experiment in the rollout type menu.
4. Swap in your draft. In the Changes section, click Add changes and choose Checkout & accounts. Under Replace the current configuration, select your draft, then click Apply.
5. Keep the 50/50 split. It's the default. Set Traffic allocation to 100% of eligible visitors. Visitors are grouped at random, and a cookie keeps each one in the same group when they come back on the same browser.
6. Set the dates in whole weeks. An Experiment ends 90 days after it starts unless you change it. Shopify's own A/B testing guide says to run at least one, ideally two, full business cycles, in full-week increments. Before you save, Shopify shows what happens when the rollout ends. Read it.
7. Leave it alone. Shopify warns that editing the control or the treatment during an active Experiment can affect its results. New idea halfway through? Save it for test two.
8. Apply it or roll it back. Shopify's glossary describes a roll back as the automatic undo when a rollout reaches its end date, so expect the treatment to disappear then unless you apply it. End the test early and Shopify asks you to pick Roll back changes or Apply changes. Applying is permanent. To keep the treatment and drop the end date, change the rollout type to Launch.

Reading the results: a live test, two days in

Shopify lists one metric for a checkout Experiment, checkout conversion rate: online store sessions with completed purchases relative to the visits that reached checkout. Everyone counted already got to checkout, so the rate looks huge next to your store's usual conversion rate. Revenue and average order value only show up for a Launch.

Two days into our client's test, Control sat at 63.37% and the Treatment, with the new colours, at 65.69%. A badge showed the treatment up 2.32%. On the confidence interval chart, the two bars overlapped. Shopify's summary read:

"Not enough evidence to detect a difference. Treatment showed no meaningful impact on checkout conversion rate compared to Control, with both groups performing similarly at 63.37% and 65.69% respectively. More data is needed before any conclusion can be drawn."

What the confidence interval means

Any test's conversion rate is an estimate. The confidence interval is the range the true rate probably sits in, given the data so far. Less data means a wider range.

Each bar on the Rollouts chart is one of those ranges. When the bars overlap, you can't tell the two versions apart yet. The true rates could be equal, or the order could flip. That's what "Not enough evidence" is telling you. The 2.32% badge matches the gap between the two rates (65.69 minus 63.37), and two days in, it tells you nothing. Shopify doesn't publish how it calculates the badge or the chart, so I watch the bars.

Don't stop at the first good number

Looking is fine. Stopping the first time it looks like a win is the mistake. Evan Miller showed that checking constantly and stopping as soon as a result looks significant can push your false positive rate to 26.1% when you think it's 5%.

Our client's test runs to November 24, 20 days in all, one short of three full weeks. Next time I'd make it 21. I won't guess the result. If the bars still overlap at the end, that's an answer too: the colours didn't matter, and the next test gets the traffic.

How much traffic you need

Shopify's Partner blog works an example: at a 2% conversion rate, spotting a 5% relative lift takes 309,928 visitors per variation, and a 20% lift takes 19,784. A checkout test starts from a much higher rate, which helps. The same post warns that stores with fewer than 100 purchases a month will have a hard time A/B testing at all, because purchases are the real sample.

If that's your store, don't force a test. There's a better use of your afternoon, and it's in the playbook at the bottom of this post.

Five things that trip people up

• Only an Experiment has a control group. A Launch merges your change into the store for good, so there's nothing to compare it with.
• You can't pick the audience. The markets come from the change you added, and there's no targeting by device, customer tag or segment. Everyone eligible is grouped at random.
• Rollouts can overlap, and some changes can't share a visitor. Shopify says the treatment can end up with less traffic than the percentage you set. Run one checkout experiment at a time.
• Experiments only count the online store. POS, headless and other channels are left out, so a store that sells a lot elsewhere is testing a slice of itself.
• The group assignment lives in a cookie. The same person on a new browser or device can land in the other group, and a counted treatment session doesn't prove that visitor saw the change. Small test, noisy numbers. Run full weeks.

So what do you test first?

The thing that makes a nervous shopper feel safe at the moment they pay, before anything you're merely curious about. Two changes do that on any Shopify plan, with no Plus and no app, and they're the first thing we do on every checkout we touch. A few more only Plus can make.

I'm not putting them in this post. They're the whole point of the Checkout Safety Playbook, a free PDF: the two changes step by step, each with a checklist, the Plus-only moves, the Rollouts setup above as a checklist, ten checkout tests in priority order, and the math to tell whether your store is big enough to test.

Rather go through your checkout with a developer? Get a free CRO teardown, see how we run conversion rate optimization month to month, or talk to FireNet.

On Grow or higher? Duplicate your checkout this week and start your first Experiment. Not enough traffic yet? Make the two changes anyway. The playbook signup is right below this post.