How to Use Shogun to A/B Test a Product Page Template

The short answer: if your product pages draw enough traffic to settle a test within a few weeks, Shogun's A/B Testing app runs a template test from $39 a month on its Pro plan, with three active tests and a 14-day trial, per the pricing page checked on 26 October 2026, and your live template stays untouched until you choose a winner. The steps: install the app, create a Template Test, copy the original as Variant B, change one thing on it, pick a goal and a 50/50 split, launch, read the result once confidence reaches 95%, then publish the winner.

Disclosure: I may earn a commission through some links on this site. It does not change what this page says.

The job: one product template serves every product page in the store, so a change to it, reviews moved above the fold or a shorter description, lands on the whole catalogue at once. That is why it deserves a test before it ships. This guide follows Shogun's A/B help centre; what Shogun sells beyond testing is in the Shogun glossary entry.

Three checks before the test:

1. The months you will pay for. For example, a test that needs four weeks, started on day one of the 14-day trial, runs into one paid month of Pro at $39, using the pricing page. Two tests side by side still fit Pro's three active slots.
2. The template you already run. Variant B starts as a copy of your current product template, edited in Shopify's theme editor or in the Edit in Shogun beta, so the test begins from what shoppers see today, per the first test article.
3. Today's numbers. Note add-to-cart rate and conversion rate on product pages for the last 30 days in Shopify. Shogun's own figures will differ, because it counts only shoppers inside the test, so treat Shopify's as the context and Shogun's as the verdict.

What you need

• Permission in Shopify to install apps and accept charges, which the installation article names as the only requirement.
• An Online Store 2.0 theme if you want to edit the variant in Edit in Shogun; older themes use Shopify's editor, per the Edit in Shogun article.
• One hypothesis, written as one sentence: what changes, and which number it should move.

Step 1: Install and let the app sync

Install Shogun A/B Testing from the Shopify App Store and accept its permissions. The app then syncs with your published theme, and its pages and templates become available to test.

Step 2: Create a Template Test

Choose Template Test and pick your current product template as the Original. For the variant, choose Copy original, and Shogun creates an editable duplicate of that template. Each visitor stays assigned to the same variant for the whole test, per the first test article.

Step 3: Change one thing on Variant B

Open the variant and make the single change from your hypothesis. Shogun's testing practices article asks for one variable per test, puts above-the-fold changes and trust elements such as reviews, badges and return policies first, and asks you to check the variant on a phone, since most traffic arrives there. In Edit in Shogun you can describe the change in chat or edit sections directly, with a live preview.

Step 4: Pick the goal

For a product template, Add to Cart Rate reads buying intent and Conversion Rate reads orders, per the goals article. Add to Cart Rate counts additions anywhere in the session, not only on the tested page. Revenue per visitor and average order value are Advanced goals on the pricing page, so on Pro choose between the first two.

Step 5: Split, check and launch

Set the traffic split, 50/50 for a two-way test, then preview both variants on desktop and mobile before you start. The clean test design article warns against running a second test on the same templates at the same time and against large edits once the test is live.

Step 6: Read the result

The test page shows which variant leads and Shogun's confidence. Shogun aims for 95% before it declares a winner; metrics refresh about every minute and confidence about every 15, and filters by device or traffic source change the cards but never the confidence, per the results article. Early swings are normal on smaller stores.

Step 7: Publish the winner

Click Publish Winner. For a template test you then choose to record the winner only, overwrite the original template with it, or point the original's template suffix at the winning template, and you can delete the losing templates, per the publishing winners article. Check the store on several devices afterwards and note what you learned in the test's notes.

A step for an agent: Shogun MCP

On the Advanced testing plan at $119 a month, the Shogun MCP server lets Claude or ChatGPT read test results and create tests, per the MCP guide. A prompt in the guide's own style: "Create a template test. Set the variant to my 'pdp-reviews-first' template, set the goal to Add to Cart Rate and the traffic split to 50/50." The test starts as a draft that gets no traffic until you ask the assistant to start it and confirm, and the guide recommends keeping write tools on Needs approval.

The human check matters at the end as much as the start. One sample prompt in that MCP guide promotes winners automatically at 90% confidence, below the 95% the results article aims for, so keep the call on winners with a person, at 95%.

What Shogun's help centre warns about

• Stopping early. Results can swing on low traffic, so do not end a test the moment one variant pulls ahead.
• Overlapping tests. Two tests on the same templates muddle each other's data.
• Large edits mid-test. Pause or restart instead.
• Numbers that differ from Shopify and GA4. Shogun reports test participants only, as the analytics article explains at length.

Fitting the test inside the trial

A 14-day trial covers two weekends, while the clean test design article asks for at least a few business cycles before calling a result, so on most stores the first verdict lands in the first paid month: the trial is long enough to get a test running and short enough to end before it finishes. Make the free days count: build the variant and choose the goal before you start the trial, so all fourteen days collect data.

The other free check is your order export. The MCP guide suggests matching the Shopify order IDs attributed to each variant against your own export; on Pro, compare the order counts the results article tells you to watch with the orders Shopify shows for the same dates. Later, if you want the gap between what a test reports and what the business booked checked continuously, a causal attribution tool like Causality Engine does it (disclosure: I founded it).

FAQ

How long should a Shogun test run?

Until Shogun reaches 95% confidence and at least a few business cycles have passed, such as weekends or a promotion, per the clean test design article.

Why don't Shogun's numbers match Shopify's?

Shogun counts only shoppers who entered the test, and only from the point they became eligible, while Shopify counts every session in the store.

Can Shogun test prices?

Yes, on the Unlimited plan at $499 a month, per the pricing page. One price test can run at a time, alongside other tests, per the test types article.