Shogun With Causality Engine: Check What the Winning Test Sold

The short answer: Variant B won, the template went live, and next month Meta and Google still claim more orders than Shopify booked, with no sign of the lift. Both can be true. Shogun, a Shopify page builder sold beside separate A/B testing and personalization apps, counts only the shoppers who take part in a test, credits their orders to the variant each one was assigned, and aims for 95% confidence on a single goal before it names a winner (analyzing results). That answers which page converts better. It says nothing about which channel sent the buyers. Causality Engine, a causal attribution tool that reads one GA4 export, answers the second question, and its first finding is free at app.causalityengine.ai/start.

Disclosure: I founded Causality Engine and own it, and this page recommends it. Every claim about it links to Causality Engine's own pages.

Before a test winner moves any ad budget:

1. Price the winner in orders, not percentages. For example, 18,000 participating sessions split evenly, converting at 2.4% and 2.7%, give 9,000 × 0.024 = 216 against 9,000 × 0.027 = 243 orders. The winner is worth 27 orders over the test, a page fact, not a channel fact.
2. Use the filter Shogun already ships. The Insights tab filters every card by traffic source (Direct, Paid social, Organic social, Agentic, Email, Referral and Organic search) and recalculates them for that traffic only, per the results article.
3. Compare like with like. Theme, price, product details and checkout tests are eligible on any page, so store-wide Shopify numbers are a fair yardstick for them; page, template and URL redirect tests are not, per Why Analytics Look Different.

Shogun and the read, row by row

Two scoreboards, two rules

In the Page Builder, Page Analytics reports Total Sales as "the total sales from shoppers who visited the page during their session", per the page analytics article, which adds that only a Shogun Add to Cart button is tracked and that data arrives four hours late. The rule is the session, so a visit through three Shogun pages credits one order to all three.

The A/B Testing app is stricter about who counts. Each test is judged on one goal, from Conversion Rate to Revenue Per Visitor, and add to carts count across the whole session, not only on the tested page (goal definitions). An order enters a test only when the shopper took part, is attributable to their variant and consented to tracking. Shoppers who decline still see a variant; Shogun records neither their visit nor their order.

In a URL redirect test that redirects only on the first visit, a Variant B shopper who later buys from the original URL is still credited to Variant B, while a Shopify page report counts the original (analytics differences).

The verdict rests on confidence in the goal metric alone, recalculated about every 15 minutes, with 95% as the bar (results article). Publish Winner can overwrite the original on page and template tests; for other types it only records the result (publishing winners).

What a Shogun page changes in GA4

Very little. Shogun controls only the body of a page, so the Google Analytics code set in the platform settings loads on Shogun pages too, with Shogun having no say over its settings (tracking code article). A visit keeps the source and medium it arrived with. GA4 then files it by Google's rules: Paid Social, for instance, needs a social source and a medium such as cpc, ppc or one starting with "paid" (default channel group).

What GA4 does not get is the variant. The A/B help articles read on 11 November 2026 describe no setting that sends it there, and a template test serves both versions at one address. Landing page reports miss participants too: a shopper who lands on the homepage and reaches the tested page later is in the test but outside a report filtered by landing page (analytics differences).

So the export Causality Engine reads holds channels, not templates: the read can say what Paid Social caused, never whether Variant B did.

A worked example: one winning template, four claims

For example, with figures invented to show the arithmetic and no store behind them: a four-week template test on a best-selling product page. Shogun reports 268 orders for Variant B and 231 for the Original, at 96% confidence on Conversion Rate. Over the same four weeks:

The ad platforms alone claim 690 + 520 = 1,210 orders, which is 1,210 - 1,140 = 70 more than the store booked, before email or organic search say a word. The test covered 499 ÷ 1,140 = 43.8% of orders, and the lift it found is 268 - 231 = 37 orders on an even split. All three figures can be right under their own rules.

The read adds the channel view. Per its how-it-works page, it splits those 301 Direct orders back to the channels that sent the buyers and shows each channel's caused share next to a last-click rebuild, with a data-health score from 0 to 100 and a next step. It will not split Meta's share by template. If the read rates Meta well below its own 690, keep the winning template and move spend in steps, tested in one region first.

The free check: let the test do its job

The fairest check Shogun offers is the one it sells: a test. A theme or template test copies your live page as Variant B (first test article), and the Pro plan of the A/B app carries a 14-day free trial (pricing, checked on 11 November 2026). Pick the goal before launch, leave it until confidence reaches the 95% the app aims for, and read the Traffic sources filter before publishing. A winner that holds for Paid social and collapses for Email is a page that suits one audience.

Without the app, a by-hand before-and-after works with limits: four weeks on each page, same ad budgets, no sale in either window. Seasonality lands inside the difference, so read it as a direction.

Once the page question is settled, the channel question is the export. Drop the Attribution paths CSV on the start page: the first finding is worked out in your browser at no charge. Causality Engine, which I founded, sells the full read of the same file for €99 once, excluding VAT, refundable within 30 days (pricing, checked on 11 November 2026).

Where the pairing stops

• History and events. The read wants 12 months or more of GA4 data (how it works), and it takes key events from GA4's Advertising section (one-off read), so purchase must be a key event.
• Small channels. A channel under 1% of credit on both methods is called too small to act on (how it works).
• Pages and variants. Templates, goals, profit per variant and the MCP connection that lets an assistant start and stop tests stay in Shogun.

Shogun's Profit card uses $0 for fulfillment costs nobody has entered, and the results article itself warns that profit can then look higher than it is. So the first profit figure a new subscriber reads is the most generous the app can produce, read in the same weeks the trial turns into a bill. Type the costs in before the first test.

See also

• Shogun: the three apps, their plans and complaints
• How to use Shogun: a product template test from copy to winner
• Shogun on the marketplace: Shogun's affiliate program terms
• Causality Engine: what the read is and what it costs
• How to use Causality Engine: from GA4 export to a budget call
• Causality Engine on the marketplace: its affiliate program terms
• GemPages vs Shogun: unlimited pages against built-in testing
• Conversion rate calculator: orders over sessions, your inputs

FAQ

Can Shogun pass its test results to Causality Engine?

No, and nothing has to. Causality Engine reads the channels in a GA4 export (integrations); a Shogun variant never becomes one of them.

Why does a Shogun test show fewer orders than Shopify?

Shogun counts participants only. Shoppers outside the test's audience, in another concurrent test, who never reached the tested page or who declined tracking are left out, per Why Analytics Look Different.

Should a winning test change the ad budget?

Not by itself. A winner says the page converts better for those who saw it; whether Meta sent buyers who would not have come anyway is a channel question for the read.