How to Use Dynamic Yield to Test Product Recommendations

The short answer: Dynamic Yield sells no self-serve plan and publishes no price, so this is a job for a team already under contract, and the first week belongs to a developer, because the feed, the page context and the events are developer steps in the implementation journey; after that, build one campaign on the product page with two recommendation variations, keep 10% of traffic in a control group, the share the dynamic allocation guide recommends, make Add to Cart the primary metric, give Predictive Targeting the 14 days it needs before it reports audiences (Predictive Targeting guide), and let the Analyst agent draft the readout while a person decides what ships.

Disclosure: I may earn a commission through some links on this site. It does not change what this page says.

A product page is where a recommendation widget earns its slot or loses it. Dynamic Yield offers a long list of algorithms for that slot, and only a test on your own catalogue says which one belongs there. Ownership, prices and rivals are in the Dynamic Yield glossary entry.

Before you start:

• The traffic sum. For example, 300,000 product-page users a month with a 10% control leaves 300,000 - 30,000 = 270,000 for the test, or 135,000 per variation. The engine samples 300,000 results per variation to compute a Probability to Be Best, a probability that A beats B rather than a p-value (Bayesian guide).
• A feed that already syncs, by CSV or API, with the Add to Cart and Purchase events reported (implementation journey).
• Last month's recommendation revenue, direct (a clicked item bought within 30 days) and assisted (any purchase in the session after a click), from the dashboard (Experience OS homepage guide).
• Someone with the legal call on consent.

1. Check the plumbing

Your site exists in Experience OS as a section, and the implementation journey lists what has to be true before a campaign renders: feed synced, page context set so the platform knows a product page from a category page, pageviews and events reported, the user cookie set from your own backend. Confirm the script loads with the validation guide before anyone builds a variation.

2. Settle consent and where the data sits

For visitors who have to be asked, Dynamic Yield documents an active cookie consent mode in which nothing is stored until the visitor agrees (consent guide). Article 5(3) of the ePrivacy Directive is the rule behind that mode in the EU. A .eu console runs in AWS Frankfurt, a .com console in North Virginia (Experience APIs guide).

3. Create the campaign on the product page

A campaign holds up to 20 experiences, each with targeting, who sees it and where, and variations, what they see; the highest-priority match wins and anyone who matches nothing gets the default (campaign overview). Target the product page type and all visitors. Segments come later, and the engine proposes them.

4. Build two strategies

A strategy is an algorithm plus filters, created under Assets, then Strategies (strategies guide). For variation A take Purchased Together, which scores items bought in the same transaction over the past 6 months and recalculates every 24 hours. For variation B take User Affinity, which scores items against the visitor's own views, carts and purchases in real time, or NextML if your customer success manager has switched it on. Give both the same two filters, match the viewed item's category and exclude recent purchases, so any difference comes from the algorithm.

5. Allocate, and keep the control

Choose A/B test, 45% per variation and 10% control. In an A/B test the control is served no variation at all, which is the comparison you want, widget against no widget (traffic allocation guide). Dynamic allocation is the other route: a bandit that recalculates shares at least hourly on a 10/90 explore and exploit split, finds a leader fast and tells you less about why (dynamic allocation guide).

6. Make Add to Cart the primary metric

The primary metric declares the winner. The vendor's guidance is to measure the action the page is for, and for a product page that is Add to Cart, because revenue as the metric drops every visitor who carted and did not buy and takes longer to conclude (primary metric guide). Create a goal on the Add to Cart event and select it; purchases and revenue still appear as secondary metrics.

7. Publish and leave it alone

Publish and let whole weeks pass. The engine declares a winner when the Probability to Be Best crosses your threshold, or a tie when the expected loss from picking either is small enough (Bayesian guide). Peeking is allowed in this model, with the guide's caution attached, so look, but do not act on day three.

8. Read the audiences after 14 days

While the test runs, Predictive Targeting scans every audience for one that prefers the variation losing overall, ranks the resulting opportunities and shows the top 5 as cards beside the report (Predictive Targeting guide). The vendor's worked example is a test B leads overall while mobile users prefer A, where serving each group its own variation lifts expected purchases by 25%. The checks are strict: primary metric only, outliers removed, more than 14 days live, a declared winner on both halves, and at least 1% better than serving the leader to everyone.

9. Let the Analyst agent draft the readout, then decide

Experience OS Agents went live for every Experience OS customer on 29 July 2026: Dana for personalization strategy, David the analyst, Manu for design, Gemma for copy, Alex for implementation. David "analyzes performance, explains what changed, and recommends the next optimization opportunity", and the announcement keeps the marketer as the one who reviews, edits and approves what goes live (agents announcement). Ask David for the control group's purchases per user beside each variation's and read that row before his summary; no variation goes to all traffic on an agent's say-so.

The agents come included, and the first thing Dana is built to do is propose the next campaign; a programme that keeps launching experiences is a licence that renews, which is why the approval step the announcement leaves with you is the one that matters. On the shopper's side, Shopping Muse can answer product questions inside ChatGPT through an MCP server Dynamic Yield generates and hosts, with no development work (ChatGPT discovery guide).

What the guides warn about

• A control set to No experience cannot be combined with CTR as the primary metric or an attribution window that starts on a click (traffic allocation guide).
• Under dynamic allocation nothing resets when you edit; an edited or added variation gets a fresh share, and the score is computed from the last edit (dynamic allocation guide).
• Experience Email revenue can be attributed to more than one campaign and counted twice (Experience OS homepage guide).

Measuring what the widget is worth

The Experience OS Impact report runs a global comparison between users who get personalized experiences and users who do not (Experience OS homepage guide). It is still the platform reporting on the platform. The free check is in your own experience report: purchases per user in the control row against the winning variation, over whole weeks, with the attribution window written next to the number. Later, if you want that holdout comparison run across every channel rather than one widget, a causal attribution tool like Causality Engine does it. Disclosure: I founded it.

FAQ

Can I run this without a developer?

Not the first time. Feed sync, page context, events and cookies are developer steps in the implementation journey; after that the console does the rest.

Why does my test show no winner?

The engine declares a winner when the Probability to Be Best passes the threshold, or a tie when the expected loss drops below one; until then the report shows probabilities, not a verdict (Bayesian guide).

Does the control group cost sales?

In an A/B test the control sees no widget, and that is the price of knowing what the widget did. Under dynamic allocation a random-variation control of up to 10% is taken from the traffic the bandit already spends exploring, so the guide says it does not hurt the experience (dynamic allocation guide).