A/B Testing

Personyze A/B testing splits your audience across multiple variations of a campaign, measures each against your primary goal and supporting KPIs, and lets you promote the winner — manually after reviewing results, or automatically once a variation is conclusive.

Four short videos cover A/B testing end to end:

A/B testing, start to finish · 2:19
Who takes part, two versions of a store bar in groups A and B, an Original group that sees neither, auto-pick on purchases, and publishing to staging.

Auto-pick the winner · 1:43
What counts as winning, and handing traffic to the AI optimizer.

Reading the results · 1:50
The verdict in words, the confidence threshold, and who it applies to.

A full test, built from scratch · 4:26
A B2B example end to end: who takes part, what each version does, how you know.

An A/B Testing campaign starts with its test on, and any other campaign becomes a test with Turn on A/B testing on its Content step. But a good test doesn’t begin with the variations — it begins with targeting, because who you test on determines whether the results mean anything.

In New campaign, Toolkits › A/B testing holds five ways to test: a change on your site, a landing page, recommendations, an email and a push notification. Each opens ready to test. The playbook A/B test a change on your site tests two or three versions with 10% held back as a control group. See Playbooks.

New here? This quick walkthrough shows the whole flow. The full detail — with every screen — is below.

1. Start with targeting — aim the test at the right audience

Every campaign begins on the Visitor Targeting screen, where you define who the campaign — and therefore the test — applies to. For a test this is the most important step, because the audience is what keeps your data clean.

The tighter and more relevant your audience, the less noise in the results. Testing a pricing headline on everyone mixes in visitors who’d never buy, diluting the signal and slowing you down. Testing it only on high-intent visitors to the pricing page means:

No data pollution — the people in the test are the people the change is for, so the winner reflects real intent, not random traffic.
Faster significance — with less noise, each variation reaches a trustworthy result sooner.
Decisions you can act on — “B won for pricing-page visitors on mobile” is far more useful than “B won for all traffic.”

Everything below is the full targeting toolkit — the same rules, forecast, and controls available to any campaign.

The Visitor Targeting screen is where you define who the campaign applies to. Rules combine 70+ real-time visitor attributes into an expression that Personyze evaluates on every page view.

Adding rules: start empty, then pick the signals you need

A new campaign’s audience starts empty — it would match every visitor until you narrow it. Click + Add audience rule and pick a rule type: each pick becomes a new rule, joined to the others with “and”. To add a condition to a rule you already have, use its own + Condition.

The Add audience rule menu open on the Target step — + Add audience rule opens this menu: a search box, the tabs All, Visitor context, Behavior, Acquisition and the rest, the pinned Audiences, Existing campaign and Templates tabs, and Audience settings (Audience persistence, Overlaps) at its foot. Click to enlarge.

The menu has a search box that looks through every rule, and tabs for the rule families — Visitor context, Behavior, Acquisition, Conversions, E-commerce, Content and Integrations — plus Audiences, Existing campaign and Templates for rules that already exist (see the Add audience rule menu). The main families:

Visitor context & attributes — who and where the visitor is, right now:

Pages Visited — match on the full URL, a path, or a query parameter; check it on this page or during the visit.
Countries / Cities — geo-location down to city level from the visitor’s IP.
User / CRM / ABM data — known-visitor profile fields synced from your CRM or CDP (lifecycle stage, plan tier, lifetime value, industry, and custom fields).
Device & System — device type (mobile, tablet, desktop), operating system, and browser.
Visitor Type — new vs. returning, plus session count.
User Lists — match against a spreadsheet/CSV of users you upload.
Date & Time — a date range, day of week, or hour of day (useful for launches, sales windows, and business hours).
IP Addresses — specific IPs or CIDR ranges (handy for internal QA or named accounts).
Weather — the visitor’s local conditions by IP or GPS.

Behavior & engagement — what the visitor has actually done, this session or over time:

E-commerce — products they’ve viewed, added to cart, or purchased.
Product Interactions — counts across products (distinct, same, or total).
Last Product Interaction — their most recent shown / cart / purchase event, for “pick up where you left off” targeting.
Content Interaction — article views, favorites, and subscriptions.
Session Attributes — cookies, JavaScript variables, and form values captured on the page — your own client-side data.
Time on Site — total time in the current session.
Landing Page — the first page of the session.
Traffic Source — referrer, search engine, or campaign the visit came from.

Integrations — signals pulled live from connected tools. Known-visitor data flows in through our integrations with HubSpot, Salesforce, Pardot, Zoho, Tealium, Segment, and Zeotap; for anonymous B2B visitors, ABM providers can infer company name and firmographics so you can target by account before anyone fills out a form.

Each rule is a card under Audience rules, with its own Include / Exclude, Any condition / All conditions and, where the rule offers it, On this page / During the visit. Rules join with a connector pill — and, or or only one — and can be grouped like brackets in math. A rule can be as simple as “is on the pricing page” or as layered as “viewed the pricing page in the last 14 days and is on a paid plan, excluding anyone who opened a support ticket in the last 7 days.” How each part works: how a rule is built, and how rules combine.

An example audience

Here’s what the step looks like once several rules have been added. This is one example — not a default or a required set. On the right, the Audience story reads the rules back as one sentence, above the live forecast:

Audience rules as rule cards, with an Exclude rule, a group and the connectors — Four audience rules: Countries / cities set to Exclude (in red), a group in which Referring site (two conditions, Any condition) or Referrer type must match, and Pages visited checked On this page. On the right, the Audience story reads them back as one sentence. Click to enlarge.

AI Targeting Assistant

If you already know the audience you want in plain English, describe it to the AI assistant and it generates a matching rule set. You review, edit, and refine before saving — it’s a starting point, not a black box.

For example, typing “returning US visitors who viewed the pricing page but didn’t sign up in the last 30 days” produces a rule set like: Visitor Type is returning AND Countries is United States AND Pages Visited contains /pricing (in this session) AND NOT a sign-up event in the last 30 days — each as an editable rule you can adjust.

The assistant is a row at the top of Audience rules. Type or dictate the audience and press Generate rules: the new rules are added to yours. Turn on Replace my current rules to start fresh instead — the button then reads Generate and replace. When the description is unclear, the assistant asks a question first; Refine carries on the conversation, Start over forgets it, and Undo puts the rules back as they were.

The live audience forecast

As you edit rules, the Audience forecast in the right-hand column updates continuously, showing how much of your site’s last 90 minutes of traffic would match versus would not match. But the counters are only the summary — the real value is opening the forecast to see the actual visitors behind the numbers.

Expand it and you get two lists: the visitors who would match your current rules and the ones who would not. Click into any visitor to see their full detail — the pages they viewed, their session and device, their CRM/profile attributes, and exactly which of your rules they passed or failed. That’s how you confirm you’re targeting the right people, not just the right number of them: you can spot a visitor who slipped into the audience by accident, or a visitor you expected to match who didn’t, and trace it straight to the rule responsible.

Used this way, the forecast prevents the two most common audience mistakes before the campaign ever goes live:

Too narrow — a compound rule set that matches only a handful of visitors. The campaign runs but nobody sees it. An empty would-match list makes this obvious immediately, instead of weeks later.
Too broad — a rule you thought was specific that actually matches half your traffic. Reading the would-match list surfaces visitors who clearly don’t belong, so you can tighten the rules before launch.
Right count, wrong people — the number looks healthy but the audience is the wrong crowd. Only inspecting individual visitors catches this, and it’s the mistake the counters alone can never reveal.

The forecast also has a live-traffic mode — “Test rules on live visitors.” Instead of a rolling sample, Personyze evaluates your rule set against visitors arriving in real time and shows which ones would qualify, without saving anything. You can open a single live session and see it checked rule-by-rule, or scan the whole list of recent sessions flagged as matching or not — a final sanity check before you publish.

“Test rules on live visitors” checks your rules against real recent sessions. Opening one session shows it rule-by-rule — which conditions matched and which didn’t — so you can see exactly why a visitor would or wouldn’t qualify. Click to enlarge.

The session list view shows recent live sessions flagged as matching or not matching, so you can confirm your targeting is catching the right visitors before launch. Click to enlarge.

Group priority for overlapping audiences

When two campaigns can match the same visitor, an audience group decides which one wins. The Overlaps pill under the rules ticks the groups this campaign is in; Manage audience groups opens the groups themselves. Add the overlapping audiences to a shared group and give each a priority (lower number = higher precedence). The group’s conflict-resolution rule then picks the winner for a shared visitor — higher priority first, with ties broken by “first occurred” or “last occurred” — so a visitor who qualifies for both sees only the intended content.

When audiences overlap, the Audience groups modal lets you group the conflicting campaigns and set priority and decision order, so a visitor sees only the intended one. — The Audience groups modal: (1) your groups and “+ New group”; (2) the conflict-resolution rule (priority order, then first/last occurred); and (3) the audiences attached to the group. Click to enlarge.

Audience settings: persistence and overlaps

Under the rules, Audience settings holds two pills, also listed at the foot of the Add audience rule menu. Audience persistence controls how long a visitor stays in the audience after they match:

Default — the rules are checked on every page.
Only once during the session, where rules first matched.
Stays for a number of pages, then the rules are checked again — or with no re-entry this session.
Stays for a number of sessions or hours, then the rules are checked again.
Permanent — stays in the audience forever once matched.

For most acquisition campaigns, per-session persistence is right; for retention or loyalty campaigns, a permanent stay makes the experience follow the visitor across future sessions. The stay is remembered in the visitor’s browser, so changing the rules later does not remove anyone already captured. Overlaps puts the campaign in an audience group (above). An email campaign has neither: it matches its recipients when it sends.

Audience settings with the Audience persistence panel open — Under Audience settings, Audience persistence says how long a visitor stays in the audience once they match. Overlaps is the pill beside it. Click to enlarge.

How often a matched visitor sees the campaign is the Campaign cap, on the Content step’s Campaign settings card. Per-session caps keep the same popup from firing twice in one visit; per-visitor caps keep a returning visitor from seeing the same message every time; no cap suits content that should run every time, like a header banner. See Frequency caps.

Full rule-by-rule reference: Targeting Rules in Personyze.

2. Build the variations

With the audience set, open the campaign’s Content step. On a test it holds the A/B test card: the traffic-split bar on top, then a column per group — Group A, Group B and so on — beside Shown in every group and the Original (the card in full). Inside each group you add and edit actions exactly as you would in any campaign — everything below applies per group.

Once the audience is defined, the Content step is where you decide what those visitors actually see. Content in Personyze is built from actions — and a single campaign can hold as many as you need. A header bar, a personalized recommendation widget, a countdown timer, and an exit-intent popup can all live in the same campaign, each firing for the same audience. Every action carries its own configuration, delivery health, and click-through tracking.

Adding an action is a two-part choice: what type of action it is (which decides the editor you work in), and how it behaves (placement, trigger, timing). The rest of this step walks through the action types, starting with the two you’ll reach for most.

The A/B test card's columns: Shown in every group, Group A, Group B and the Original — Shown in every group (green) holds what everyone who matches sees, whichever group they are in. Under In the test, each group has its own column of action cards, and the Original comes last. Click to enlarge.

Move actions between groups. You don’t have to rebuild a variation from scratch — grab an action by its drag handle (the “≡” at the top-right of the card) and drop it into another group’s Add action zone. Hovering a card also shows quick controls to Edit, Detach, or Preview it. This makes it easy to start from the campaign’s base content and tweak just one element per variation.

Move to… opened on an action card in an A/B test — Each action card has Edit, Preview, Move to… and Detach. Move to… sends the action to another group or to Shown in every group, and you can also drag a card by its handle. Nothing is saved until you save the campaign. Click to enlarge.

The Content screen (above) is the campaign’s control room. The main list holds every action, each with a status pill (Live, Staging, Draft or Off), its delivery health, and the last time it fired, plus quick controls to edit, duplicate, or remove it. The right-hand rail keeps the last 90 minutes of activity, the campaign’s primary conversion goal, and shortcuts to version history and the full performance report always in view. To add an action, you pick its type — which is where the two main editors come in.

The action type you pick determines the editor you work in. Two cover most campaigns — the Live Editor and the Popup, Banner & HTML Builder — with a set of specialized action types for everything else.

Live Editor (WYSIWYG)

The Live Editor is Personyze’s most-used content tool: it opens your actual live site inside the editor and lets you edit any element on the page — text, style, layout, images, links, buttons — with click-to-edit precision. Changes are captured as a variation that Personyze injects for your audience only; the underlying site code isn’t touched.

The same content can carry different versions by case. With Split Cases, a single action holds a Default version plus extra cases — each shown only when its condition is met (most often the visitor’s location, but also device, audience, language, or a CRM / feed value). You switch between cases and edit each one right in the WYSIWYG editor (the Popup & Banner editor works the same way).

The real Personyze Variations (audience-based) modal with an If-visitor-is condition and Then-content-is value. — Inside Split Cases: each case has an “If visitor is…” condition (attribute / operator / value, with “+ Add rule”) and a “Then content is…” value. “+ Add case” adds more variations. Click to enlarge.

What the Live Editor covers:

Navigate vs. Edit modes — Edit mode makes every element clickable so you can pick something to change; Navigate mode disables editing so you can click links and interact with the page as a real visitor. Press Esc to return to Edit.
Basics panel — edit text (with a rich-text editor for bold/italic/lists/links), change styling (color, font, size, background), duplicate or delete an element, or drop into raw HTML source when needed.
Layout panel — move & resize by dragging corners, adjust spacing (margin/padding), change alignment for flex & grid layouts, hide/show per device breakpoint, preview responsive breakpoints, or reposition with z-index and pinning.
Insert panel — drop in new elements: images (upload or URL), links, buttons, reusable blocks, videos (YouTube / Vimeo / MP4), or raw custom HTML snippets.
Behavior panel — attach behavior to any element: run custom JavaScript on click, scroll to an anchor, delay or hover-triggered actions, count clicks toward the action’s stats, register the element as a conversion goal, or fire a custom event with a name and payload.
AI-assisted editing — for any selected element, ask the AI for style suggestions (“Match brand accent,” “Modern sans look,” “More contrast,” “Softer, rounded”) or text rewrites (“Make it punchier,” “Shorter,” “Benefit-focused,” “More urgent,” “Match brand voice”). The AI proposes changes you review, apply, or discard.
Page-level AI — for wholesale rewrites, describe the goal (“Rewrite this whole page for enterprise buyers”) and the AI proposes an integrated set of changes across the page.
Variation history — every element tracks how many variations have been created and lets you switch between them for comparison.

Popup, Banner & HTML Builder

When you want to put a new element on the page — a popup, a sticky header/footer bar, a floating bubble, a slide-in notification, a message box, an embedded in-page banner, or a fully custom HTML block — use the Popup, Banner & HTML Builder. It ships with production-ready templates for each format and supports live personalization tokens (e.g., “Hi Alex — your search for ‘noise-cancelling headphones’ deserves 20% off”) that resolve at render time from the visitor’s profile, session, or CRM data.

Despite the name, this builder isn’t limited to overlays that float on top of the page. Every content item can be pinned to a placeholder — a CSS selector you point at any element on your site. Set a placeholder and the content is injected at that spot, and it can replace the existing content there rather than only adding alongside it. That makes the same builder useful both for classic overlays (a popup, a corner toast) and for swapping in-page content (a hero banner, a promo strip, a block inside a product page) without touching your site’s code.

Edit any action with AI

Any action — a popup, banner, WYSIWYG edit, or block of HTML — can be edited with AI instead of by hand. Click Edit action with AI, describe the change in plain English, and Personyze rewrites the action’s HTML for you to review and apply.

The Edit-this-action-with-AI panel with a prompt box and Apply AI changes. — Open **Edit action with AI** and describe the change — the AI sees the current HTML and rewrites it. Examples: *“Make the background purple,” “Shorten the timer to 1 hour,” “Change CTA to Save now.”* Click to enlarge.

The AI can:

Restyle — colors, fonts, spacing, layout, button styles (“match our brand accent,” “more contrast,” “modern sans look”).
Rewrite copy — headlines, body, CTAs (“make it punchier,” “benefit-focused,” “more urgent,” “match brand voice”).
Change the creative — swap or adjust images, timers, coupon codes, and other template elements.
Insert personalization — add Personyze variables like the visitor’s first name, city, or a searched keyword so the action fills in per visitor.

The AI proposes the change; you review, apply, or discard — nothing goes live until you accept it.

The builder walks through three stages:

1 · Template. Start from a format-specific template, filterable by type: Bar (slim sticky top/bottom bar, or a rich bar with image & CTA), Embedded Banner (inline in a specific page slot), Message Box (inline messaging with CTAs), Popup (modal overlay — signup / lead form, or countdown promo), Notification (slide-in corner toast), Floating (floating bubble / launcher), and Custom HTML / Blank (raw markup). Any template can embed a product-recommendation widget powered by your catalog (Bought Together, Countdown, Grid, Slider, and more), and there’s an “Edit this action with AI” option to generate or restyle it from a prompt.

2 · Customize look & feel. Adjust the design — layout, colors, fonts, imagery, buttons, and responsive behavior across base / hover states and phone / tablet / desktop breakpoints — and drop in personalization tokens that resolve per visitor.

3 · Placement, trigger & frequency. This stage decides where, when, and how often the content shows:

Placement — for overlays, choose a screen position (top/bottom, left/center/right, or centered). For in-page content, set a placeholder selector to inject into — or replace — an element on the page. Personyze also lists every other content item already using that same placeholder, so you can see what competes for the slot.
Trigger — what makes it appear: as soon as the page loads, after a delay, after the visitor is idle for a while, after scrolling to a point, when they scroll back to the top, when text is copied, on predicted leave intent (exit intent), on a custom JS event, or from a click-to-open launcher.
Frequency & close behavior — frequency caps keep it from feeling like spam: show a close (×) button and, if closed, suppress it for N sessions; hide it after the visitor has closed it a set number of times; or hide it after they’ve clicked it a set number of times.

Full guide: Popup & Banner Action Guide.

Other action types

Personyze also supports these action types, each with its own dedicated guide:

Product Recommendations — dedicated recommendation widgets (Bought Together, Also Viewed, Trending, Personalized by browsing history, Category-based, and more) powered by your product catalog feed.
Countdown Timer — add urgency with a live countdown to a specific date/time, a per-visitor session countdown, or a rolling window.
Redirect to URL — send matching visitors to a different URL entirely. Useful for A/B testing landing pages or routing paid traffic to a segment-specific page.
JavaScript Action — run arbitrary JavaScript when the audience matches. Used for anything the built-in action types don’t cover — talking to a third-party API, mutating page state, firing tracking events, or wiring up something custom.
Email actions — personalized email sends including cart abandonment, re-targeting, and campaign-driven sends, with recommendations embedded at open-time.
Web Push Notifications — reach visitors who’ve granted push permissions with time-of-day or event-triggered notifications.

3. Configure the test

An A/B Testing campaign starts with its test on: two groups at 50% each. Any other campaign becomes a test from its Content step: open the Campaign settings card (Change on its folded line) and, on the A/B test row, press Turn on A/B testing. The row then names what is being tested — each group as a chip with its share — and Turn off ends the test, moving the groups’ actions into the content everyone sees. On an email campaign, Turn off asks which email to keep; the others are switched off, not deleted.

The test itself is the A/B test card on the Content step. From top to bottom:

The header — where the test stands (Draft — starts when you publish, or Running · day N), Test type (A/B test or Multivariate), and Full / Compact. Compact shows each action as one short row, so more fits and dragging between groups is easier.
Traffic split — a bar with each group’s share in its own colour. Drag the handles, or use − / + on a group. + Add group adds a version. The shares add up to at most 100%, and what they leave goes to the Original. Assign by sits beside it (below).
Picking the winner — one line with the goal, what decides and how close the test is; Configure opens it (below).
Shown in every group — the green column on the left: content outside the test, shown to everyone the campaign matches, whichever group they are in. Nothing in it is compared. Its Add action is always there, and actions drag into it and back out.
In the test — a column per group in the group’s colour, with its name, its share and Remove group, and the Original last. Each visitor sees one of them, by the traffic split.

The A/B test card of a running test — The A/B test card: Running · day 34, Test type (A/B test or Multivariate), Full / Compact, the Traffic split bar (here AI-managed) with + Add group and Assign by, the Picking the winner line with Configure, and the groups’ columns. Click to enlarge.

The Original is your site with none of the tested actions; everything under Shown in every group still shows. Its card spells out its share — 100% minus the groups’ shares. At 0% nobody is in it, and the groups are compared only with each other.

Changed something mid-test? New groups, a different split or an edited action change what is being measured. Read the results from the change with the Performance step’s From last edit range.

How visitors are assigned: Assign by

Assign by, beside the traffic split, decides how a person is put in a group. Email and push campaigns do not have it.

Choice	What it does
Visitor	Each visitor stays in one group, on every visit. The classic A/B test, for funnels and anything where consistency matters.
Visit	A visitor can get a different group on each visit. Good for banners and headlines.
Page view	Each page view picks a group at random, so a visitor can see several. Rarely what a real test needs.

Under Same group across campaigns, two more choices hold a shared control group across your whole program — so you can measure the total lift of your personalization, not just one test:

Visitor (fixed id) — a visitor is in the same group in every campaign with this mode and the same split. Give the Original the same share in each, and the same visitors are held out everywhere.
Visit (fixed id) — the same, for each visit.

A fixed-id split keeps each group’s place, so it cannot be used with the AI optimizer, which moves the shares every night: choose Pick a winner under Picking the winner to use it.

A campaign saved with its groups on No split shows every group to everyone, which is no test at all: its Content step says No A/B test, and Turn on A/B testing makes it a visitor split at equal shares.

The Assign by menu of an A/B test — Assign by: Visitor, Visit or Page view, and under Same group across campaigns, Visitor (fixed id) and Visit (fixed id), which work with Pick a winner, not with the AI optimizer. Click to enlarge.

Picking the winner

Folded, Picking the winner is one line: the goal, and whether Auto-pick is on and with which preset, or the AI optimizer. Configure opens two columns — the choices on the left, what they lead to on the right; Collapse folds it again. It turns “is B really better?” into explicit rules instead of a gut call.

Goals — pick one or more. The first is the primary goal, and the winner is judged on it; the others are still tracked, so a test can’t “win” by trading your real objective for a vanity metric. + New goal makes a goal only for this campaign, without using one of your account’s goal slots.
How this test decides:
- Pick a winner — an experiment with an end. Traffic stays as you set it while the versions compete; when one proves itself at the thresholds below, it is promoted and the test concludes.
- Optimize continuously (AI optimizer) — no end. Every night the split shifts toward the version performing now, always keeping 10% exploring, and the split is marked AI-managed. It needs at least two versions besides the Original, and starts moving the split once the campaign is live.
Auto-pick winner — on, Personyze picks the winner and gives it the traffic once the result is significant on your primary goal. Off, it only recommends; the strategy, the readiness and When it decides stay on screen in grey, so you can see what turning it on would do.
Decision strategy — how strict the bar is, as a trade-off between speed and certainty:
- Cautious — highest confidence and the most data before deciding; fewest false winners, slowest to call. Best for high-stakes changes (pricing, checkout).
- Balanced — the recommended middle ground for most tests.
- Fast — lower thresholds so you decide sooner; good for low-risk copy or creative tweaks where a wrong call is cheap.
- Custom — set the confidence, sessions, days and minimum improvement over the Original yourself.
Shift traffic toward the leader moves the split toward the likely winner while the test runs.
When it decides — Give the winner all tested traffic, pause the rest, and Send an email when it decides to the addresses you list, so a decision never happens silently.

Each preset is really a combination of three thresholds, all of which must be satisfied before a winner is declared:

Confidence — the statistical certainty that the leader’s lift is real and not noise (for example, 95%). Higher confidence = fewer false positives, but more data needed.
Minimum sessions — a floor on how many sessions each version must collect, so a tiny early sample can’t trip a decision on a lucky streak.
Minimum duration — a floor on how long the test runs, so it spans full weekly cycles (weekday vs. weekend behavior) rather than a single unrepresentative day.

Readiness to decide shows how close the test is to clearing each one — statistical confidence, sessions in test, test duration and the improvement vs. the Original — with the current leader. A version can lead on conversion yet still be “not ready” because it hasn’t hit the session floor or the minimum run time, which is exactly the mistake the thresholds exist to prevent.

What happens when a winner is decided. Promoting a winner isn’t just a notification — it changes how traffic is served:

The winner takes the tested traffic. The winning version takes all the traffic that is in the test, and the other versions stop showing. If the test has an Original share, it keeps it, so the campaign can still be measured.
The winner becomes the campaign’s content. Once promoted, that version is effectively the live experience; you can keep the campaign running as a normal (non-test) campaign from there.
With auto-pick on, Personyze does this automatically once the thresholds are cleared, and emails whoever you listed. With auto-pick off, nothing shifts on its own — you get the recommendation, and the traffic only moves when you promote it. Until then the versions keep splitting traffic as before.

On an email or push campaign, Picking the winner is the confidence alone: the winner is called on click rate once each email has been sent enough times, and nothing is switched automatically — moving the split after messages went out would send the winner to people who already got another version.

Count the Original only on a page group

A group’s action can be limited to a page group with Restrict to in the action’s editor. Such a group is counted only on those pages, while the Original is counted on every page the campaign matches — so the Original fills with visitors who never reached the group’s pages, and the two sides compare different people.

On a website campaign whose tested content is limited this way, the Original’s card offers Count the Original only on <page group>. Ticked, the Original is counted only on the pages the tested content can show on, so both sides compare the same visitors. Visitors see nothing different — only who is counted changes — and it applies from the next publish. The card then reads Counted only on <page group>.

Groups limited to different page groups: the Original is counted wherever any group can show, so each group is compared with a slightly different set of visitors.
A group that shows on every page keeps the Original counted everywhere.
Not offered on email or landing page campaigns, or while the Original’s share is 0%.

Testing a landing page

On a landing page campaign, Turn on A/B testing asks What do you want to test?

The actions on the page — everyone lands on the same page; what changes between groups is the popup, bar or exit-intent shown on it.
The page itself — each group gets its own page, all served at the same campaign address. Group A takes the page you already have; for Group B pick A copy of your page (the usual way: change one thing and compare), A blank page, or A page I already have on your own site, which needs the Personyze tracking code to be measured. You can leave the choice for later: the group’s card asks again.

Then Start testing. A landing page test has no Original: everyone who opens the address gets one group.

A survey, review or booking page made only for its page is tested as a landing page is: Test your survey page (review page, booking page), no Original, the split always totals 100%, and Group B is a copy of the page or a fresh form from the gallery — see A survey’s own page › A/B testing the page.

What do you want to test? on a landing page campaign — On a landing page campaign, Turn on A/B testing asks What do you want to test? Picking The page itself then asks Where does Group B’s page come from?: A copy of your page, A blank page or A page I already have. Click to enlarge.

Multivariate: test several elements at once

An A/B test compares whole versions. Multivariate tests individual elements in combination: pick Multivariate as the Test type and create variables — each variable is one thing you’re varying — with an option for each way it could look. A visitor sees one option from each variable, and Personyze builds every combination of them.

The A/B test card set to Multivariate — Test type Multivariate, in the Compact view: two variables with two options each, each variable’s control marked CONTROL, and the Combinations table that crosses them (2 × 2 = 4). The variable and option names are examples. Click to enlarge.

Multivariate measures each element’s individual impact and how elements interact, so you learn which specific change drove the result — at the cost of needing more traffic, since every combination has to accumulate enough data.

4. Read the results

The results dashboard is built to answer one question first — which variation won, and can I trust it?

The A/B test results overview. — The results dashboard is built to answer one question first — which variation won, and can I trust it? Click to enlarge.

The A/B results dashboard with control, variations, winner, and daily trend. — The results for the deciding goal: each variation’s conversion rate against the baseline, with the lift, how sure the test is and the 95% range of the lift, then the rate over time. Tabs switch between goals. Click to enlarge.

Tab per KPI — the primary goal first, then supporting KPIs (clicks, conversions, revenue).
Winner recommendation — surfaced when one variation leads on every conclusive metric with no negative side effects.
Delivery health — confirms each variation was actually injected, so a “loser” isn’t just a variation that failed to render.
Data readiness — for any KPI short of 95% confidence, an estimate of how much longer it needs.
By segment — break results down by new vs. returning, country, device, and more.

A campaign with no control. A campaign whose content all sits in one always-shown group has no test version: every visitor gets the same content, so there is no control group (Site original) and nothing to compare. Its A/B testing tab says so instead of showing results. To measure it, put the content in a test version with a share of the traffic on the Content step — the visitors left over are the control.

A campaign-dedicated goal can decide the test. A goal attached to just this campaign can be picked as the test’s goal, and with auto-pick winner on it is offered as the default winner criterion — so a test is settled by the goal built for exactly that campaign. See conversion tracking.

Reading the numbers: A/B and multivariate tests on the Performance Metrics page explains every figure on this screen and how to read it.

5. Promote the winner

Manual — review the dashboard and promote the recommended variation; it becomes the campaign’s content and the others stop.
Auto-pick — turn on Auto-pick winner under Picking the winner and Personyze does the same automatically once the test is conclusive under your decision strategy.

If you fixed something mid-test (say, a selector), read the results from the fix with the Performance step’s From last edit range.

A different winner for some audiences? When Audience Discovery is on, Audiences → A/B tests by audience shows where another version wins — on phones, say — and lets you apply each audience’s winner. See Apply A/B test winners to their audiences.

What to test

Copy — headlines, CTAs, value props, social-proof badges.
Design — hero images, layouts, color schemes, button styles.
Offers — discount sizes, trial length, “no credit card” framing, urgency.
Recommendations — different recommendation algorithms against each other.
Targeting — the same content against different audience definitions, to find which segment converts.

QA before going live

The campaign’s QA step says whether it is Ready or Almost ready, with the split between its test versions under Who it reaches → Details — see The QA step.

Before you publish a test to live, publish it to staging and use the QA Simulator to preview each variation as different visitor profiles — confirming the right variation shows for the right audience on the real page.

The QA Simulator previewing a campaign as a simulated visitor. — The QA Simulator opens the real page with the campaign injected and lets you impersonate visitors — so you can confirm each variation renders correctly for the audience it targets. Click to enlarge.

Impersonate the visitor profiles each variation is meant for, and confirm the right one shows:

QA Simulator session-data panel for impersonating a visitor. — Impersonate a visitor: (1) switch identity with a new user ID; (2) control the session; and (3) override CRM / custom attributes — to test that the test targets the right people. Click to enlarge.

QA Simulator targetings panel showing rule-by-rule match. — The targetings panel shows, rule-by-rule, whether the simulated visitor qualifies — so a “not matched” result is never a mystery. Click to enlarge.

You can also confirm the variation’s actions actually fire and render on the real page:

QA Simulator campaign-content panel listing each action and status. — The campaign-content panel lists each action, its type, whether it executed, and its status — and lets you trigger one manually to preview it. Click to enlarge.

Publish to staging and QA it there first, fix anything the simulator surfaces, then publish the campaign to live.

Frequently asked questions

How does A/B testing work in Personyze?

Build two or more variations of your content, then turn on A/B testing to split your targeted audience across them. Personyze tracks impressions, clicks, and conversions per variation and reports which performs best, with statistical confidence.

What can I A/B test?

Anything you can build as a Personyze action – popups, banners, in-page content changes, product and content recommendations, landing page elements, CTAs, and more. You can also test personalization against a control group that sees nothing.

How are visitors assigned to variations?

You choose under Assign by: Visitor (a visitor always sees the same group), Visit or Page view. You set each group’s share on the split bar, and the share the groups leave is the Original — your site without the tested content — which measures true lift versus no personalization.

How does Personyze decide the winner?

You set a decision strategy – a preset (Cautious, Balanced, Fast) or custom thresholds for statistical confidence, minimum sessions, and minimum duration. When a variation meets the bar, Personyze can automatically shift traffic to it.

What is a control group and should I use one?

A control group is a slice of your audience that sees no personalization. Including one lets you measure the real lift your variations produce versus doing nothing – recommended whenever you want to prove impact.

What’s the difference between A/B and multivariate testing?

A/B testing compares whole variations against each other. Multivariate testing varies several elements at once and measures how combinations perform, helping you find which specific elements drive results.

Targeting and Personalization — the full campaign lifecycle.
Targeting Rules in Personyze — every audience rule type.
Personyze QA Simulator — preview as different visitor profiles before going live.
AI Personalization: Versions and Results — versions written for each segment, each measured against your page as it is

A/B or Multivariate — two test types

The Content step’s Test type toggle picks the experiment shape:

A/B test — compares whole variants (A vs B vs C). Each group holds its own set of actions, the traffic split slider divides visitors, and Assign by chooses how a person gets a group: Visitor (the classic — each person keeps their group), Visit (a different group per visit — good for banners) or Page view (every page view draws fresh).
Multivariate — names the individual things you’re changing, as variables with options, and tests every combination.

How multivariate works

A variable is one thing you are changing — the layout, the headline, the price line — and each option in it is one way that thing could look. Personyze builds every combination: two variables with two options each make 2 × 2 = 4 combinations, and each visitor lands in exactly one, seeing one option from each variable. Because every option is tried alongside every other, the results can say what each variable did on its own — pooled across all combinations it appears in — and where two changes only work together, which comparing whole variants cannot separate.

The cost is traffic: every combination needs its own sample, so adding a variable multiplies what the test needs, while adding an option to an existing variable only adds to it. The editor shows the live combination count, lets you mark a control per variable, switch options off (they leave the split but keep their history), and estimates how long the test will take at your traffic.

Building the variations — with the live editor and AI

Variants don’t have to be popups: a test group can hold a WYSIWYG live-editor action that changes your page itself — and the new live editor does much of the work for you:

Page-level AI — one prompt per variant: “rewrite this page for enterprise buyers” for group B, “make the calls to action more urgent” for group C. Every AI edit lands as a reviewable, undoable change.
Per-field AI — ask for a replacement headline or button label while editing one element.
Show original flips between the variant and the untouched page, so you always know what you’re testing against.

AI edit applied and logged — The AI rewrote the hero headline to “Sound. Everywhere.”, explained what it did, and logged it as a reviewable TEXT change with Undo / Show original — with the monthly AI-actions quota shown. Click to enlarge.

The full AI roster is on AI in Personyze.

Reading the results

The campaign’s Performance → A/B testing tab is where the test reports itself: total visitors and the live traffic split, each variation’s conversion rate, and per-goal bars comparing every group against the holdout baseline with lift and confidence intervals. The verdict banner says exactly where things stand — e.g. “Keep running — no conclusive winner yet: the leader is at 62% confidence on the primary goal, below your 98% threshold” — so you never have to eyeball significance yourself.

A/B test performance — A live test: traffic split, per-variation conversion, and the primary-goal comparison with confidence. Click to enlarge.