Personyze Wiki Personyze Wiki docs
Open Personyze
Campaigns

A/B Testing

Split your audience across multiple variations of any Personyze campaign, measure performance against your primary goal and supporting KPIs, and promote the winner manually or automatically. Native A/B and multivariate…

Updated 7 days ago 24 min read
A
by Admin


Personyze A/B testing splits your audience across multiple variations of a campaign, measures each against your primary goal and supporting KPIs, and lets you promote the winner — manually after reviewing results, or automatically once a variation is conclusive.

There’s no separate “activate testing” step. The moment you add a second variation to a campaign, it starts in Test mode and begins splitting traffic. But a good test doesn’t begin with the variations — it begins with targeting, because who you test on determines whether the results mean anything.

1. Start with targeting — aim the test at the right audience

Every campaign begins on the Visitor Targeting screen, where you define who the campaign — and therefore the test — applies to. For a test this is the most important step, because the audience is what keeps your data clean.

The tighter and more relevant your audience, the less noise in the results. Testing a pricing headline on everyone mixes in visitors who’d never buy, diluting the signal and slowing you down. Testing it only on high-intent visitors to the pricing page means:

  • No data pollution — the people in the test are the people the change is for, so the winner reflects real intent, not random traffic.
  • Faster significance — with less noise, each variation reaches a trustworthy result sooner.
  • Decisions you can act on — “B won for pricing-page visitors on mobile” is far more useful than “B won for all traffic.”

Everything below is the full targeting toolkit — the same rules, forecast, and controls available to any campaign.

The Visitor Targeting screen is where you define who the campaign applies to. Rules combine 70+ real-time visitor attributes into an expression that Personyze evaluates on every page view.

Adding rules: start empty, then pick the signals you need

A new campaign’s audience starts empty — it would match every visitor until you narrow it. You build the audience by clicking Add targeting rule and choosing from the rule categories in the picker. Each category you add becomes a rule group on the screen; a campaign only shows the groups you’ve actually added, in whatever combination you need.

The Add-targeting-rule picker, grouped into Visitor context, Behavior, and Integrations, with tabs to filter.
Click “Add targeting rule” to open the picker. Each category you choose adds a rule group to the audience. Click to enlarge.

The picker organizes rules under three tabs — Visitor context, Behavior, and Integrations — so you can jump straight to the kind of signal you want. Every rule answers a different question about the visitor:

Visitor context & attributes — who and where the visitor is, right now:

  • Pages Visited — match on the full URL, a path, or a query parameter; scope it to the current page or anywhere in the session.
  • Countries / Cities — geo-location down to city level from the visitor’s IP.
  • User / CRM / ABM data — known-visitor profile fields synced from your CRM or CDP (lifecycle stage, plan tier, lifetime value, industry, and custom fields).
  • Device & System — device type (mobile, tablet, desktop), operating system, and browser.
  • Visitor Type — new vs. returning, plus session count.
  • User Lists — match against a spreadsheet/CSV of users you upload.
  • Date & Time — a date range, day of week, or hour of day (useful for launches, sales windows, and business hours).
  • IP Addresses — specific IPs or CIDR ranges (handy for internal QA or named accounts).
  • Weather — the visitor’s local conditions by IP or GPS.

Behavior & engagement — what the visitor has actually done, this session or over time:

  • E-commerce — products they’ve viewed, added to cart, or purchased.
  • Product Interactions — counts across products (distinct, same, or total).
  • Last Product Interaction — their most recent shown / cart / purchase event, for “pick up where you left off” targeting.
  • Content Interaction — article views, favorites, and subscriptions.
  • Session Attributes — cookies, JavaScript variables, and form values captured on the page — your own client-side data.
  • Time on Site — total time in the current session.
  • Landing Page — the first page of the session.
  • Traffic Source — referrer, search engine, or campaign the visit came from.

Integrations — signals pulled live from connected tools. Known-visitor data flows in through our integrations with HubSpot, Salesforce, Pardot, Zoho, Tealium, Segment, and Zeotap; for anonymous B2B visitors, ABM providers can infer company name and firmographics so you can target by account before anyone fills out a form.

Each rule supports include/exclude per condition. Parentheses nest logic; rule groups combine with AND, OR, or XOR. A rule can be as simple as “is on the pricing page” or as sophisticated as “has viewed the pricing page in the last 14 days AND is on a paid plan AND has NOT opened a support ticket in the last 7 days.”

An example configured audience

Here’s what the screen looks like once several rule groups have been added and configured. This is one example — not a default or a required set — combining session, CRM, geo, device, pages-visited, and visitor-type rules:

Example Personyze Visitor Targeting screen with several configured rule groups numbered and described.
An example audience with several rule groups added and configured — your own campaign will contain whatever groups you choose. Click to enlarge.
Personyze page and session targeting rules with the current-page vs. during-session scope selector.
Page & session rules decide who qualifies based on what a visitor is doing. The scope selector on each rule — “On current page” vs. “Anywhere in this session” — is the key: the first checks only the page they’re on now, the second checks their whole visit. Click to enlarge.

AI Targeting Assistant

Annotated AI Targeting Assistant showing the plain-English input, Append/Replace toggle, and Generate rules button.
The AI Targeting Assistant: describe the audience in plain English, choose Append or Replace, then Generate rules. Click to enlarge.

If you already know the audience you want in plain English, describe it to the AI assistant and it generates a matching rule set. You review, edit, and refine before saving — it’s a starting point, not a black box.

For example, typing “returning US visitors who viewed the pricing page but didn’t sign up in the last 30 days” produces a rule set like: Visitor Type is returning AND Countries is United States AND Pages Visited contains /pricing (in this session) AND NOT a sign-up event in the last 30 days — each as an editable rule group you can adjust.

The assistant has two modes, shown on the bar: Append adds the generated rules to whatever you already have, while Replace swaps in a fresh rule set. Use Append to layer on an extra condition, Replace to start over from a description.

The live audience forecast

As you edit rules, the forecast updates continuously, showing how much of a rolling sample of your recent traffic would match versus would not match. But the counters are only the summary — the real value is opening the forecast to see the actual visitors behind the numbers.

Expand it and you get two lists: the visitors who would match your current rules and the ones who would not. Click into any visitor to see their full detail — the pages they viewed, their session and device, their CRM/profile attributes, and exactly which of your rules they passed or failed. That’s how you confirm you’re targeting the right people, not just the right number of them: you can spot a visitor who slipped into the audience by accident, or a visitor you expected to match who didn’t, and trace it straight to the rule responsible.

Used this way, the forecast prevents the two most common audience mistakes before the campaign ever goes live:

  • Too narrow — a compound rule set that matches only a handful of visitors. The campaign runs but nobody sees it. An empty would-match list makes this obvious immediately, instead of weeks later.
  • Too broad — a rule you thought was specific that actually matches half your traffic. Reading the would-match list surfaces visitors who clearly don’t belong, so you can tighten the rules before launch.
  • Right count, wrong people — the number looks healthy but the audience is the wrong crowd. Only inspecting individual visitors catches this, and it’s the mistake the counters alone can never reveal.

The forecast also has a live-traffic mode — “Test rules on live visitors.” Instead of a rolling sample, Personyze evaluates your rule set against visitors arriving in real time and shows which ones would qualify, without saving anything. You can open a single live session and see it checked rule-by-rule, or scan the whole list of recent sessions flagged as matching or not — a final sanity check before you publish.

“Test rules on live visitors” checks your rules against real recent sessions. Opening one session shows it rule-by-rule — which conditions matched and which didn’t — so you can see exactly why a visitor would or wouldn’t qualify.
“Test rules on live visitors” checks your rules against real recent sessions. Opening one session shows it rule-by-rule — which conditions matched and which didn’t — so you can see exactly why a visitor would or wouldn’t qualify. Click to enlarge.
The session list view shows recent live sessions flagged as matching or not matching, so you can confirm your targeting is catching the right visitors before launch.
The session list view shows recent live sessions flagged as matching or not matching, so you can confirm your targeting is catching the right visitors before launch. Click to enlarge.

Group priority for overlapping audiences

When two campaigns can match the same visitor, an audience group decides which one wins. Add the overlapping audiences to a shared group and give each a priority (lower number = higher precedence). The group’s conflict-resolution rule then picks the winner for a shared visitor — higher priority first, with ties broken by “first occurred” or “last occurred” — so a visitor who qualifies for both sees only the intended content.

When audiences overlap, the Audience groups modal lets you group the conflicting campaigns and set priority and decision order, so a visitor sees only the intended one.
The Audience groups modal: (1) your groups and “+ New group”; (2) the conflict-resolution rule (priority order, then first/last occurred); and (3) the audiences attached to the group. Click to enlarge.

Audience persistence & frequency cap

Audience persistence controls how long a visitor stays in the audience after they match. Options range from “just this page view” (re-evaluate on every page) to “this session” to “for N hours” to “for N sessions” to “permanently once matched.” For most acquisition campaigns you want per-session persistence; for retention or loyalty campaigns you often want persistent membership so the experience follows the visitor across future sessions.

Audience persistence controls how long a matched visitor stays in the audience — re-checked on every page (default), once per session, after N pages or sessions, or permanently — alongside the frequency cap.
Audience persistence controls how long a matched visitor stays in the audience — re-checked on every page (default), once per session, after N pages or sessions, or permanently — alongside the frequency cap. Click to enlarge.

Frequency cap limits how often the campaign is allowed to show to a single visitor. Per-session caps prevent the same popup from firing twice in one visit. Per-visitor lifetime caps prevent a returning visitor from seeing the same message every time they come back. You can also remove the cap for content that should run every time (a persistent header banner, for example).

Full rule-by-rule reference: Targeting Rules in Personyze.

2. Build the variations

With the audience set, open the campaign’s Content step — which is already the A/B test view. A strip of variation groups runs across the top with the traffic-split bar above them; the campaign is in Test mode the moment a second variation exists. Inside each variation group you add and edit actions exactly as you would in any campaign — everything below applies per variation.

Once the audience is defined, the Content step is where you decide what those visitors actually see. Content in Personyze is built from actions — and a single campaign can hold as many as you need. A header bar, a personalized recommendation widget, a countdown timer, and an exit-intent popup can all live in the same campaign, each firing for the same audience. Every action carries its own configuration, delivery health, and click-through tracking.

Adding an action is a two-part choice: what type of action it is (which decides the editor you work in), and how it behaves (placement, trigger, timing). The rest of this step walks through the action types, starting with the two you’ll reach for most.

The Content step before A/B testing: your campaign’s actions are listed, with an “Activate A/B testing” bar to split traffic across variations.
The A/B test lives right in the Content step: (1) choose A/B or Multivariate; (2) drag the traffic-split bar to allocate each group’s share; (3) the green “+” adds a new variation group; (4) each group header is one version; and (5) “Add action” drops actions into a group. Click to enlarge.

Move actions between groups. You don’t have to rebuild a variation from scratch — grab an action by its drag handle (the “≡” at the top-right of the card) and drop it into another group’s Add action zone. Hovering a card also shows quick controls to Edit, Detach, or Preview it. This makes it easy to start from the campaign’s base content and tweak just one element per variation.

Dragging an action card from one A/B group to another.
Reuse actions across variations by dragging them: (1) grab a card by its drag handle; (2) the hover toolbar lets you Edit, Detach, or Preview; and (3) drop it into another group’s “Add action” zone. Click to enlarge.
Adding an action opens the action menu — Popup/Banner & HTML, WYSIWYG visual editor, Run JavaScript, Countdown Timer, Redirect, Product Recommendations, Replace text/links/images, Lead Capture Form, and Inject Custom CSS.
Adding an action opens this panel — search or pick a type: Popup/Banner & HTML Builder, WYSIWYG Visual Editor, Run JavaScript, Countdown Timer, Redirect, Product Recommendations, Replace text/links/images, Lead Capture Form, and more. Click to enlarge.

The Content screen (above) is the campaign’s control room. The main list holds every action, each with a status pill (Live / Draft / Test), its delivery health, and the last time it fired, plus quick controls to edit, duplicate, or remove it. The right-hand rail keeps the last 90 minutes of activity, the campaign’s primary conversion goal, and shortcuts to version history and the full performance report always in view. To add an action, you pick its type — which is where the two main editors come in.

The action type you pick determines the editor you work in. Two cover most campaigns — the Live Editor and the Popup, Banner & HTML Builder — with a set of specialized action types for everything else.

Live Editor (WYSIWYG)

The Live Editor is Personyze’s most-used content tool: it opens your actual live site inside the editor and lets you edit any element on the page — text, style, layout, images, links, buttons — with click-to-edit precision. Changes are captured as a variation that Personyze injects for your audience only; the underlying site code isn’t touched.

Personyze Live Editor (WYSIWYG) annotated: click any element, Navigate/Edit mode toggle, floating toolbar with Basics/Layout/Insert/Behavior panels, and AI editing.
The Live Editor opens your real page — click any element to edit it. Click to enlarge.

The same content can carry different versions by case. With Split Cases, a single action holds a Default version plus extra cases — each shown only when its condition is met (most often the visitor’s location, but also device, audience, language, or a CRM / feed value). You switch between cases and edit each one right in the WYSIWYG editor (the Popup & Banner editor works the same way).

Vary-by-audience icon in the Popup/Banner editor opens Split Cases; preview shows the personalized result.
Every property in the Popup/Banner editor has a “vary by audience” icon (1). Click it to open Split Cases; the live preview (3) shows the personalized result — here the city, first name, and searched product are all filled per visitor. Click to enlarge.
The real Personyze Variations (audience-based) modal with an If-visitor-is condition and Then-content-is value.
Inside Split Cases: each case has an “If visitor is…” condition (attribute / operator / value, with “+ Add rule”) and a “Then content is…” value. “+ Add case” adds more variations. Click to enlarge.

What the Live Editor covers:

  • Navigate vs. Edit modes — Edit mode makes every element clickable so you can pick something to change; Navigate mode disables editing so you can click links and interact with the page as a real visitor. Press Esc to return to Edit.
  • Basics panel — edit text (with a rich-text editor for bold/italic/lists/links), change styling (color, font, size, background), duplicate or delete an element, or drop into raw HTML source when needed.
  • Layout panel — move & resize by dragging corners, adjust spacing (margin/padding), change alignment for flex & grid layouts, hide/show per device breakpoint, preview responsive breakpoints, or reposition with z-index and pinning.
  • Insert panel — drop in new elements: images (upload or URL), links, buttons, reusable blocks, videos (YouTube / Vimeo / MP4), or raw custom HTML snippets.
  • Behavior panel — attach behavior to any element: run custom JavaScript on click, scroll to an anchor, delay or hover-triggered actions, count clicks toward the action’s stats, register the element as a conversion goal, or fire a custom event with a name and payload.
  • AI-assisted editing — for any selected element, ask the AI for style suggestions (“Match brand accent,” “Modern sans look,” “More contrast,” “Softer, rounded”) or text rewrites (“Make it punchier,” “Shorter,” “Benefit-focused,” “More urgent,” “Match brand voice”). The AI proposes changes you review, apply, or discard.
  • Page-level AI — for wholesale rewrites, describe the goal (“Rewrite this whole page for enterprise buyers”) and the AI proposes an integrated set of changes across the page.
  • Variation history — every element tracks how many variations have been created and lets you switch between them for comparison.

Popup, Banner & HTML Builder

When you want to put a new element on the page — a popup, a sticky header/footer bar, a floating bubble, a slide-in notification, a message box, an embedded in-page banner, or a fully custom HTML block — use the Popup, Banner & HTML Builder. It ships with production-ready templates for each format and supports live personalization tokens (e.g., “Hi Alex — your search for ‘noise-cancelling headphones’ deserves 20% off”) that resolve at render time from the visitor’s profile, session, or CRM data.

Despite the name, this builder isn’t limited to overlays that float on top of the page. Every content item can be pinned to a placeholder — a CSS selector you point at any element on your site. Set a placeholder and the content is injected at that spot, and it can replace the existing content there rather than only adding alongside it. That makes the same builder useful both for classic overlays (a popup, a corner toast) and for swapping in-page content (a hero banner, a promo strip, a block inside a product page) without touching your site’s code.

Personyze Popup, Banner & HTML Builder — Template step, showing format types and the three build stages.
Step 1 of the builder — pick a template by format. Click to enlarge.
Step 2 of the Popup/Banner builder — Customize Look & Feel, editing elements in a live preview.
Step 2 — Customize Look & Feel: click any element in the live preview (or pick it from the sidebar) to edit its text, style, and layout. Personalization tokens like the city and search keyword resolve right in the preview. Click to enlarge.

Edit any action with AI

Any action — a popup, banner, WYSIWYG edit, or block of HTML — can be edited with AI instead of by hand. Click Edit action with AI, describe the change in plain English, and Personyze rewrites the action’s HTML for you to review and apply.

The Edit-this-action-with-AI panel with a prompt box and Apply AI changes.
Open Edit action with AI and describe the change — the AI sees the current HTML and rewrites it. Examples: “Make the background purple,” “Shorten the timer to 1 hour,” “Change CTA to Save now.” Click to enlarge.

The AI can:

  • Restyle — colors, fonts, spacing, layout, button styles (“match our brand accent,” “more contrast,” “modern sans look”).
  • Rewrite copy — headlines, body, CTAs (“make it punchier,” “benefit-focused,” “more urgent,” “match brand voice”).
  • Change the creative — swap or adjust images, timers, coupon codes, and other template elements.
  • Insert personalization — add Personyze variables like the visitor’s first name, city, or a searched keyword so the action fills in per visitor.

The AI proposes the change; you review, apply, or discard — nothing goes live until you accept it.

Personyze Popup/Banner builder — Placement, Trigger & Frequency: placeholder selector to inject or replace content, trigger options, and frequency caps.
Step 3 — placement (with placeholder selector), triggers, and frequency caps. Click to enlarge.

The builder walks through three stages:

1 · Template. Start from a format-specific template, filterable by type: Bar (slim sticky top/bottom bar, or a rich bar with image & CTA), Embedded Banner (inline in a specific page slot), Message Box (inline messaging with CTAs), Popup (modal overlay — signup / lead form, or countdown promo), Notification (slide-in corner toast), Floating (floating bubble / launcher), and Custom HTML / Blank (raw markup). Any template can embed a product-recommendation widget powered by your catalog (Bought Together, Countdown, Grid, Slider, and more), and there’s an “Edit this action with AI” option to generate or restyle it from a prompt.

2 · Customize look & feel. Adjust the design — layout, colors, fonts, imagery, buttons, and responsive behavior across base / hover states and phone / tablet / desktop breakpoints — and drop in personalization tokens that resolve per visitor.

3 · Placement, trigger & frequency. This stage decides where, when, and how often the content shows:

  • Placement — for overlays, choose a screen position (top/bottom, left/center/right, or centered). For in-page content, set a placeholder selector to inject into — or replace — an element on the page. Personyze also lists every other content item already using that same placeholder, so you can see what competes for the slot.
  • Trigger — what makes it appear: as soon as the page loads, after a delay, after the visitor is idle for a while, after scrolling to a point, when they scroll back to the top, when text is copied, on predicted leave intent (exit intent), on a custom JS event, or from a click-to-open launcher.
  • Frequency & close behavior — frequency caps keep it from feeling like spam: show a close (×) button and, if closed, suppress it for N sessions; hide it after the visitor has closed it a set number of times; or hide it after they’ve clicked it a set number of times.

Full guide: Popup & Banner Action Guide.

Other action types

Personyze also supports these action types, each with its own dedicated guide:

  • Product Recommendations — dedicated recommendation widgets (Bought Together, Also Viewed, Trending, Personalized by browsing history, Category-based, and more) powered by your product catalog feed.
  • Countdown Timer — add urgency with a live countdown to a specific date/time, a per-visitor session countdown, or a rolling window.
  • Redirect to URL — send matching visitors to a different URL entirely. Useful for A/B testing landing pages or routing paid traffic to a segment-specific page.
  • JavaScript Action — run arbitrary JavaScript when the audience matches. Used for anything the built-in action types don’t cover — talking to a third-party API, mutating page state, firing tracking events, or wiring up something custom.
  • Email actions — personalized email sends including cart abandonment, re-targeting, and campaign-driven sends, with recommendations embedded at open-time.
  • Web Push Notifications — reach visitors who’ve granted push permissions with time-of-day or event-triggered notifications.

3. Configure the test

The test controls sit at the top of the variations.

The A/B test controls — test type, traffic split, assignment, and goal.
The A/B test controls: (1) test type — A/B or Multivariate; (2) the traffic-split bar, dragged to allocate each variation’s share; (3) how visitors are assigned; and (4) the Goal & auto-pick bar. Click to enlarge.
  • Traffic split — drag the handles on the colored bar to set each variation’s share (they always total 100%). Holding some traffic on the original measures lift against doing nothing. Use “+” to add a variation and “×” to remove one.
  • Goal & auto-pick — choose the KPI the test optimizes for (clicks, a conversion goal, revenue) and whether Personyze auto-picks the winner or leaves it to you.
  • Reset test data — if you change variations, the split, or an action mid-flight, reset so measurement restarts cleanly rather than mixing old and new numbers.

How visitors are assigned

The Assign by control decides how visitors are bucketed into variations — and how a shared control cohort is held out across campaigns.

The Assign-by dropdown with all rotation modes.
The “Assign by” dropdown decides how visitors are bucketed into variations — and, at the bottom, how a shared control cohort is held out across campaigns. Click to enlarge.
  • Rotate users — different users see different versions, and each user stays on the same version. The classic A/B test; use it for most experiments where you want a consistent per-visitor experience.
  • Visits — the same person can see different variants on each visit. Good for banners and creative where per-visit variety is fine and you care about impressions more than a stable per-user experience.
  • Random — every page view picks a variation at random, with no user or visit consistency. Use only when consistency doesn’t matter.

The bottom of the menu sets up a shared control group across campaigns — a hold-out cohort that never sees any of your personalization, so you can measure the total lift of your whole program, not just one test:

  • Control users — the same users are held out across every campaign (for example, a shared 10% control cohort everywhere).
  • Control visits — a shared visit-level control cohort across campaigns.

The decision strategy

The decision panel controls how sure Personyze must be before it calls a winner — and what happens when it does. It turns “is B really better?” into explicit, auditable rules instead of a gut call.

The A/B decision strategy panel.
The Decision strategy panel: (1) primary goal; (2) auto-pick winner on/off; (3) a preset (Cautious / Balanced / Fast / Custom) that sets the confidence, sessions, and duration thresholds; (4) readiness to decide; and (5) who’s notified when a winner is picked. Click to enlarge.

It has four parts:

  • Primary goal — the single metric the winner is judged on (a conversion goal, clicks, or revenue). Supporting KPIs are still tracked, but the decision is made on this one so a test can’t “win” by trading your real objective for a vanity metric.
  • Auto-pick winner — a toggle. On, Personyze promotes the winner automatically the moment the thresholds below are met. Off, it only flags the recommendation and waits for you to promote manually.
  • Decision preset — how strict the bar is, as a trade-off between speed and certainty:
    • Cautious — highest confidence and the most data before deciding; fewest false winners, slowest to call. Best for high-stakes changes (pricing, checkout).
    • Balanced — the default; a sensible middle ground for most tests.
    • Fast — lower thresholds so you decide sooner; good for low-risk copy or creative tweaks where a wrong call is cheap.
    • Custom — set the exact thresholds yourself (see below).
  • Notifications — who gets told when a winner is picked, so a decision never happens silently.

Each preset is really a combination of three thresholds, all of which must be satisfied before a winner is declared:

  • Confidence — the statistical certainty that the leader’s lift is real and not noise (for example, 95%). Higher confidence = fewer false positives, but more data needed.
  • Minimum sessions — a floor on how many sessions each variation must collect, so a tiny early sample can’t trip a decision on a lucky streak.
  • Minimum duration — a floor on how long the test runs, so it spans full weekly cycles (weekday vs. weekend behavior) rather than a single unrepresentative day.

The readiness-to-decide meter shows how close the current leader is to clearing all three at once. A variation can lead on conversion yet still be “not ready” because it hasn’t hit the session floor or the minimum run time — which is exactly the mistake the thresholds exist to prevent. When every condition is met, this is where the decision actually takes effect.

What happens when a winner is decided. Promoting a winner isn’t just a notification — it changes how traffic is served:

  • Traffic shifts to the winner. The winning variation’s share is raised — up to 100% of the targeted audience — so the losing variations stop being served and every matching visitor now gets the winner. The original/baseline is retired along with the other losers.
  • The winner becomes the campaign’s content. Once promoted, that variation is effectively the live experience; you can keep the campaign running as a normal (non-test) campaign from there.
  • With auto-pick on, Personyze does this reallocation automatically the instant the thresholds are cleared, and notifies whoever you listed. With auto-pick off, nothing shifts on its own — you get the recommendation (and the notification), and the traffic only moves when you click to promote. If you never allocate, the variations keep splitting traffic as before; the “decision” without promotion is just an alert.

If you promoted early on bad data, or changed the content afterward, reset the test data so measurement — and any future auto-decision — starts clean.

Multivariate: test several elements at once

An A/B test compares whole variations. Multivariate tests individual elements in combination: you create groups — each group is one thing you’re varying — and drag the alternative actions into each. A visitor sees one action from each group, and Personyze combines the groups into every possible variant.

A multivariate test with groups of element variations.
A multivariate test: (1) the Multivariate toggle; (2) content shown to everyone; (3) the rule that a visitor sees one action from each group; and groups (4) and (5), each varying one element — Personyze combines them into every variant. Click to enlarge.

Multivariate measures each element’s individual impact and how elements interact, so you learn which specific change drove the result — at the cost of needing more traffic, since every combination has to accumulate enough data.

4. Read the results

The results dashboard is built to answer one question first — which variation won, and can I trust it?

The A/B test results overview.
The results dashboard is built to answer one question first — which variation won, and can I trust it? Click to enlarge.
The A/B results dashboard with control, variations, winner, and daily trend.
The results dashboard for the primary goal: a Control group (holdout) as the true baseline, each variation’s conversion rate with confidence and 95% CI, a WINNER badge on the leading variation, and a daily conversion-rate trend. Tabs at top switch between goals (trial signups, CTA clicks, demo requests). Click to enlarge.
Performance-by-segment breakdown of an A/B test.
Performance by segment shows where each variation wins — split by device, country, city, URL, or browser. The winning cell per row is highlighted, and each segment gets its own leader and confidence, since a variation can win overall but lose on mobile. Click to enlarge.
  • Tab per KPI — the primary goal first, then supporting KPIs (clicks, conversions, revenue).
  • Winner recommendation — surfaced when one variation leads on every conclusive metric with no negative side effects.
  • Delivery health — confirms each variation was actually injected, so a “loser” isn’t just a variation that failed to render.
  • Data readiness — for any KPI short of 95% confidence, an estimate of how much longer it needs.
  • By segment — break results down by new vs. returning, country, device, and more.

5. Promote the winner

  • Manual — review the dashboard and promote the recommended variation; it becomes the campaign’s content and the others stop.
  • Auto-pick — turn on auto-promote and Personyze does the same automatically once the test is conclusive under your decision strategy.

If results are inconclusive or you cleared bad early data (say, after fixing a selector), reset the test data and let it gather cleanly.

What to test

  • Copy — headlines, CTAs, value props, social-proof badges.
  • Design — hero images, layouts, color schemes, button styles.
  • Offers — discount sizes, trial length, “no credit card” framing, urgency.
  • Recommendations — different recommendation algorithms against each other.
  • Targeting — the same content against different audience definitions, to find which segment converts.

QA before going live

Before switching a test to Live, use the QA Simulator to preview each variation as different visitor profiles — confirming the right variation shows for the right audience on the real page.

The QA Simulator previewing a campaign as a simulated visitor.
The QA Simulator opens the real page with the campaign injected and lets you impersonate visitors — so you can confirm each variation renders correctly for the audience it targets. Click to enlarge.

Impersonate the visitor profiles each variation is meant for, and confirm the right one shows:

QA Simulator session-data panel for impersonating a visitor.
Impersonate a visitor: (1) switch identity with a new user ID; (2) control the session; and (3) override CRM / custom attributes — to test that the test targets the right people. Click to enlarge.
QA Simulator targetings panel showing rule-by-rule match.
The targetings panel shows, rule-by-rule, whether the simulated visitor qualifies — so a “not matched” result is never a mystery. Click to enlarge.

You can also confirm the variation’s actions actually fire and render on the real page:

QA Simulator campaign-content panel listing each action and status.
The campaign-content panel lists each action, its type, whether it executed, and its status — and lets you trigger one manually to preview it. Click to enlarge.

QA in Test mode first, fix anything the simulator surfaces, then switch the campaign to Live.

Related