Experiments — A/B Test Your Videos and Find What Converts

Experiments lets you run A/B tests directly inside Vidalytics — no external CRO tools, no manual traffic splitting, no re-embedding. Test different video versions against a single business metric, get a clear read on what is working, and roll out the winner with confidence.

How to Create an Experiment 

Go to Experiments in your Vidalytics account and click Create Experiment.

Set your experiment basics — give it a clear name (we recommend a "what + where" format, such as "Hook Test — Homepage VSL") and optionally add a description or hypothesis.

Select your variants — choose your Control (the existing video) and your Variant (the challenger video you want to test against it).

Choose your success metric — pick the one metric that matters most for this test:

• Click Rate
• Opt-In Rate
• Conversion Rate
• Average Percentage Watched
• Unmute Rate
• Bounce Rate

Set your traffic allocation — the default is 50/50. Adjust if needed.

Save and launch — if your Control video is already embedded on your page, you can start immediately. No re-embedding required.

Step by step walkthrough video on how to setup your Experiment.

Managing Your Experiment

Once your experiment is running, you have full control over its lifecycle:

• Start — begins traffic splitting and collects results wherever the Control embed is installed
• Pause — stops traffic splitting. All visitors see the Control video. Collected data is preserved
• Resume — restarts traffic splitting. May take a few minutes to propagate across all embeds
• Finish — ends the run, locks results, and prompts you to select an outcome

Reading Your Results 

Timeline View Track how each variant performs over time, spot when a winner starts to emerge, and avoid making decisions based on short-term spikes.

Video Engagement View Compare both videos side by side, frame by frame — including audience behavior, drop-off points, conversions, and available segments. This is the clearest way to see exactly where viewers are engaging, dropping off, or converting on each variant.

Winner Recommender

Instead of guessing when your test has enough data, the Winner Recommender analyzes your results in real time and tells you exactly where things stand. Each experiment is assigned one of four statuses that update automatically as new data comes in:

• Too Early — not enough data collected yet to draw any conclusions
• Too Close — both variants are performing similarly with no meaningful difference
• Gaining Confidence — a leading variant is emerging but the data is not yet conclusive
• Ready to Finish — enough data to safely declare a winner

This removes the uncertainty of knowing when to call it and helps you make faster, more confident decisions.

Declaring a Winner

When you finish an experiment, you choose one of two outcomes:

No winner — the Control experience stays live and the experiment ends with no changes.

Declare a winner — select the winning variant and optionally apply it to 100% of traffic.

Settings Experiments — Test Video Settings, Not Just Videos

You can also A/B test individual video settings — such as CTA timing, autoplay, thumbnail, or play gate — without duplicating your video or managing two separate library entries. The original settings become your Control, the changed settings become your Variant, and traffic is split 50/50 on the same embed.

Best Practices 

• Run experiments on high-traffic pages for faster, more reliable results
• Test one variable at a time — hook, offer, CTA — so you know exactly what drove the difference
• Let tests run long enough to gather meaningful data before finishing
• Document your learnings after each test to build a repeatable optimization habit

Have questions about Experiments? Join our live Q&A session — Register here.

If you have any questions, feedback, or need assistance, feel free to reach out to our Customer Happiness Team at hi@vidalytics.com.