A/B Test Overview

A/B Testing is a controlled, randomized experimental methodology used to compare two versions of a system component typically referred to as variant A and variant B. A/B Testing compares these two versions to see which one works better.

This regulated experiment involves randomly separating users into two distinct groups. One group sees version A, and the other sees version B. This test tracks how people interact with each version and then the data is analyzed to check if the differences between A and B are practically significant. This helps teams make smart choices about how to improve their systems and make them easier for people to use.

It helps them find the answer to a key question: Will a specific change in search, LLMs, prompts, or another configuration improve the quality of the user experience?

To know more about A/B Testing, see What is A/B Test and Implementing A/B Test for Search.

You can run A/B Testing on a Search Client to see which search-type (Keyword, Neural, or Hybrid) leads to higher CTR, ACP, and Deflection.

Just Around the Corner

SearchUnify will soon be rolling out additional A/B Testing support for:

• LLMs (Coming Soon)

• Prompts (Coming Soon)

• Embeddings models (Coming Soon)

• Search Configurations (Coming soon)

• Search Tuning (Coming Soon)

These new additions will make it possible to test more areas, helping improve the platform’s smart search features further.