Relevance Index

The Relevance Index report displays how well the search results align with the search users' expectations. It is calculated based on the conversions, average click position, and relevancy scores. By default, the report displays the data for up to seven quarters.

Fig. The Relevancy Index Report in Search Analytics Conversions.

The Relevance Index ranges from 0 to 1 and reflects how effectively search results meet users' needs.

  • A value closer to 1 indicates that users can quickly find the information they are looking for on the search results page.

  • A value closer to 0 suggests that users are struggling to find relevant results for their queries. To improve the Relevance Index, admins can take various actions, such as:

    • Tuning search results

    • Using content annotation

    • Rewriting articles

    • Implementing other optimization strategies

Calculating the Relevancy Index

The Relevance Index is based on three factors:

  • Conversions - 40% Weight

    Conversions measure how often users click on search results. It is calculated as:

    Example: If there are 100 searches and 20 clicks, then conversion is:

  • Average Click Position – 30% Weight

    This average click position measures the average ranking position of clicked search results. It is calculated as:

    Example: If a user clicks the results on ranks 1, 2, 4, and 8, then the average click position is 3.75:

    The Average Click Position is normalized before it can be plugged into the Relevance Index equation. The formula for the normalization is:

    The Average Click Position Weight is 30. Continuing with our example, the Normalized Average Click Position will be 0.875.

  • Query Match - 30% Weight

    Query Match is the average relevancy score of all the clicked results. It is calculated as:

    Example: If three clicked documents have relevancy scores of 0.2, 0.6, and 0.8, then:

The Relevance Index is the sum of these three values after applying their respective weights. The Relative Index equation is:

Continuing with our example, the Relevance Index is 0.5015.

Viewing the Relevancy Index Report

  1. Log into SearchUnify.

  2. Navigate to Analytics > Search Analytics.

  3. In the Search Analytics screen, click the Conversions tab.

  4. Scroll down and you can see the Relevance Index report.

    Fig. The Relevancy Index Report in Search Analytics Conversions.

The Relevance Index report is always displayed, even if the Relevance Score toggle in Search Clients > Edit > Relevancy is turned off. However, it is hidden if unchecked in Search Clients > Edit > Analytics Settings.