Experimenting with your Keyword Search Algorithm – 1 of 2

This series will discuss an approach to experimenting with your Keyword Search Algorithm using the Optimizely Search & Navigation, Commerce Cloud and Experimentation products.

Search & Navigation

At a basic level the following happens on a keyword search:

• Customer attempts a keyword search on your Commerce Cloud website
• Commerce Cloud code builds a Search & Navigation query and posts to API
• Search & Navigation executes the query on it’s Elasticsearch index taking into account factors such as Statistics gathered through Click Tracking to assign a relevancy score to the products determined to be the best search results

Assuming click tracking is already implemented, the variable we have control over to tweak in optimising search performance is the search query sent to Search & Navigation.

Search Query Optimisation

To CMS manage the query, we create an Algorithm Settings Page that can integrate with the search query sent to the API.

Algorithm Settings Page

The following is an example of a Search Algorithm Settings page PoC from my Github:

https://github.com/johnnymullaney/Foundation/blob/search-algorithm-experiment/src/Foundation/Features/Search/Search/SearchAlgorithmSettingsPage.cs

I’ve divided the content type into three tabs representing important aspects of the query.

1 Property Weightings

Property weightings specify what perceived value we assign to a keyword match in various content property’s.

For example we may specify in the query that a keyword match in a Product Title should be twice as relevant as a match in the Description. This will be a key factor when Search & Navigation assigns relevancy score to search results based on that request.

2 Boosting

Your query can instruct Search & Navigation to boost the relevancy scores further of search results that meet a pre-defined conditions. For example a query may include a relevancy boosting for products that are marked in the CMS as “Popular”.

3 Query Optimisation

I am categorising query optimisations as anything else that may improve the quality of search results such as Synonyms, Fuzzy Search, And/Or operators etc.

There is a very useful Labs Github repository which contains a number of extensions that can be integrated with your Search & Navigation solution to improve search performance.

https://github.com/episerver/EPiServer.Labs.Find.Toolbox

Integrate Settings Page with Search Query

We can integrate our Search Algorithm Settings Page into the query that we build in code.

var query = _findClient.Search<GenericProduct>()
             .For(filterOptions.Q)
             .MinimumShouldMatch(searchAlgorithmSettings.MinimumShouldMatch)
                .InField(x => x.DisplayName,     searchAlgorithmSettings.DisplayNameWeighting)
                .InField(x => x.Brand, searchAlgorithmSettings.BrandWeighting)
                .InField(x => x.Description, searchAlgorithmSettings.DescriptionWeighting);

            if (searchAlgorithmSettings.EnableSynonymsImproved)
            {
                query = query.UsingSynonymsImproved();
            }
            else
            {
                query = query.UsingSynonyms();
            }

            if (searchAlgorithmSettings.EnableDisplayNameRelevanceImproved)
            {
                query = query.UsingRelevanceImproved(x => x.DisplayName);
            }

            if (searchAlgorithmSettings.EnableDisplayNameFuzzyMatch)
            {
                query = query.FuzzyMatch(x => x.DisplayName);
            }

            if (searchAlgorithmSettings.EnableDisplayNameWildcardMatch)
            {
                query = query.WildcardMatch(x => x.DisplayName);
            }

            if (searchAlgorithmSettings.PopularProductBoostingValue > 0)
            {
                typeSearch = typeSearch.BoostMatching(x => x.PopularProduct.Match(true), searchAlgorithmSettings.PopularProductBoostingValue);
            }

Next Post

In the next post, we’ll create an Optimizely Experiment to AB test the success of search query variations.