Friday, March 11, 2016

Sitecore Lucene highlight search term for best matching field

In a previous post I outlined an example of some code to highlight the search term in the search results using Lucene in Sitecore. This works well if you have one field you want to display on the search results page (the page description for example). But if you have multiple fields which could contain the search terms, it's a good idea to check them all to see which has the best match.
Luckily the GetBestTextFragments method in Lucene will provide a score for each piece of content you search against, which allows you to show the most relevant content.
/// <summary>
/// Checks a list of content for the best match for a search term
/// </summary>
/// <param name="searchTerm">search term</param>
/// <param name="searchContent">search content</param>
/// <returns>Best match</returns>
private static string GetBestMatch(string searchTerm, List<string> searchContent)
{
    // create analyzer
    var analyzer = new StandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30);

    // create FuzzyQuery using the BooleanQuery for multiple words
    var booleanQuery = new BooleanQuery();
    var segments = searchTerm.ToLower().Split(new[] { " " }, StringSplitOptions.RemoveEmptyEntries);
    foreach (var segment in segments)
    {
        var fuzzyQuery = new FuzzyQuery(new Term("", segment), 0.7f, 3);
        booleanQuery.Add(new BooleanClause(fuzzyQuery, Occur.SHOULD));
    }
    // create highlighter - using strong tag to highlight in this case (change as needed)
    IFormatter formatter = new SimpleHTMLFormatter("<strong>", "</strong>");

    // excerpt set to 200 characters in length
    var fragmenter = new SimpleFragmenter(200);
    var scorer = new QueryScorer(booleanQuery);
    var highlighter = new Highlighter(formatter, scorer) { TextFragmenter = fragmenter };
    TextFragment bestMatch = null;

    // Loop through all text fields and find the best matched content
    foreach (var contentItem in searchContent.Where(x => x != null))
    {
        string rawPageContent = Sitecore.StringUtil.RemoveTags(contentItem); // remove html tags from content
        Lucene.Net.Analysis.TokenStream stream = analyzer.TokenStream("", new StringReader(rawPageContent));

        var highlightedFragment = highlighter.GetBestTextFragments(stream, rawPageContent, true, 1).FirstOrDefault();

        if (highlightedFragment != null) // Will come back as null if no match found
        {
            if (bestMatch == null)
            {
                bestMatch = highlightedFragment; // first best match
            }
            else if (bestMatch.Score < highlightedFragment.Score)
            {
                bestMatch = highlightedFragment; // new best match
            }
        } 
    }
            
    if (bestMatch == null)
    {
        return searchContent.FirstOrDefault(); // null is returned if no matching text found
    }

    return bestMatch.ToString();
}
When passing through the list of content to search against, you should send the most relevant fields first. Because if there are no matches the first piece of content is returned, and if 2 pieces of content have the same score, the one higher in the list will be sent back.

No comments:

Post a Comment