Sitecore allowing content authors to exclude items from search

Third party search engines like Google will find pages to index via a sitemap or what is linked off of other pages on the internet. This makes it easier to control what pages are index-able and able to be found by users. With Sitecore any item which has a template included in the search index (or not excluded) will be indexed - as long as it's published to the web database. For system pages or campaign pages for example, you might not want users to be able to find them in your custom site search (on a case by case basis). There is a simple solution to this, which empowers content authors to exclude a given page/item as required.

Template changes

The first change to make is to add the following field to any templates which will be indexed by the Lucene search index. This can be done with a base template or adding it the relevant template(s) manually.

It is an exclude from search checkbox field, that when checked will be used to indicate a result that should not appear on the search front end. Technically the item is still inside the Lucene index, but we will use this field in the custom search code.

Lucene search index (XML)

Now we need to index the exclude from search field and set the storage type to yes so that the field value is accessible without having to get each individual Sitecore item. For a full sample of a Lucene index see Sitecore 8 search implementation.

<!--exclude from search-->
  <field fieldName="exclude from search" storageType="YES" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.Boolean"
          settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider"
          patch:after="field[last()]" />
<field fieldName="exclude_from_search" storageType="YES" indexType="TOKENIZED" vectorType="NO" boost="1f" type="System.Boolean"
          settingType="Sitecore.ContentSearch.LuceneProvider.LuceneSearchFieldConfiguration, Sitecore.ContentSearch.LuceneProvider"
          patch:after="field[last()]" />
</fieldNames>

Search code

In a previous post I introduced the concept of using a predicate builder to query the items in the Lucene index. This concept works well because we can have all of our normal search logic using or statements (where field A is like or field B is like) and then on top of that have an and statement where exclude from search is false.

public static Expression<Func<SearchModel, bool>> GetSearchPredicate(string searchTerm)
{
    var predicate = PredicateBuilder.True<SearchModel>(); // Items which meet the predicate

    // search logic cut down
    predicate = predicate.Or(x => x.DispalyName.Like(searchTerm)).Boost(60);

    // Only show items which are not excluded from search
    predicate = predicate.And(x => x.ExcludeFromSearch == false);

    return predicate;
}

Now when we query the results, any items with the exclude from search check box checked, will not be returned.

var searchIndex = ContentSearchManager.GetIndex("MySearchIndex"); // Get the search index
var searchPredicate = GetSearchPredicate(searchTerm); // Build the search predicate

using (var searchContext = searchIndex.CreateSearchContext()) // Get a context of the search index
{
    var searchResults = searchContext.GetQueryable<SearchModel>().Filter(searchPredicate); // Search the index for items which match the predicate

    // This will get all of the results, which is not reccomended
    var fullResults = searchResults.GetResults();

    // This is better and will get paged results - page 1 with 10 results per page
    //var pagedResults = searchResults.Page(1, 10).GetResults();
}