By Cathy Dew
Don’t Neglect Your Search Indexer and Results
After design, content, and SEO most people stop and take a break. But there is one more critical area for your web site and that is search and the results you are returning to your users. This is true for both intranets and public-facing websites. Do you know how your visitors are searching for content? Do you know what the most looked-for items/pages are? What are the search trends on your site? Have you tried to mimic your visitors searches and viewed the results in real-time?
A great intranet or public website needs monitoring, and the one neglected area we see with new clients and existing sites is Search. This post is included in a series of articles on Kentico Search and the ins-and-outs of this tool and how you can improve your user experience with a well-tuned search feature.
Our first article discussed how to hide global elements from the Kentico Search Crawler so that these ubiquitous items do not affect your search results. In this article, we’ll dive into the details of identifying and boosting your search results to improve your user search experience.
Just a bit of history – Apache Lucene
Under the covers, Kentico uses a .Net port of the Java based Lucene Search Engine. The Lucene search engine was originally developed by Doug Cutting and was later brought into the Apache Software Foundation project family for continued development and refinement. There have been various project breakouts, mergers, and ports of the Lucene Search Engine since then, and Lucene has become the de facto standard for integrated search engines for web based applications – both .Net and Java.
Scoring – The Basics
The Lucene Search Engine starts out by building an index of all the words it finds on your Kentico website, linking those words back to the pages it found them on. The key to the Lucene indexing is the use of “fields”, where words within a page or document are assigned to fields, e.g. the field “Title” is paired with “Tweaking Kentico CMS: Boosting Search Results”. All of these key-value pair assignments are stored in Lucene’s index, against which all the search queries are run. This ensures that the relevant data is returned to the visitor as fast as possible.
Internally, Lucene implements the tf-idf (term frequency-inverse document frequency) scoring model. What this means is – by understanding how frequent a term (or word) is used in one document, as well as across all documents, we can understand how important that term is on both a micro and macro level. Lucene, and others, have built on this statistical simplicity and from that development we now have a robust indexing engine we can fine tune for our individual needs.
Where you can Score Boost
Because the basic key-value pair assignments made during and index build or update, we can change the relevancy of the returned results by add or subtracting score values to specific fields. For Kentico, we have three target areas where we can influence the scoring of a term:
- Field Level
- Document Level
- Query Level
In order to boost terms at the field and document level, you need to bypass the default Kentico Indexer and write your own because these scoring methods are saved directly into the Lucene index during the web site crawl. Since we’ve already buried you with technical bits and bytes, we’ll let the more adventurous learn custom smart search indexes on their own time.
That leaves us with Query Level boosting. The score boosting is done at the time of the query and can be added via the Search Settings :: Search Conditions as part of adding the Search Web Part to your master page. However, in order to properly utilize the Query Level boosting, you need to make sure you have your Kentico Page Types configured properly, not only to ensure that the page attributes are available to the Search Engine for crawling, but so that similar fields across page types can be boosted with the same search name or alias.
Configuring Page Types
Each Kentico page type has a Search Field tab, where you can enable or disable whether the search engine can see the target page type fields. Depending on how you are using the page types to build and display content, you many need to enable fields for the search indexer to crawl. Additionally, page types can have different names for the same field type, e.g. Title could be ArticleTitle, BlogTitle, StoryTitle, ProductTitle, etc. In order to not have to write a query level boost expression for each individual page type, after enabling search to crawl each of these fields – give them all the same alias “title”.
It’s here, that you’ll want to take a half step back and look at all the page types in use on your Kentico website and decide which fields need to be crawled, and which fields might benefit from boosting. Once you have identified all the fields and configured them, you are ready to apply the score boosting
Search Boost Expression
Our crawl through search has brought us to final part of the score boosting discussion, the K# (K Sharp) expression that we’ll implement to increase the relevancy of terms found within our Title fields. For this example, we’ll boost the score of terms found within our shared title fields by “2”.
(title: {% QueryString["searchtext"] @%})^2
If you want to boost the score if just one specific page type, you can do that my calling out that page type for a score boost:
(ClassName:CMS.PageTypeName)^1
And if you want current (and recently updated) articles to have a high relevancy than older articles, you can do that with the following expression:
DocumentModifiedWhen:[{% ToSearchDateTime(CurrentDateTime. AddMonths(-6)) @%} TO {% ToSearchDateTime(CurrentDateTime) %}])^0.5
As you come to understand the power and breath of what you can do with K# expressions and macros, you’ll find that you can extend your Kentico implementation with very power but simple methods right from the admin interface – no under the covers development required. We’ll touch on these power options in other articles.
As you can see, tweaking Kentico search to suit your visitors needs can be quite simple or quite complex, depending on the nature of what you are changing and how large your Kentico CMS web site is. If you need help learning the basics of these features or configuring your Kentico web site for more effective indexing and searching, 2Plus2 can help.
Go online to schedule a free consultation with our team or call 510-652-7700 today.