Blog

Tweaking Kentico CMS: Hiding Navigation from the Search Crawler

By Cathy Dew on July 14, 2017

Don’t Neglect Your Search Indexer and Search Results

After design, content, and SEO most people stop there and take a break. But there is one more critical area for your web site and that is search and the results you are returning to your users. This is true for both intranets and public facing websites. Do you know how your visitors are searching for content? Do you know what the most looked-for items / pages are? What are the search trends on your site? Have you tried to mimic your visitors searches and viewed the results in real-time?

A great intranet or public website needs monitoring, and the one neglected area we see with new clients and existing sites is in the area of search. This post starts a series of articles on Kentico Search and the ins and outs of this tool and how you can improve your user experience with a well-tuned search feature.

Help! My global elements are always at the top of my search results

When you have dynamic content that is designed around the user (localization, for our current discussion, can be language / culture dynamic or office / geographic dynamic), you’ll want to make sure your searches include that dynamic content but don’t include the global elements that appear on every page. The best practice with indexing is to use the Kentico Pages Crawler and specific user profiles built around your user dynamic elements. For example, if you are serving content in different languages, you want to setup an index crawler for each language you are presenting.

The Pages Crawler is a great tool to index your web site “as your target audience sees your site”. However, unless you hide your global elements, the links in your navigation header & footer, fly out notices, etc. will all be indexed and will be related to every page in your web site. It’s very frustrating for your users to see the same non-related content appear for each search query and not the content they are actually looking for.

The Kentico Security Role is your friend

Once you go down the rabbit hole of serving dynamic content, and setup profiles for each search indexing crawler, you’ll want to assign all of the search crawlers into a single security role. The idea here is, you only need to assign the permissions to the search crawlers role instead of each search crawler individually, or make sure you craft the search crawler account name the with the same design pattern to match against. Each time you create a new search crawler profile, add the new profile to the search crawler role and it will inherit all the permissions currently setup.

Web Part Permissions and Visibility

The next step is identifying all the content you wish to hide from your search crawlers. Normally this content will reside in your Master Page, since you want it accessible to all your pages. Once you know which content you want to hide, you need to add a K# Macro to the visibility part of your target Web Part or Content Zone. (Did you know you can dynamically hide or show a whole zone and therefore the web parts contained within? Try it, it’s pretty cool.)

First open your target Master Page in design mode, and on the target Web Part or Zone, select the menu option and then Configure.

 

kentico-cms-app-screen-1-(1).jpg

 

This drill down will get you to the Properties of your Zone or Web Part. From here, you click on the “>” arrow next to Visible.

kentico-cms-app-screen-2.jpg

 

The next dialog is called the Value Editor window, and it’s here you will enter in your K# (K Sharp) macro that looks at the current user’s security roles, and hides or shows the content based on that value.

kentico-cms-app-screen-3.jpg

 

You can put anything here that is within the scope of the K# macro language, but for our needs today we need to check the security role assignment and send back a true or false to allow the web part of zone to display or not for the current user. The K# code for that would look like:

{% if(IsInRole(CurrentUser, "search_crawlers")) { return: false; } else { return true; } |(user)administrator|(hash)996f684516d00f9686dce3534945e51bdb15361cb16d554769cde40f03159b0e%}

Rebuilding the Perfect Index

Once you have updated all the Visible properties for all your target web parts and zones, you will need to run a full index scan to pick up the changes. For some sites, you will want to schedule the re-index after hours or on the weekend to not impact production use. We have clients where a full re-index for all the language and location differences takes 6 – 8 hours. So even after you have setup the structure for your perfect index, it can take time to get the changes cataloged and available to your user base.

Once you have re-indexed your site, make sure you take the time to test out sets of searching scenarios to ensure the content you need invisible to the web crawlers is actually removed. Depending on the site and the current dynamic complexity, there are always a web part or two that slips through the cracks and needs to be updated after the fact.  Because of the re-indexing time sink, the longer a re-index takes the more important it is to update all the truly global contact on the first pass.

As you can see, fixing Kentico search can be quite simple or quite complex, depending on the nature of what you are changing and how large your Kentico CMS web site is. If you need help learning the basics of these features or configuring your Kentico web site for more effective indexing and searching, 2Plus2 can help.

Go online to schedule a free consultation with our team or call 510-652-7700 today.

Cathy Dew
Cathy Dew – CEO + Information Architect
Cathy focuses the company on our mission – Real results. Every time. Information architect and strategist, Cathy is passionate about making software work well – the function, the feel, the result.