Search Done Right

An interesting conversation has emerged across the blogs this week regarding how well—or not—enterprise search solutions perform at finding information to the satisfaction of the individual user within the confines of an organization.

The ball got rolling when Udi Manber, a vice president of engineering at Google said that his company used its own solution for internal search, adding: “It’s not that good—I’m complaining about it.” (Ouch. Well at least he has those great cafeterias!)

We at Vivisimo use a CRM system from a vendor I won’t name.  For a long time, we have been crawling its contents in order to enable searching our CRM account as a tab on our own internal enterprise search; CRM search results are also returned on our default ‘All’ tab.  Clicking on a search result takes me into the native CRM system, when I can’t get what I want just from the search results page itself.  Of course, our internal search is powered by Velocity, since we eat our own dog food.

I once joked with a boss of mine that everything bought and sold by businesses can be boiled down to having just one of two benefits—it either saves you money or makes you money. Last Monday, I was reminded of this conversation while sitting on a panel, “Enterprise Search: Running Your Own Search Engine”, with a few other folks at the Search Engine Strategies Conference in San Jose. One of the panelists, Bill French, CTO from Myst Technology Partners, brought up statistics from IDC in his presentation, observing that employees may spend up to two hours a day searching for information, to illustrate the point of why enterprise search is so critical. Now these are not new numbers, they are the same figures that everyone in the industry has bandied about for quite some time as the reason d’etre for search—if enterprise search solutions can save each employee x amount of time, then multiply x by y (employee hourly salary) to get the theoretical dollar savings per employee search can provide. This is the “saves you money” argument.

In my last post, The Search Gap – Part 1, I discussed the enormous gap between web search and “behind the firewall” search—i.e enterprise search. Now, as promised, I’ll lay out how you can bridge the search gap in three critical areas—coverage, findability and usability.

Deadly Sin #6: Brand Confusion

As site search sins go, brand confusion is one of the easiest to find and least expensive to fix. Brand confusion occurs when either the search page title or search URL string reflects the brand of the company providing the search technology versus the brand of the company whose pages site visitors are searching.

You can see this sin illustrated on the search results page of United Technologies Corporation’s website in the graphic after the jump.

Do a simple experiment: Ask yourself or a colleague who works in a large organization this question: “How often do you use a web search engine versus a search engine that is ‘behind the firewall’ (i.e., to find information only available inside your organization)?” The two answers are typically very different. In fact, most internet-savvy people use a web search engine every day. The same people rarely or never use a search engine behind the firewall. But maybe they just don’t need it?

Deadly Sin #5: Egoism

This one needs a bit of an upfront definition. Egoism is a philosophy about making personal welfare and interests your primary concern, sometimes at the expense of others. Companies can be guilty of this as well as individuals.

We can see examples of egoism in site search implementations where the company asks far too much from a site visitor who wants to search, as in the following example from the Cadillac website.

First, in order to search the Cadillac website, you have to hunt for the search function that is hidden on the bottom left of the home page (circled in red):

Enterprise search - or search within businesses - is a decade behind web search in terms of usage. Interesting questions are why? and whither? - what’s the trend?

There is roughly one web search per person per day in the U.S., counting the web searches at Google, Yahoo, Live, etc., but excluding searches at eBay, YouTube, CMU.edu, WashPost.com, USA.gov and the like. The analogue in business is searches done on the general intranet search, not at point solutions like Outlook search, desktop search, single-repository search, and so on. Our experience is that daily, general intranet searches lag web searches by orders of magnitude.

Vivisimo introduced high-quality text clustering into the search engine market in the year 2000, after a couple of years of computer science research on new algorithms by the founders at Carnegie Mellon. The research breakthrough was labelling the clusters, i.e, grouping search results into folder topics. Before that breakthrough, search result clusters had poor labels and so the technology was unusable. The technology was first demonstrated on a university website and later at vivisimo.com, with excellent reviews.

Microsoft has stated that it bought Fast to become a one-stop shop for enterprise search at the high end as well as at the low/medium end (”infrastructure search”). This is a compelling $1.2B argument that high-end search is what enterprises need, contrary to previous claims about infrastructure search being good enough.

[An alternative hypothesis is that Microsoft thinks low-end search is good enough for the enterprise and will try eventually to switch over the customer base from Fast, but wanted to immediately jump-start its play into enterprise search. If so, both Microsoft and Fast customers may well be in for a surprise.]