Search Done Right

Lynda Moulton of the Gilbane Group writes today on how confusing the enterprise search market is for buyers. She is right. My own conversations with prospects and attendees at tradeshows and conferences reveal that people just don’t know what questions even to ask about a product or vendor. But there are two different sources of confusion:

1. products that aren’t quite up to the job (Lynda’s example)
2. the fog of buying from a myriad of options, terminologies, technologies, etc.

Let’s take Lynda’s example of a really difficult search problem:

• Looking up an address in a directory
• Finding an image for a presentation
• Retrieving a press release your department issued last year on a new product
• Locating a configuration change to a piece of equipment in manufacturing
• and so on…

She asks: “Can you imagine any single search interface or product from the tools you know that would give you the means to find all of these pieces of information?”

Here’s a solution sketch:

1. Index all the addresses in the directory. If the directory is contained in a spreadsheet, then index each entry as a separate search result, matching search queries against the person or organization’s name. Then display the hits as a spotlight on the display page.
2. Crawl images separately and enable access via a tab in the user interface. Typically, image meta-data will be stored somewhere, so index that meta-data and associate the meta-data with the image location. In the absolute worst case, of image files stored in a file system with no meta-data whatsoever, associate the path name with the image.
3. Probably all press releases are on the corporate website, so crawl it and create a separate website search engine.
4. Assuming configuration changes are documented in some content management system, just index its contents, while attaching the meta-data to the index.

It may make most sense to create separate search collections for the above. But when the user types a query, a federated search will retrieve the top, say, 100 results from each, which can be clustered into topics to overcome the problem of information overload (one can’t rely on ranking alone, least of all on intranets).

Something like this is seen at FirstGov.gov, where a query on Smithsonian triggers separate FAQ and image searches whose results are woven into the display page as shown.

Enterprise search products are capable now of setting up such an intranet search, not with lots of custom programming and endless services, but rather via a web-based administrative interface.The enterprise search “fog” is, I believe, due not so much to the inability of enterprise search to handle the complexity of enterprises, but rather to the generic challenges of buying any complex or rapidly evolving product compounded with the career-damaging risks of choosing wrong.

At the start of the school year, I was tasked with buying a clarinet for my grade-schooler instead of continuing to rent one, which is costlier. Not knowing the first thing about clarinets, I discovered an article “How to buy a clarinet” which cleared it all up for me. I bought a satisfactory clarinet on ebay and spent no more than a couple of hours on it all.

Maybe we need an article “How to buy enterprise search” that lets you buy confidently in a couple of hours. That’s a topic for a future post …

This entry was posted on Tuesday, January 9th, 2007 at 3:27 pm and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.