Search Done Right

Back in the early to mid 90s, during the emergence of the world-wide web, the number of websites was exploding. This profusion led to the January ‘94 founding of Yahoo, which brought some order to the web by offering a directory whose categories people could browse top-down, much like taxonomies in libraries. Then, Altavista was founded in late 1995, offering users single-search-box access to the world-wide web, bypassing the need for directories like Yahoo’s. Later Google was founded in late 1998 based on a better ranking algorithm than Altavista’s, soon sweeping it and others away

The history looks clear cut, but there were disagreements and confusion along the way. Some observers believed that after novice users became familiar with the web, they would go directly to their preferred sites, bypassing the web directories and search engines. Their analogy was to the everyday world, in which people rarely use the yellow pages to find new destinations; instead they re-visit familiar places or brands: Starbucks, Wal-Mart, Barnes & Noble, Carnegie Museum, etc.  According to this view, the web would become like that, so search engines were an interesting novelty but wouldn’t be of much importance in the end.

Boy, were they spectacularly wrong.  But what did those observers miss?

They overlooked that the world-wide web of information was going to be very unlike the physical world. People’s information needs vary from one moment to the next. At one moment I’m figuring out how best to donate our old family car with 200,000 miles on it, another time I need the address of a school where my daughter has a volleyball game, and the next time I’m verifying conversational claims that northern hemisphere tornadoes always whirl in the opposite direction from southern hemisphere ones (apparently, not entirely true). None of these needs is well met by familiar single websites or web directories. But they are met well by world-wide web search engines, such as launched in the mid 90s.

So what about the enterprise?

The information within companies today is like the web before Altavista:  there are information silos (repositories) that are analogous to the ebay, newyorktimes.com, cmu.edu, etc. silos of old, except that now these enterprise silos are called Sharepoint, Lotus Notes, Documentum, Confluence, Exchange, Hummingbird, Oracle, internal websites, etc. etc.  And, just as individual websites used to have, at best, mediocre site search engines, today’s repository vendors typically offer mediocre search engines to search their own repositories. Like before, skeptics see no problem with this state of affairs. I recall an audience member (not I) at a government information sharing conference asking a panel when users would be able to search the content across multiple intelligence repositories. One panelist dangled his key chain, pointing out that we have separate keys for different locks, so what’s so unusual? Again, the false analogy to the physical world.

Nearly 15 years after Altavista made it possible to search the world-wide web, corporations are now realizing that it’s both technically feasible and hugely beneficial for users to search across enterprise-wide repositories, i.e., to poll their knowledge environments for facts, trends, relationships, practices, prior work, etc. To be a success, it doesn’t mean that each and every information repository must be accessible from a single search box immediately,  just like web search engines did not access proprietary content for a long time until deals were struck with publishers.  But the goal is the same: one box searches all, or at least, lots of the information, to be added to over time. And just like on the web, corporate information that isn’t indexed by the enterprise-wide search engine will be ignored or forgotten, its value to users and the business becoming nil.

It has been said that the fox knows many things, but the hedgehog knows one big thing. This is my hedgehog moment.

Within the next few years, every high-performing organization will need to offer multi-repository, enterprise-wide search to its employees or risk falling severely behind in knowledge-worker productivity.

This entry was posted on Friday, April 10th, 2009 at 8:42 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.