Search Done Right

The headline was my father’s reaction to the launch of Wolfram Alpha, the new “answer engine” from Wolfram Research. My father did not like the fact that in many cases WA rewrites your query to find something it can actually answer.

The problem my father has highlighted is that WA can only answer a small portion of the questions asked of it. A lot has been said about its limitations in NLP but I actually see WA’s limitation more related to the data – how it collects it and how it uses it. WA relies on “curated data” and not on the overall web knowledge. It is purely rule-based and not statistical-based.

Most new developments in language technology (translation, audio transcription, question/answering, conceptual search, etc.) heavily rely on statistical/large dataset approaches rather than being purely rule-based. To simplify, the system doesn’t really try to understand the language, it just tries to find good matches in a large database. One big advantage is that with large amounts of data, there is no need to get it absolutely right.

But look at these quick examples:

• If you type, “top languages in the world,” WA does not return anything (you have to type “languages spoken in the world”). But Google doesn’t have a problem finding relevant results.

• If you type “language most similar to German,” WA responds “English” because it apparently does not know anything about Dutch, while once again Google “dumbly” finds the right answer.

Being rule-based on curated data means higher accuracy but very low coverage (high precision but low recall in IR terms). It also means a very high cost for a limited impact. The only way to get the economics to work is to have the system be part of a much broader engine so that the number of queries answered can justify the upfront investment.

Let’s say WA can answer 1 percent of all requests. On Google that’s still a few million queries a day and can justify investing tens of millions a year. Compare that to the number of people who will know to switch to WA for the 1 percent of queries that it can answer… In other words, Google can outspend WA at its own game.

This illustrates the new “Google economics” and the amazing barrier to entry posed by Google. It makes much more sense for Google – as opposed to a niche engine – to develop many new features with significant upfront costs.

A prime example of that was “Street View.” As many have forgotten, Street View was first introduced by Amazon in its now discontinued A9 web/map service. It’s a neat feature, but very costly and used in less than 10 percent of map searches. For Google, on the other hand, it makes perfect economical sense to throw tens of millions of dollars at it because it will likely reap the rewards.

So what’s the bottom line for WA? It needs to quickly partner (and not compete) with major search vendors so that its investment can be leveraged in large volumes of requests (which I guarantee it won’t garner on its own).

Wolfram, I know a couple of search providers who would be happy to speak with you…

This entry was posted on Tuesday, May 19th, 2009 at 11:48 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.