Search Done Right

We don’t have all the details about the latest terrorist attempt on our country but we do know one thing: it was preventable.

President Obama calls it a “mix of human and systemic failure”. Despite a clear warning coming from the terrorist’s father, the state department did not revoke his visa, the NCTC did not put him on the 4,000 people “no fly” list or the 14,000 second screening list and everybody ignored the fact that he was put on the big 550,000 “linked to terrorism” list. Humans, all too human, made a series of bad judgment calls.

Can we really blame them? In their first assessments of the issue, Obama and Napolitano talk about the “selectee list”, the “generic list”, the “watch list”, the “no fly list”… Lists, lists, lists… Who wouldn’t get confused? Even the first inspector general of the Department of Homeland Security, in his NYTimes Op-Ed does not understand the purpose of these lists when arguing that anybody on the huge 550,000 watch list should be put on the small 4,000 no-fly list—ready to unleash a travel Armageddon.

And didn’t we figure out 10 years ago—when Google overtook Yahoo—that lists don’t work for large amounts of information? The problem with lists is that they require making all-or-nothing decisions based on a limited context. Google’s search is superior to Yahoo’s directory because it does not constrain the information accessible based on arbitrary decisions. Is this site interesting? Maybe, maybe not. It depends on the user needs. Let the user decide. Don’t filter, just tag and sort.

Similarly, when government agencies put names on lists, they often do so based on isolated events. Should the warning of a father put someone on a “no-fly” list? Maybe, maybe not. Who knows what motivates a father? One can’t make that choice lightly given the high cost of false positives.

Obama and Napolitano talk about “process”, “system”, “protocols”,… implying that human judgment should be taken out of the picture. But the problem is not human judgment. After all it is quick judgment from the passengers that averted a tragedy. The problem lies in an antiquated system that does not allow proper judgment. In retrospect, with all the information that is available, any sane human would make the right call. It’s incredible to see how quickly after the event a wealth of information was easily collected about the would-be terrorist. The information is available. But it’s not available at the time critical decisions are made.

A father’s warning may be benign, but not before traveling to the US without luggage, buying a ticket in cash, coming back from Yemen, and after being denied a visa renewal in UK. Using secure federated search all these disparate pieces of information could have been made available to whoever cleared the passenger list. But we didn’t trust or empower that person. We just asked them to check a list.

Terrorists are nimble. They adapt quickly and use the latest technologies. The only way we can win this war is to do the same. Drop the “lists” Mr. President. Use search.

This entry was posted on Monday, January 4th, 2010 at 9:23 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.