Knowledge abundance

By David Weinberger

Hubert Dreyfus is the author of one of my favorite books, "What Computers Can't Do," a convincing philosophical argument against the possibility of the interesting types of AI. Now he's written a brief against the Internet called "On the Net," which in a hundred pages tries to let the air out of the cultural Internet bubble. And, he gets it wrong, in my opinion. Fortunately, he gets it wrong in a way that sheds light on one of the assumptions that keeps businesses from taking advantage of the knowledge the Net is making accessible.

I find it heartening to see an academic philosopher wade into a big issue without pussyfooting around with lots of "On the other hand" and "Now, it may seem that." Professor Dreyfus has written a polemic that doesn't confuse fairness with indecisiveness. No, he thinks the Net has been overhyped by precisely the same forces that overhyped AI, and he's out to correct the deep presumptions that have enabled these enthusiasms to grow out of proportion.

In the first chapter of this slim and readable book, Dreyfus wants to show us how the Net by its nature is going to remain an information quagmire. He argues that we can only discern what's relevant to us because we have a body and we have interests. Search engines have neither. So, he goes back to the theme of "What Computers Can't Do" to prove that attempts at AI solutions to this problem are doomed: Lacking bodies, computers have to be fed impossibly large amounts of explicit information if they are to be able to make sense of even the simplest of situations. This is what Douglas Lenat's CYC project has been trying to do, so we have empirical evidence that Dreyfus' theoretical argument is correct. And I am fully on board with him.

One lesson KM could learn from this: Implicit knowledge isn't explicit knowledge that we're not currently thinking about. Implicit knowledge isn't there the way ore is buried. It's "there" only in the sense that we can generate it when required. Most simply: That we can come up with an answer doesn't mean that the answer was lying dormant in us all along. Answering questions is a creative act. Similarly, the context that is our world is not made up of facts and knowledge, although it is only from that context that facts and knowledge can emerge. As Dreyfus says: " ... the world is not a meaningless collection of billions of facts. Rather, it is a field of significance organized by and for beings like us with our bodies, desires, interests and purposes." (p. 26)

Dreyfus talks here about the importance of the body for discovering what's relevant because he wants to make a point about searching the Web. He says that when your favorite search engine pulls up a list of hits, perhaps 30% of the pages are in fact relevant. And the percentage of relevant pages that it's found is also quite small. That is, if there are 100 pages that are relevant to your query, the search engine would be doing quite well if it built a list of 100 of which 30 are relevant.

Then why are we so ecstatic about Google? Dreyfus explains, quoting from an e-mail message from David Blair that cites information retrieval pioneer Don Swanson who calls this "the fallacy of abundance." With so many pages to choose from, the search engine is bound to find some that are relevant to the query. The fact that other pages that may be even more relevant are ignored isn't obvious to the user.

And now we're at the heart of it. In the traditional way of thinking, we've built a mythology around the idea that knowledge is an asset, a jewel. It has value because it's so hard to come by. The possessors of knowledge are rich. The world of knowledge is finite and circumscribed. But in the new world, knowledge is a dime a dozen, so to speak. Everyone has access to it. In the new economy of knowledge abundance, you don't need to find all the relevant information or even the most relevant information. You just need what's good enough. So, "information retrieval" doesn't require a great white hunter to bring back the info; you just have to stick your ladle into the stream.

Of course, this has wider implications, for knowledge no longer is power, at least not in the old sense. It's not confined to the top. It's showing up all over. Knowledge isn't an asset, it's a really fun argument in the hallway or an e-mail thread that the participants are enjoying so much that they won't let it die.

Delivering the right information to the right people at the right time? Nah, the definite article—the "the"—doesn't make sense in the economy of knowledge abundance. Dip your ladle in the stream and you'll get some information, enough information. And if you don't like what you got, just stick it in again.

David Weinberger edits "The Journal of the Hyperlinked Organization" (hyperorg.com), e-mail self@evident.com.