Amazon vs. The Librarians! The Fight of the Century!

Not really. It just somehow worked out that at the Aspen Ideas Festival in June, I ended up identified with The Librarians, which would be fine with me, except it was for the wrong reasons.

I was on a panel about “Who will create the market of the future?” Mark Schlageter from Thomson Reuters began by pointing to the way big data enables offers to be more closely tailored to each user’s needs and preferences. I swore fealty to big data but said something Cluetrain-y—in fact, I’m pretty sure I quoted Doc Searls’ line from the book: “Markets are conversations”—about customer-to-customer conversations shifting the power balance.

But then the question was: Does big data pose any dangers to markets? So I raised a hypothetical: Suppose Amazon notes that after reading Gone Girl, users are more likely to buy a book that has a blue cover. Amazon has no idea why that it is. In fact, it might be a trick of our lizard brains. Nevertheless, Amazon will recommend those blue-covered books because its aim is to sell as many books as it can.

Stretching a bit

But (I continued), if you were in the library, a librarian would probably make two recommendations. First, the librarian would suggest a book very much like Gone Girl. Then s/he would recommend a book that would stretch your interests. For example, the librarian might say, “Have you ever read Patricia Highsmith? She wrote in the 1950s, but I think you might like her.” [Unfortunately, I only came up with this example after the panel ended.] A librarian would make suggestions like these because librarians have your interests at heart. They’re not trying to sell you a book. And they’d add something like the second suggestion—the one that stretches you a little bit—because they recognize the cultural value of seeing beyond your current sphere of experience.

Andrew McAfee, the third panelist and a friend, harumphed and said that he would take an algorithmic recommendation by Amazon any day.

Commercial interests

The conversation moved on from this point, with the result that I got pegged as the “Ask a librarian” guy, even though for the past four and a half years I’ve been working on ways to guide readers through the content of libraries by combining algorithms, community usage patterns and, aspirationally, the recommendations of librarians. My point on the panel was not that humans beat algorithms, but that algorithms that come from commercial entities are corrupted by their commercial interests.

For example, the Harvard Library Innovation Lab I’ll be leaving in September has developed a library browser that “heatmaps” books based on how often they have been used in various ways by the Harvard community, and defaults to ranking them by that usage. The result is a list that reflects the usage of a research community. If you are a student or researcher, this should turn up more useful results than a list from a company that simply wants to sell you another book.

The influence of librarians is only indirectly felt in our Lab’s browser: Some works are more used because librarians recommend them. We’ve long wanted to be able to factor in the “lib guides” prepared by library staff. But the real point is not librarians or no librarians, or humans or no humans. Rather, it’s whose interest the recommendation engine is serving.

Priorities

Now, clearly Amazon wants you to be happy with the books you’ve bought based on Amazon’s recommendation. So there is a convergence of interests there. But Amazon will be happy to sell you your 35th western or sf novel or book of cat photos. In fact, if you’ve bought 34 books of cat photos, Amazon has extremely strong reasons to show you a list of nothing but cat photo books. That’s in Amazon’s commercial interest. But many of us believe—I do—that we have a cultural and societal interest in expanding our horizons. A librarian is likely to help us to that end.

But not because the librarian is a human. I am not a romantic about humans and our tiny brains. Amazon’s algorithms know more about more books than any human ever could. It’s not about humans vs. machines. It’s about whose interests are being put first.