Deep learning and ignorance

“Yes, we don’t know how an AI builds its model, but neither do I know how I catch a ball.” 

This was Tim O’Reilly’s tweeted response to an article of mine (https://goo.gl/5vqNfg) about the inscrutability of the results of the type of machine learning called Deep Learning. It’s a good point –O’Reilly is the founder of O’Reilly Media and is an exceptionally smart person—both in general and when considering Deep Learning.

Knowledge is always the surface of a deep pool of ignorance. This is for at least two reasons.

First, there’s the sort of ignorance that Tim points to. Catching a ball is a matter of know-how that—as we’ve acknowledged at least since Polanyi—is bodily. If you know how to play a Chopin polonaise, there’s a real sense in which it’s your fingers that have the knowledge, or, in a more real sense, it’s your entire body. When we try to know how we do it and examine what got us to press the fourth finger of our left hand against the keys at exactly the right moment, it’s a mystery to us. Of course, one reason it’s a mystery is that we’ve supposed that the mind is non-physical so there’s a gap to be overcome between it and our body, but when we try to catch our fingers in the act of knowing, at best we make our fingers clumsy.

Knowledge we build together

But we can get a little closer to knowing how we play the piano or catch a ball. One of the responses to Tim’s tweet said that it’s an easy heuristic: Keep the ball in the center of your visual field and you’ll catch it. If that’s right, that’s pretty impressive: such a simple rule covering so many different types of tosses. Elegant! 

Yet, it does not address Tim’s point: We could catch the ball even before we read the tweet that explained how we do it. Know-how doesn’t have to be transparent to work.

Neither does the sort of knowledge my article is talking about: knowledge that gets expressed in sentences rather than in a successful catch and, in particular, the knowledge that we build together as a culture. That body of knowledge is not entirely transparent to us, in two ways. 

First, we often can be asked questions about it that will quickly expose our deeper pool of ignorance. “Shakespeare wrote Hamlet in 1599.” True, but: When exactly in 1599? Is that when he began it or completed it? Exactly where in the order of plays does it stand? How do we know when he wrote it? What we know is always just a little of what there is to be known.

Second, and more troubling, is that for most of what we know, we cannot explain exactly how we know it. I read about Hamlet’s creation date in some book and it stuck in my mind because it’s so close to being a round number. Which book? I have no idea. Of course, I just looked it up online to confirm my memory, but I didn’t then research where that page got its information or where its source got its information, all the way back to its origin. I am, ultimately, ignorant about the date.

That is, as they say, a feature not a bug. If we had to do original research each time we needed to know something, our species would not have advanced. We have had pretty reliable systems for transmitting knowledge without requiring us to reacquire its source. Without these systems, we’d always be beginning again and would not have time to get anywhere new. Our willingness to not know enables what we do know.

A radical advance in ignorance

But with Deep Learning, we are creating a new type of ignorance to support our knowledge. My knowledge of Hamlet’s creation date is extremely superficial because it doesn’t matter that much to me. If I’m wrong about it, nothing in the world will change. If, however, the piece of knowledge matters, then someone will dig into it, tracing the sources and authorities and redoing the tests if necessary. I may not have done that for Hamlet’s date, but I could have. It is a possibility. 

With Deep Learning, it can be impossible. Deep Learning systems create models of the domain they’re looking at by examining what can be tens of thousands of variables. The result can then be re-passed through the system for further refinement. We know this works when it comes to playing the game of Go because Google’s AlphaGo program keeps beating the world’s best players. We know it works for predicting the weather because our predictions are getting better. We know it works for face and handwriting recognition because our computers generally get those right. We have reasons to think that it works when ascertaining whether we have in fact discovered a Higgs boson. 

But with at least some instances of Deep Learning, we’re not sure the computer’s output is right or optimal—imagine Deep Learning being used to guide hiring decisions—and our brains simply can’t hold in place the thousands of variables and perhaps millions of connections passed through an artificial neural network multiple times. We have no way of checking up on the process by which Deep Learning came up with its conclusion, so we will not always be able to get past our ignorance, even in theory.

This is not an argument against Deep Learning. Deep Learning is an awe-inspiring advance in what we can know. But it is also a radical advance in the ignorance that always undergirds knowledge. And that raises questions that we are now only beginning to confront.