Data and our future: too much of a good thing? Not enough? How will we know?
A portrait of chaos
The medical hegemony of the myth of the humors lasted close to 2000 years, until finally in the 18th century, Enlightenment scientists developed the foundations for the modern understanding of the behavior of fluids in the body through the circulatory system. It wasn’t until the latter half of the 19th century that Rudolf Virchow described the workings of cellular pathology and became the founder of contemporary pathology practice and the dissemination of the germ theory of disease.
The discovery of DNA by Watson and Crick (and Rosalind Franklin) in the early 1950s and the eventual sequencing of human DNA by the Human Genome Project, completed in 2003 opened an entirely new universe of complexity and opportunity in the understanding of human health and the practice of medicine.
In terms of numbers and complexity: We’ve gone from four humors, to hundreds of toxins revealed by the germ theory of disease, to 23 pairs of chromosomes in each human being, with 25,000 genes altogether in the human genome. And we still don’t understand how much of it works. This is the portrait of the growth of chaos and complexity that Weinberger addresses.
Humans versus machines
In human efforts to understand reality—to create models of the world that capture behaviors and can be used for predictions of the future or solving problems in the present—the traditional approach has been to simplify, simplify, simplify. Faced with rich complexity such as the realities of genetics and novel approaches to treatment, it is inevitable that we try the approach we have always used in the past. But Weinberger admonishes that today, armed with machine learning and deep neural networks, we may be better off leaving simplifications behind. In the vast array of intricate data our world is generating, we now have machine tools that have no need of simplifying. The machines are just as happy analyzing everything. And we benefit more from their ability to surface interesting connections in that mass of chaotic complexity than we do from insisting on simplifying.
As much as Weinberger’s book lays out the broad challenges to our thinking brought forward by our new relationship with data, it looks at changes that will be happening in the ways we process knowledge. Its story-filled chapters take on approaches to prediction and understanding the future, the building of mental models, the role of expectation, the loosening of the bonds among cause and effect, the new pressures on business strategy and innovation, and many more subtle changes to come.
Consider our data-driven world as lurching forward in a state of “everyday chaos.” Weinberger has given us a beautifully written set of mental maps to help keep us aligned with our own futures.