Knowing the World: Intuition, Theories, Models, and Data

The following guest editorial was written by Emanuel Derman, a professor at Columbia University and a senior adviser at Prisma Capital Partners in New York City. It was published in the November/December 2013 issue of the Financial Analysts Journal.

Some 70 years ago, cybernetics was a hot field; 30 years ago, catastrophe theory was on everyone’s lips. Those Greek-derived words for disciplines that once brought hope of explaining human behavior now evoke a quaint nostalgia, like Polaroids of long-haired young people in bell-bottom jeans and tie-dyed T-shirts. The new buzzword nowadays is big data, the fashionable term for capturing and analyzing the vast collections of information that people reveal about themselves when shopping online at Amazon.com and Travelocity or when writing about themselves on Facebook and Twitter. Big data involves a mix of computer science, information technology, mathematics, and applied statistics. It is increasingly used to sell us products or to persuade us to vote for politicians by tailoring the products’ or politicians’ images to our particular data-generated personas. Some talking heads like to say that computer-aided analysis of patterns will soon replace our traditional methods of discovering the truth in many fields, including medicine, the social sciences, and physics.

Classic Modes of Understanding

What are the classic ways of understanding? Recall the great triumph at the dawn of modern science: the understanding of gravitation and motion. How did that come about?

Intuition. For millennia after the ancient Greeks, scientists’ prejudices led them to describe all planetary movements in terms of circles around a stationary earth. But the motion of a planet, as seen from the orbiting earth itself, is too complicated for a single circle — sometimes a planet seems to move backward relative to the earth — and so it needs circles moving on circles moving on circles (i.e., epicycles). Eventually, Galileo pointed out that the earth was not stationary, that the earth and planets orbited the sun, and that the planets’ weird, apparently retrograde motions were not intrinsically theirs but, rather, were a consequence of their being observed from the orbiting earth.

In the early 1600s, Kepler examined the data on planetary positions and formulated his three astonishing laws of planetary motion: Planets move in ellipses (not circles) with the sun at one of the foci, the line between the sun and a planet sweeps out equal areas in equal times, and the square of a planet’s orbital period is proportional to the cube of its distance from the sun.

If you want to glimpse the miracle of discovery, think about Kepler’s second law: The line between the sun and a planet sweeps out equal areas in equal times. This deep symmetry of planetary motion implies that the closer a planet is to the sun, the more rapidly it moves. The astonishing thing is that there is no line between a planet and the sun that Kepler could have observed. His data consisted of planetary positions in the night sky. How then did he decide to describe the motion of the planets in terms of an invisible, imaginary line? No one knows exactly, but it must have involved long immersion, struggle, strange associative thinking that arose from somewhere inside him, and then — aha! — intuition, followed by checking the data.

Intuition is the first mode of understanding. The observer becomes so close to the object observed that he begins to experience its existence from both outside and inside it. Intuition is a quantum-like merging of the observer with the observed — the ability to be in two places at the same time.

Theories. Kepler’s laws describe the patterns of the planets but not their cause. Newton found their cause; he showed that Kepler’s laws are a mathematical consequence of Newton’s own theories of gravitation (the inverse square law of attraction) and motion (Force = Mass × Acceleration).

How did Newton discover his theories? Certainly, the orbiting planets and falling apples did not announce the laws that drove them. John Maynard Keynes wrote of Newton, “I fancy his pre-eminence is due to his muscles of intuition being the strongest and most enduring with which a man has ever been gifted.” Keynes understood something about the discovery of truth that many of his more formal economist-disciples have never learned.

Theories are descriptions of the laws of the world; they can be right, partly right, or totally wrong. What all theories have in common is that — like God’s voice to Moses in the desert — they proclaim, I am what I am. Theories stand on their own feet.

Newton’s laws have been supplanted by Einstein’s, but that does not mean that Newton is an approximation to Einstein. Newton is to Einstein as cursive is to typing or as navigation by the stars is to the Global Positioning System. Two different approaches reach the same end by different means and with different accuracies. One does not approximate the other. Both are theories that describe facts.

Models. The final mode of understanding is a model. A model compares something we don’t yet understand with something we already do. For example, the famous liquid drop model of the atomic nucleus pretends that the nucleus is a drop of water that can vibrate and rotate and even fission into two drops — useful, picturesque, but not entirely true. Similarly, the Black–Scholes financial option model compares the uncertain movement of stock prices with the diffusion of smoke from the tip of a burning cigarette — useful up to a point, but not fact. Models are metaphors, graven images of reality but not reality itself. Models are analogies whose incautious use can unleash all the dangers of idolatry that God warned against in the second of his Ten Commandments.

Data. Some would argue that yet another mode of understanding is statistics, the analysis that lies behind big data. Statistics seeks to find past tendencies and correlations in data and assumes they will persist. As a famous unattributed phrase puts it, however, correlation does not imply causation.

“Philosophy is a battle against the bewitchment of our intelligence by means of language,” wrote Ludwig Wittgenstein. I take that to mean that language can deceive our natural intuition, and we need philosophy to reclaim it. In a similar sense, I would argue, science is a battle against the bewitchment of our intelligence by data. Although useful, big data is not a replacement for the classic ways of understanding the world. Data have no voice. There are no “raw” data. Choosing what data to collect takes insight; making good sense of the data collected requires the classic methods.

We still need a model, a theory, or intuition to find a cause.

Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.

Photo credit: ©iStockphoto.com/esenkartal