Practical analysis for investment professionals
09 December 2013

Knowing the World: Intuition, Theories, Models, and Data

The following guest editorial was written by Emanuel Derman, a professor at Columbia University and a senior adviser at Prisma Capital Partners in New York City. It was published in the November/December 2013 issue of the Financial Analysts Journal.


Some 70 years ago, cybernetics was a hot field; 30 years ago, catastrophe theory was on everyone’s lips. Those Greek-derived words for disciplines that once brought hope of explaining human behavior now evoke a quaint nostalgia, like Polaroids of long-haired young people in bell-bottom jeans and tie-dyed T-shirts. The new buzzword nowadays is big data, the fashionable term for capturing and analyzing the vast collections of information that people reveal about themselves when shopping online at Amazon.com and Travelocity or when writing about themselves on Facebook and Twitter. Big data involves a mix of computer science, information technology, mathematics, and applied statistics. It is increasingly used to sell us products or to persuade us to vote for politicians by tailoring the products’ or politicians’ images to our particular data-generated personas. Some talking heads like to say that computer-aided analysis of patterns will soon replace our traditional methods of discovering the truth in many fields, including medicine, the social sciences, and physics.

Classic Modes of Understanding

What are the classic ways of understanding? Recall the great triumph at the dawn of modern science: the understanding of gravitation and motion. How did that come about?

Intuition. For millennia after the ancient Greeks, scientists’ prejudices led them to describe all planetary movements in terms of circles around a stationary earth. But the motion of a planet, as seen from the orbiting earth itself, is too complicated for a single circle — sometimes a planet seems to move backward relative to the earth — and so it needs circles moving on circles moving on circles (i.e., epicycles). Eventually, Galileo pointed out that the earth was not stationary, that the earth and planets orbited the sun, and that the planets’ weird, apparently retrograde motions were not intrinsically theirs but, rather, were a consequence of their being observed from the orbiting earth.

In the early 1600s, Kepler examined the data on planetary positions and formulated his three astonishing laws of planetary motion: Planets move in ellipses (not circles) with the sun at one of the foci, the line between the sun and a planet sweeps out equal areas in equal times, and the square of a planet’s orbital period is proportional to the cube of its distance from the sun.

If you want to glimpse the miracle of discovery, think about Kepler’s second law: The line between the sun and a planet sweeps out equal areas in equal times. This deep symmetry of planetary motion implies that the closer a planet is to the sun, the more rapidly it moves. The astonishing thing is that there is no line between a planet and the sun that Kepler could have observed. His data consisted of planetary positions in the night sky. How then did he decide to describe the motion of the planets in terms of an invisible, imaginary line? No one knows exactly, but it must have involved long immersion, struggle, strange associative thinking that arose from somewhere inside him, and then — aha! — intuition, followed by checking the data.

Intuition is the first mode of understanding. The observer becomes so close to the object observed that he begins to experience its existence from both outside and inside it. Intuition is a quantum-like merging of the observer with the observed — the ability to be in two places at the same time.

Theories. Kepler’s laws describe the patterns of the planets but not their cause. Newton found their cause; he showed that Kepler’s laws are a mathematical consequence of Newton’s own theories of gravitation (the inverse square law of attraction) and motion (Force = Mass × Acceleration).

How did Newton discover his theories? Certainly, the orbiting planets and falling apples did not announce the laws that drove them. John Maynard Keynes wrote of Newton, “I fancy his pre-eminence is due to his muscles of intuition being the strongest and most enduring with which a man has ever been gifted.” Keynes understood something about the discovery of truth that many of his more formal economist-disciples have never learned.

Theories are descriptions of the laws of the world; they can be right, partly right, or totally wrong. What all theories have in common is that — like God’s voice to Moses in the desert — they proclaim, I am what I am. Theories stand on their own feet.

Newton’s laws have been supplanted by Einstein’s, but that does not mean that Newton is an approximation to Einstein. Newton is to Einstein as cursive is to typing or as navigation by the stars is to the Global Positioning System. Two different approaches reach the same end by different means and with different accuracies. One does not approximate the other. Both are theories that describe facts.

Models. The final mode of understanding is a model. A model compares something we don’t yet understand with something we already do. For example, the famous liquid drop model of the atomic nucleus pretends that the nucleus is a drop of water that can vibrate and rotate and even fission into two drops — useful, picturesque, but not entirely true. Similarly, the Black–Scholes financial option model compares the uncertain movement of stock prices with the diffusion of smoke from the tip of a burning cigarette — useful up to a point, but not fact. Models are metaphors, graven images of reality but not reality itself. Models are analogies whose incautious use can unleash all the dangers of idolatry that God warned against in the second of his Ten Commandments.

Data. Some would argue that yet another mode of understanding is statistics, the analysis that lies behind big data. Statistics seeks to find past tendencies and correlations in data and assumes they will persist. As a famous unattributed phrase puts it, however, correlation does not imply causation.

Philosophy is a battle against the bewitchment of our intelligence by means of language,” wrote Ludwig Wittgenstein. I take that to mean that language can deceive our natural intuition, and we need philosophy to reclaim it. In a similar sense, I would argue, science is a battle against the bewitchment of our intelligence by data. Although useful, big data is not a replacement for the classic ways of understanding the world. Data have no voice. There are no “raw” data. Choosing what data to collect takes insight; making good sense of the data collected requires the classic methods.

We still need a model, a theory, or intuition to find a cause.


Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.

Photo credit: ©iStockphoto.com/esenkartal

About the Author(s)

7 thoughts on “Knowing the World: Intuition, Theories, Models, and Data”

  1. Umed Saidov says:

    Although we have come a long way from relying on our five senses to perceive reality (e.g. infrared, ultrasound etc.), I believe we are still far from understanding the world in its most fundamental/true form.

    Intuition, theories, models and big data are helpful tools in developing a more granular insight, but without judgement these tools make little sense on their own.

    Limitation imposed by the structure of our languages and (rational/irrational) thinking in general (may be physiological?) make the task more difficult. On some level we have to be less human to understand the world… So tells me my intuition.

    1. Janice says:

      Yes we have to let go of our human attachment to the world so we can access more expansive intelligence and guidance. The physical plane is dense and reveals only a sliver of all that is.

  2. Evan DeBarra says:

    Excellent article. Pausing to test the validity of belief in big data is important. Yet its availability in greater quantities over shorter periods of time will also enable us to test whether intuition in a particular instance is the powerful contribution to understanding you eloquently describe, or it is the cause of a distortion of a theory or model.

  3. David Newcomer says:

    An extended deep meditation of this forms the book “The Master and His Emissary” by McGilchrist. If you want more consider Hofstadter’s “Surfaces and Essences: Analogy as the Fuel and Fire of Thinking”.

  4. Jamal Munshi says:

    i thought it was copernicus who established heliocentrism in the 16th century and explained the retrograde motions of planets that way. it had been proposed in ancient times by aristarchus as well. also, newton derived his the inverse square model from kepler’s laws and it is not clear whether he or hooke or boulliau was the first to do it. regarding the statement about newtonian gravity and general relativity that “One does not approximate the other. Both are theories that describe facts”, please note that according to general relativity, newtonian gravity only approximates universal gravitation and falls apart when gravity becomes too strong.

    anyway, it was brilliant insight or “intuition” anyway you cut it. very interesting article indeed, and many thanks for posting it. not very easy to grasp but i am working on it.

Leave a Reply to David Newcomer Cancel reply

Your email address will not be published. Required fields are marked *



By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.

Close