Practical analysis for investment professionals
20 October 2014

Nobel Laureate Myron Scholes on the Common Mistakes Research Analysts Make

In the first half of my interview with Nobel Laureate Myron Scholes, he shared the intuition behind the Black–Scholes option pricing model and its extensions. We then moved on to discuss some more philosophical issues about research methods, such as the common mistakes research analysts make. This was probably the most thought-provoking conversation I’ve had in a long time.

CFA Institute: What are some of the things that fundamental analysts and quantitative analysts are each good at?

Myron Scholes: I look at three dimensions. One is forecasting the future, which is the alpha component, the abnormal return component. It takes a lot of skill to be able to forecast the future, and there are a lot of constraints on managers when they forecast that stocks are rich or cheap. How much of the future can they forecast? Is it over a short horizon or long horizon? Most active managers spend their time pursuing alpha or looking for rich and cheap securities within a sector.

The second dimension is risk management. What factors affect returns? Are those factors changing, and why? Most quantitative investors don’t really spend much time on changing risk. Quantitative managers do measure and monitor factor risks. They use quantitative techniques to understand factors that affect returns such as cash flows, earnings, small stocks, or valuation. Most managers compare their performance to a benchmark. This is relative performance. They ignore absolute performance and risk. And, many managers are constrained to stay close to the benchmark. This affects returns.

The third dimension is the intermediation business — I call this “omega.” In the omega world, investors are willing to pay others to take risks from them. Examples here include merger arbitrage or credit quality or spinoffs or corporate reorganizations. Investors pay others to take the risks, [are] willing to give up returns. Fundamental analysts or risk modelers don’t understand the omega business — they don’t have the expertise needed to delve into these opportunities.

Nobel Laureate Myron Scholes on the Common Mistakes Research Analysts Make2

What do you think are the common mistakes that analysts tend to make in developing models?

There are three important problems in quantitative science, which apply more generally than just to finance. And these problems apply equally to qualitative analysis and how we build our views and models of the world around us.

One is data mining: You use historical data to build your views or to build models that spot anomalies. The problem is, we build our views and test our views using historical data. What value is this? We might be lucky and take a random sample of past data. But this is unlikely. As a result, our results are biased and have little predictive power. Time series analysis, therefore is fraught with worries. The correct data can provide a perspective of the future. But, we never know the extent of data mining. Some claim that with long periods, such as over 50 or 60 years, we can identify risk factors that generate returns. I’m skeptical too, like you. The best defense is to use economic theory to develop a model of why something should be happening, and then garner data to test the model. Even then we must be careful about building a model, for our views are not independent of previous observations.

Two is the cross section. Cross sectional data, or panels, produce assumed clusters that are NP-complete in the sense that once I know the clusters, I can put elements into the clusters. But how many clusters really exist is unknown. If you don’t know the clusters beforehand, then you are making an error that could lead to wrong quantitative analysis and false conclusions. It is believed to be easier to put elements into clusters. But be careful. For example, prior to the financial crisis of 2008, the rating agencies assumed that home owners in Stockton, California, defaulted on homes independently of those in Las Vegas or Miami. Or Spain would be in difficulty independently of Italy. Or diversification in the stock market would provide protection in a crisis. Well, both the time series and cross section were wrong.

The third problem is that a model by definition is an incomplete description of reality. Every model has an error term. People will reverse-engineer the model and see whether they can use the error to make money against the modeler.

Can you give us an example?

When rating agencies started rating mortgage pools in the United States, they used the time series, the history of defaults on housing. In addition, they assumed that clusters were independent, which provided diversification against defaults. And the third component is that for a mortgage structure to achieve a AAA rating, it had needed to contain a certain number of quality mortgages. As a result, mortgage structures reverse-engineered the error of the model to deduce how to just pass to achieve an AAA rating. They started to put in more and more “stones in the wheat for delivery” and put worse and worse credits into their structures to just pass the AAA rating. The rating agencies did not adjust their models fast enough to correct their errors.

Then the defaults occurred. First, the credits were much worse than they assumed. Second, Las Vegas, Stockton, and Miami all defaulted together, so they lacked independence, which would have reduced risk. And third, because they used a short time period to estimate the probability of defaults, there were many more defaults than they ever thought possible. Housing prices fell to a much greater extent than was ever built into their models. I keep these three dimensions of risk of both qualitative and quantitative analysis in mind in thinking and modeling. There are risks in all of the sciences including aspects of our lives generally.

One way of dealing with the time series issue: What if we start from thinking how the world works, and then based on our experience, we put together a model? We will then test it with data and try not to tweak it after the tests. Even then I thought we may still not be in the clear because our experience is just like the rest of the data. It’s still data mining.

That is right because we all peek. [Laughter] Science is induction and then deduction. We must use induction first before we deduce anything. Our experience is induction, so that’s already data mining. That’s the problem for all science. We need to be careful.

There’s no way around that.

The only way around it is to have a first principles model. There’s no way around it. It’s just the drudgery of doing quantitative analysis. Fundamental and other forms of qualitative analysis have the same three issues. The best way around the problem is to be lucky or to build models where the assumptions are not too onerous and give insights on how to enhance our understanding.

In spirit, it is the same thing.

In spirit, it is the same thing. That’s what makes the investment world interesting.

It’s a very difficult business. Could the way to get out of this be somehow combining the fundamental and quantitative analyses?

I would think so. I think that basically every model has an error. Therefore, if I tell you my model, you can reverse-engineer the error and game against me. In some sense the market protects you. You trade at market prices thinking that the stock is cheap (by your model), but it isn’t. Unless you are perverse, you will make random returns. So if an investor has a model, which is systematic, and prices using that model, he will lose — for someone else will have better intuition or new data that will game against the model. This is why non-market pricing is so difficult. When I play golf, I have a model of the golf game, which has a much greater error, however, than does the model that Tiger Woods [uses]. So the question is whether the convex combination is better than running the model alone. The purists say, no, you can’t do that. We could tell them that their models are also wrong. [Laughter]

Nobel Laureate Myron Scholes on the Common Mistakes Research Analysts Make2

You know the risk model is wrong, but it still gives you a good starting point. Whenever you look at a risk model trying to understand what’s going on, you get a better sense of why this is moving, why this is not moving, and why this probably just reflects the fact that the model is overspecified. At the same time, you still need to have a feel for what is going on.

Intuition is a model, too!

Indeed! Going back to what you said, everybody peeks. Since the global financial crisis, the world has changed in many ways. Some old models seem to have stopped working.

The herd goes on. The herd has a model, and the herd changes its location. That’s what makes science interesting. If we knew everything, [if] everything were static, we’d all give up. We’d be too bored.

It’s a constant pursuit. Thank you so much. It’s been really fascinating.

Read the first installment of the Myron Scholes-Larry Cao, CFA, interview, “Nobel Laureate Myron Scholes on the Black–Scholes Option Pricing Model.”

If you liked this post, consider subscribing to the Enterprising Investor.

Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.

Photo credits: ©CFA Institute.

About the Author(s)
Larry Cao, CFA

Larry Cao, CFA, senior director of industry research, CFA Institute, conducts original research with a focus on the investment industry trends and investment expertise. His current research interests include multi-asset strategies and FinTech (including AI, big data, and blockchain). He has led the development of such popular publications as FinTech 2017: China, Asia and Beyond, FinTech 2018: The Asia Pacific Edition, Multi-Asset Strategies: The Future of Investment Management and AI Pioneers in Investment management. He is also a frequent speaker at industry conferences on these topics. During his time in Boston pursuing graduate studies at Harvard and as a visiting scholar at MIT, he also co-authored a research paper with Nobel laureate Franco Modigliani that was published in the Journal of Economic Literature by American Economic Association. Larry has more than 20 years of experience in the investment industry. Prior to joining CFA Institute, Larry worked at HSBC as senior manager for the Asia Pacific region. He started his career at the People’s Bank of China as a USD fixed-income portfolio manager. He also worked for US asset managers Munder Capital Management, managing US and international equity portfolios, and Morningstar/Ibbotson Associates, managing multi-asset investment programs for a global financial institution clientele. Larry has been interviewed by a wide range of business media, such as Bloomberg, CNN, the Financial Times, South China Morning Post and the Wall Street Journal.

6 thoughts on “Nobel Laureate Myron Scholes on the Common Mistakes Research Analysts Make”

  1. Jamal Munshi says:

    i agree with him about the data mining. great interview. thank you. btw, i wrote a paper on teaching finance with monte carlo simulation instead of math formulas and would be grateful for some feedback. here is the link to my paper.

  2. Mark says:

    Um remember when Myron Schoals was a principal at Long Term Capital Management? Not sure he’s the best person to tell analysts about the mistakes they’re making creating models; his personal track record not so great on that front, like when their models almost brought down the financial system. “Markets are efficient, it’s ok to lever Russian bonds at 25x”.

    1. Rachman U Budiman says:

      Hope he has learned his mistakes from LTCM and his thought has reflected that experiences.

  3. Larry Cao, CFA says:

    Thank you all for visiting our blog!


Leave a Reply

Your email address will not be published. Required fields are marked *

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.