Skill Ratio: A New Measure for the (Lack of) Persistence in Active Management
One of the most difficult tasks in investment management is choosing an outperforming active manager before they outperform. A new investment metric helps identify and differentiate those managers who achieve alpha through skill and those who generate it by luck.
Spoiler Alert: Most Are Lucky
In Thinking, Fast and Slow, Daniel Kahneman states that “the diagnostic for the existence of any skill is the consistency of individual differences in achievement.” And in order for skill to be acquired, Kahneman says two factors must be present:
- An environment that is high-validity, or sufficiently regular to be predictable.
- An opportunity to learn these regularities through prolonged practice.
In an empty gym, Steph Curry will drain three-pointers more consistently and at a much higher rate than the average person. The three-point line is the same distance, the ball is the same weight, the rim is the same size. Within seconds of Curry shooting his shot, the rim will provide feedback. The environment is static, Curry has had the opportunity to learn from the feedback provided, and thus skill can be acquired. In this example, skill is most certainly present.
So how do we identify skilled active fund managers?
Kahneman describes the stock market as a “low-validity environment.” That means, unlike Steph Curry’s empty gym, it “entails a significant degree of uncertainty and unpredictability.”
The SPIVA US Scorecard demonstrates that the chances of outperforming are slim. Additionally, the SPIVA Persistence Scorecard shows that consistent outperformance displays properties of randomness.
But some managers do outperform. And the annualized numbers for some of the top-performing funds over the last 15 to 20 years can be very seductive to investors.
So for those investors enticed by the siren song of an outperforming active manager, I pose this question: “Once your outperformance arrives, what did the journey look like?”
Dimeo, Schneider & Associates published “The Next Chapter in the Active vs. Passive Debate” in 2015. The research found that 92% of 10-year top-quartile funds spent at least one three-year period in the bottom half of their peer group, while 56% languished in the bottom half for at least one five-year period.
One way to visualize a fund’s journey to its ultimate performance number is to examine its rolling five-year excess returns relative to a benchmark.
So what does the frequency distribution of rolling five-year excess returns of one of the oldest, largest, and most successful active US equity mutual funds — the Dodge & Cox Stock Fund — look like?
DODGX Rolling Excess Five-Year Returns vs. S&P 500 Distribution Frequency*
Of the 169 observations, 75 — approximately 44% — are negative excess returns. And when we look at the excess returns chronologically, we can see where these periods of positive and negative excess returns fall.
Over these 14 years, DODGX has an average excess return of 2.48%. Not too bad. But we also want to be able to explain the variation, or (in)consistency, of these excess returns. The standard deviation of excess returns for DODGX is 5.50%.
The Skill Ratio
This is where the new metric, the Skill Ratio, comes in. By dividing the average excess rolling returns by the standard deviation of excess rolling returns, the Skill Ratio isolates and measures a manager’s consistency (or skill) and, like the Sharpe ratio, distills it into a single number.
In this instance, DODGX’s Skill Ratio is caluculated as follows.
Ideally, a skilled manager should generate high average excess returns on a consistent basis (i.e., with low volatility). A Skill Ratio greater than 1 indicates such ability.
The Skill Ratio can help differentiate between managers whose successful track records are the result of a couple of “lucky” runs and those who can consistently outperform a benchmark.
So what are the Skill Ratios for some of the top-performing US large-cap equity; Europe, Australasia, and Far East (EAFE) equity; and emerging market (EM) equity fund managers?
Top 20 US Large-Cap Funds
Top 20 Europe, Australasia, and Far East (EAFE) Funds
Top 20 Emerging Market (EM) Funds
And what are the rolling returns for some of the same top-performing US large-cap equity, EAFE equity, and EM equity fund managers?
CGM Focus was dubbed the “Best Stock Fund of the Decade” by the Wall Street Journal in 2009. During a period when the S&P 500 was more or less flat, CGMFX returned 18.2% on an annualized basis. Unfortunately, its volatility was significantly higher than the S&P 500’s, which caused sporadic flows in and out of the fund.
After a run of excellent performance, lots of assets would pour in just in time for a large drawdown. This led to a dollar-weighted return (or average investor return) of -11%. The inconsistent performance had a detrimental effect on the investor experience.
The Oppenheimer Developing Markets Fund has been one of the top performers in the EM space since 2001 and also has a very impressive Skill Ratio. But despite listing the MSCI EM Index as its benchmark, ODMAX has often held 30% to 40% of its portfolio in companies that MSCI would classify as EAFE stocks.
When we apply the Skill Ratio to factor investing, specifically the MSCI Diversified Multi-Factor indexes, which are used in the iShares Multifactor product suite, the results are especially interesting. (Disclaimer: These index returns are backtested and also fail to reflect the frictions generated by fees, taxes, and trading costs.)
Since 1999 for the US and EAFE indexes, and since 2001 for the EM index, each of the three MSCI Diversified Multifactor indexes ranked in the top 10% of their categories in terms of performance, with Skill Ratios among the top five of all funds.
Kahneman posits that investors are better off following a simple algorithm or formula rather than the judgment of experts, especially in low-validity environments. The MSCI Diversified Multifactor indexes successfully apply the same systematic, rules-based approach across three different asset classes.
And most importantly, the results appear to be delivered consistently, or “skillfully.”
If a manager is underperforming, how can we reliably determine whether it is simply a bad spell that will eventually turn into outperformance, or if the underperformance will persist and potentially lead to obsolescence?
A quick cautionary tale when considering this quandary: There were 1,227 US large-cap funds to choose from as of 1 January 1999. Over the next 19 years, 745 — 60% — merged or liquidated. And each of these obsolescent funds no doubt had CFAs, MBAs, and PhDs on its roster, a sound investment thesis, and an approach that attracted some assets.
So when choosing an active manager, remember: Every corpse on Mount Everest was once a highly motivated individual.
*All fund data was sourced using Morningstar Direct. MSCI Diversified Multifactor Index returns were sourced from Bloomberg and uploaded using Morningstar Direct.
If you liked this post, don’t forget to subscribe to the Enterprising Investor.
All posts are the opinion of the author. As such, they should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute or the author’s employer.
Image credit: ©Getty Images/erhui1979
Professional Learning for CFA Institute Members
CFA Institute members are empowered to self-determine and self-report professional learning (PL) credits earned, including content on Enterprising Investor. Members can record credits easily using their online PL tracker.
Daniel:
Fantastic article! It is another confirmation that we should spend our time doing our most valuable work for our clients, to wit: investment counseling. Trying to consistently beat the market is, in the words of Charles Ellis, a “Loser’s Game”.
Greg Dorriety, CFA
Nice piece and a great closing line. I really liked the way you decomposed the returns of ODMAX and looked at it from a couple of angles. We have been using a similar metric using 3 year data. Well articulated.
How can i do to calculate the avg. rolling 5-year excess return and the avg. rolling 5-year excess return std. dev. ? or where can i find these informations to calculate the skill ratio ?
Is the skill ratio much different to the information ratio?
This is exactly the same in principle:
– information ratio is average excess return divided by tracking error. The tracking error being exactly the standard error of excess returns
– the skill ratio presented here is based on 5yr rolling excess returns. I is therefore a kind of ‘rolling 5yr’ information ratio.
There pros and cons for using such a rolling statistics. The most striking negative of such a stati is apparent on charts above: they are ‘smoothed’ over time. Statisticians call for auto-correlations in data which generally dampens the precision of std error or average estimates.
The main positive is to provide a medium term view which is key to assess skill.
I wonder is there are dfferent smoothness degree among active Funds in excess returns. Put another way: how quickly are the active manages excess returns mean revert?
My guess:
– the slower the worse in terms of ‘skill’ – (you have to wait longer to realize the skill as an investor).
– Factor investing is built to maximize information ratios (or skill ratio), but might be slower to mean revert tahn active managers excess returns.
Looks like the Information Ratio, which is OLD. In real world IR over 0.5 is good.
It seems to me your Skill Ratio is eerily similar to the Information Ratio. What is the difference?
Daniel an article I very much enjoyed reading for its clarity and insight. Thank you for putting it together. Have a nice weekend Savio
I think 5-year periods are not adequate. The correct period should be the length of economic cycles, e.g. 2000-2007 or 2007-20017.
This metric also makes no sense, if a manager achieved the return of the respective index or a bit more but significanty reduced the volatilty and drawdowns during crises, e.g. halving it. Even though most investors would prefer that, this skill is not reflected in this metric, as it would calculate values much below 1.
What is the use of applying such metrics to backtests such as with the MF indices here? In see no value at all except for their promotion. Because the highest hurdle for active management is to keep up its edge when market inefficiencies, which it tries to exploit, are eroded by overcrowding. This mechanism is completely absent in backtests. They are over-optimized to deliver such virtual results in the first place! However, no skill, which is discussed here, is involved.
Interesting article. But rolling period analyses completely ignore and miss the most important driver of long term investor returns – the power of compounding. Would like to see the cumulative Dollar return from beginning of the data set. It is possible, indeed mathematically certain that one can compound at a lower return than the benchmark index for several years and still be climbing away from the index on a cumulative since inception basis, IF the base at start of the underperformance period is higher enough than index due to previous outlerformance periods. Rolling periods completely expinge the returns that came before, which is not the investor’s experience, assuming they have been invested for a longer period than the rolling term being analysed. Previous outperforming returns are still in the base, they do not disappear. Investor behaviour is the bigger problem – very few stay invested through a full cycle, being driven all to often by backward looking three year rolling period surveys.
I absolutely agree with this comment. It occurred to me to build an analysis over the period of 48 years in a theoretical fund that invests 100% of its assets in shares of BRK. In this scenario, the ratio is above 1 only in 5 years and it is well known how well an investor would have made had it invested this way. I do welcome any attempts to quantify asset management skill though as it would be of great help to the Industry.
At some point, investors will learn that tracking error of investment return is not investment risk. Tracking error assumes that the investor’s preferred path to her objective is the same path as the benchmark. Don’t believe me…compare information ratio for two investments with the same average return over time but one having no variability of return (constant return year-to-year) and the other having the same variability as the S&P 500. That’s an extreme example but it illustrates that IR will suggest you buy the risky asset instead of the guaranteed one!
The other issue that’s left out of the author’s analysis is the level of absolute returns when the periods of underperformance occurred. Let’s assume an investor’s target return to achieve her objective is 7% over time. And let’s also assume that an active strategy underperforms by 2% per year when the benchmark is up 12% or more. Horrible investment, right? Just go passive. Yet the strategy is still crushing the investor’s target return. And what if that same strategy has outperformed by 4% per year when the benchmark return was 4% or less. Lower highs and higher lows makes for a much more pleasant ride and yet tracking error, information ratio and “skill ratio” penalize managers for not producing a pattern of returns more like the benchmark!
I figure that at some point in history, someone decided that a completely relevant statistic like tracking error for uses like engineering precision could be applied to investments. Unfortunately, they didn’t ask “to what benefit”? Tracking error, or its equivalent, is a great tool to ensure the precision sizing of jet engine ball bearings – and variability of flight path due to engine failure is certainly risk. But suggesting that deviation from a volatile asset’s return is “risk” is a failing of statistical application.
How well does selecting mutual funds based on past skill ratio work on a rolling basis?