Does Selling Theoretical Performance Put Investors First?
An investment adviser touts the performance of a proprietary trading strategy that he claims posted solid returns over the past decade and suffered minimal losses during the last market swoon.
It sounds impressive, but after reading the marketing materials’ fine print, you discover that the results shown are theoretical and reflect the outcome of a backtest rather than a real client account.
Confused, you ask an important question: “Am I looking at this correctly?”
Marketing an unproven trading strategy using theoretical results and tiny disclosures sounds underhanded, but it is fairly common in the investment industry. Consider so-called ETF managed portfolios. Many of the strategists behind these products do not use best practices when displaying their results and/or employ strategies that are yet to be proven in real time.
A quick query using Morningstar’s ETF Managed Portfolios Screener brings up a database of more than 670 different strategies. But that number shrinks to fewer than 340 if you screen to only include strategies that have been in existence for more than three years and claim compliance with the Global Investment Performance Standards (GIPS®).
Moreover, one of the most prominent ETF strategists tracked by Morningstar sent a letter to clients admitting that they overstated the past performance of their main strategy in marketing materials and misrepresented that their strategy tracked real money.
Half-baked performance presentations are a red flag for two reasons: First, if the adviser is not using best practices when selling you a trading strategy, they might not be using best practices when developing and implementing it. Second, aggressive marketing material can make a trading strategy seem like it matches your risk tolerance when it really does not. This mismatch will likely cause you to abandon the strategy at an inopportune time, regardless of how well it works long term.
An article by David D. Spaulding, CIPM, and Steven W. Stone, “Model, Hypothetical, and Backtested Performance — Best Practices” provides seven best practices investors should expect ethical advisers to adhere to when presenting a strategy’s theoretical results. They are as follows:
- Clearly label all theoretical results as such (e.g., Backtested Global 130/30 Strategy).
- Do not link theoretical with actual performance in any way. This means more than just not geometrically linking the returns. This means that if you must include theoretical and actual performance in the same presentation, then show them on separate pages, and clearly label them.
- Do not state that “past performance is not indicative of future results.” Even though we are accustomed to this language, in the context of model performance, it implies that what is being shown is actual performance.
- Provide clear and prominent disclosure that the returns are theoretical, and describe all the assumptions made and their limits.
- Theoretical results should be shown only to consultants and sophisticated clients or prospects that have sufficient experience and knowledge to assess the product, presentation, and risks.
- Maintain sufficient records to support calculations and presentations.
- Consult with attorneys and your compliance department regarding applicable laws and regulations.
Investors can take much away from the list above, but one key insight is that you should not have to read the fine print to tell that results are theoretical.
Theoretical results should be apparent with a quick glance over the strategy’s marketing material. The adviser should label all theoretical results as such by using the word “model,” “hypothetical,” or “backtested” in the title of the exhibit. In addition, they should not link together theoretical and actual results in any way.
How Do You Evaluate Theoretical Results?
There are a number of issues that make gauging the quality of theoretical results a thorny task.
For example, if you are looking at the results of a backtest, consider that the adviser might have run hundreds of trials and model configurations when developing the trading strategy — tweaking the final one to deliver excellent returns over the sample time period shown. Research shows that leaning too heavily on trial-and-error when developing a trading strategy might have a detrimental effect on its real world efficacy.
Transaction costs are another issue. According to Aswath Damodaran, a valuation expert and finance professor at NYU’s Stern School of Business, transaction costs are the biggest culprit when theoretical strategies do not live up to investor expectations in the real world. The full impact of transaction costs — such as bid/ask spreads and the price impact you might have when you trade — can be difficult for researchers to quantify in theoretical models. For this reason, the results of “model” or “index” portfolios might not include all of the costs associated with buying and selling investments as recommended by the adviser’s trading strategy.
The unrealistic advantage of perfect hindsight and a lack of transaction costs are just two of the many issues that can make theoretical results fool’s gold.
According to the CFA Institute’s 2014 Global Market Sentiment Survey, investment professionals indicated that the mis-selling of financial products is the most serious ethical issue facing their local market this year. The good news is that identifying whether an adviser is putting your interests first is a lot easier than identifying whether they have uncovered the next best trading scheme. You should expect advisers to adhere to the seven best practices listed above when presenting the theoretical results of a trading strategy. Furthermore, you should be skeptical if an adviser uses the theoretical results of a backtest, model portfolio, or proprietary index as their main selling point, even if they are using best practices.
For a detailed discussion of best practices in investment performance measurement and reporting, investment professionals are invited to attend the GIPS Standards Annual Conference in Boston this September.
If you enjoyed this post, you should follow the Enterprising Investor here.
Please note that the content of this site should not be construed as investment advice, nor do the opinions expressed necessarily reflect the views of CFA Institute.
Photo credit: ©iStockphoto.com/retrorocket
Hello David
Thank you for the guidelines listed in this posting.
I am about to start the marketing of an investment strategy I have backtested, and even though I am marketing this only to professional money managers the points you have listed are helpful to ensure that the backtested results are accurately presented.
I also thought your point about being unable to model all transaction costs, such as the price impact of a trade, or bid-ask spreads, was a good one. I have used minimum volume filters to address both of these issues but as you point out this is not actual trading.
Thanks again for taking the time to put this post together – it is timely in my case.
Best wishes
Savio
Savio,
Thank you for reading the post! I am glad it helped. I would also suggest checking out the website below:
http://www.gipsstandards.org/Pages/index.aspx.
Regards,
Dave
Good article. The emergence of smart beta ETF strategies have enabled an industry to grow up in which both retail and institutional investors are being miss-sold products using back-tests which are the largely product of data mining exercises.
Those back-tests are packaged up by major index providers, for a lucrative fee, to give the small ETF providers the marketing benefit of being associated with a trusted major brand. By claiming to simply “track the index”, the ETF provider benefits from a loophole which enables them to present hypothetical performance as akin to a real money track record.
Unsurprisingly, the out of sample performance of many of these products has been disastrous.
A scandal waiting to happen.