Views on improving the integrity of global capital markets
01 March 2016

In Fintech Era, Harvesting Signals in Words to Forecast Volatility and Returns

Posted In: Market Structure

How many news articles have you read today? How many blog posts like this one? Of what you’ve read, how much has formed your opinion of likely returns and volatility in the stock market today? This week? This year?

A recent paper suggests that the news we read contains useful signals about returns and volatility, and that using natural language-processing techniques can harvest these insights far more efficiently than any human can, burdened as we are with what the Columbia University authors, Harry Mamaysky and Paul Glasserman, refer to as “rational inattention.”

Mamaysky and Glasserman go beyond using computers to assess the sentiment expressed in news articles (simply counting the number of negative or positive words to arrive at a sentiment indicator) to consider the “unusualness” of words, defined as the relative frequency of appearance of a particular phrase prior to its publication in the current news article of interest. The study finds that the interaction of sentiment and unusualness does a better job of forecasting volatility than either measure alone, at both a single-stock and aggregate market level; the interaction between unusualness and negative sentiment is especially powerful. And, rather than being incorporated and reflected in prices quickly as sentiment indicators generally are, the combination of unusualness and sentiment apparently reflects information that is absorbed slowly by markets, with effects out as far as six months.

Investors, Policymakers: Take Note

The implications for investors are important. We have limits to how much information we can consume and process as individuals, and indeed, much of investment analysis is a sort of triage process of trying to identify what information is relevant and what can be discounted and ignored. But Mamaysky and Glasserman cite academic work that suggests that investors tend to focus on a narrow set of stocks or industries and miss information that is relevant when it is aggregated over many stocks over time. This “rational inattention” is the logical end result of the cost-benefit analysis that we all perform, consciously or not, to decide what to pay attention to.

Policymakers should take interest in this application of technology as well. Finding evidence of brewing trouble in financial markets through this sort of analysis of news in the public domain could be useful to anticipate and mitigate systemic shocks, although further refinement is probably necessary. Knowing that a storm is approaching is useful, but understanding just where in the financial ecosystem pressures are exceeding tolerances is essential to mitigation.

Investors also have a challenging task to put this insight to use. Macro states may be relevant yet unobservable, with only imperfect hedging vehicles (VIX, for example) available. For specific portfolio holdings, the authors make a preliminary assessment (with appropriate caution about the small sample size) of the embedded value in news for predicting prices, again suggesting that combining both sentiment and a measure of unusualness adds predictive power.


Investment practitioners have a new dimension of fintech to think about. Natural language-processing techniques threaten to make interpreting data and news (a bulwark of our added value to clients) a quaint anachronism limited by what looks like a really short attention span relative to what the humming servers in the cloud can do. If finding signals in the cacophony of data and news isn’t our best use in the future, applying those signals systematically and profitably might still be. Even at this nascent stage with academia teasing out some of the possibilities, we’d be wise to think about how to embrace and apply this change to how we do business.

If you liked this post, consider subscribing to Market Integrity Insights.

Photo Credit:

About the Author(s)
Bob Dannhauser, CFA

Bob Dannhauser, CFA, was the head of global private wealth management at CFA Institute.

Leave a Reply

Your email address will not be published. Required fields are marked *

By continuing to use the site, you agree to the use of cookies. more information

The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.