Measuring Qualitative Information in Capital Markets Research: Comparison of Alternative Methodologies to Measure Disclosure Tone

Abstract

This study evaluates alternative measures of the tone of financial narrative. We present evidence that word-frequency tone measures based on domain-specific wordlists—compared to general wordlists—better predict the market reaction to earnings announcements, have greater statistical power in short-window event studies, and exhibit more economically consistent post-announcement drift. Further, inverse document frequency weighting, advocated in Loughran and McDonald (2011), provides little improvement to the alternative approach of equal weighting. We also provide evidence that word-frequency tone measures are as powerful as the Naïve Bayesian machine-learning tone measure from Li (2010) in a regression of future earnings on MD&A tone. Overall, although more complex techniques are potentially advantageous in certain contexts, equal-weighted, domain-specific, word-frequency tone measures are generally just as powerful in the context of financial disclosure and capital markets. Such measures are also more intuitive, easier to implement, and, importantly, far more amenable to replication.