Explainable Machine Learning Exploiting News and Domain-Specific Lexicon for Stock Market Forecasting
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
In this manuscript, we propose a Machine Learning approach to tackle a binary classification problem whose goal is to predict the magnitude (high or low) of future stock price variations for individual companies of the S&P 500 index. Sets of lexicons are generated from globally published articles with the goal of identifying the most impactful words on the market in a specific time interval and within a certain business sector. A feature engineering process is then performed out of the generated lexicons, and the obtained features are fed to a Decision Tree classifier. The predicted label (high or low) represents the underlying company's stock price variation on the next day, being either higher or lower than a certain threshold. The performance evaluation we have carried out through a walk-forward strategy, and against a set of solid baselines, shows that our approach clearly outperforms the competitors. Moreover, the devised Artificial Intelligence (AI) approach is explainable, in the sense that we analyze the white-box behind the classifier and provide a set of explanations on the obtained results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Efficient capital markets: A review of theory and empirical work
Fama, E. F. (1970) · The Journal of Finance · doi:10.2307/2325486 · Cited by 13,635
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516 · cited 1×
“The increasing number of studies in the last couple of decades can be attributed to the development of techniques that allow an effective automatic processing of textual information, such as probabilistic topic models [16] and word-embeddings [17].”
BackgroundExplaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 1×
“In this section, we show a qualitative evaluation of the explainability of the proposed model [50], [51], by inspecting the white box of the Decision Tree and by providing examples of explanation.”
MethodToward Human-Understandable, Explainable AI
Hagras, H. (2018) · Computer · doi:10.1109/mc.2018.3620965 · Cited by 350 · cited 1×
“This kind of approach meets the recent and exponentially increasing trend of Explainable AI [42], [43], consisting of opening up the given black-box model by explaining how the method arrives to a certain final output.”
Method
Cited by
Machine learning techniques and data for stock market forecasting: A literature review
Kumbure, M. M., Lohrmann, C., Luukka, P., & Porras, J. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116659 · Cited by 535
Trust in AI: progress, challenges, and future directions
Afroogh, S., Akbari, A., Malone, E., Kargar, M., & Alambeigi, H. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04044-8 · Cited by 507
Explainable artificial intelligence (XAI) in finance: a systematic literature review
Černevičienė, J., & Kabašinskas, A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10854-8 · Cited by 289
Weber, P., Carl, K. V., & Hinz, O. (2024) · Management Review Quarterly · doi:10.1007/s11301-023-00320-0 · Cited by 292
Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models
Koa, K. J. L., Ma, Y., Ng, R., & Chua, T.-S. (2024) · Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589334.3645611 · Cited by 47
“Even among works that deal with explainable stock predictions [6, 40], the "explanations" are often simply defined as the specific texts that caused the price movement, which are usually obtained by analyzing learnt attention weights [15, 56].”
Background
Metadata
- Authors
- Carta, S. M., Consoli, S., Piras, L., Podda, A. S., & Recupero, D. R.
- Venue
- IEEE Access
- Published
- 2021
- Volume
- 9
- Pages
- 30193-30205
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3130367027
- Cited by
- 83
- References
- 7
- FWCI
- 9.24
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Explainable Machine Learning Exploiting News and Domain-Specific Lexicon for Stock Market Forecasting. https://app.cochat.ai/papers/10.1109/access.2021.3059960
- The paper
- Carta, S. M., Consoli, S., Piras, L., Podda, A. S., & Recupero, D. R. (2021). Explainable Machine Learning Exploiting News and Domain-Specific Lexicon for Stock Market Forecasting. IEEE Access. https://doi.org/10.1109/access.2021.3059960
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.