The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.
Text mining of news-headlines for FOREX market prediction: A Multi-layer Dimension Reduction Algorithm with semantics and sentiment
Abstract
FOREX prediction through text mining of news is viable and effective.Feature-selection by abstraction of word-hypernyms increases prediction accuracy.Feature-weighting based on the sum of pos and neg sentiment scores is effective.Feature-reduction based on maximum optimization for prediction-target is crucial. In this paper a novel approach is proposed to predict intraday directional-movements of a currency-pair in the foreign exchange market based on the text of breaking financial news-headlines. The motivation behind this work is twofold: First, although market-prediction through text-mining is shown to be a promising area of work in the literature, the text-mining approaches utilized in it at this stage are not much beyond basic ones as it is still an emerging field. This work is an effort to put more emphasis on the text-mining methods and tackle some specific aspects thereof that are weak in previous works, namely: the problem of high dimensionality as well as the problem of ignoring sentiment and semantics in dealing with textual language. This research assumes that addressing these aspects of text-mining have an impact on the quality of the achieved results. The proposed system proves this assumption to be right. The second part of the motivation is to research a specific market, namely, the foreign exchange market, which seems not to have been researched in the previous works based on predictive text-mining. Therefore, results of this work also successfully demonstrate a predictive relationship between this specific market-type and the textual data of news. Besides the above two main components of the motivation, there are other specific aspects that make the setup of the proposed system and the conducted experiment unique, for example, the use of news article-headlines only and not news article-bodies, which enables usage of short pieces of text rather than long ones; or the use of general financial breaking news without any further filtration.In order to accomplish the above, this work produces a multi-layer algorithm that tackles each of the mentioned aspects of the text-mining problem at a designated layer. The first layer is termed the Semantic Abstraction Layer and addresses the problem of co-reference in text mining that is contributing to sparsity. Co-reference occurs when two or more words in a text corpus refer to the same concept. This work produces a custom approach by the name of Heuristic-Hypernyms Feature-Selection which creates a way to recognize words with the same parent-word to be regarded as one entity. As a result, prediction accuracy increases significantly at this layer which is attributed to appropriate noise-reduction from the feature-space.The second layer is termed Sentiment Integration Layer, which integrates sentiment analysis capability into the algorithm by proposing a sentiment weight by the name of SumScore that reflects investors' sentiment. Additionally, this layer reduces the dimensions by eliminating those that are of zero value in terms of sentiment and thereby improves prediction accuracy.The third layer encompasses a dynamic model creation algorithm, termed Synchronous Targeted Feature Reduction (STFR). It is suitable for the challenge at hand whereby the mining of a stream of text is concerned. It updates the models with the most recent information available and, more importantly, it ensures that the dimensions are reduced to the absolute minimum.The algorithm and each of its layers are extensively evaluated using real market data and news content across multiple years and have proven to be solid and superior to any other comparable solution. The proposed techniques implemented in the system, result in significantly high directional-accuracies of up to 83.33%.On top of a well-rounded multifaceted algorithm, this work contributes a much needed research framework for this context with a test-bed of data that must make future research endeavors more convenient. The produced algorithm is scalable and its modular design allows improvement in each of its layers in future research. This paper provides ample details to reproduce the entire system and the conducted experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421 · cited 1×
“Last but not least, the system is using a standard Support Vector Machine (SVM) algorithm (Cortes & Vapnik, 1995) for model creation and label prediction.”
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · cited 2×
“Specifically, the semantics of categories are represented by senses of terms appearing in the category labels as well as the interpretation of them by WordNet (Miller, 1995).”
BackgroundTwitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206
SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining
Baccianella, S., Esuli, A., & Sebastiani, F. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2jktnrykui9s · Cited by 2,666
SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining
Esuli, A., & Sebastiani, F. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/54vo5wjya4cw · Cited by 2,480
Is All That Talk Just Noise? The Information Content of Internet Stock Message Boards
Antweiler, W., & Frank, M. Z. (2004) · The Journal of Finance · doi:10.1111/j.1540-6261.2004.00662.x · Cited by 2,373
More Than Words: Quantifying Language to Measure Firms' Fundamentals
TETLOCK, P. C., SAAR‐TSECHANSKY, M., & MACSKASSY, S. (2008) · The Journal of Finance · doi:10.1111/j.1540-6261.2008.01362.x · Cited by 2,304
Financial time series forecasting using support vector machines
Kim, K.-J. (2003) · Neurocomputing · doi:10.1016/s0925-2312(03)00372-2 · Cited by 1,734
Yahoo! for Amazon: Sentiment Extraction from Small Talk on the Web
Das, S. R., & Chen, M. Y. (2007) · Management Science · doi:10.1287/mnsc.1070.0704 · Cited by 1,396
LI, F. (2010) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2010.00382.x · Cited by 1,360
New Avenues in Opinion Mining and Sentiment Analysis
Cambria, E., Schuller, B., Xia, Y., & Havasi, C. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.30 · Cited by 1,285
Application of support vector machines in financial time series forecasting
Tay, F. E. H., & Cao, L. (2001) · Omega · doi:10.1016/s0305-0483(01)00026-3 · Cited by 1,268
Developing a Test Collection for the Evaluation of Integrated Search
Lykke, M., Larsen, B., Lund, H., & Ingwersen, P. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-12275-0_63 · Cited by 969
Textual analysis of stock market prediction using breaking financial news: The AZFin text system
Schumaker, R. P., & Chen, H. (2009) · ACM Transactions on Information Systems · doi:10.1145/1462198.1462204 · Cited by 777
The impact of preprocessing on text classification
Uysal, A. K., & Gunal, S. (2014) · Information Processing & Management · doi:10.1016/j.ipm.2013.08.006 · Cited by 657
More than words: Social networks’ text mining for consumer brand sentiments
Mostafa, M. M. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.01.019 · Cited by 623
Text mining for market prediction: A systematic review
Khadjeh Nassirtoussi, A., Aghabozorgi, S., Ying Wah, T., & Ngo, D. C. L. (2014) · Expert Systems with Applications · doi:10.1016/j.eswa.2014.06.009 · Cited by 615
Self-citationFeature selection for text classification with Naïve Bayes
Chen, J., Huang, H., Tian, S., & Qu, Y. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.06.054 · Cited by 576
The impact of social and conventional media on firm equity value: A sentiment analysis approach
Yu, Y., Duan, W., & Cao, Q. (2013) · Decision Support Systems · doi:10.1016/j.dss.2012.12.028 · Cited by 496
Ghiassi, M., Skinner, J., & Zimbra, D. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.05.057 · Cited by 485
Evidence on the speed of convergence to market efficiency
Chordia, T., Roll, R., & Subrahmanyam, A. (2005) · Journal of Financial Economics · doi:10.1016/j.jfineco.2004.06.004 · Cited by 453
Text feature selection using ant colony optimization
Aghdam, M. H., Ghasem-Aghaee, N., & Basiri, M. E. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2008.08.022 · Cited by 389
How is macro news transmitted to exchange rates?
Evans, M. D. D., & Lyons, R. K. (2008) · Journal of Financial Economics · doi:10.1016/j.jfineco.2007.06.001 · Cited by 366
Ontology-based sentiment analysis of twitter posts
Kontopoulos, E., Berberidis, C., Dergiades, T., & Bassiliades, N. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.01.001 · Cited by 365
Hagenau, M., Liebmann, M., & Neumann, D. (2013) · Decision Support Systems · doi:10.1016/j.dss.2013.02.006 · Cited by 356
An improved K-nearest-neighbor algorithm for text categorization
Jiang, S., Pang, G., Wu, M., & Kuang, L. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.08.040 · Cited by 322
Evaluating sentiment in financial news articles
Schumaker, R. P., Zhang, Y., Huang, C.-N., & Chen, H. (2012) · Decision Support Systems · doi:10.1016/j.dss.2012.03.001 · Cited by 318
Is All That Talk Just Noise? The Information Content of Internet Stock Message Boards
Antweiler, W., & Frank, M. Z. (2001) · SSRN Electronic Journal · doi:10.2139/ssrn.282320 · Cited by 297
Forecasting Intraday stock price trends with text mining techniques
Mittermayer, M.-A. (2004) · 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the · doi:10.1109/hicss.2004.1265201 · Cited by 264
Sound and complete relevance assessment for XML retrieval
Piwowarski, B., Trotman, A., & Lalmas, M. (2008) · ACM Transactions on Information Systems · doi:10.1145/1416950.1416951 · Cited by 238
Twitter Mood as a Stock Market Predictor
Bollen, J., & Mao, H. (2011) · Computer · doi:10.1109/mc.2011.323 · Cited by 220
Emotion detection in suicide notes
Desmet, B., & Hoste, V. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.05.050 · Cited by 213
An intraday market risk management approach based on textual analysis
Groth, S. S., & Muntermann, J. (2011) · Decision Support Systems · doi:10.1016/j.dss.2010.08.019 · Cited by 167
Sermpinis, G., Theofilatos, K., Karathanasopoulos, A., Georgopoulos, E. F., & Dunis, C. (2013) · European Journal of Operational Research · doi:10.1016/j.ejor.2012.10.020 · Cited by 167
Daily stock market forecast from textual web data
Wüthrich, B., Cho, V., Leung, S., Permunetilleke, D., Sankaran, K., & Zhang, J. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1998.725072 · Cited by 163
Twitter user profiling based on text and community mining for market analysis
Ikeda, K., Hattori, G., Ono, C., Asoh, H., & Teruo Higashino, Y. (2013) · Knowledge-Based Systems · doi:10.1016/j.knosys.2013.06.020 · Cited by 158
News Sensitive Stock Trend Prediction
Fung, G. P. C., Yu, J. X., & Lam, W. P. (2002) · Lecture notes in computer science · doi:10.1007/3-540-47887-6_48 · Cited by 153
Is the k -NN classifier in high dimensions affected by the curse of dimensionality?
Pestov, V. (2012) · Computers & Mathematics with Applications · doi:10.1016/j.camwa.2012.09.011 · Cited by 135
Combining News and Technical Indicators in Daily Stock Price Trends Prediction
Zhai, Y., Hsu, A. L., & Halgamuge, S. K. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-72395-0_132 · Cited by 129
An Experiment in Integrating Sentiment Features for Tech Stock Prediction in Twitter
Vu, T. T., Chang, S., Ha, Q.-T., & Collier, N. (2012) · Open Research Online - ORO (The Open University) · Cited by 127
Yu, H., Nartea, G. V., Gan, C. E. C., & Yao, L. J. (2012) · International Review of Economics & Finance · doi:10.1016/j.iref.2012.07.016 · Cited by 122
Chaos-based support vector regressions for exchange rate forecasting
Huang, S.-C., Chuang, P.-J., Wu, C.-F., et al. (2010) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.06.001 · Cited by 120
Stock prediction: Integrating text mining approach using real-time news
Fung, G. P. C., Yu, J. X., & Lam, W. P. (2003) · IEEE Conference on Computational Intelligence for Financial Engineering & Economics · doi:10.1109/cifer.2003.1196287 · Cited by 113
Design and implementation of fuzzy expert system for Tehran Stock Exchange portfolio recommendation
Fasanghari, M., & Montazer, G. A. (2010) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.02.114 · Cited by 110
Chordia, T., Goyal, A., Lehmann, B. N., et al. (2013) · Journal of Financial Markets · doi:10.1016/j.finmar.2013.06.004 · Cited by 105
Anastasakis, L., & Mort, N. (2009) · Expert Systems with Applications · doi:10.1016/j.eswa.2009.03.057 · Cited by 67
Forex-foreteller: currency trend modeling using news articles
Jin, F., Self, N., Saraf, P., Butler, P., Wang, W., & Ramakrishnan, N. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487710 · Cited by 65
Financial news semantic search engine
Lupiani, E., García-Manotas, I., Valencia‐García, R., García‐Sánchez, F., Castellanos–Nieves, D., Fernández‐Breis, J. T., & Camón-Herrero, J. B. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.06.003 · Cited by 55
Financial Forecasting Using Character N-Gram Analysis and Readability Scores of Annual Reports
Butler, M., & Kešelj, V. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01818-3_7 · Cited by 45
Mabu, S., Hirasawa, K., Obayashi, M., & Kuremoto, T. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.05.037 · Cited by 45
Enhancing stockmarket trading performance with ANNs
Vanstone, B. J., & Finnie, G. R. (2010) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.02.124 · Cited by 34
How fast do stock prices adjust to market efficiency? Evidence from a detrended fluctuation analysis
Reboredo, J. C., Castro, M. A. R., Miranda, J. G. V., & Garcia-Rubio, R. (2012) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2012.11.038 · Cited by 31
Adapting centroid classifier for document categorization
Tan, S., Wang, Y., & Wu, G. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2011.02.114 · Cited by 23
Intraday Foreign Exchange Rate Forecasting Using Sparse Grids
Garcke, J., Gerstner, T., & Griebel, M. (2012) · Lecture notes in computational science and engineering · doi:10.1007/978-3-642-31703-3_4 · Cited by 11
Parallel rare term vector replacement: Fast and effective dimensionality reduction for text
Berka, T., & Vajteršic, M. (2012) · Journal of Parallel and Distributed Computing · doi:10.1016/j.jpdc.2012.08.008 · Cited by 7
Cited by
A survey on opinion mining and sentiment analysis: Tasks, approaches and applications
Ravi, K., & Ravi, V. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.06.015 · Cited by 1,306
Computational Intelligence and Financial Markets: A Survey and Future Directions
Cavalcante, R. C., Brasileiro, R. C., Souza, V. L. F., Nobrega, J. P., & Oliveira, A. L. I. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.02.006 · Cited by 589
A Survey of Forex and Stock Price Prediction Using Deep Learning
Hu, Z., Zhao, Y., & Khushi, M. (2021) · Applied System Innovation · doi:10.3390/asi4010009 · Cited by 340
Text-based crude oil price forecasting: A deep learning approach
Li, X., Shang, W., & Wang, S. (2019) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2018.07.006 · Cited by 304
Predicting short-term stock prices using ensemble methods and online data sources
Weng, B., Lu, L., Wang, X., Megahed, F. M., & Martinez, W. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.06.016 · Cited by 278
Metadata
- Authors
- Khadjeh Nassirtoussi, A., Aghabozorgi, S., Ying Wah, T., & Ngo, D. C. L.
- Venue
- Expert Systems with Applications
- Publisher
- Elsevier BV
- Published
- 2015-01
- Volume
- 42
- Issue
- 1
- Pages
- 306-324
- ISSN
- 0957-4174 (ISSN-L)
- OpenAlex
- W2095847349
- Cited by
- 275
- References
- 56
- FWCI
- 18.89
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.