Mastering Stock Markets with Efficient Mixture of Diversified Trading Experts
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
Quantitative stock investment is a fundamental financial task that highly relies on accurate prediction of market status and profitable investment decision making. Despite recent advances in deep learning (DL) have shown stellar performance on capturing trading opportunities in the stochastic stock market, the performance of existing DL methods is unstable with sensitivity to network initialization and hyperparameter selection. One major limitation of existing works is that investment decisions are made based on one individual neural network predictor with high uncertainty, which is inconsistent with the workflow in real-world trading firms. To tackle this limitation, we propose AlphaMix, a novel three-stage mixture-of-experts (MoE) framework for quantitative investment to mimic the efficient bottom-up hierarchical trading strategy design workflow of successful trading companies. In Stage one, we introduce an efficient ensemble learning method, whose computational and memory costs are significantly lower comparing to traditional ensemble methods, to train multiple groups of trading experts with personalised market understanding and trading styles. In Stage two, we collect diversified investment suggestions through building a pool of trading experts utilizing hyperparameter level and initialization level diversity of neural networks for post hoc ensemble construction. In Stage three, we design three different mechanisms, namely as-needed router, with-replacement selection and integrated expert soup, to dynamically pick experts from the expert pool, which takes the responsibility of a portfolio manager. Through extensive experiments on US and Chinese stock markets, we demonstrate that AlphaMix significantly outperforms many state-of-the-art baselines in terms of 7 popular financial criteria.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
EFFICIENT CAPITAL MARKETS: A REVIEW OF THEORY AND EMPIRICAL WORK*
Malkiel, B. G., & Fama, E. F. (1970) · The Journal of Finance · doi:10.1111/j.1540-6261.1970.tb00518.x · Cited by 15,699
Efficient capital markets: A review of theory and empirical work
Fama, E. F. (1970) · The Journal of Finance · doi:10.2307/2325486 · Cited by 13,635
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
Adaptive Mixtures of Local Experts
Jacobs, R. A., Jordan, M. I., Nowlan, S. J., & Hinton, G. E. (1991) · Neural Computation · doi:10.1162/neco.1991.3.1.79 · Cited by 5,384
Hansen, L. K., & Salamon, P. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.58871 · Cited by 4,260 · cited 1×
“Ensemble methods, which combine the output of different models to improve the overall generalization performance, have been studied for decades [15, 16, 21, 31, 43, 45, 46].”
Method5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503
Film: Visual reasoning with a general conditioning layer
Perez, E., Strub, F., De Vries, H., Dumoulin, V., & Courville, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11671 · Cited by 2,490
A Dual-Stage Attention-Based Recurrent Neural Network for Time Series Prediction
Qin, Y., Song, D., Chen, H., Cheng, W., Jiang, G., & Cottrell, G. W. (2017) · Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/366 · Cited by 1,585
Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220007 · Cited by 1,233
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010)
Adams, R. P., Dahl, G. E., & Murray, I. (2010) · Cited by 966
Stock market's price movement prediction with LSTM neural networks
Nelson, D. M. Q., Pereira, A. C. M., & de Oliveira, R. A. (2017) · 2017 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2017.7966019 · Cited by 951
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · cited 2×
“[38] present a sparsely-gated MoE layer to enable training extremely large number of experts.”
BackgroundEnsemble selection from libraries of models
Caruana, R., Niculescu-Mizil, A., Crew, G., & Ksikes, A. (2004) · Twenty-first international conference on Machine learning - ICML '04 · doi:10.1145/1015330.1015432 · Cited by 861
Evaluating multiple classifiers for stock price direction prediction
Ballings, M., Van den Poel, D., Hespeels, N., & Gryp, R. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.05.013 · Cited by 611
Computational Intelligence and Financial Markets: A Survey and Future Directions
Cavalcante, R. C., Brasileiro, R. C., Souza, V. L. F., Nobrega, J. P., & Oliveira, A. L. I. (2016) · Expert Systems with Applications · doi:10.1016/j.eswa.2016.02.006 · Cited by 589
Temporal Relational Ranking for Stock Prediction
Feng, F., He, X., Wang, X., Luo, C., Liu, Y., & Chua, T.-S. (2019) · ACM Transactions on Information Systems · doi:10.1145/3309547 · Cited by 553 · cited 1×
“For instance, Feng et al. [12] enhance graph convolutional networks (GCNs) with temporal convolutions for mining inter-stock relations.”
BackgroundSelf-citationStock Price Prediction via Discovering Multi-Frequency Trading Patterns
Zhang, L., Aggarwal, C., & Qi, G.-J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098117 · Cited by 447
Stock Movement Prediction from Tweets and Historical Prices
Xu, Y., & Cohen, S. B. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1183 · Cited by 434 · cited 10×
“We analyze AlphaMix’s performance (TR & SR) variation with the number of experts 𝑒 on ACL18 and SZ50 in Figure 10.”
BackgroundMethodInfluentialA comprehensive evaluation of ensemble learning for stock-market prediction
Nti, I. K., Adekoya, A. F., & Weyori, B. A. (2020) · Journal of Big Data · doi:10.1186/s40537-020-00299-5 · Cited by 329
Deep Ensembles: A Loss Landscape Perspective
Fort, S., Hu, H., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02757 · Cited by 311
Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., Namkoong, H., Farhadi, A., Carmon, Y., Kornblith, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.05482 · Cited by 307
Deep Learning with Gated Recurrent Unit Networks for Financial Sequence Predictions
Shen, G., Tan, Q., Zhang, H., Zeng, P., & Xu, J. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.04.298 · Cited by 301
OpitzDavid, & MaclinRichard (1999) · doi:10.5555/3013545.3013549 · Cited by 293
Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction
Hu, Z., Liu, W., Bian, J., Liu, X., & Liu, T.-Y. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159690 · Cited by 270
A Survey of Uncertainty in Deep Neural Networks
Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03342 · Cited by 232
Accurate Multivariate Stock Movement Prediction via Data-Axis Transformer with Multi-Level Contexts
Yoo, J., Soun, Y., Park, Y.-C., & Kang, U. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467297 · Cited by 222
Qlib: An AI-oriented Quantitative Investment PlatformPreprint
Yang, X., Liu, W., Zhou, D., Bian, J., & Liu, T. (2020) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2009.11189 · Cited by 210
Hierarchical Multi-Scale Gaussian Transformer for Stock Movement Prediction
Ding, Q., Wu, S., Sun, H., Guo, J., & Guo, J. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/640 · Cited by 206
What is being transferred in transfer learning?
Neyshabur, B., Sedghi, H., & Zhang, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.11687 · Cited by 183
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Du, N., Huang, Y., Dai, A. M., Tong, S., Lepikhin, D., Xu, Y., Krikun, M., Zhou, Y., Yu, A. W., Fırat, O., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.06905 · Cited by 157
THE THEORY OF THE TRADING FIRM REVISITED
Dumas, B. (1978) · The Journal of Finance · doi:10.1111/j.1540-6261.1978.tb02041.x · Cited by 151
Sawhney, R., Agarwal, S., Wadhwa, A., & Shah, R. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.676 · Cited by 146
Learning Factored Representations in a Deep Mixture of Experts
Eigen, D., Ranzato, M., & Sutskever, I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1312.4314 · Cited by 127
Learning Multiple Stock Trading Patterns with Temporal Routing Adaptor and Optimal Transport
Lin, H., Zhou, D., Liu, W., & Bian, J. (2021) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3447548.3467358 · Cited by 115
Snapshot Ensembles: Train 1, get M for free
Huang, G. M., Li, Y., Pleiss, G., Liu, Z., Hopcroft, J. E., & Weinberger, K. Q. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.00109 · Cited by 100
Chen, C., Zhao, L., Bian, J., Xing, C., & Liu, T.-Y. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330663 · Cited by 73
Wang, J., Sun, T., Liu, B., Cao, Y., & Zhu, H. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/514 · Cited by 73 · cited 1×
“[40] combine attention mechanism with LSTM to model correlated time steps.”
MethodStock market index prediction using deep neural network ensemble
Yang, B., Gong, Z.-J., & Yang, W. (2017) · 2017 36th Chinese Control Conference (CCC) · doi:10.23919/chicc.2017.8027964 · Cited by 58
Multi-scale Two-way Deep Neural Network for Stock Trend Prediction
Liu, G., Mao, Y., Sun, Q., Huang, H., Gao, W., Li, X., Shen, J., Li, R., & Wang, X. (2020) · Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/628 · Cited by 50
Self-citationREST: Relational Event-driven Stock Trend Forecasting
Xu, W., Liu, W., Xu, C., Bian, J., Yin, J., & Liu, T.-Y. (2021) · Proceedings of the Web Conference 2021 · doi:10.1145/3442381.3450032 · Cited by 49
Sawhney, R., Khanna, P., Aggarwal, A., Jain, T., Mathur, P., & Shah, R. R. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.643 · Cited by 36
Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks
Whitaker, T., & Whitley, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20842 · Cited by 35
Xu, J., Zhou, J., Jia, Y., Li, J., & Xiong, H. (2020) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde48307.2020.00058 · Cited by 19
Predicting the Stock Market using Multiple Models
Xiang, C., & Fu, W. M. (2006) · International Conference on Control, Automation, Robotics and Vision (ICARCV) · doi:10.1109/icarcv.2006.345431 · Cited by 14
Research works printed but not linked: 13
- Fuli Feng, Huimin Chen, Xiangnan He, Ji Ding, Maosong Sun, and Tat-Seng Chua. 2018. Enhancing stock movement prediction with adversarial training. arXiv preprint arXiv:1810.09936 (2018). · arXiv
- Raphael Gontijo-Lopes, Yann Dauphin, and Ekin Dogus Cubuk. 2022. No One Representation to Rule Them All: Overlapping Features of Training Methods. In International Conference on Learning Representations.
- Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. In 33th Conference on Neural Information Processing Systems (NeurIPS).
- Anders Krogh and Jesper Vedelsby. 1994. Neural network ensembles, cross validation, and active learning. Advances in Neural Information Processing Systems 7 (1994).
- Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30 (2017).
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436-444.
- Mahdi Pakdaman Naeini, Hamidreza Taremian, and Homa Baradaran Hashemi. 2010. Stock market value prediction using neural networks. In 2010 Interna- tional Conference on Computer Information Systems and Industrial Management Applications. 132-136.
- Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Doro- gush, and Andrey Gulin. 2018. CatBoost: Unbiased boosting with categorical features. In 34th Conference on Neural Information Processing Systems (NeurIPS).
- Carlos Riquelme Ruiz, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. 2021. Scaling vision with sparse mixture of experts. In 37th Conference on Neural Information Processing Systems (NeurIPS).
- Xudong Wang, Long Lian, Zhongqi Miao, Ziwei Liu, and Stella Yu. 2020. Long- tailed recognition by routing diverse distribution-aware experts. In 9th Interna- tional Conference on Learning Representations (ICLR).
- Yeming Wen, Dustin Tran, and Jimmy Ba. 2019. BatchEnsemble: an Alternative Approach to Efficient Ensemble and Lifelong Learning. In International Conference on Learning Representations.
- Yeming Wen, Paul Vicol, Jimmy Ba, Dustin Tran, and Roger Grosse. 2018. Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches. In Interna- tional Conference on Learning Representations.
- Florian Wenzel, Jasper Snoek, Dustin Tran, and Rodolphe Jenatton. 2020. Hyper- parameter ensembles for robustness and uncertainty quantification. Advances in Neural Information Processing Systems (2020), 6514-6527.
Cited by
A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist
Zhang, W., Zhao, L., Xia, H., Sun, S., Sun, J., Qin, M., Li, X., Zhao, Y., Zhao, Y., Cai, X., et al. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671801 · Cited by 74
“Recently, there has been growing interest in enhancing LLM agents with external tools and modular methods as AI agents.”
Self-citationEarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading
Qin, M., Sun, S., Zhang, W., Xia, H., Wang, X., & An, B. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i13.29384 · Cited by 70
Self-citationEarnHFT: Efficient Hierarchical Reinforcement Learning for High Frequency Trading
Qin, M., Sun, S., Zhang, W., Xia, H., Wang, X., & An, B. (2023) · arXiv · doi:10.48550/arxiv.2309.12891 · Cited by 29
Self-citationYang, J., Zhang, M., Fang, R., Zhang, W., & Zhou, J. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112116 · Cited by 16
Jeon, J., Park, J., Park, C., & Kang, U. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671668 · Cited by 21
Metadata
- Authors
- Sun, S., Wang, X., Xue, W., Lou, X., & An, B.
- Affiliations
- Nanyang Technological University (SG)
- Venue
- Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
- Publisher
- ACM
- Published
- 2023-08-04
- Pages
- 2109-2119
- OpenAlex
- W4385562522
- Cited by
- 16
- References
- 35
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 9
- FWCI
- 8.79
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion, cc-by)
- DR-NTU (Nanyang Technological University) (publishedVersion)
- Singapore Management University Institutional Knowledge (InK) (Singapore Management University) (submittedVersion, cc-by)
- Funders
- National Research Foundation Singapore
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Stock Market Forecasting Methods
- Social Sciences › Economics, Econometrics and Finance › Finance › Financial Markets and Investment Strategies
- Social Sciences › Business, Management and Accounting › Accounting › Financial Analysis and Corporate Governance
- Keywords
- mixture of experts, trading strategy design, ensemble learning, expert routing, portfolio management, stock market prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.