CoChat
Sign Up
PreprintCited by 87Anomaly Detection Techniques and ApplicationsTopic ModelingArtificial Intelligence Applications

Large Language Models for Forecasting and Anomaly Detection: A Systematic Literature Review

Open access
Su, Jing, Jiang, Chufeng, Jin, Xin, Qiao, Yuxin, Xiao, Tingsong, Ma, Hongda, Wei, Rong, Jing, Zhi +2 more·2024·arXiv
DOI 10.48550/arxiv.2402.10350arXiv 2402.10350
Open this paper in CoChat — free accountOpen PDF

Abstract

This systematic literature review comprehensively examines the application of Large Language Models (LLMs) in forecasting and anomaly detection, highlighting the current state of research, inherent challenges, and prospective future directions. LLMs have demonstrated significant potential in parsing and analyzing extensive datasets to identify patterns, predict future events, and detect anomalous behavior across various domains. However, this review identifies several critical challenges that impede their broader adoption and effectiveness, including the reliance on vast historical datasets, issues with generalizability across different contexts, the phenomenon of model hallucinations, limitations within the models' knowledge boundaries, and the substantial computational resources required. Through detailed analysis, this review discusses potential solutions and strategies to overcome these obstacles, such as integrating multimodal data, advancements in learning methodologies, and emphasizing model explainability and computational efficiency. Moreover, this review outlines critical trends that are likely to shape the evolution of LLMs in these fields, including the push toward real-time processing, the importance of sustainable modeling practices, and the value of interdisciplinary collaboration. Conclusively, this review underscores the transformative impact LLMs could have on forecasting and anomaly detection while emphasizing the need for continuous innovation, ethical considerations, and practical solutions to realize their full potential.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2402.10350

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Class imbalance and the limited availability of annotated training samples restrict the applicability of conventional supervised machine learning techniques.

    “Compounding the issue are problems like class imbalance -since anomalies are rare by definition -and the limited availability of annotated training samples, which restricts the applicability of conventional supervised machine learning techniques [17,18].”

    From Log anomaly detection in AIOps: A real-world implementation using Large Language Models (2026) · cites this paper as [17]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Part of this paper’s reference list could not be split into single entries.

Research works printed but not linked: 212

  1. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
  2. Matheus Gomes Sousa, Kenzo Sakiyama, Lucas de Souza Rodrigues, Pedro Henrique Moraes, Eraldo Rezende Fernandes, and Edson Takashi Matsubara. Bert for stock market sentiment analysis. In 2019 IEEE 31st international conference on tools with artificial intelligence (ICTAI), pages 1597-1601. IEEE, 2019.
  3. Xin Jin and Yuchen Wang. Understand legal documents with contextualized large language models. arXiv preprint arXiv:2303.12135, 2023. · arXiv
  4. Lei Zhang, Mahsa Radnejad, and Andriy Miranskyy. Identifying flakiness in quantum programs. arXiv preprint arXiv:2302.03256, 2023. · arXiv
  5. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436-444, 2015.
  6. Jiashu Liao, Victor Sanchez, and Tanaya Guha. Self-supervised frontalization and rotation gan with random swap for pose-invariant face recognition. In 2022 IEEE International Conference on Image Processing (ICIP), pages 911-915. IEEE, 2022.
  7. Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng, Wen Ye, and Yan Liu. TEMPO: Prompt- based generative pre-trained transformer for time series forecasting. In The Twelfth International Conference on Learning Representations, 2024.
  8. Song Chen and Hai Liao. BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model. Applied Artificial Intelligence, 36(1):2145642, December 2022.
  9. Pranav K Nayak, Krishn V Kher, M Bharat Chandra, MV Rao, and Lei Zhang. Q-pac: Automated detection of quantum bug-fix patterns. arXiv preprint arXiv:2311.17705, 2023. · arXiv
  10. William Pourmajidi, Lei Zhang, John Steinbacher, Tony Erwin, and Andriy Miranskyy. Immutable log storage as a service on private and public blockchains. IEEE Transactions on Services Computing, 2021.
  11. Penghui Zhang, Zhibo Sun, Sukwha Kyung, Hans Walter Behrens, Zion Leonahenahe Basque, Haehyun Cho, Adam Oest, Ruoyu Wang, Tiffany Bao, Yan Shoshitaishvili, et al. I'm spartacus, no, i'm spartacus: Proactively protecting users from phishing by intentionally triggering cloaking behavior. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 3165-3179, 2022.
  12. Bryan Lim and Stefan Zohren. Time-series forecasting with deep learning: a survey. Philosophical Transactions of the Royal Society A, 379(2194):20200209, 2021.
  13. Lei Zhang and Douglas G Down. Smva: a stable mean value analysis algorithm for closed systems with load-dependent queues. Systems Modeling: Methodologies and Tools, pages 11-28, 2019.
  14. Jing Su, Suku Nair, and Leo Popokh. Optimal resource allocation in sdn/nfv-enabled networks via deep reinforcement learning. In 2022 IEEE Ninth International Conference on Communications and Networking (ComNet), pages 1-7. IEEE, 2022.
  15. Shun Liu, Kexin Wu, Chufeng Jiang, Bin Huang, and Danqing Ma. Financial time-series forecasting: Towards synergizing performance and interpretability within a hybrid machine learning approach. arXiv preprint arXiv:2401.00534, 2023. · arXiv
  16. Weixia Dang, Biyu Zhou, Weigang Zhang, and Songlin Hu. Time Series Anomaly Detection Based on Language Model. In Proceedings of the Eleventh ACM International Conference on Future Energy Systems, E-Energy '20, pages 544-547, New York, NY, USA, June 2020. Association for Computing Machinery.
  17. Weixia Dang, Biyu Zhou, Lingwei Wei, Weigang Zhang, Ziang Yang, and Songlin Hu. TS-Bert: Time Series Anomaly Detection via Pre-training Model Bert. In Maciej Paszynski, Dieter Kranzlmüller, Valeria V. Krzhizhanovskaya, Jack J. Dongarra, and Peter M. A. Sloot, editors, Computational Science -ICCS 2021, Lecture Notes in Computer Science, pages 209-223, Cham, 2021. Springer International Publishing.
  18. Jungwon Kim, Arlene Ong, and Richard E Overill. Design of an artificial immune system as a novel anomaly detector for combating financial fraud in the retail sector. In The 2003 Congress on Evolutionary Computation, 2003. CEC'03., volume 1, pages 405-412. IEEE, 2003.
  19. Xin Jin, Charalampos Katsis, Fan Sang, Jiahao Sun, Ashish Kundu, and Ramana Kompella. Edge security: Challenges and issues. arXiv preprint arXiv:2206.07164, 2022. · arXiv
  20. Zhibo Sun, Carlos E Rubio-Medrano, Ziming Zhao, Tiffany Bao, Adam Doupé, and Gail-Joon Ahn. Understand- ing and predicting private interactions in underground forums. In Proceedings of the Ninth ACM Conference on Data and Application Security and Privacy, pages 303-314, 2019.
  21. Kexin Wu and Kun Chi. Enhanced e-commerce customer engagement: A comprehensive three-tiered recommen- dation system. Journal of Knowledge Learning and Science Technology ISSN: 2959-6386 (online), 2(3):348-359, 2023.
  22. Zhibo Sun, Adam Oest, Penghui Zhang, Carlos Rubio-Medrano, Tiffany Bao, Ruoyu Wang, Ziming Zhao, Yan Shoshitaishvili, Adam Doupé, Gail-Joon Ahn, et al. Having your cake and eating it: An analysis of {Concession- Abuse-as-a-Service}. In 30th USENIX Security Symposium (USENIX Security 21), pages 4169-4186, 2021.
  23. Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Eric Sun, and Yue Zhang. A survey on large language model (llm) security and privacy: The good, the bad, and the ugly. arXiv preprint arXiv:2312.02003, 2023. · arXiv
  24. Xin Jin, Charalampos Katsis, Fan Sang, Jiahao Sun, Elisa Bertino, Ramana Rao Kompella, and Ashish Kundu. Prometheus: Infrastructure security posture analysis with ai-generated attack graphs. arXiv preprint arXiv:2312.13119, 2023. · arXiv
  25. Xinqi Dong, Bo Dang, Hengyi Zang, Shaojie Li, and Danqing Ma. The prediction trend of enterprise financial risk based on machine learning arima model. Journal of Theory and Practice of Engineering Science, 4(01):65-71, 2024.
  26. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
  27. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Cantón Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony S. Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel M. Kloumann, A. V. Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, R. Subramanian, Xia Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zhengxu Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. ArXiv, abs/2307.09288, 2023. · arXiv
  28. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, L'elio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of experts. ArXiv, abs/2401.04088, 2024. · arXiv
  29. Barbara A. Kitchenham. Systematic review in software engineering: Where we are and where we should be going. In Proceedings of the 2nd International Workshop on Evidential Assessment of Software Technologies, EAST '12, pages 1-2, New York, NY, USA, 2012. Association for Computing Machinery.
  30. Barbara Kitchenham, O. Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. Systematic literature reviews in software engineering -A systematic literature review. Information and Software Technology, 51(1):7-15, 2009.
  31. Pearl Brereton, Barbara A Kitchenham, David Budgen, Mark Turner, and Mohamed Khalil. Lessons from applying the systematic literature review process within the software engineering domain. Journal of systems and software, 80(4):571-583, 2007.
  32. Barbara Kitchenham, Rialette Pretorius, David Budgen, O Pearl Brereton, Mark Turner, Mahmood Niazi, and Stephen Linkman. Systematic literature reviews in software engineering-a tertiary study. Information and software technology, 52(8):792-805, 2010.
  33. Aldeida Aleti, Barbora Buhnova, Lars Grunske, Anne Koziolek, and Indika Meedeniya. Software architecture optimization methods: A systematic literature review. IEEE Transactions on Software Engineering, 39(5):658- 683, 2012.
  34. Annalisa Cocchia. Smart and digital city: A systematic literature review. Smart city: How to create public and economic value with high technology in urban space, pages 13-43, 2014.
  35. Chaomei Chen. Science mapping: A systematic review of the literature. Journal of data and information science, 2(2):1-40, 2017.
  36. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
  37. Nate Gruver, Marc Anton Finzi, Shikai Qiu, and Andrew Gordon Wilson. Large Language Models Are Zero-Shot Time Series Forecasters. In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
  38. Tian Zhou, Peisong Niu, Xue Wang, Liang Sun, and Rong Jin. One Fits All: Power General Time Series Analysis by Pretrained LM. In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
  39. Xiaoming Shi, Siqiao Xue, Kangrui Wang, Fan Zhou, James Y. Zhang, Jun Zhou, Chenhao Tan, and Hongyuan Mei. Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning. In Thirty-Seventh Conference on Neural Information Processing Systems, November 2023.
  40. Hao Xue and Flora D. Salim. PromptCast: A new prompt-based learning paradigm for time series forecasting. IEEE Transactions on Knowledge and Data Engineering, pages 1-14, 2023.
  41. Hao Xue, Bhanu Prakash Voutharoja, and Flora D. Salim. Leveraging language foundation models for human mobility forecasting. In Proceedings of the 30th International Conference on Advances in Geographic Information Systems, SIGSPATIAL '22, pages 1-9, New York, NY, USA, November 2022. Association for Computing Machinery.
  42. Harold Ott, Jasmin Bogatinovski, Alexander Acker, Sasho Nedelkoski, and Odej Kao. Robust and Transferable Anomaly Detection in Log Data using Pre-Trained Language Models. In 2021 IEEE/ACM International Workshop on Cloud Intelligence (CloudIntelligence), pages 19-24, May 2021.
  43. Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y. Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-LLM: Time series forecasting by reprogramming large language models. In The Twelfth International Conference on Learning Representations, 2024.
  44. Qizhi Li, Xianyong Li, Yujia Song, Maolin Zhang, Longqi Chen, Gang Wang, and Yajun Du. Evaluating BERT on cloud-edge time series forecasting and sentiment analysis via prompt learning. In 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCit. IEEE, 2022.
  45. Ting Zhang, Xin Huang, Wen Zhao, Shaohuang Bian, and Peng Du. LogPrompt: A Log-based Anomaly Detection Framework Using Prompts. In 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, 2023.
  46. Shaohan Huang, Yi Liu, Carol Fung, He Wang, Hailong Yang, and Zhongzhi Luan. Improving Log-Based Anomaly Detection by Pre-Training Hierarchical Transformers. IEEE Transactions on Computers, 72(9):2656- 2667, September 2023.
  47. Pranjal Gupta, Harshit Kumar, Debanjana Kar, Karan Bhukar, Pooja Aggarwal, and Prateeti Mohapatra. Learning Representations on Logs for AIOps. In 2023 IEEE 16th International Conference on Cloud Computing (CLOUD). IEEE, 2023.
  48. Yangyi Shao, Wenbin Zhang, Peishun Liu, Ren Huyue, Ruichun Tang, Qilin Yin, and Qi Li. Log Anomaly Detection method based on BERT model optimization. In 2022 7th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA). IEEE, 2022.
  49. Shiming He, Ying Lei, Ying Zhang, Kun Xie, and Pradip Kumar Sharma. Parameter-Efficient Log Anomaly Detection based on Pre-training model and LORA. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2023.
  50. Egil Karlsen, Rafael Copstein, Xiao Luo, Jeff Schwartzentruber, Bradley Niblett, Andrew Johnston, Malcolm I. Heywood, and Nur Zincir-Heywood. Exploring semantic vs. Syntactic features for unsupervised learning on application log files. In 2023 7th Cyber Security in Networking Conference (CSNet), pages 219-225, 2023.
  51. Jiaxiang Dong, Haixu Wu, Haoran Zhang, Li Zhang, Jianmin Wang, and Mingsheng Long. SimMTM: A simple pre-training framework for masked time-series modeling. In Thirty-Seventh Conference on Neural Information Processing Systems, 2023.
  52. Caiping Hu, Xuekui Sun, Hua Dai, Hangchuan Zhang, and Haiqiang Liu. Research on log anomaly detection based on sentence-BERT. Electronicsweek, 12(3580), 2023.
  53. Crispin Almodovar, Fariza Sabrina, Sarvnaz Karimi, and Salahuddin Azad. LogFiT: Log anomaly detection using fine-tuned language models. IEEE Transactions on Network and Service Management, pages 1-1, 2024.
  54. Mingyang Zhang, Jianfei Chen, Jianyi Liu, Jingchu Wang, Rui Shi, and Hua Sheng. LogST: Log semi-supervised anomaly detection based on sentence-BERT. In 2022 7th International Conference on Signal and Image Processing (ICSIP), pages 356-361, 2022.
  55. Van-Hoang Le and Hongyu Zhang. Log-based anomaly detection without log parsing. In 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 492-504, 2021.
  56. Shaohan Huang, Yi Liu, Carol Fung, Rong He, Yining Zhao, Hailong Yang, and Zhongzhi Luan. HitAnomaly: Hierarchical Transformers for Anomaly Detection in System Log. IEEE Transactions on Network and Service Management, 17(4):2064-2076, December 2020.
  57. Yong Yu, Xiaosheng Si, Changhua Hu, and Jianxun Zhang. A review of recurrent neural networks: Lstm cells and network architectures. Neural computation, 31(7):1235-1270, 2019.
  58. Danqing Ma, Bo Dang, Shaojie Li, Hengyi Zang, and Xinqi Dong. Implementation of computer vision technology based on artificial intelligence for medical image analysis. International Journal of Computer Science and Information Technology, 1(1):69-76, 2023.
  59. Yi Zhu, Rong Wei, Ge Gao, Lian Ding, Xiaodong Zhang, Xiaoying Wang, and Jue Zhang. Fully automatic segmentation on prostate mr images based on cascaded fully convolution network. Journal of Magnetic Resonance Imaging, 49(4):1149-1156, 2019.
  60. Ruixue Yuan, Hongming Li, Zhenyu Sun, and Huiwen Zhang. Application of graph convolutional network in the construction of knowledge graph for higher mathematics teaching. Sensors and Materials, 35(12):4269-4290, 2023.
  61. Ajay Shrestha and Ausif Mahmood. Review of deep learning algorithms and architectures. IEEE access, 7:53040-53065, 2019.
  62. François Gardères, Maryam Ziaeefard, Baptiste Abeloos, and Freddy Lecue. Conceptbert: Concept-aware representation for visual question answering. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 489-498, 2020.
  63. Clara Llorens Quintana, Siyu Chen, JUNHONG Lin, James G Fujimoto, Yan Li, and David Huang. Anterior topographic limbal demarcation with ultrawide-field oct. Investigative Ophthalmology & Visual Science, 63(7):1195-A0195, 2022.
  64. Chenchen Ding, Hnin Thu Zar Aye, Win Pa Pa, Khin Thandar Nwet, Khin Mar Soe, Masao Utiyama, and Eiichiro Sumita. Towards burmese (myanmar) morphological analysis: Syllable-based tokenization and part-of-speech tagging. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 19(1):1-34, 2019.
  65. Tao Gui, Qi Zhang, Haoran Huang, Minlong Peng, and Xuan-Jing Huang. Part-of-speech tagging for twitter with adversarial neural networks. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2411-2420, 2017.
  66. Alan Ritter, Sam Clark, Oren Etzioni, et al. Named entity recognition in tweets: an experimental study. In Proceedings of the 2011 conference on empirical methods in natural language processing, pages 1524-1534, 2011.
  67. Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 34(1):50-70, 2020.
  68. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1-35, 2023.
  69. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1-23, 2021.
  70. Shanchan Wu and Yifan He. Enriching pre-trained language model with entity information for relation classifi- cation. In Proceedings of the 28th ACM international conference on information and knowledge management, pages 2361-2364, 2019.
  71. Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, and Ziqing Yang. Pre-training with whole word masking for chinese bert. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3504-3514, 2021.
  72. Wei Bao, Hongshu Che, and Jiandong Zhang. Will_go at semeval-2020 task 3: An accurate model for predicting the (graded) effect of context in word similarity based on bert. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 301-306, 2020.
  73. Chongyang Tao, Lili Mou, Dongyan Zhao, and Rui Yan. Ruber: An unsupervised method for automatic evaluation of open-domain dialog systems. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
  74. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon llm: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01116, 2023. · arXiv
  75. Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023. · arXiv
  76. Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health, 2(2):e0000198, 2023.
  77. Jize Xiong, Mingyang Feng, Xiaosong Wang, Chufeng Jiang, Ning Zhang, and Zhiming Zhao. Decoding sentiments: Enhancing covid-19 tweet analysis through bert-rcnn fusion. Journal of Theory and Practice of Engineering Science, 4(01):86-93, 2024.
  78. Kangrui Ruan, Cynthia He, Jiyang Wang, Xiaozhou Joey Zhou, Helian Feng, and Ali Kebarighotbi. S2e: Towards an end-to-end entity resolution solution from acoustic signal. In ICASSP 2024, 2024.
  79. Leigang Qu, Shengqiong Wu, Hao Fei, Liqiang Nie, and Tat-Seng Chua. Layoutllm-t2i: Eliciting layout guidance from llm for text-to-image generation. In Proceedings of the 31st ACM International Conference on Multimedia, pages 643-654, 2023.
  80. Zoie Zhao, Sophie Song, Bridget Duah, Jamie Macbeth, Scott Carter, Monica P Van, Nayeli Suseth Bravo, Matthew Klenk, Kate Sick, and Alexandre LS Filipowicz. More human than human: Llm-generated narratives outperform human-llm interleaved narratives. In Proceedings of the 15th Conference on Creativity and Cognition, pages 368-370, 2023.
  81. Zhuoyan Li, Hangxiao Zhu, Zhuoran Lu, and Ming Yin. Synthetic data generation with large language models for text classification: Potential and limitations. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 10443-10461, Singapore, December 2023. Association for Computational Linguistics.
  82. Stavros Demetriadis and Yannis Dimitriadis. Conversational agents and language models that learn from human dialogues to support design thinking. In International Conference on Intelligent Tutoring Systems, pages 691-700. Springer, 2023.
  83. Vojtěch Hudeček and Ondřej Dušek. Are large language models all you need for task-oriented dialogue? In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pages 216-228, 2023.
  84. Shoetsu Sato, Jin Sakuma, Naoki Yoshinaga, Masashi Toyoda, and Masaru Kitsuregawa. Vocabulary adaptation for domain adaptation in neural machine translation. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4269-4279, 2020.
  85. Ye Zhang, Mengran Zhu, Yulu Gong, and Rui Ding. Optimizing science question ranking through model and retrieval-augmented generation. International Journal of Computer Science and Information Technology, 1(1):124-130, 2023.
  86. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D Manning. What does bert look at? an analysis of bert's attention. arXiv preprint arXiv:1906.04341, 2019. · arXiv
  87. Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. What does bert learn about the structure of language? In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics, 2019.
  88. Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao Li, Shuailiang Zhang, Xi Zhou, and Xiang Zhou. Semantics- aware bert for language understanding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 9628-9635, 2020.
  89. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019. · arXiv
  90. Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan Huang. Evaluating commonsense in pre-trained language models. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 9733-9740, 2020.
  91. Anastasia Chan. Gpt-3 and instructgpt: Technological dystopianism, utopianism, and "contextual" perspectives in ai ethics and industry. AI and Ethics, 3(1):53-64, 2023.
  92. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
  93. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. Explainability for large language models: A survey. ACM Transactions on Intelligent Systems and Technology, 2023.
  94. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022. · arXiv
  95. Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33:22243-22255, 2020.
  96. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744, 2022.
  97. Tomáš Mikolov et al. Statistical language models based on neural networks. Presentation at Google, Mountain View, 2nd April, 80(26), 2012.
  98. Adam Pauls and Dan Klein. Faster and smaller n-gram language models. In Proceedings of the 49th annual meeting of the Association for Computational Linguistics: Human Language Technologies, pages 258-267, 2011.
  99. Antoine Tremblay and Benjamin V Tucker. The effects of n-gram probabilistic measures on the recognition and production of four-word sequences. The Mental Lexicon, 6(2):302-324, 2011.
  100. Felix Wang, Tu-Thach Quach, Jason Wheeler, James B Aimone, and Conrad D James. Sparse coding for n-gram feature extraction and training for file fragment classification. IEEE Transactions on Information Forensics and Security, 13(10):2553-2562, 2018.
  101. Khalil Sima'an and Luciano Buratto. Backoff parameter estimation for the dop model. In European Conference on Machine Learning, pages 373-384. Springer, 2003.
  102. Alon Orlitsky and Ananda Theertha Suresh. Competitive distribution estimation: Why is good-turing good. Advances in Neural Information Processing Systems, 28, 2015.
  103. Rebekah George Benjamin. Reconstructing readability: Recent developments and recommendations in the analysis of text difficulty. Educational Psychology Review, 24:63-88, 2012.
  104. Wei Xu and Alex Rudnicky. Can artificial neural networks learn language models? In Interspeech, 2000.
  105. Yoshua Bengio, Réjean Ducharme, and Pascal Vincent. A neural probabilistic language model. Advances in neural information processing systems, 13, 2000.
  106. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137-1155, 2003.
  107. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. Recurrent neural network based language model. In Interspeech, volume 2, pages 1045-1048. Makuhari, 2010.
  108. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.
  109. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014. · arXiv
  110. Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In International Conference on Machine Learning, pages 933-941. PMLR, 2017.
  111. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249-256. JMLR Workshop and Conference Proceedings, 2010.
  112. Ilya Sutskever, James Martens, and Geoffrey E Hinton. Generating text with recurrent neural networks. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 1017-1024, 2011.
  113. Guanghui Wang, Yulu Gong, Mingwei Zhu, Jiaqiang Yuan, and Kuo Wei. Unveiling the future navigating next-generation ai frontiers and innovations in application. International Journal of Computer Science and Information Technology, 1(1):147-156, 2023.
  114. Sepp Hochreiter. The vanishing gradient problem during learning recurrent neural nets and problem solutions. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(02):107-116, 1998.
  115. Hongjie Niu, Hao Li, Jiufan Wang, Xiaonan Xu, and Huan Ji. Enhancing computer digital signal processing through the utilization of rnn sequence algorithms. International Journal of Computer Science and Information Technology, 1(1):60-68, 2023.
  116. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. In Marilyn Walker, Heng Ji, and Amanda Stent, editors, Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 2227-2237, New Orleans, Louisiana, June 2018. Association for Computational Linguistics.
  117. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014. · arXiv
  118. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025, 2015. · arXiv
  119. Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. Attention-based models for speech recognition. Advances in neural information processing systems, 28, 2015.
  120. Tianrui Liu, Changxin Xu, Yuxin Qiao, Chufeng Jiang, and Weisheng Chen. News recommendation with attention mechanism. arXiv preprint arXiv:2402.07422, 2024. · arXiv
  121. Xiaochen Wang, Tingsong Xiao, Jie Tan, Deqiang Ouyang, and Jie Shao. Mrmrp: multi-source review-based model for rating prediction. In Database Systems for Advanced Applications: 25th International Conference, DASFAA 2020, Jeju, South Korea, September 24-27, 2020, Proceedings, Part II 25, pages 20-35. Springer, 2020.
  122. Xiaochen Wang, Tingsong Xiao, and Jie Shao. Emrm: Enhanced multi-source review-based model for rating prediction. In Knowledge Science, Engineering and Management: 14th International Conference, KSEM 2021, Tokyo, Japan, August 14-16, 2021, Proceedings, Part III 14, pages 487-499. Springer, 2021.
  123. Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6077-6086, 2018.
  124. Meng-Hao Guo, Tian-Xing Xu, Jiang-Jiang Liu, Zheng-Ning Liu, Peng-Tao Jiang, Tai-Jiang Mu, Song-Hai Zhang, Ralph R Martin, Ming-Ming Cheng, and Shi-Min Hu. Attention mechanisms in computer vision: A survey. Computational visual media, 8(3):331-368, 2022.
  125. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30.
  126. Qilong Wang, Banggu Wu, Pengfei Zhu, Peihua Li, Wangmeng Zuo, and Qinghua Hu. Eca-net: Efficient channel attention for deep convolutional neural networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11534-11542, 2020.
  127. Maolin Li, Peng Ling, Shangsheng Wen, Xiandong Chen, and Fei Wen. Bubble-wave-mitigation algorithm and transformer-based neural network demodulator for water-air optical camera communications. IEEE Photonics Journal, 2023.
  128. Xizhou Zhu, Dazhi Cheng, Zheng Zhang, Stephen Lin, and Jifeng Dai. An empirical study of spatial attention mechanisms in deep networks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6688-6697, 2019.
  129. Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3-19, 2018.
  130. Xin Jin, Jonathan Larson, Weiwei Yang, and Zhiqiang Lin. Binary code summarization: Benchmarking chatgpt/gpt-4 and other large language models. arXiv preprint arXiv:2312.09601, 2023. · arXiv
  131. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. · arXiv
  132. Xiaodong Liu, Pengcheng He, Weizhu Chen, and Jianfeng Gao. Multi-task deep neural networks for natural language understanding. In Anna Korhonen, David Traum, and Lluís Màrquez, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4487-4496, Florence, Italy, July 2019. Association for Computational Linguistics.
  133. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
  134. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 27730-27744.
  135. Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo. Gpt-4 passes the bar exam. Available at SSRN 4389233, 2023.
  136. Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capabilities of gpt-4 on medical challenge problems. arXiv preprint arXiv:2303.13375, 2023. · arXiv
  137. Yu Wang, Nedim Lipka, Ryan A Rossi, Alexa Siu, Ruiyi Zhang, and Tyler Derr. Knowledge graph prompting for multi-document question answering. arXiv preprint arXiv:2308.11730, 2023. · arXiv
  138. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology, 2023.
  139. Steve Rathje, Dan-Mircea Mirea, Ilia Sucholutsky, Raja Marjieh, Claire Robertson, and Jay Joseph Van Bavel. GPT is an effective tool for multilingual psychological text analysis. PsyArXiv, May 2023.
  140. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022. · arXiv
  141. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38-45, Online, October 2020. Association for Computational Linguistics.
  142. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. arXiv preprint arXiv:2010.15980, 2020. · arXiv
  143. Xingchen Song, Di Wu, Binbin Zhang, Zhendong Peng, Bo Dang, Fuping Pan, and Zhiyong Wu. Zeroprompt: Streaming acoustic encoders are zero-shot masked lms. arXiv preprint arXiv:2305.10649, 2023. · arXiv
  144. Tianyu Gao, Adam Fisch, and Danqi Chen. Making pre-trained language models better few-shot learners. arXiv preprint arXiv:2012.15723, 2020. · arXiv
  145. Gamaleldin F Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. Adversarial reprogramming of neural networks. arXiv preprint arXiv:1806.11146, 2018. · arXiv
  146. Pin-Yu Chen. Model reprogramming: Resource-efficient cross-domain machine learning. arXiv preprint arXiv:2202.10629, 2022. · arXiv
  147. Guanhua Zhang, Yihua Zhang, Yang Zhang, Wenqi Fan, Qing Li, Sijia Liu, and Shiyu Chang. Fairness reprogramming. Advances in Neural Information Processing Systems, 35:34347-34362, 2022.
  148. Peyman Bateni, Raghav Goyal, Vaden Masrani, Frank Wood, and Leonid Sigal. Improved few-shot visual classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14493-14502, 2020.
  149. Yunxiao Deng, Carl Kesselman, Suvrajeet Sen, and Jiajun Xu. Computational operations research exchange (core): A cyber-infrastructure for analytics. In 2019 Winter Simulation Conference (WSC), pages 3447-3456. IEEE, 2019.
  150. Jiajun Xu and Suvrajeet Sen. Decision intelligence for nationwide ventilator allocation during the covid-19 pandemic. SN Computer Science, 2(6):423, 2021.
  151. Jiajun Xu and Suvrajeet Sen. Ensemble variance reduction methods for stochastic mixed-integer programming and their application to the stochastic facility location problem. INFORMS Journal on Computing, 2023.
  152. Jianmo Ni, Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 188-197, Hong Kong, China, November 2019. Association for Computational Linguistics.
  153. Julien Herzen, Francesco Lässig, Samuele Giuliano Piazzetta, Thomas Neuer, Léo Tafti, Guillaume Raille, Tomas Van Pottelbergh, Marek Pasieka, Andrzej Skrodzki, Nicolas Huguenin, Maxime Dumonal, Jan Kościsz, Dennis Bader, Frédérick Gusset, Mounir Benheddi, Camila Williamson, Michal Kosinski, Matej Petrik, and Gaël Grosch. Darts: User-friendly modern machine learning for time series. The Journal of Machine Learning Research, 23(1):124:5442-124:5447, January 2022.
  154. Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11106-11115, 2021.
  155. Spyros Makridakis and Michèle Hibon. The M3-Competition: Results, conclusions and implications. Interna- tional Journal of Forecasting, 16(4):451-476, 2000.
  156. Spyros Makridakis, Evangelos Spiliotis, and Vassilios Assimakopoulos. The M4 Competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting, 36(1):54-74, 2020.
  157. Rakshitha Wathsadini Godahewa, Christoph Bergmeir, Geoffrey I. Webb, Rob Hyndman, and Pablo Montero- Manso. Monash time series forecasting archive. In Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
  158. Adam Oliner and Jon Stearley. What supercomputers say: A study of five system logs. In 37th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN'07), pages 575-584, 2007.
  159. Wei Xu, Ling Huang, Armando Fox, David Patterson, and Michael I. Jordan. Detecting large-scale system problems by mining console logs. In Proceedings of the ACM SIGOPS 22nd Symposium on Operating Systems Principles, SOSP '09, pages 117-132, New York, NY, USA, 2009. Association for Computing Machinery.
  160. Min Du, Feifei Li, Guineng Zheng, and Vivek Srikumar. DeepLog: Anomaly detection and diagnosis from system logs through deep learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, pages 1285-1298, New York, NY, USA, 2017. Association for Computing Machinery.
  161. Ya Su, Youjian Zhao, Chenhao Niu, Rong Liu, Wei Sun, and Dan Pei. Robust anomaly detection for multivariate time series through stochastic recurrent neural network. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD '19, pages 2828-2837, New York, NY, USA, 2019. Association for Computing Machinery.
  162. Jinghua Sheng. An augmentable domain-specific models for financial analysis. In 2023 16th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), pages 1-4, 2023.
  163. Xin Jin, Kexin Pei, Jun Yeon Won, and Zhiqiang Lin. Symlm: Predicting function names in stripped binaries via context-sensitive execution-aware code embeddings. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 1631-1645, 2022.
  164. Xin Jin, Sunil Manandhar, Kaushal Kafle, Zhiqiang Lin, and Adwait Nadkarni. Understanding iot security from a market-scale perspective. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 1615-1629, 2022.
  165. James A Hanley and Barbara J McNeil. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1):29-36, 1982.
  166. I. de Zarzà, J. de Curtò, Gemma Roig, and Carlos T. Calafate. LLM Multimodal Traffic Accident Forecasting. Sensors, 23(22):9225, January 2023.
  167. Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, and Hongyuan Zha. Transformer hawkes process. In International conference on machine learning, pages 11692-11702. PMLR, 2020.
  168. Qiang Zhang, Aldo Lipani, Omer Kirnap, and Emine Yilmaz. Self-attentive hawkes process. In International conference on machine learning, pages 11183-11193. PMLR, 2020.
  169. Chenghao Yang, Hongyuan Mei, and Jason Eisner. Transformer embeddings of irregularly spaced events and their participants. In Proceedings of the Tenth International Conference on Learning Representations (ICLR), 2022.
  170. Siqiao Xue, Yan Wang, Zhixuan Chu, Xiaoming Shi, Caigao JIANG, Hongyan Hao, Gangwei Jiang, Xiaoyun Feng, James Y Zhang, and JUN ZHOU. Prompt-augmented temporal point process for streaming event sequence. In Thirty-Seventh Conference on Neural Information Processing Systems, 2023.
  171. Nishanth Nakshatri, Siyi Liu, Sihao Chen, Dan Roth, Dan Goldwasser, and Daniel Hopkins. Using llm for improving key event discovery: Temporal-guided news stream clustering with event summaries. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 4162-4173, 2023.
  172. Kangrui Ruan and Xuan Di. Learning human driving behaviors with sequential causal imitation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 4583-4592, 2022.
  173. Adam Szirmai. The dynamics of socio-economic development: an introduction. Cambridge University Press, 2005.
  174. Tingsong Xiao, Lu Zeng, Xiaoshuang Shi, Xiaofeng Zhu, and Guorong Wu. Dual-graph learning convolutional networks for interpretable alzheimer's disease diagnosis. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 406-415. Springer, 2022.
  175. Lu Zeng, Hengxin Li, Tingsong Xiao, Fumin Shen, and Zhi Zhong. Graph convolutional network with sample and feature weights for alzheimer's disease diagnosis. Information Processing & Management, 59(4):102952, 2022.
  176. Lavender Yao Jiang, Xujin Chris Liu, Nima Pour Nejatian, Mustafa Nasir-Moin, Duo Wang, Anas Abidin, Kevin Eaton, Howard Antony Riina, Ilya Laufer, Paawan Punjabi, Madeline Miceli, Nora C. Kim, Cordelia Orillac, Zane Schnurman, Christopher Livia, Hannah Weiss, David Kurland, Sean Neifert, Yosef Dastagirzada, Douglas Kondziolka, Alexander T. M. Cheung, Grace Yang, Ming Cao, Mona Flores, Anthony B. Costa, Yindalon Aphinyanaphongs, Kyunghyun Cho, and Eric Karl Oermann. Health system-scale language models are all-purpose prediction engines. Nature, 619(7969):357-362, July 2023.
  177. Van-Hoang Le and Hongyu Zhang. Log Parsing: How Far Can ChatGPT Go? In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2023.
  178. Mukhsimbayev Bobur, Kuralbayev Aibek, Bekbaganbetov Abay, and Fuad Hajiyev. Anomaly Detection Between Judicial Text-Based Documents. In 2020 IEEE 14th International Conference on Application of Information and Communication Technologies (AICT). IEEE, 2020.
  179. Sarda Komal, Namrud Zakeya, Rouf Raphael, Ahuja Harit, Rasolroveicy Mohammadreza, Litoiu Marin, Shwartz Larisa, and Watts Ian. ADARMA auto-detection and auto-remediation of microservice anomalies by leveraging large language models. In Proceedings of the 33rd Annual International Conference on Computer Science and Software Engineering, CASCON '23, pages 200-205, USA, 2023. IBM Corp.
  180. Pitikorn Khlaisamniang, Prachaya Khomduean, Kriangkrai Saetan, and Supasin Wonglapsuwan. Generative AI for self-healing systems. In 2023 18th International Joint Symposium on Artificial Intelligence and Natural Language Processing (iSAI-NLP), pages 1-6, 2023.
  181. Lei Zhang, Sean Howard, Tom Montpool, Jessica Moore, Krittika Mahajan, and Andriy Miranskyy. Automated data validation: An industrial experience report. Journal of Systems and Software, 197:111573, 2023.
  182. Zhibo Sun, Hong Zhang, Zixia Liu, Chen Xu, and Liqiang Wang. Migrating gis big data computing from hadoop to spark: an exemplary study using twitter. In 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), pages 351-358. IEEE, 2016.
  183. Zhuoyan Li, Zhuoran Lu, and Ming Yin. Modeling human trust and reliance in ai-assisted decision making: a markovian approach. In Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. AAAI Press, 2023.
  184. Ye Zhang, Yulu Gong, Dongji Cui, Xinrui Li, and Xinyu Shen. Deepgi: An automated approach for gastrointesti- nal tract segmentation in mri scans. arXiv preprint arXiv:2401.15354, 2024. · arXiv
  185. Junhong Lin. Deep-learning Enabled Accurate Bruch's Membrane Segmentation in Ultrahigh-Resolution Spectral Domain and Ultrahigh-Speed Swept Source Optical Coherence Tomography. PhD thesis, Massachusetts Institute of Technology, 2022.
  186. Siyu Chen, Benjamin Potsaid, Junhong Lin, Yunchan Hwang, Eric Moult, Yan Li, David Huang, and James G Fujimoto. High-speed, long-range swept-source optical coherence tomography for the anterior segment of the eye. Investigative Ophthalmology & Visual Science, 62(11):75-75, 2021.
  187. Siyu Chen, Benjamin Potsaid, Yan Li, Junhong Lin, Yunchan Hwang, Eric M Moult, Jason Zhang, David Huang, and James G Fujimoto. High speed, long range, deep penetration swept source oct for structural and angiographic imaging of the anterior eye. Scientific reports, 12(1):992, 2022.
  188. Siyu Chen, Omar Abu-Qamar, Deepayan Kar, Jeffrey D Messinger, Yunchan Hwang, Eric M Moult, Junhong Lin, Caroline R Baumal, Andre Witkin, Michelle C Liang, et al. Ultrahigh resolution oct markers of normal aging and early age-related macular degeneration. Ophthalmology Science, 3(3):100277, 2023.
  189. Pujin Cheng, Li Lin, Junyan Lyu, Yijin Huang, Wenhan Luo, and Xiaoying Tang. Prior: Prototype representation joint learning from medical images and reports. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023.
  190. Yongfei Liu, Haoyu Yang, and Chenwei Wu. Unveiling patterns: A study on semi-supervised classification of strip surface defects. IEEE Access, 11:119933-119946, 2023.
  191. Jing Su, Suku Nair, and Leo Popokh. Edgegym: A reinforcement learning environment for constraint-aware nfv resource allocation. In 2023 IEEE 2nd International Conference on AI in Cybersecurity (ICAIC), pages 1-7. IEEE, 2023.
  192. Junhong Lin, Bang Wang, Guang Yang, and Mu Zhou. Indoor localization based on weighted surfacing from crowdsourced samples. Sensors, 18(9):2990, 2018.
  193. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
  194. Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Lo, Piotr Dollár, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. · arXiv
  195. Jiashu Liao, Tanaya Guha, and Victor Sanchez. Self-supervised random mask attention gan in tackling pose- invariant face recognition. Available at SSRN 4583223, 2023.
  196. Yongfei Liu, Haoyu Yang, and Chenwei Wu. Unveiling patterns: A study on semi-supervised classification of strip surface defects. IEEE Access, 11:119933-119946, 2023.
  197. Zhongkai Shangguan, Yue Zhao, Wei Fan, and Zhehan Cao. Dog image generation using deep convolutional generative adversarial networks. In 2020 5th international conference on universal village (UV), pages 1-6. IEEE, 2020.
  198. Lei Zhang and Douglas G Down. Approximate mean value analysis for multi-core systems. In 2015 International Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS), pages 1-8. IEEE, 2015.
  199. Hengyi Zang. Precision calibration of industrial 3d scanners: An ai-enhanced approach for improved measure- ment accuracy. Global Academic Frontiers, 2(1):27-37, 2024.
  200. Carl Herickhoff, Junhong Lin, and Jeremy Dahl. Low-cost sensor-enabled freehand 3d ultrasound. In 2019 IEEE International Ultrasonics Symposium (IUS), pages 498-501. IEEE, 2019.

200 of 212 shown

Cited by

Sorted by citations

Authors
Su, J., Jiang, C., Jin, X., Qiao, Y., Xiao, T., Ma, H., Wei, R., Jing, Z., Xu, J., & Lin, J.
Affiliations
  • Southern Methodist University (US)
  • The University of Texas at Austin (US)
  • The Ohio State University (US)
  • Universidad Isabel I (ES)
  • University of Florida (US)
  • Peking University (CN)
  • Carnegie Mellon University (US)
  • University of Southern California (US)
  • Massachusetts Institute of Technology (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2402.10350
arXiv
2402.10350
OpenAlex
W4391940723
Cited by
87
References
0
Citations by year
2026: 5 · 2025: 30 · 2024: 52
FWCI
3.01
Type
journal article
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
large language models, forecasting, anomaly detection, model hallucination, generalizability, multimodal data, model explainability, computational efficiency, real-time processing, ethical considerations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review