CoChat
Sign Up
PreprintCited by 19Topic ModelingAdvanced Database Systems and QueriesNatural Language Processing Techniques

The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.

A Survey of LLM $\times$ DATA

Open access
Zhou, Xuanhe, He, Junxuan, Zhou, Wei, Chen, Haodong, Tang, Zirui, Zhao, Haoyu, Tong, Xin, Li, Guoliang, Chen, Youmin, Zhou, Jun, Sun, Zhaojun, Hui, Binyuan, Wang, Shuo, He, Conghui, Liu, Zhiyuan, Zhou, Jingren, Wu, Fan·2025·arXiv
DOI 10.48550/arxiv.2505.18458arXiv 2505.18458
Open this paper in CoChat — free accountOpen PDF

Abstract

The integration of large language model (LLM) and data management (DATA) is rapidly redefining both domains. In this survey, we comprehensively review the bidirectional relationships. On the one hand, DATA4LLM, spanning large-scale data processing, storage, and serving, feeds LLMs with high quality, diversity, and timeliness of data required for stages like pre-training, post-training, retrieval-augmented generation, and agentic workflows: (i) Data processing for LLMs includes scalable acquisition, deduplication, filtering, selection, domain mixing, and synthetic augmentation; (ii) Data Storage for LLMs focuses on efficient data and model formats, distributed and heterogeneous storage hierarchies, KV-cache management, and fault-tolerant checkpointing; (iii) Data serving for LLMs tackles challenges in RAG (e.g., knowledge post-processing), LLM inference (e.g., prompt compression, data provenance), and training strategies (e.g., data packing and shuffling). On the other hand, in LLM4DATA, LLMs are emerging as general-purpose engines for data management. We review recent advances in (i) data manipulation, including automatic data cleaning, integration, discovery; (ii) data analysis, covering reasoning over structured, semi-structured, and unstructured data, and (iii) system optimization (e.g., configuration tuning, query rewriting, anomaly diagnosis), powered by LLM techniques like retrieval-augmented prompting, task-specialized fine-tuning, and multi-agent collaboration.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2505.18458

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 342

  1. [2][2] https://arxiv.org/. 4. ai/nova/understanding/. -
  2. [45][45] A. Abbas, E. Rusak, K. Tirumala, W. Brendel, K. Chaudhuri, and A. S. Morcos. Effective pruning of web-scale datasets based on complexity of concept clusters. arXiv preprint arXiv:2401.04578 , 2024. - · arXiv
  3. [47][47] A. Abdallah, J. Mozafari, B. Piryani, and A. Jatowt. Asrank: Zero-shot re-ranking with answer scent for document retrieval. arXiv preprint arXiv:2501.15245 , 2025. - · arXiv
  4. [46][46] A. Abbas, K. Tirumala, D. Simig, S. Ganguli, and A. S. Morcos. Semdedup: Data-efficient learning at web-scale through semantic deduplication. arXiv preprint arXiv:2303.09540 , 2023. - · arXiv
  5. [48][48] S. Abiteboul. Querying semi-structured data. In Database Theory-ICDT'97: 6th International Conference Delphi, Greece, January 8-10, 1997 Proceedings 6 , pages 1-18. Springer, 1997. -
  6. [51][51] T. Ahmed, K. S. Pai, P. Devanbu, and E. Barr. Automatic semantic augmentation of language model prompts (for code summarization). In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering , ICSE '24, New York, NY, USA, 2024. Association for Computing Machinery.
  7. [52][52] - A. Akbik, T. Bergmann, D. Blythe, K. Rasul, S. Schweter, and R. Vollgraf. Flair: An easy-to-use framework for stateof-the-art nlp. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (demonstrations) , pages 54-59, 2019. -
  8. [53][53] P. Akioyamen, Z. Yi, and R. Marcus. The unreasonable effectiveness of llms for query optimization. CoRR , abs/2411.02862, 2024. - · arXiv
  9. [55][55] A. Albalak, Y. Elazar, S. M. Xie, S. Longpre, N. Lambert, X. Wang, N. Muennighoff, B. Hou, L. Pan, H. Jeong, et al. A survey on data selection for language models. arXiv preprint arXiv:2402.16827 , 2024. - · arXiv
  10. [54][54] M. M. Alam and W. Wang. A comprehensive survey on data provenance: State-of-the-art approaches and their deployments for iot security enforcement. J. Comput. Secur. , 29(4):423-446, 2021. -
  11. [58][58] Q. An, C. Ying, Y. Zhu, Y. Xu, M. Zhang, and J. Wang. LEDD: large language model-empowered data discovery in data lakes. CoRR , abs/2502.15182, 2025. - · arXiv
  12. [57][57] K. An, F. Yang, L. Li, J. Lu, S. Cheng, S. Si, L. Wang, P. Zhao, L. Cao, Q. Lin, et al. Thread: A logic-based data organization paradigm for how-to question answering with retrieval augmented generation. arXiv preprint arXiv:2406.13372 , 2024. - · arXiv
  13. [59][59] R. Angles. A comparison of current graph database models. In 2012 IEEE 28th International Conference on Data Engineering Workshops , pages 171-177, 2012. -
  14. [61][61] Z. Ankner, C. Blakeney, K. Sreenivasan, M. Marion, M. L. Leavitt, and M. Paul. Perplexed by perplexity: Perplexitybased data pruning with small reference models. arXiv preprint arXiv:2405.20541 , 2024. - · arXiv
  15. [60][60] R. Angles and C. Gutierrez. Survey of graph database models. ACM Comput. Surv. , 40(1), Feb. 2008. -
  16. [64][64] M. Artetxe, S. Bhosale, N. Goyal, T. Mihaylov, M. Ott, S. Shleifer, X. V. Lin, J. Du, S. Iyer, R. Pasunuru, et al. Efficient large scale language modeling with mixtures of experts. arXiv preprint arXiv:2112.10684 , 2021. - · arXiv
  17. [63][63] S. Arora, B. Yang, S. Eyuboglu, A. Narayan, A. Hojel, I. Trummer, and C. R´ e. Language models enable simple systems for generating structured views of heterogeneous data lakes. Proc. VLDB Endow. , 17(2):92-105, 2023. -
  18. [65][65] G. A. Atemezing. Empirical evaluation of a cloud-based graph database: the case of neptune. In Knowledge Graphs and Semantic Web: Third Iberoamerican Conference and Second IndoAmerican Conference, KGSWC 2021, Kingsville, Texas, USA, November 22-24, 2021, Proceedings 3 , pages 31-46. Springer, 2021. -
  19. [67][67] A. Audibert, Y. Chen, D. Graur, A. Klimovic, J. ˇ Simˇ sa, and C. A. Thekkath. tf. data service: A case for disaggregating ml input data processing. In Proceedings of the 2023 ACM Symposium on Cloud Computing , pages 358-375, 2023. -
  20. [66][66] J.-M. Attendu and J.-P. Corbeil. Nlu on data diets: Dynamic data subset selection for nlp classification tasks. arXiv preprint arXiv:2306.03208 , 2023. - · arXiv
  21. [70][70] S. Bai, K. Chen, X. Liu, et al. Qwen2.5-vl technical report. CoRR , abs/2502.13923, 2025. - · arXiv
  22. [69][69] J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609 , 2023. - · arXiv
  23. [71][71] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 , 2022. - · arXiv
  24. [73][73] A. Barbaresi. Trafilatura: A web scraping library and command-line tool for text discovery and extraction. In H. Ji, J. C. Park, and R. Xia, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: System Demonstrations , pages 122-131, Online, Aug. 2021. Association for Computational Linguistics. -
  25. [72][72] M. I. L. Balaka, D. Alexander, Q. Wang, Y. Gong, A. Krisnadhi, and R. C. Fernandez. Pneuma: Leveraging llms for tabular data representation and retrieval in an end-to-end system. arXiv preprint arXiv:2504.09207 , 2025. - · arXiv
  26. [74][74] T. Bendinelli, A. Dox, and C. Holz. Exploring llm agents for cleaning tabular machine learning datasets. In ICLR 2025 Workshop on Foundation Models in the Wild , 2025. arXiv:2503.06664. - · arXiv
  27. [76][76] J. Bevendorff, S. Gupta, J. Kiesel, and B. Stein. An empirical comparison of web content extraction algorithms. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , SIGIR '23, page 2594-2603, New York, NY, USA, 2023. Association for Computing Machinery. -
  28. [75][75] M. Berglund and B. van der Merwe. Formalizing bpe tokenization. arXiv preprint arXiv:2309.08715 , 2023. - · arXiv
  29. [77][77] S. Biderman, U. Prashanth, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, and E. Raff. Emergent and predictable memorization in large language models. Advances in Neural Information Processing Systems , 36, 2024. -
  30. [79][79] D. Borthakur et al. Hdfs architecture guide. Hadoop apache project , 53(1-13):2, 2008. -
  31. [78][78] F. Biester, M. Abdelaal, and D. D. Gaudio. Llmclean: Contextaware tabular data cleaning via llm-generated ofds. In ADBIS (Short Papers) , volume 2186 of Communications in Computer and Information Science , pages 68-78. Springer, 2024. -
  32. [80][80] D. Brandfonbrener, H. Zhang, A. Kirsch, J. R. Schwarz, and S. Kakade. Color-filter: Conditional loss reduction filtering for targeted language model pre-training. arXiv preprint arXiv:2406.10670 , 2024. - · arXiv
  33. [81][81] A. Z. Broder. On the resemblance and containment of documents. In Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No. 97TB100171) , pages 21-29. IEEE, 1997. -
  34. [84][84] Y. Cao, Y. Kang, C. Wang, and L. Sun. Instruction mining: Instruction data selection for tuning large language models. arXiv preprint arXiv:2307.06290 , 2023. - · arXiv
  35. [86][86] C. Chai, J. Wang, Y. Luo, Z. Niu, and G. Li. Data management for machine learning: A survey. IEEE Trans. Knowl. Data Eng. , 35(5):4646-4667, 2023. -
  36. [88][88] M. S. Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the thiry-fourth annual ACM symposium on Theory of computing , pages 380-388, 2002. -
  37. [89][89] T.-Y. Che, X.-L. Mao, T. Lan, and H. Huang. A hierarchical context augmentation method to improve retrieval-augmented llms on scientific papers. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 243-254, 2024. -
  38. [91][91] D. Chen, H. Wang, Y. Huang, C. Ge, Y. Li, B. Ding, and J. Zhou. Data-juicer sandbox: A feedback-driven suite for multimodal data-model co-development. arXiv preprint arXiv:2407.11784 , 2024. - · arXiv
  39. [90][90] D. Chen, Y. Huang, Z. Ma, H. Chen, X. Pan, C. Ge, D. Gao, Y. Xie, Z. Liu, J. Gao, et al. Data-juicer: A one-stop data processing system for large language models. In Companion of the 2024 International Conference on Management of Data , pages 120-134, 2024. -
  40. [92][92] H. Chen, A. Waheed, X. Li, Y. Wang, J. Wang, B. Raj, and M. I. Abdin. On the diversity of synthetic data and its impact on training large language models. arXiv preprint arXiv:2410.15226 , 2024. - · arXiv
  41. [94][94] J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu. M3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. ACL , 2024. -
  42. [93][93] J. Chen, Z. Chen, J. Wang, K. Zhou, Y. Zhu, J. Jiang, Y. Min, W. X. Zhao, Z. Dou, J. Mao, et al. Towards effective and efficient continual pre-training of large language models. arXiv preprint arXiv:2407.18743 , 2024. - · arXiv
  43. [95][95] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 , 2021. - · arXiv
  44. [96][96] S. Chen, J. Fan, B. Wu, N. Tang, C. Deng, P. Wang, Y. Li, J. Tan, F. Li, J. Zhou, and X. Du. Automatic database configuration debugging using retrieval-augmented language models. CoRR , abs/2412.07548, 2024. - · arXiv
  45. [98][98] Z. Chen, T. Liu, M. Tian, W. Luo, Z. Liu, et al. Advancing mathematical reasoning in language models: The impact of problem-solving data, data synthesis methods, and training stages. In The Thirteenth International Conference on Learning Representations , 2025. -
  46. [97][97] T. Chen, H. Wang, S. Chen, W. Yu, K. Ma, X. Zhao, H. Zhang, and D. Yu. Dense x retrieval: What retrieval granularity should we use? arXiv preprint arXiv:2312.06648 , 2023. - · arXiv
  47. [99][99] D. Cheng, Y. Gu, S. Huang, J. Bi, M. Huang, and F. Wei. Instruction pre-training: Language models are supervised multitask learners. arXiv preprint arXiv:2406.14491 , 2024. - · arXiv
  48. [101][101] X. Cheng, X. Wang, X. Zhang, T. Ge, S.-Q. Chen, F. Wei, H. Zhang, and D. Zhao. xrag: Extreme context compression for retrieval-augmented generation with one token. arXiv preprint arXiv:2405.13792 , 2024. - · arXiv
  49. [102][102] A. Chevalier, A. Wettig, A. Ajith, and D. Chen. Adapting language models to compress contexts. arXiv preprint arXiv:2305.14788 , 2023. - · arXiv
  50. [103][103] J. Choi, J. Yun, K. Jin, and Y. Kim. Multi-news+: Costefficient dataset cleansing via llm-based data annotation. In EMNLP , pages 15-29. Association for Computational Linguistics, 2024. -
  51. [105][105] M. Christ, S. Gunn, and O. Zamir. Undetectable watermarks for language models. In The Thirty Seventh Annual Conference on Learning Theory , pages 1125-1139. PMLR, 2024. -
  52. [107][107] E. F. Codd. A relational model of data for large shared data banks. Commun. ACM , 13(6):377-387, June 1970. -
  53. [106][106] K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021. - · arXiv
  54. [108][108] B. Colson, P. Marcotte, and G. Savard. An overview of bilevel optimization. Annals of operations research , 153:235-256, 2007. -
  55. [113][113] G. Cui, L. Yuan, N. Ding, G. Yao, B. He, W. Zhu, Y. Ni, G. Xie, R. Xie, Y. Lin, et al. Ultrafeedback: Boosting language models with scaled ai feedback. In Forty-first International Conference on Machine Learning , 2024. -
  56. [112][112] A. I. Cowen-Rivers, W. Lyu, Z. Wang, R. Tutunov, J. Hao, J. Wang, and H. Bou-Ammar. HEBO: heteroscedastic evolutionary bayesian optimisation. CoRR , abs/2012.03826, 2020. - · arXiv
  57. [114][114] J. Cui, Z. Li, Y. Yan, B. Chen, and L. Yuan. Chatlaw: Opensource legal large language model with integrated external knowledge bases. CoRR , abs/2306.16092, 2023. - · arXiv
  58. [116][116] H. Ding, Z. Wang, G. Paolini, V. Kumar, A. Deoras, D. Roth, and S. Soatto. Fewer truncations improve language modeling. arXiv preprint arXiv:2404.10830 , 2024. - · arXiv
  59. [118][118] Y. Ding, Z. Wang, W. Ahmad, M. K. Ramanathan, R. Nallapati, P. Bhatia, D. Roth, and B. Xiang. CoCoMIC: Code Completion by Jointly Modeling In-file and Cross-file Context. In N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , pages 34333445, Torino, Italia, May 2024. ELRA and ICCL. -
  60. [117][117] N. Ding, Y. Chen, B. Xu, Y. Qin, Z. Zheng, S. Hu, Z. Liu, M. Sun, and B. Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233 , 2023. - · arXiv
  61. [121][121] F. Dong, Y. He, Y. Liang, Z. Liu, Y. Wu, P. Chen, and T. Yang. Simisketch: Efficiently estimating similarity of streaming multisets. arXiv preprint arXiv:2405.19711 , 2024. - · arXiv
  62. [120][120] J. Dodge, M. Sap, A. Marasovi´ c, W. Agnew, G. Ilharco, D. Groeneveld, M. Mitchell, and M. Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. arXiv preprint arXiv:2104.08758 , 2021. - · arXiv
  63. [122][122] G. Dong, D. Pan, Y. Sun, S. Zhang, Z. Liang, X. Wu, Y. Shen, F. Yang, H. Sun, T. Li, et al. Baichuanseed: Sharing the potential of extensive data collection and deduplication by introducing a competitive large language model baseline. arXiv preprint arXiv:2408.15079 , 2024. - · arXiv
  64. [123][123] G. Dong, H. Yuan, K. Lu, C. Li, M. Xue, D. Liu, W. Wang, Z. Yuan, C. Zhou, and J. Zhou. How abilities in large language models are affected by supervised fine-tuning data composition. arXiv preprint arXiv:2310.05492 , 2023. - · arXiv
  65. [125][125] M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazar´ e, M. Lomeli, L. Hosseini, and H. J´ egou. The faiss library. arXiv preprint arXiv:2401.08281 , 2024. - · arXiv
  66. [127][127] D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130 , 2024. - · arXiv
  67. [126][126] Q. Du, C. Zong, and J. Zhang. Mods: Model-oriented data selection for instruction tuning. arXiv preprint arXiv:2311.15653 , 2023. - · arXiv
  68. [128][128] M. Eibich, S. Nagpal, and A. Fred-Ojala. Aragog: Advanced rag output grading. arXiv preprint arXiv:2404.01037 , 2024. - · arXiv
  69. [130][130] L. Engstrom, A. Feldmann, and A. Madry. Dsdm: Modelaware dataset selection with datamodels. arXiv preprint arXiv:2401.12926 , 2024. - · arXiv
  70. [129][129] M. Y. Eltabakh, Z. A. Naeem, M. S. Ahmad, M. Ouzzani, and N. Tang. Retclean: Retrieval-based tabular data cleaning using llms and data lakes. Proc. VLDB Endow. , 17(12):4421-4424, 2024. -
  71. [131][131] J. Fairoze, S. Garg, S. Jha, S. Mahloujifar, M. Mahmoody, and M. Wang. Publicly detectable watermarking for language models. arXiv preprint arXiv:2310.18491 , 2023. - · arXiv
  72. [132][132] C. Fan, Z. Pan, W. Sun, C. Yang, and W. Chen. Latuner: An llm-enhanced database tuning system based on adaptive surrogate model. In ECML/PKDD (5) , volume 14945 of Lecture Notes in Computer Science , pages 372-388. Springer, 2024. -
  73. [134][134] M. Fan, X. Han, J. Fan, C. Chai, N. Tang, G. Li, and X. Du. Cost-effective in-context learning for entity resolution: A design space exploration. In ICDE , pages 3696-3709. IEEE, 2024. -
  74. [136][136] T. Fan, J. Wang, X. Ren, and C. Huang. Minirag: Towards extremely simple retrieval-augmented generation. arXiv preprint arXiv:2501.06713 , 2025. - · arXiv
  75. [135][135] S. Fan, M. Pagliardini, and M. Jaggi. Doge: Domain reweighting with generalization estimation. arXiv preprint arXiv:2310.15393 , 2023. - · arXiv
  76. [137][137] M. Fayyaz, E. Aghazadeh, A. Modarressi, M. T. Pilehvar, Y. Yaghoobzadeh, and S. E. Kahou. Bert on a data diet: Finding important examples by gradient-based pruning. arXiv preprint arXiv:2211.05610 , 2022. - · arXiv
  77. [140][140] W. Feng, C. Hao, Y. Zhang, Y. Han, and H. Wang. Mixture-ofloras: An efficient multitask tuning for large language models. arXiv preprint arXiv:2403.03432 , 2024. - · arXiv
  78. [139][139] S. Feng, S. Prabhumoye, K. Kong, D. Su, M. Patwary, M. Shoeybi, and B. Catanzaro. Maximize your data's potential: Enhancing llm accuracy with two-phase pretraining. arXiv preprint arXiv:2412.15285 , 2024. - · arXiv
  79. [143][143] B. Feuer, Y. Liu, C. Hegde, and J. Freire. Archetype: A novel framework for open-source column type annotation using large language models. Proc. VLDB Endow. , 17(9):2279-2292, 2024. -
  80. [142][142] S. Ferr´ e. First steps of an approach to the arc challenge based on descriptive grid models and the minimum description length principle. arXiv preprint arXiv:2112.00848 , 2021. - · arXiv
  81. [144][144] A. Finn, N. Kushmerick, and B. Smyth. Fact or fiction: Content classification for digital libraries. In DELOS Workshops / Conferences , 2001. -
  82. [146][146] S. Y. Gadre, G. Ilharco, A. Fang, J. Hayase, G. Smyrnis, T. Nguyen, R. Marten, M. Wortsman, D. Ghosh, J. Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing Systems , 36:27092-27112, 2023. -
  83. [147][147] S. Gandhi, M. Zhao, A. Skiadopoulos, and C. Kozyrakis. Recycle: Resilient training of large dnns using pipeline adaptation. In Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles , pages 211-228, 2024. -
  84. [149][149] L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027 , 2020. - · arXiv
  85. [148][148] B. Gao, Z. He, P. Sharma, Q. Kang, D. Jevdjic, J. Deng, X. Yang, Z. Yu, and P. Zuo. { Cost-Efficient } large language model serving for multi-turn conversations with { CachedAttention } . In 2024 USENIX Annual Technical Conference (USENIX ATC 24) , pages 111-126, 2024. -
  86. [151][151] S. Gao, C. Gao, Y. He, J. Zeng, L. Nie, X. Xia, and M. Lyu. Code structure-guided transformer for source code summarization. ACM Transactions on Software Engineering and Methodology , 32(1):1-32, Jan. 2023. -
  87. [153][153] Z. Ge, S. Liu, F. Wang, Z. Li, and J. Sun. Yolox: Exceeding yolo series in 2021. arXiv preprint arXiv:2107.08430 , 2021. - · arXiv
  88. [155][155] V. Giannakouris and I. Trummer. DBG-PT: A large language model assisted query performance regression debugger. Proc. VLDB Endow. , 17(12):4337-4340, 2024. -
  89. [156][156] V. Giannankouris and I. Trummer. λ -tune: Harnessing large language models for automated database system tuning. CoRR , abs/2411.03500, 2024. - · arXiv
  90. [158][158] D. Graur, D. Aymon, D. Kluser, T. Albrici, C. A. Thekkath, and A. Klimovic. Cachew: Machine learning input data processing as a service. In 2022 usenix annual technical conference (usenix atc 22) , pages 689-706, 2022. -
  91. [157][157] S. Goyal, P. Maini, Z. C. Lipton, A. Raghunathan, and J. Z. Kolter. Scaling laws for data filtering-data curation cannot be compute agnostic. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 22702-22711, 2024. -
  92. [159][159] D. Graur, O. Mraz, M. Li, S. Pourghannad, C. A. Thekkath, and A. Klimovic. Pecan: { Cost-Efficient }{ ML } data preprocessing with automatic transformation ordering and hybrid placement. In 2024 USENIX Annual Technical Conference (USENIX ATC 24) , pages 649-665, 2024. -
  93. [160][160] J. Gu, Z. Yang, C. Ding, R. Zhao, and F. Tan. Cmr scaling law: Predicting critical mixture ratios for continual pre-training of language models. arXiv preprint arXiv:2407.17467 , 2024. - · arXiv
  94. [162][162] D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 , 2025. - · arXiv
  95. [161][161] R. Gu, K. Zhang, Z. Xu, Y. Che, B. Fan, H. Hou, H. Dai, L. Yi, Y. Ding, G. Chen, et al. Fluid: Dataset abstraction and elastic acceleration for cloud-native deep learning training jobs. In 2022 IEEE 38th International Conference on Data Engineering (ICDE) , pages 2182-2195. IEEE, 2022. -
  96. [166][166] W. L. Hamilton, R. Ying, and J. Leskovec. Inductive representation learning on large graphs. In Proceedings of the 31st International Conference on Neural Information Processing Systems , NIPS'17, page 1025-1035, Red Hook, NY, USA, 2017. Curran Associates Inc. -
  97. [168][168] Y. He, Z. Wang, Z. Shen, G. Sun, Y. Dai, Y. Wu, H. Wang, and A. Li. Shed: Shapley-based automated dataset refinement for instruction fine-tuning. arXiv preprint arXiv:2405.00705 , 2024. - · arXiv
  98. [167][167] N. He, W. Xiong, H. Liu, Y. Liao, L. Ding, K. Zhang, G. Tang, X. Han, and W. Yang. Softdedup: an efficient data reweighting method for speeding up language model pre-training. arXiv preprint arXiv:2407.06654 , 2024. - · arXiv
  99. [169][169] D. Hernandez, T. Brown, T. Conerly, N. DasSarma, D. Drain, S. El-Showk, N. Elhage, Z. Hatfield-Dodds, T. Henighan, T. Hume, et al. Scaling laws and interpretability of learning from repeated data. arXiv preprint arXiv:2205.10487 , 2022. - · arXiv
  100. [170][170] J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. de Las Casas, L. A. Hendricks, J. Welbl, A. Clark, T. Hennigan, E. Noland, K. Millican, G. van den Driessche, B. Damoc, A. Guy, S. Osindero, K. Simonyan, E. Elsen, O. Vinyals, J. W. Rae, and L. Sifre. Training computeoptimal large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems , NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. -
  101. [172][172] M. J. Hosseini, Y. Gao, T. Baumg¨ artner, A. Fabrikant, and R. K. Amplayo. Scalable and domain-general abstractive proposition segmentation. arXiv preprint arXiv:2406.19803 , 2024. - · arXiv
  102. [173][173] Z. Hou, X. Lv, R. Lu, J. Zhang, Y. Li, Z. Yao, J. Li, J. Tang, and Y. Dong. Advancing language model reasoning through reinforcement learning and inference scaling. arXiv preprint arXiv:2501.11651 , 2025. - · arXiv
  103. [175][175] H. Hua, Y. Tang, C. Xu, and J. Luo. V2xum-llm: Crossmodal video summarization with temporal prompt instruction tuning. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 39, pages 3599-3607, 2025. -
  104. [177][177] X. Huang, H. Li, J. Zhang, X. Zhao, Z. Yao, Y. Li, Z. Yu, T. Zhang, H. Chen, and C. Li. E2etune: End-to-end knob tuning via fine-tuned generative language model. CoRR , abs/2404.11581, 2025. - · arXiv
  105. [176][176] J. Huang, D. Guo, C. Wang, J. Gu, S. Lu, J. P. Inala, C. Yan, J. Gao, N. Duan, and M. R. Lyu. Contextualized datawrangling code generation in computational notebooks. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering , ASE '24, page 1282-1294. ACM, Oct. 2024. -
  106. [178][178] Y. Huang, X. Lin, Z. Liu, Q. Cao, H. Xin, H. Wang, Z. Li, L. Song, and X. Liang. Mustard: Mastering uniform synthesis of theorem and proof data. arXiv preprint arXiv:2402.08957 , 2024. - · arXiv
  107. [179][179] Y. Huang, X. Liu, Y. Gong, Z. Gou, Y. Shen, N. Duan, and W. Chen. Key-point-driven data synthesis with its enhancement on mathematical reasoning. arXiv preprint arXiv:2403.02333 , 2024. - · arXiv
  108. [181][181] H. Husain, H.-H. Wu, T. Gazit, M. Allamanis, and M. Brockschmidt. CodeSearchNet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436 , 2019. - · arXiv
  109. [184][184] A. Ilyas, S. M. Park, L. Engstrom, G. Leclerc, and A. Madry. Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.00622 , 2022. - · arXiv
  110. [186][186] I. Jang, Z. Yang, Z. Zhang, X. Jin, and M. Chowdhury. Oobleck: Resilient distributed training of large models using pipeline templates. In Proceedings of the 29th Symposium on Operating Systems Principles , pages 382-395, 2023. -
  111. [187][187] Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, and P. Fung. Survey of hallucination in natural language generation. ACM Computing Surveys , 55(12):1-38, 2023. -
  112. [189][189] H. Jiang, Q. Wu, C.-Y. Lin, Y. Yang, and L. Qiu. Llmlingua: Compressing prompts for accelerated inference of large language models. arXiv preprint arXiv:2310.05736 , 2023. - · arXiv
  113. [191][191] J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim. A survey on large language models for code generation. arXiv preprint arXiv:2406.00515 , 2024. - · arXiv
  114. [195][195] X. Jiao, Y. Yin, L. Shang, X. Jiang, X. Chen, L. Li, F. Wang, and Q. Liu. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351 , 2019. - · arXiv
  115. [194][194] Z. Jiang, H. Lin, Y. Zhong, Q. Huang, Y. Chen, Z. Zhang, Y. Peng, X. Li, C. Xie, S. Nong, et al. { MegaScale } : Scaling large language model training to more than 10,000 { GPUs } . In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24) , pages 745-760, 2024. -
  116. [196][196] R. L. J. X. Y. C. P. H. C. H. M. D. Z. Jie Tan, Kangfei Zhao and Y. Rong. Can large language models be query optimizer for relational databases? CoRR , abs/2502.05562, 2025. - · arXiv
  117. [198][198] D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences , 11(14):6421, 2021. -
  118. [197][197] C. Jin, Z. Zhang, X. Jiang, F. Liu, X. Liu, X. Liu, and X. Jin. Ragcache: Efficient knowledge caching for retrieval-augmented generation. arXiv preprint arXiv:2404.12457 , 2024. - · arXiv
  119. [199][199] A. Joulin, E. Grave, P. Bojanowski, and T. Mikolov. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759 , 2016. - · arXiv
  120. [200][200] D. Jung, Q. Liu, T. Huang, B. Zhou, and M. Chen. Familiarityaware evidence compression for retrieval augmented generation. arXiv preprint arXiv:2409.12468 , 2024. - · arXiv
  121. [202][202] A. Kay. Tesseract: an open-source optical character recognition engine. Linux J. , 2007(159):2, July 2007. -
  122. [201][201] J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 , 2020. - · arXiv
  123. [203][203] M. Kayali, A. Lykov, I. Fountalis, N. Vasiloglou, D. Olteanu, and D. Suciu. CHORUS: foundation models for unified data discovery and exploration. Proc. VLDB Endow. , 17(8):21042114, 2024. -
  124. [205][205] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, and T.-Y. Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems , 30, 2017. -
  125. [204][204] M. Kayali, F. Wenz, N. Tatbul, and C ¸. Demiralp. Mind the data gap: Bridging llms to enterprise data integration. In Proceedings of the 2025 Conference on Innovative Data Systems Research (CIDR) , Chaminade, California, 2025. CIDR 2025. -
  126. [206][206] J. D. M.-W. C. Kenton and L. K. Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT , volume 1, page 2. Minneapolis, Minnesota, 2019. -
  127. [207][207] A. Khan, R. Underwood, C. Siebenschuh, Y. Babuji, A. Ajith, K. Hippe, O. Gokdemir, A. Brace, K. Chard, and I. Foster. Lshbloom: Memory-efficient, extreme-scale document deduplication. arXiv preprint arXiv:2411.04257 , 2024. - · arXiv
  128. [211][211] T. N. Kipf and M. Welling. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR) , 2017. Published as a conference paper at ICLR 2017. -
  129. [210][210] J. Kim and J. Lee. Strategic data ordering: Enhancing large language model performance through curriculum learning. arXiv preprint arXiv:2405.07490 , 2024. - · arXiv
  130. [212][212] J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, and T. Goldstein. A watermark for large language models. In International Conference on Machine Learning , pages 1706117084. PMLR, 2023. -
  131. [214][214] D. Kocetkov, R. Li, L. Ben Allal, J. Li, C. Mou, C. Mu˜ noz Ferrandis, Y. Jernite, M. Mitchell, S. Hughes, T. Wolf, D. Bahdanau, L. von Werra, and H. de Vries. The stack: 3 tb of permissively licensed source code. Preprint , 2022. -
  132. [213][213] S. Kirkpatrick, C. D. Gelatt Jr, and M. P. Vecchi. Optimization by simulated annealing. science , 220(4598):671-680, 1983. -
  133. [215][215] K. Kolluru, M. Mohammed, S. Mittal, S. Chakrabarti, et al. Alignment-augmented consistent translation for multilingual open information extraction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 2502-2517, 2022. -
  134. [217][217] K. Korini and C. Bizer. Evaluating knowledge generation and self-refinement strategies for llm-based column type annotation. CoRR , abs/2503.02718, 2025. - · arXiv
  135. [216][216] W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603 , 2024. - · arXiv
  136. [218][218] M. M. Krell, M. Kosec, S. P. Perez, and A. Fitzgibbon. Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance. arXiv preprint arXiv:2107.02027 , 2021. - · arXiv
  137. [219][219] A. V. Kumar and M. Sivathanu. Quiver: An informed storage cache for deep learning. In 18th USENIX Conference on File and Storage Technologies (FAST 20) , pages 283-296, 2020. -
  138. [224][224] H. Lauren¸ con, L. Saulnier, L. Tronchon, S. Bekman, A. Singh, A. Lozhkov, T. Wang, S. Karamcheti, A. Rush, D. Kiela, et al. Obelics: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural Information Processing Systems , 36, 2024. -
  139. [223][223] J. Lao, Y. Wang, Y. Li, J. Wang, Y. Zhang, Z. Cheng, W. Chen, M. Tang, and J. Wang. Gptuner: A manual-reading database tuning system via gpt-guided bayesian optimization. Proc. VLDB Endow. , 17(8):1939-1952, 2024. -
  140. [227][227] Y. Leviathan, M. Kalman, and Y. Matias. Fast inference from transformers via speculative decoding. In ICML , volume 202 of Proceedings of Machine Learning Research , pages 19274-19286. PMLR, 2023. -
  141. [226][226] F. Lei, X. Li, Y. Wei, S. He, Y. Huang, J. Zhao, and K. Liu. S3HQA: A three-stage approach for multi-hop text-table hybrid question answering. In A. Rogers, J. Boyd-Graber, and N. Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages 1731-1740, Toronto, Canada, July 2023. Association for Computational Linguistics. -
  142. [228][228] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. K¨ uttler, M. Lewis, W.-t. Yih, T. Rockt¨ aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems , 33:9459-9474, 2020. -
  143. [230][230] D. Li, S. Cao, T. Griggs, S. Liu, X. Mo, S. G. Patil, M. Zaharia, J. E. Gonzalez, and I. Stoica. Llms can easily learn to reason from demonstrations structure, not content, is what matters! arXiv preprint arXiv:2502.07374 , 2025. - · arXiv
  144. [229][229] B. Li, Y. Luo, C. Chai, G. Li, and N. Tang. The dawn of natural language to sql: Are we fully ready? Proceedings of the VLDB Endowment , 17(11):3318-3331, July 2024. -
  145. [231][231] G. Li, X. Zhou, S. Li, and B. Gao. Qtune: A query-aware database tuning system with deep reinforcement learning. Proc. VLDB Endow. , 12(12):2118-2130, 2019. -
  146. [233][233] H. Li, Q. Dong, Z. Tang, C. Wang, X. Zhang, H. Huang, S. Huang, X. Huang, Z. Huang, D. Zhang, et al. Synthetic data (almost) from scratch: Generalized instruction tuning for language models. arXiv preprint arXiv:2402.13064 , 2024. - · arXiv
  147. [232][232] H. Li, Y. Chen, Q. Ai, Y. Wu, R. Zhang, and Y. Liu. Lexeval: A comprehensive chinese legal benchmark for evaluating large language models. arXiv preprint arXiv:2409.20288 , 2024. - · arXiv
  148. [236][236] J. Li, A. Fang, G. Smyrnis, M. Ivgi, M. Jordan, S. Gadre, H. Bansal, E. Guha, S. Keh, K. Arora, et al. Datacomp-lm: In search of the next generation of training sets for language models. arXiv preprint arXiv:2406.11794 , 2024. - · arXiv
  149. [237][237] L. Li, L. Fang, and V. I. Torvik. Autodcworkflow: Llmbased data cleaning workflow auto-generation and benchmark. CoRR , abs/2412.06724, 2024. - · arXiv
  150. [239][239] M. Li, Y. Zhang, Z. Li, J. Chen, L. Chen, N. Cheng, J. Wang, T. Zhou, and J. Xiao. From quantity to quality: Boosting llm performance with self-guided data selection for instruction tuning. arXiv preprint arXiv:2308.12032 , 2023. - · arXiv
  151. [238][238] M. Li, Y. Zhang, S. He, Z. Li, H. Zhao, J. Wang, N. Cheng, and T. Zhou. Superfiltering: Weak-to-strong data filtering for fast instruction-tuning. arXiv preprint arXiv:2402.00530 , 2024. - · arXiv
  152. [241][241] S. Li, X. Ning, L. Wang, T. Liu, X. Shi, S. Yan, G. Dai, H. Yang, and Y. Wang. Evaluating quantized large language models. In ICML . OpenReview.net, 2024. -
  153. [243][243] Y. Li, H. Li, P. Zhao, J. Zhang, X. Zhang, T. Ji, L. Sun, C. Li, and H. Chen. Is large language model good at database knob tuning? A comprehensive experimental evaluation. CoRR , abs/2408.02213, 2024. - · arXiv
  154. [248][248] Z. Li, H. Yuan, H. Wang, G. Cong, and L. Bing. LLM-R2: A large language model enhanced rule-based rewrite system for boosting query efficiency. Proc. VLDB Endow. , 18(1):53-65, 2024. -
  155. [250][250] J. Lian, X. Liu, Y. Shao, et al. Chatbi: Towards natural language to complex business intelligence SQL. CoRR , abs/2405.00527, 2024. - · arXiv
  156. [249][249] Z. Li, X. Zhang, Y. Zhang, D. Long, P. Xie, and M. Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281 , 2023. - · arXiv
  157. [251][251] H. Liang, K. Zhao, Y. Yang, B. Cui, G. Dong, Z. Zhou, and W. Zhang. Data proportion detection for optimized data management for large language models. arXiv preprint arXiv:2409.17527 , 2024. - · arXiv
  158. [253][253] H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050 , 2023. - · arXiv
  159. [254][254] X. Lin, W. Wang, Y. Li, S. Yang, F. Feng, Y. Wei, and T.S. Chua. Data-efficient fine-tuning for llm-based recommendation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages 365-374, 2024. -
  160. [256][256] A. Liu, L. Pan, X. Hu, S. Li, L. Wen, I. King, and S. Y. Philip. Anunforgeable publicly verifiable watermark for large language models. In The Twelfth International Conference on Learning Representations , 2023. -
  161. [255][255] A. Liu, J. Liu, Z. Pan, Y. He, R. Haffari, and B. Zhuang. Minicache: Kv cache compression in depth dimension for large language models. Advances in Neural Information Processing Systems , 37:139997-140031, 2024. -
  162. [259][259] H. Liu, Q. Peng, Q. Yang, K. Liu, and H. Xu. Bucket pretraining is all you need. arXiv preprint arXiv:2407.07495 , 2024. - · arXiv
  163. [260][260] H. Liu, Y. Zhang, Y. Luo, and A. C.-C. Yao. Augmenting math word problems via iterative question composing. arXiv preprint arXiv:2401.09003 , 2024. - · arXiv
  164. [261][261] J. Liu and B. Mozafari. Query rewriting via large language models. CoRR , abs/2403.09060, 2024. - · arXiv
  165. [263][263] Q. Liu, X. Zheng, N. Muennighoff, G. Zeng, L. Dou, T. Pang, J. Jiang, and M. Lin. Regmix: Data mixture as regression for language model pre-training. arXiv preprint arXiv:2407.01492 , 2024. - · arXiv
  166. [264][264] W. Liu, W. Zeng, K. He, Y. Jiang, and J. He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. arXiv preprint arXiv:2312.15685 , 2023. - · arXiv
  167. [265][265] Y. Liu, H. Li, Y. Cheng, S. Ray, Y. Huang, Q. Zhang, K. Du, J. Yao, S. Lu, G. Ananthanarayanan, et al. Cachegen: Kv cache compression and streaming for fast large language model serving. In Proceedings of the ACM SIGCOMM 2024 Conference , pages 38-56, 2024. -
  168. [267][267] Y. Liu, E. Pe˜ na, A. S. R. Santos, E. Wu, and J. Freire. Magneto: Combining small and large language models for schema matching. CoRR , abs/2412.08194, 2024. - · arXiv
  169. [268][268] Z. Liu, Y. Huang, X. Yu, L. Zhang, Z. Wu, C. Cao, H. Dai, L. Zhao, Y. Li, P. Shu, et al. Deid-gpt: Zero-shot medical text de-identification by gpt-4. arXiv preprint arXiv:2303.11032 , 2023. - · arXiv
  170. [270][270] Z. Liu, Q. Liao, W. Gu, and C. Gao. Software vulnerability detection with gpt and in-context learning. In 2023 8th International Conference on Data Science in Cyberspace (DSC) , pages 229-236, 2023. -
  171. [269][269] Z. Liu, A. Karbasi, and T. Rekatsinas. Tsds: Data selection for task-specific model finetuning. arXiv preprint arXiv:2410.11303 , 2024. - · arXiv
  172. [273][273] D. Lu, H. Wu, J. Liang, Y. Xu, Q. He, Y. Geng, M. Han, Y. Xin, and Y. Xiao. Bbt-fin: Comprehensive construction of chinese financial domain pre-trained language model, corpus and benchmark. arXiv preprint arXiv:2302.09432 , 2023. - · arXiv
  173. [272][272] L. Long, R. Wang, R. Xiao, J. Zhao, X. Ding, G. Chen, and H. Wang. On llms-driven synthetic data generation, curation, and evaluation: A survey. arXiv preprint arXiv:2406.15126 , 2024. - · arXiv
  174. [274][274] W. Lu, J. Zhang, J. Fan, Z. Fu, Y. Chen, and X. Du. Large language model for table processing: A survey. Frontiers of Computer Science , 19(2):192350, 2025. -
  175. [276][276] Z. Luo, X. Zhang, X. Liu, H. Li, Y. Gong, C. Qi, and P. Cheng. Velocitune: A velocity-based dynamic domain reweighting method for continual pre-training. arXiv preprint arXiv:2411.14318 , 2024. - · arXiv
  176. [275][275] N. Lukas, A. Salem, R. Sim, S. Tople, L. Wutschitz, and S. Zanella-B´ eguelin. Analyzing leakage of personally identifiable information in language models. In 2023 IEEE Symposium on Security and Privacy (SP) , pages 346-363. IEEE, 2023. -
  177. [277][277] C. Ma, S. Chakrabarti, A. Khan, and B. Moln´ ar. Knowledge graph-based retrieval-augmented generation for schema matching. CoRR , abs/2501.08686, 2025. - · arXiv
  178. [279][279] L. Ma, N. Thakurdesai, J. Chen, J. Xu, E. K¨ orpeoglu, S. Kumar, and K. Achan. Llms with user-defined prompts as generic data operators for reliable data processing. In IEEE Big Data , pages 3144-3148. IEEE, 2023. -
  179. [278][278] G. Ma, Y. Ma, X. Wu, Z. Su, M. Zhou, and S. Hu. Task-level distributionally robust optimization for large language modelbased dense retrieval. arXiv preprint arXiv:2408.10613 , 2024. - · arXiv
  180. [280][280] Y. Ma, Y. Cao, Y. Hong, and A. Sun. Large language model is not a good few-shot information extractor, but a good reranker for hard samples! In Findings of the Association for Computational Linguistics: EMNLP 2023 . Association for Computational Linguistics, 2023. -
  181. [282][282] P. Maini, S. Seto, H. Bai, D. Grangier, Y. Zhang, and N. Jaitly. Rephrasing the web: A recipe for compute and data-efficient language modeling. arXiv preprint arXiv:2401.16380 , 2024. - · arXiv
  182. [283][283] U. Manber and G. Myers. Suffix arrays: a new method for online string searches. siam Journal on Computing , 22(5):935948, 1993. -
  183. [285][285] M. Marion, A. ¨ Ust¨ un, L. Pozzobon, A. Wang, M. Fadaee, and S. Hooker. When less is more: Investigating data pruning for pretraining llms at scale. arXiv preprint arXiv:2309.04564 , 2023. - · arXiv
  184. [286][286] J. L. McClelland, B. L. McNaughton, and R. C. O'Reilly. Why there are complementary learning systems in the hippocam- pus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review , 102(3):419, 1995. -
  185. [288][288] D. Mekala, A. Nguyen, and J. Shang. Smaller language models are capable of selecting instruction-tuning training data for larger language models. arXiv preprint arXiv:2402.10430 , 2024. - · arXiv
  186. [287][287] M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation , volume 24, pages 109165. Elsevier, 1989. -
  187. [289][289] S. Minaee, T. Mikolov, N. Nikzad, M. Chenaghlu, R. Socher, X. Amatriain, and J. Gao. Large language models: A survey. arXiv preprint arXiv:2402.06196 , 2024. - · arXiv
  188. [291][291] J. Mohan, A. Phanishayee, and V. Chidambaram. CheckFreq: Frequent, Fine-Grained DNN checkpointing. In 19th USENIX Conference on File and Storage Technologies (FAST 21) , pages 203-216. USENIX Association, Feb. 2021. -
  189. [290][290] A. Mitra, L. Del Corro, G. Zheng, S. Mahajan, D. Rouhana, A. Codas, Y. Lu, W.-g. Chen, O. Vrousgos, C. Rosset, et al. Agentinstruct: Toward generative teaching with agentic flows. arXiv preprint arXiv:2407.03502 , 2024. - · arXiv
  190. [292][292] J. Monteiro, F. S´ a, and J. Bernardino. Graph databases assessment: Janusgraph, neo4j, and tigergraph. In Perspectives and Trends in Education and Technology: Selected Papers from ICITED 2022 , pages 655-665. Springer, 2023. -
  191. [293][293] J. Mu, X. Li, and N. Goodman. Learning to compress prompts with gist tokens. Advances in Neural Information Processing Systems , 36, 2024. -
  192. [295][295] C. Na, I. Magnusson, A. H. Jha, T. Sherborne, E. Strubell, J. Dodge, and P. Dasigi. Scalable data ablation approximations for language models through modular training and merging. arXiv preprint arXiv:2410.15661 , 2024. - · arXiv
  193. [297][297] T. Nguyen, C. V. Nguyen, V. D. Lai, H. Man, N. T. Ngo, F. Dernoncourt, R. A. Rossi, and T. H. Nguyen. CulturaX: A cleaned, enormous, and multilingual dataset for large language models in 167 languages. In N. Calzolari, M.-Y. Kan, V. Hoste, A. Lenci, S. Sakti, and N. Xue, editors, Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024) , pages 4226-4237, Torino, Italia, May 2024. ELRA and ICCL. -
  194. [296][296] R. Navigli, S. Conia, and B. Ross. Biases in large language models: origins, inventory, and discussion. ACM Journal of Data and Information Quality , 15(2):1-21, 2023. -
  195. [298][298] I. Nunes, M. Heddes, P. Verg´ es, D. Abraham, A. Veidenbaum, A. Nicolau, and T. Givargis. Dothash: Estimating set similarity metrics for link prediction and document deduplication. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , pages 1758-1769, 2023. -
  196. [300][300] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 , 2023. - · arXiv
  197. [301][301] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems , 35:27730-27744, 2022. -
  198. [303][303] Z. Pan, Q. Wu, H. Jiang, M. Xia, X. Luo, J. Zhang, Q. Lin, V. R¨ uhle, Y. Yang, C.-Y. Lin, et al. Llmlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. arXiv preprint arXiv:2403.12968 , 2024. - · arXiv
  199. [302][302] R. Pan, J. Zhang, X. Pan, R. Pi, X. Wang, and T. Zhang. Scalebio: Scalable bilevel optimization for llm data reweighting. arXiv preprint arXiv:2406.19976 , 2024. - · arXiv
  200. [304][304] M. Parciak, B. Vandevoort, F. Neven, L. M. Peeters, and S. Vansummeren. Schema matching with large language models: an experimental study. In VLDB Workshops . VLDB.org, 2024. -

200 of 342 shown

Cited by

Sorted by citations

Metadata

Authors
Zhou, X., He, J., Zhou, W., Chen, H., Tang, Z., Zhao, H., Tong, X., Li, G., Chen, Y., Zhou, J., Sun, Z., Hui, B., Wang, S., He, C., Liu, Z., Zhou, J., & Wu, F.
Affiliations
  • Shanghai Artificial Intelligence Laboratory (CN)
  • Tsinghua University (CN)
  • Shanghai Jiao Tong University (CN)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2505.18458
arXiv
2505.18458
OpenAlex
W4414581331
Cited by
19
References
2
Citations by year
2026: 13 · 2025: 5
Type
posted content
Language
en
Open access
Licence
creativecommons.org/licenses/by/4.0/legalcode
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
large language models, data management, data deduplication, synthetic data augmentation, KV cache management, checkpointing, retrieval-augmented generation, prompt compression, data provenance, data integration, data discovery, query rewriting, anomaly diagnosis, agent cooperation, retrieval-augmented prompting, LLM fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review