Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 6

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling

Kavathekar, Ishan, Donakanti, Raghav, Kumaraguru, Ponnurangam, Vaidhyanathan, Karthik·2025·Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering
DOI 10.1145/3756681.3757001
Open this paper in CoChat — free account

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3756681.3757001

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 34

  1. Marah Abdin Jyoti Aneja Hany Awadalla Ahmed Awadallah Ammar Ahmad Awan Nguyen Bach Amit Bahree and Arash Bakhtiari et al.2024. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arxiv:https://arXiv.org/abs/2404.14219 [cs.CL] https://arxiv.org/abs/2404.14219 · arXiv
  2. 10.18653/v1/2024.emnlp-industry.94 · DOI
  3. Muhammad Azeem Akbar Arif Ali Khan Najmul Islam and Sajjad Mahmood. 2024. DevOps project management success factors: A decision-making framework. Software: Practice and Experience 54 2 (2024) 257–280. · DOI
  4. Satanjeev Banerjee and Alon Lavie. 2004. Meteor: an automatic metric for MT evaluation with high levels of correlation with human judgments. Proceedings of ACL-WMT (2004) 65–72.
  5. Kinjal Basu Ibrahim Abdelaziz Subhajit Chaudhury Soham Dan Maxwell Crouse Asim Munawar Sadhana Kumaravel Vinod Muthusamy Pavan Kapanipathi and Luis A Lastras. 2024. API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs. arXiv preprint arXiv:https://arXiv.org/abs/2402.15491 (2024). · DOI
  6. Hezekiah J Branch Jonathan Rodriguez Cefalu Jeremy McHugh Leyla Hujer Aditya Bahl Daniel del Castillo Iglesias Ron Heichman and Ramesh Darwishi. 2022. Evaluating the susceptibility of pre-trained language models via handcrafted adversarial examples. arXiv preprint arXiv:https://arXiv.org/abs/2209.02128 (2022). · arXiv
  7. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, and et al.2020. Language models are few-shot learners. In Proceedings of the 34th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NIPS ’20). Curran Associates Inc., Red Hook, NY, USA, Article 159, 25 pages.
  8. 10.1109/ICSE.2017.57 · DOI
  9. Xiang Chen Chaoyang Gao Chunyang Chen Guangbei Zhang and Yong Liu. 2025. An Empirical Study on Challenges for LLM Application Developers. arxiv:https://arXiv.org/abs/2408.05002 [cs.SE] https://arxiv.org/abs/2408.05002 · arXiv
  10. Tri Dao Daniel Y. Fu Stefano Ermon Atri Rudra and Christopher Ré. 2022. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. arxiv:https://arXiv.org/abs/2205.14135 [cs.LG] https://arxiv.org/abs/2205.14135 · arXiv
  11. Tim Dettmers Mike Lewis Younes Belkada and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems 35 (2022) 30318–30332.
  12. Yuxian Gu Li Dong Furu Wei and Minlie Huang. 2023. Knowledge distillation of large language models. arXiv preprint arXiv:https://arXiv.org/abs/2306.08543 (2023). · arXiv
  13. Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Y. Wu Y. K. Li Fuli Luo Yingfei Xiong and Wenfeng Liang. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming – The Rise of Code Intelligence. arxiv:https://arXiv.org/abs/2401.14196 [cs.SE] https://arxiv.org/abs/2401.14196 · arXiv
  14. Zhicheng Guo Sijie Cheng Hao Wang Shihao Liang Yujia Qin Peng Li Zhiyuan Liu Maosong Sun and Yang Liu. 2024. StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models. arXiv preprint arXiv:https://arXiv.org/abs/2403.07714 (2024). · arXiv
  15. Zixu Hao Huiqiang Jiang Shiqi Jiang Ju Ren and Ting Cao. 2024. Hybrid SLM and LLM for Edge-Cloud Collaborative Inference. Proceedings of the Workshop on Edge and Mobile Foundation Models (2024). https://api.semanticscholar.org/CorpusID:270405086 · DOI
  16. 10.1109/ICSE43902.2021.00107 · DOI
  17. Takeshi Kojima Shixiang Shane Gu Machel Reid Yutaka Matsuo and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022) 22199–22213.
  18. 10.18653/v1/2021.acl-long.353 · DOI · Paper page
  19. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out. 74–81.
  20. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation. In Thirty-seventh Conference on Neural Information Processing Systems. https://openreview.net/forum?id=1qvx610Cu7
  21. Yupei Liu, Yuqi Jia, Runpeng Geng, Jinyuan Jia, and Neil Zhenqiang Gong. 2024. Formalizing and benchmarking prompt injection attacks and defenses. In 33rd USENIX Security Symposium (USENIX Security 24). 1831–1847.
  22. Zuxin Liu Thai Hoang Jianguo Zhang Ming Zhu Tian Lan Shirley Kokane Juntao Tan Weiran Yao Zhiwei Liu Yihao Feng Rithesh Murthy Liangwei Yang Silvio Savarese Juan Carlos Niebles Huan Wang Shelby Heinecke and Caiming Xiong. 2024. APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets. arxiv:https://arXiv.org/abs/2406.18518 [cs.CL] https://arxiv.org/abs/2406.18518 · arXiv
  23. Anton Lozhkov Raymond Li Loubna Ben Allal Federico Cassano and et al.2024. StarCoder 2 and The Stack v2: The Next Generation. arxiv:https://arXiv.org/abs/2402.19173 [cs.SE] https://arxiv.org/abs/2402.19173 · arXiv
  24. Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy Ambrosio Blanco Colin Clement Dawn Drain Daxin Jiang Duyu Tang et al. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:https://arXiv.org/abs/2102.04664 (2021). · arXiv
  25. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311–318.
  26. Yujia Qin Shihao Liang Yining Ye Kunlun Zhu Lan Yan Yaxi Lu Yankai Lin Xin Cong Xiangru Tang Bill Qian et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:https://arXiv.org/abs/2307.16789 (2023). · arXiv
  27. Qiaoyu Tang Ziliang Deng Hongyu Lin Xianpei Han Qiao Liang Boxi Cao and Le Sun. 2023. Toolalpaca: Generalized tool learning for language models with 3000 simulated cases. arXiv preprint arXiv:https://arXiv.org/abs/2306.05301 (2023). · arXiv
  28. Yiwei Wang Muhao Chen Nanyun Peng and Kai-Wei Chang. 2024. DeepEdit: Knowledge Editing as Decoding with Constraints. arxiv:https://arXiv.org/abs/2401.10471 [cs.CL] · arXiv
  29. Yaqing Wang Quanming Yao James T Kwok and Lionel M Ni. 2020. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur) 53 3 (2020) 1–34. · DOI · Paper page
  30. 10.5555/2349018 · doi:10.5555/2349018
  31. Biwei Yan Kun Li Minghui Xu Yueyan Dong Yue Zhang Zhaochun Ren and Xiuzhen Cheng. 2024. On protecting the data privacy of large language models (llms): A survey. arXiv preprint arXiv:https://arXiv.org/abs/2403.05156 (2024). · DOI
  32. Kechi Zhang Huangzhao Zhang Ge Li Jia Li Zhuo Li and Zhi Jin. 2023. Toolcoder: Teach code generation models to use api search tools. arXiv preprint arXiv:https://arXiv.org/abs/2305.04032 (2023). · arXiv
  33. Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al. 2023. A survey of large language models. arXiv preprint arXiv:https://arXiv.org/abs/2303.18223 (2023). · arXiv
  34. Lucen Zhong Zhengxiao Du Xiaohan Zhang Haiyi Hu and Jie Tang. 2025. ComplexFuncBench: Exploring Multi-Step and Constrained Function Calling under Long-Context Scenario. arXiv preprint arXiv:https://arXiv.org/abs/2501.10132 (2025). · arXiv

Cited by

Metadata

Authors
Kavathekar, I., Donakanti, R., Kumaraguru, P., & Vaidhyanathan, K.
Affiliations
International Institute of Information Technology, Hyderabad, India
ORCID
Venue
Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering
Publisher
ACM
Published
2025-06-17
Pages
1117-1126
DOI
10.1145/3756681.3757001
Cited by
6
References
11

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.