The DOI matches its Crossref record; 15 of the 21 references it cites were found in a registry or an index.
Are Small Language Models Ready to Compete with Large Language Models for Practical Applications?
Open accessAbstract
The rapid rise of Language Models (LMs) has expanded their use in several applications.Yet, due to constraints of model size, associated cost, or proprietary restrictions, utilizing stateof-the-art (SOTA) LLMs is not always feasible.With open, smaller LMs emerging, more applications can leverage their capabilities, but selecting the right LM can be challenging as smaller LMs don't perform well universally.This work tries to bridge this gap by proposing a framework to experimentally evaluate small, open LMs in practical settings through measuring semantic correctness of outputs across three practical aspects: task types, application domains and reasoning types, using diverse prompt styles.It also conducts an in-depth comparison of 10 small, open LMs to identify best LM and prompt style depending on specific application requirement using the proposed framework.We also show that if selected appropriately, they can outperform SOTA LLMs like DeepSeek-v2, GPT-4o-mini, Gemini-1.5-Pro,and even compete with GPT-4o. 1
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Lightweight SLMs provide a suitable choice for online control and filtering.
“In this setting, lightweight SLMs provide a suitable choice (Sinha et al., 2025) for online control and filtering, while heavier abstraction and consolidation can be deferred to offline processing.”
From Lightweight LLM Agent Memory with Small Language Models (2026) · cites this paper as Sinha et al., 2025
The Introduction says small, open language models can offer faster inference and be an appealing alternative for practical applications; the Conclusion says properly selected models can be effective. But the paper does not establish their suitability specifically for online control and filtering. The cited work supports the general choice of lightweight models, not that particular use. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIn this setting, lightweight SLMs provide a suitable choice for online control and filtering, while heavier abstraction and consolidation can be deferred to offline processing.
“In this setting, lightweight SLMs provide a suitable choice (Sinha et al., 2025) for online control and filtering, while heavier abstraction and consolidation can be deferred to offline processing.”
From Lightweight LLM Agent Memory with Small Language Models (2026) · cites this paper as Sinha et al., 2025
The Introduction describes small, open language models as offering on-device use and faster inference, and Section 2.5 emphasizes computational efficiency. This supports their general suitability for lightweight applications, but the paper does not establish that they are suitable specifically for online control and filtering. The citation is attached to the suitability claim, not the separate proposal to defer abstraction and consolidation to offline processing. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 4
- Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved cor- relation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summariza- tion, pages 65-72.
- Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, volume 33, pages 1877-1901.
- Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text de- generation. In International Conference on Learning Representations.
- Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Cited by
Sorted by citations
Wan, Y., Chen, Z., Liu, Y., Chen, C., & Packianather, M. (2025) · Advanced Engineering Informatics · doi:10.1016/j.aei.2025.103212 · Cited by 138
Rahman, M. S., & Rahman, S. (2025) · doi:10.63125/5vyhgs98 · Cited by 37
StarWhisper Telescope: an AI framework for automating end-to-end astronomical observations
Wang, C., Zhang, Y., Hu, X., Mao, Y., Chen, X., Du, P., Wang, R., Li, Y., Wu, Y., Yang, H., et al. (2025) · Communications Engineering · doi:10.1038/s44172-025-00520-4 · Cited by 28
Chuang, Y.-S., Sarkar, A. R., Hsu, Y.-C., Mohammed, N., & Jiang, X. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf037 · Cited by 14
Awan, S. A., Khan, M. A., Khan, A. A., Sathio, A. A., Alsayaydeh, J. A. J., Bacarra, R., Herawan, S. G., & Aziz, R. (2025) · Journal Of Big Data · doi:10.1186/s40537-025-01330-3 · Cited by 7
Lightweight LLM Agent Memory with Small Language Models
Zhang, J., Zhang, C., Chen, S., Huang, Z., Zheng, P., Wang, Z., Guo, P., Mo, F., Bae, S.-H., Zou, J., et al. (2026) · Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2026.acl-long.588 · Cited by 35
Moorthy, S. M. K., Qi, M., Rosen, A., Malhi, Y. S., & Salguero‐Gómez, R. (2026) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.70403 · Cited by 2
Comparative Survey of Small and Large Language Models
Huen, T. H., Lim, T. M., & Tan, K. S. N. (2025) · IEEE International Conference on Computation, Big-Data and Engineering (ICCBE) · doi:10.1109/iccbe65177.2025.11255928 · Cited by 1
Bernard, J., & Sabine Graf, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-98281-1_11 · Cited by 1
Chen, C., Saha, P., Reimer, J., Wachter, S., Chen, J., Dhar, N., & Broderick, G. (2026) · Scientific Reports · doi:10.1038/s41598-026-62042-z · Cited by 1
Cargo: a Framework for Confidence-Aware Routing of Large Language Models
Barrak, A., Fourati, Y., Olchawa, M., Ksontini, E., & Zoghlami, K. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00029 · Cited by 1
Lightweight LLM Agent Memory with Small Language Models
Zhang, J., Zhang, C., Chen, S., Huang, Z., Zheng, P., Wang, Z., Guo, P., Mo, F., Bae, S.-H., Zou, J., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.07798
“In this setting, lightweight SLMs provide a suitable choice (Sinha et al., 2025) for online control and filtering, while heavier abstraction and consolidation can be deferred to offline processing.”
Language Models for Disinformation Detection: To be Large or to be Small?
Kalamakis, I., Liappi, S., Papakostas, D., & Katsaros, D. (2025) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa66935.2025.11315272
Abkenar, M. Y., Wang, W., Stede, M., Finlayson, M. A., Picca, D., & Ioannidis, P. (2026) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2e6be7q3bnzh
Small language models efficacy prototyped for Oromo word sense disambiguation
Edeti, L., Meshesha, M., & Negesse, F. (2026) · Discover Computing · doi:10.1007/s10791-026-10155-4
STARouter: Internal State based LLM Router for Software Testing Tasks
Cho, H., & Yoo, S. (2026) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw72326.2026.00015
EVALUATION OF THE IMPLEMENTATION OF AN E-PORTFOLIO–BASED ENGLISH LEARNING SYSTEM AT SMPN 2 SIDAREJA
Mustangin, M., Alfiah, N., Winarso, W., & Kulsum, N. U. (2026) · JIPI (Jurnal Ilmiah Penelitian dan Pembelajaran Informatika) · doi:10.29100/jipi.v11i3.10076
Metadata
- Authors
- Sinha, N., Jain, V., & Chadha, A.
- Affiliations
- Georgia Institute of Technology (US)
- Meta (United States) (US)
- Amazon (United States) (US)
- Venue
- Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025)
- Published
- 2025
- Pages
- 365-398
- OpenAlex
- W4411119218
- Cited by
- 9
- References
- 0
- Citations by year
- 2026: 3 · 2025: 6
- FWCI
- 8.61
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Trustworthy NLP (TrustNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- small language models, open-source language models, large language models, semantic correctness, practical application, model selection, types of reasoning, task type, application domains, benchmarking framework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
