Open-Source vs. Commercial Coding Assistants: A 2025 Comparison of DeepSeek R1, Qwen 2.5 and Claude 3.7
Open accessPublished version of a preprint
Abstract
This paper presents a comprehensive comparative analysis of state-of-the-art large language models (LLMs) for code generation, focusing on the Qwen, Claude, and DeepSeek families alongside other prominent models. Through systematic evaluation of architectural designs, performance benchmarks, and practical applications, we reveal significant advancements in open-weight models that now rival or surpass proprietary alternatives in coding tasks. Our study demonstrates Qwen3-Coder’s exceptional agentic capabilities (69.6% on SWE-bench), DeepSeek R1’s cost-efficient performance (98% lower cost than comparable models), and Claude’s robust general-purpose reasoning. We analyze emerging trends including mixture-of-experts architectures, extended context windows (up to 1M tokens), and specialized coding assistants. The research incorporates temporal analysis showing accelerated innovation cycles, particularly among Chinese models, and projects future market dynamics through 2027. Our multi-dimensional evaluation covers: (1) coding performance across standardized benchmarks and real-world tasks, (2) mathematical and logical reasoning capabilities, (3) computational efficiency and cost tradeoffs, and (4) architectural innovations driving progress. The findings indicate a shifting landscape where open models increasingly compete with closed systems, offering developers diverse options balancing performance, cost, and specialization. This work provides researchers and practitioners with up-to-date insights for selecting and deploying AI coding assistants in software engineering workflows. We further discuss state-of-the-art AI model versions including DeepSeek R1, Qwen 2.5/3 series, Claude 3.5/3.7 and Sonnet. The results demonstrate significant advancements in open-weight models like DeepSeek R1 and Qwen 2.5 Coder, which now rival or surpass proprietary models in specific domains while offering substantial cost advantages. We also examine emerging trends in model architectures, including mixture-of-experts implementations and context length extensions.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Specialized coding models outperform general-purpose LLMs in syntax accuracy and debugging reliability on complex programming tasks.
“Recent benchmarks show that specialized coding models outperform general-purpose LLMs in syntax accuracy and debugging reliability on complex programming tasks [45, 52].”
From A Multi-Agent Framework for Accessible Web Code Generation (2026) · cites this paper as [45]
The Coding Performance Comparison section reports that Qwen3-Coder outperformed many proprietary alternatives on SWE-bench, supporting a narrower claim about coding performance. The paper does not establish that specialized coding models generally outperform general-purpose LLMs in syntax accuracy or debugging reliability on complex tasks; its Benchmark Evaluation section describes bug-fixing benchmarks but gives no such comparative finding. Page numbers are unavailable in our copy. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
Cited by 3
Research works printed but not linked: 1
- "The Coding-Agent Crown Just Tipped: Qwen3-Coder Steps Up -GlobalGPT Review." Accessed: Aug. 10, 2025. [Online]. Available: https://www.glbgpt.com/resource/the- coding-agent-crown-just-tipped-qwen3-coder-steps-up
- Authors
- Joshi, S. B.
- Corresponding author
- Sanjay B. Joshi
- ORCID
- Sanjay B. Joshi
- Venue
- International Journal of Computer Applications Technology and Research
- Published
- 2025-08-28
- ISSN
- 2319-8656 (ISSN-L)
- OpenAlex
- W4413770823
- Cited by
- 3
- References
- 0
- Citations by year
- 2026: 3
- FWCI
- 3.15
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Computer Applications Technology and Research (publishedVersion)
- Preprints.org (acceptedVersion, cc-by)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- large language models, code generation, coding assistants, DeepSeek-R1, Qwen2.5, mixture of experts, inference cost, SWE-bench
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
