QuantCode-Eval: Benchmarking Quantitative Strategy Code Reproduction from Finance Papers

Abstract

Large language models (LLMs) have enabled impressive coding agents for various domains, driving growing interest in applying them to automate domain-specific workflows. In quantitative strategy development, a domain with high stakes and high potential returns, reproducing executable strategy code from financial papers and reports is valuable for both academic research and industrial gains. However, it remains unclear whether LLMs can faithfully reproduce such domain-knowledgeintensive strategy code from finance papers. These papers often omit critical implementation details, while high-quality strategy code is scarce as pretraining data for latest LLMs. This paper presents QuantCode-Eval, a benchmark for evaluating the ability of LLMs to reproduce strategies from 30 leading quantitative finance papers published (or released) between 2010 and 2026 into executable trading strategies. To support rigorous evaluation, each reproduction task includes around 20 property-based executable checkers that return a binary pass/fail result. We conduct extensive evaluation on 7 proprietary and open-source LLMs with coding-agent scaffolds. The results show that even the best-performing setup, Claude Opus 4.7 with Claude Code, achieves a 43% pass@5 rate when reproducing code that satisfies all properties. We also perform a fine-grained analysis of violated properties by jointly examining their semantics and the agent trajectories. Our findings reveal diverse failures in strategy-sensitive aspects: (i) maintaining temporal consistency, (ii) correctly interpreting and formalizing strategy definitions, and (iii) preserving the intended signal direction during trade execution.