CoChat
Sign Up
PreprintCited by 1Adversarial Robustness in Machine LearningSoftware Testing and Debugging TechniquesEthics and Social Impacts of AI

Auditing Harness Tampering in Self-Improving Agents

Open access
Wang, Xing, Zhang, Xiaoyi, Shao, Jie·2026·arXiv
DOI 10.48550/arxiv.2609.00069arXiv 2609.00069
Open this paper in CoChat — free accountOpen PDF

Abstract

Self-improving agents iteratively modify their own harness to push the frontier of their performance. However, such modifications can produce illusory performance gains or compromise integrity constraints such as authorization, provenance, and completeness without genuinely improving capability. We term this phenomenon as harness tampering, which extends the concept from reward and measurement tampering to the full self-improvement lifecycle. To systematically study this problem, we propose a two-axis taxonomy that categorizes each misaligned edit by the harness functional role in which it occurs and the obligation it violates. Then we build an annotated corpus by seeding tampered-benign edit pairs into the real trajectories of self-improving agents. We adapt and benchmark diverse audit methods on tampering classification and localization tasks. Finally we systematically audit real trajectories of self-improving agents. The results demonstrate that harness tampering consistently occurs in real runs from different agents, often persists in the lineage of the best agent, and forms distinct system-specific profiles across the taxonomy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2609.00069

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

  • Self-Harness: Harnesses That Improve Themselves

    Zhang, H., Zhang, S., Li, K., Zhang, C., Chen, Y., Zhang, Y., Bai, L., & Hu, S. (2026) · arXiv · doi:10.48550/arxiv.2606.09498 · cited 1×

    “More recent systems further generalize this search space by co-evolving task agents, search algorithms, and meta-level harnesses (Zhang et al. 2026c; Lee et al. 2026; Luo et al. 2026a), often introducing non-regression or validation checks to regulate behavioral drift (Zhang et al. 2026a).”

Flagged first, then by citations

Research works printed but not linked: 48

  1. Atinafu, Y.; and Cohen, R. 2026. RewardHackingA- gents: Benchmarking Evaluation Integrity for LLM ML- Engineering Agents. In 2026 IEEE 42nd International Con- ference on Data Engineering Workshops, 27-34.
  2. Clark, D. D.; and Wilson, D. R. 1987. A Comparison of Commercial and Military Computer Security Policies. In Proceedings of the 1987 IEEE Symposium on Security and Privacy, 184-195.
  3. Denison, C.; MacDiarmid, M.; Barez, F.; Duvenaud, D.; Kravec, S.; Marks, S.; Schiefer, N.; Soklaski, R.; Tamkin, A.; Kaplan, J.; Shlegeris, B.; Bowman, S. R.; Perez, E.; and Hubinger, E. 2024. Sycophancy to Subterfuge: Investigat- ing Reward-Tampering in Large Language Models. arXiv preprint arXiv:2406.10162. · arXiv
  4. Deshpande, D.; Kannappan, A.; and Qian, R. 2026. Bench- marking Reward Hack Detection in Code Environments via Contrastive Analysis. arXiv preprint arXiv:2601.20103. · arXiv
  5. Du, S.; Zhao, J.; Shi, J.; Xie, Z.; Jiang, X.; Bai, Y.; and He, L. 2026. A Survey on the Optimization of Large Language Model-Based Agents. ACM Computing Surveys, 58(9): 1- 37.
  6. Fang, J.; Peng, Y.; Zhang, X.; Wang, Y.; Yi, X.; Zhang, G.; Xu, Y.; Wu, B.; Liu, S.; Li, Z.; Ren, Z.; Aletras, N.; Wang, X.; Zhou, H.; and Meng, Z. 2025. A Comprehensive Sur- vey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems. arXiv preprint arXiv:2508.07407. · arXiv
  7. Fernando, C.; Banarse, D. S.; Michalewski, H.; Osindero, S.; and Rocktäschel, T. 2024. Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution. In ICML, 13481- 13544.
  8. Gao, H.-a.; Geng, J.; Hua, W.; Hu, M.; Juan, X.; Liu, H.; Liu, S.; Qiu, J.; Qi, X.; et al. 2026. A Survey of Self-Evolving Agents What, When, How, and Where to Evolve on the Path to Artificial Super Intelligenc. Transactions on Machine Learning Research, 2026-January. Google. 2026. Gemini 3.7 Flash.
  9. Greenblatt, R.; Shlegeris, B.; Sachan, K.; and Roger, F. 2024. AI Control: Improving Safety Despite Intentional Subver- sion. In ICML, 16295-16336.
  10. Hills, J.; Caspary, I.; and Stickland, A. C. 2026. Distributed Attacks in Persistent-State AI Control. arXiv preprint arXiv:2607.02514. · arXiv
  11. Hu, S.; Lu, C.; and Clune, J. 2025. Automated design of agentic systems. In ICLR, volume 2025, 21344-21377.
  12. Jia, Y.; and Harman, M. 2010. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering, 37(5): 649-678.
  13. Joulin, A.; Grave, E.; Bojanowski, P.; and Mikolov, T. 2017. Bag of Tricks for Efficient Text Classification. In Proceed- ings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, 427-431.
  14. Just, R.; Jalali, D.; and Ernst, M. D. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 2014 international symposium on software testing and analysis, 437-440.
  15. Kane, M. T. 2013. Validating the Interpretations and Uses of Test Scores. Journal of Educational Measurement, 50(1): 1-73.
  16. Kutasov, J.; Sun, Y.; Colognese, P.; van der Weij, T.; Petrini, L.; Zhang, C. B. C.; Hughes, J.; Deng, X.; Sleight, H.; Tracy, T.; Shlegeris, B.; and Benton, J. 2025. SHADE-Arena: Eval- uating Sabotage and Monitoring in LLM Agents. arXiv preprint arXiv:2506.15740. · arXiv
  17. Lee, Y.; Nair, R.; Zhang, Q.; Lee, K.; Khattab, O.; and Finn, C. 2026. Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv preprint arXiv:2603.28052. · arXiv
  18. Libon, L.; Rank, B.; Yeon, J.; Schmotz, D.; Qin, J.; Don- nelly, D.; Prinzhorn, D.; and Andriushchenko, M. 2026. Re- searchArena: Evaluating Sabotage and Monitoring in Auto- mated AI R&D. arXiv preprint arXiv:2607.19321. · arXiv
  19. Lin, M.; Wu, J.; Wang, Z.; Shi, Z.; Sang, Y.; He, B.; Liu, Z.; Wei, T.; Wu, Z.; Zhang, Z.; et al. 2026a. Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agents. arXiv preprint arXiv:2605.30621. · arXiv
  20. Lin, R.; Deng, X.; Li, Q.; Ma, J.; Feng, Y.; Qing, Y.; Li, Z.; Zhang, Y.; Cui, S.; Meng, C.; Zhang, T.; Ma, X.; Li, Q.; Xu, K.; and Ji, S. 2026b. Safety in Self-Evolving LLM Agent Systems: Threats, Amplification, and Case Studies. arXiv preprint arXiv:2606.23075. · arXiv
  21. Luo, H.; Huang, Y.; Luo, S.; Liu, F.; Li, L.; Hu, Z.; Feng, J.; and Liu, Q. 2026a. Harness-Aware Self-Evolving: Co- Evolving Model Weights, Harness, and Task Solutions. arXiv preprint arXiv:2607.03935. · arXiv
  22. Luo, X.; Xue, D.; Wang, F.; Hu, C.; and Deng, Y. 2026b. HarnessBank: Semantic Gene-Bank Search with Gated Ver- ification for Agent-Harness Self-Evolution. arXiv preprint arXiv:2607.13683. · arXiv
  23. Meng, R.; Mishra, B. D.; Chen, J.; Li, C.-L.; Goyal, P.; Parmar, M.; Song, Y.; Song, Y.; Sinha, R.; Ranganathan, P.; et al. 2026. ScientistOne: Towards Human-Level Au- tonomous Research via Chain-of-Evidence. arXiv preprint arXiv:2605.26340. · arXiv
  24. Ning, X.; Tieu, K.; Fu, D.; Wei, T.; Li, Z.; Bei, Y.; Zou, J.; Ai, M.; Liu, Z.; Li, T.-W.; et al. 2026. Code as Agent Harness. arXiv preprint arXiv:2605.18747. OpenAI. 2026a. GPT-5.6 Luna Model. OpenAI. 2026b. GPT-5.6 Sol Model. OpenAI. 2026c. GPT-5.6 Terra Model. Qwen Team. 2026. Qwen3.5-9B. Model card. · arXiv
  25. Ren, Z.; Chen, Y.; Guo, D.; Rong, G.; Li, T.; Xiong, R.; Lan, Q.; Wang, W.; Nanbo, L.; Yang, Y.; et al. 2026. Self- Improvements in Modern Agentic Systems: A Survey. arXiv preprint arXiv:2607.13104. · arXiv
  26. Roger, F.; Greenblatt, R.; Nadeau, M.; Shlegeris, B.; and Thomas, N. 2023. Benchmarks for Detecting Measurement Tampering. arXiv preprint arXiv:2308.15605. · arXiv
  27. Roth, A.; Samanta, A.; Halevy, M.; Levine, Y.; and Efroni, Y. 2026. Hack-Verifiable Environments: Towards Evaluating Reward Hacking at Scale. arXiv preprint arXiv:2605.20744. · arXiv
  28. Saltzer, J. H.; and Schroeder, M. D. 1975. The Protection of Information in Computer Systems. Proceedings of the IEEE, 63(9): 1278-1308.
  29. Shao, S.; Ren, Q.; Liu, D.; Qian, C.; Wei, B.; Guo, D.; Yang, J.; Song, X.; Zhang, L.; Zhang, W.; and Shao, J. 2026. Your Agent May Misevolve: Emergent Risks in Self-Evolving LLM Agents. In ICLR, volume 2026, 99728-99793.
  30. Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: Language Agents with Verbal Re- inforcement Learning. In NeurIPS, volume 36, 8634-8652.
  31. Thaman, K. 2026. Reward Hacking Benchmark: Measur- ing Exploits in LLM Agents with Tool Use. arXiv preprint arXiv:2605.02964. · arXiv
  32. Tu, X.; Wang, T.; Lu, Y.; Huang, K.; Qu, Y.; and Mostafavi, S. 2026. BenchGuard: Who Guards the Benchmarks? Auto- mated Auditing of LLM Agent Benchmarks. arXiv preprint arXiv:2604.24955. · arXiv
  33. Wang, G.; Xie, Y.; Jiang, Y.; Mandlekar, A.; Xiao, C.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2024a. Voyager: An Open-Ended Embodied Agent with Large Language Mod- els. Transactions on Machine Learning Research.
  34. Wang, H.; Li, H.; Mang, Q.; Cheung, A.; Sen, K.; and Song, D. 2026a. Do Androids Dream of Breaking the Game? Sys- tematically Auditing AI Agent Benchmarks with BenchJack. arXiv preprint arXiv:2605.12673. · arXiv
  35. Wang, L.; Ma, C.; Feng, X.; Zhang, Z.; Yang, H.; Zhang, J.; Chen, Z.; Tang, J.; Chen, X.; Lin, Y.; et al. 2024b. A survey on large language model based autonomous agents. Frontiers of computer science, 18(6): 186345.
  36. Wang, R. Y.; and Strong, D. M. 1996. Beyond Accuracy: What Data Quality Means to Data Consumers. Journal of Management Information Systems, 12(4): 5-33.
  37. Wang, Y.; Zhu, H.; Hu, Z.; Yuan, Y.; Chen, Z.; Senthil, S.; Hajishirzi, H.; Tsvetkov, Y.; Dasigi, P.; and Xiao, T. 2026b. Rethinking the Evaluation of Harness Evolution for Agents. arXiv preprint arXiv:2607.12227. · arXiv
  38. Xi, Z.; Chen, W.; Guo, X.; He, W.; Ding, Y.; Hong, B.; Zhang, M.; Wang, J.; Jin, S.; Zhou, E.; et al. 2025. The rise and potential of large language model based agents: A survey. Science China information sciences, 68(2): 121101.
  39. Yang, C.; Wang, X.; Lu, Y.; Liu, H.; Le, Q. V.; Zhou, D.; and Chen, X. 2024. Large language models as optimizers. In ICLR, volume 2024, 12028-12068.
  40. Yang, J.; Lieret, K.; Jimenez, C.; Wettig, A.; Khandpur, K.; Zhang, Y.; Hui, B.; Press, O.; Schmidt, L.; and Yang, D. 2026. Swe-smith: Scaling data for software engineering agents. NeurIPS, 38.
  41. Zhang, J.; Hu, S.; Lu, C.; Lange, R.; and Clune, J. 2026b. Dar- win Gödel machine: open-ended evolution of self-improving agents. In ICLR, volume 2026, 104223-104294.
  42. Zhang, J.; Xiang, J.; Yu, Z.; Teng, F.; Chen, X.; Chen, J.; Zhuge, M.; Cheng, X.; Hong, S.; Wang, J.; et al. 2025. Aflow: Automating agentic workflow generation. In ICLR, volume 2025, 34040-34077.
  43. Zhang, J.; Zhao, B.; Yang, W.; Foerster, J.; Clune, J.; Jiang, M.; Devlin, S.; and Shavrina, T. 2026c. Hyperagents. arXiv:2603.19461. · arXiv
  44. Zhao, A.; Huang, D.; Xu, Q.; Lin, M.; Liu, Y.-J.; and Huang, G. 2024. ExpeL: LLM Agents Are Experiential Learners. In AAAI, volume 38, 19632-19642.
  45. Zhao, B.; Srikanth, D.; Wu, Y.; and Jiang, Z. 2026a. Specbench: Measuring reward hacking in long-horizon cod- ing agents. arXiv preprint arXiv:2605.21384. · arXiv
  46. Zhao, W.; Zhang, Y.; Wang, Y.; Deng, Y.; Zhao, Y.; Zhi, X.; Huang, Y.; He, H.; Che, W.; Qin, B.; and Liu, T. 2026b. On Safety Risks in Experience-Driven Self-Evolving Agents. In Findings of the Association for Computational Linguistics, 42145-42169.
  47. Zhou, X.; Liu, J.; Li, L.; Rossi, R. A.; and Hu, X. 2026. Counterfactual Trace Auditing of LLM Agent Skills. arXiv preprint arXiv:2605.11946. · arXiv
  48. Zhuge, M.; Wang, W.; Kirsch, L.; Faccio, F.; Khizbullin, D.; and Schmidhuber, J. 2024. GPTSwarm: Language Agents as Optimizable Graphs. In ICML, 62743-62767.

Cited by

Sorted by citations

Authors
Wang, X., Zhang, X., & Shao, J.
Affiliations
University of Electronic Science and Technology of China (CN)
ORCID
Shao, Jie
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2609.00069
arXiv
2609.00069
OpenAlex
W7206193997
Cited by
0
References
1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
Keywords
self-improving agents, human development, AI alignment, audit procedures, tamper localization, annotated corpus, integrity constraints, authorization, provenance, completeness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review