CoChat
Sign Up
PreprintCited by 60Evolutionary Algorithms and ApplicationsAI-based Problem Solving and PlanningArtificial Intelligence Applications

EvoFlow: Evolving Diverse Agentic Workflows On The Fly

Open access
Zhang, Guibin, Chen, Kaijie, Wan, Guancheng, Chang, Heng, Cheng, Hong, Wang, Kun, Hu, Shuyue, Bai, Lei·2025·arXiv
DOI 10.48550/arxiv.2502.07373arXiv 2502.07373
Open this paper in CoChat — free accountOpen PDF

Abstract

The past two years have witnessed the evolution of large language model (LLM)-based multi-agent systems from labor-intensive manual design to partial automation (\textit{e.g.}, prompt engineering, communication topology) and eventually to fully automated design. However, existing agentic automation pipelines often lack LLM heterogeneity and focus on single-objective performance optimization, limiting their potential to combine weaker models for more customized and cost-effective solutions. To address this challenge, we propose EvoFlow, a niching evolutionary algorithm-based framework to automatically search a population of heterogeneous and complexity-adaptive agentic workflows, rather than a single homogeneous, complex workflow. Technically, EvoFlow performs \textit{(1) tag-based retrieval} to extract parent workflows from an agentic population, evolves new workflows through \textit{(2) crossover} and \textit{(3) mutation}, and employs \textit{(4) niching-based selection} to maintain population diversity and quality. Extensive evaluations across seven benchmarks demonstrate that EvoFlow is: \textbf{(I) diverse}, evolving a population of workflows ranging from simple I/O tasks to complex multi-turn interactions; \textbf{(II) high-performing}, outperforming previous handcrafted and automated workflows by $1.23\%\sim29.86\%$; \textbf{(III) economical}, surpassing powerful \llmname{o1-preview} at $12.4\%$ of its inference cost using weaker open-source models.

AI snapshot

Written from the paper’s full text.

TL;DR

EvoFlow is a niching evolutionary algorithm framework that automatically searches a population of heterogeneous, complexity-adaptive LLM agentic workflows by optimizing cost and performance as a multi-objective problem. Across math, code, and embodied benchmarks, it reports higher performance and lower cost than prior handcrafted and automated workflows.

Key findings

  • In homogeneous settings, EvoFlow outperforms handcrafted and automated workflows across six benchmarks; on MATH it exceeds vanilla gpt-4o-mini by 11.41% and surpasses SOTA baseline AFlow by 6.42%, and on ALFWorld it achieves 68.57%, outperforming second-best AgentSquare by 2.15%.
  • In heterogeneous settings using four open-source LLMs, EvoFlow surpasses o1-preview by 2.7% on MATH while its overall cost is 12.4% of o1-preview’s; its training cost is $0.45 vs AFlow’s $1.23 and inference cost $0.51 vs $2.62, while surpassing AFlow by 5.91% on MATH.
  • Cross-domain training on MATH+MBPP improves EvoFlow’s MBPP performance from 87.62% to 88.35%, whereas baselines such as GPTSwarm, DyLAN, and AFlow suffer negative gains under joint optimization.
  • Ablations show removing operator mutation drops performance by 3.5%–7.3%; increasing population size from N=5 to N=25 improves performance by 3.1% but raises per-query cost from 8e-4 to 2e-3.

Methods

  • Design: EvoFlow frames agentic workflow search as cost-performance multi-objective optimization, using operator nodes as search units; it initializes a workflow population, then iteratively applies tag-based retrieval, LLM-facilitated crossover, LLM/prompt/operator mutation, and niching-based selection to maintain diversity and quality.
  • Data and sample: evaluations use six public benchmarks covering math reasoning (GSM8K, MATH, MultiArith), code generation (HumanEval, MBPP), and embodied tasks (ALFWorld); MATH uses a harder subset of 617 problems, and datasets use a 1:4 train:test split except ALFWorld. Baselines include manual workflows (e.g., CoT, Self-Consistency, LLM-Debate, DyLAN, AgentVerse, MacNet) and autonomous workflows (e.g., GPTSwarm, AutoAgents, ADAS, AgentSquare, AFlow).
  • Main analysis: experiments compare homogeneous, heterogeneous, and cross-domain settings using gpt-4o-mini-0718, llama-3.1-70b, Qwen-2-72b, Deepseek-V2.5, and Hermes-3-70b; parameters include K=3 parent workflows, κ=5 tags, population size N=15, and E=5 in niching selection.

Limitations

  • The paper does not include a dedicated limitations section; its impact statement says the method poses no ethical concerns in motivation, design, experiments, or data usage.
  • Evident from the text: evaluation is limited to six public benchmarks in Section 5.1 and Table 5, although the abstract and contributions refer to seven benchmarks; results are tied to specific LLM backbones and API costs, with no human evaluation or real-world deployment reported.
  • Evident from the text: the framework depends on LLM-powered tag generation, crossover, and mutation prompts, and while ablations show sensitivity to removing components, prompt robustness and broader generalization are not fully characterized.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2502.07373

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

13 of 39 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 3. Preliminary · p. 3

    Existing methods typically perform single-objective optimization.

    “existing methods (Zhang et al., 2024c; Shang et al., 2024; Zhuge et al., 2024) typically perform single-objective optimization.”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Zhang et al., 2024c

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shang et al., 2024 source

      Section 3.1 (page unavailable) formulates AgentSquare's search as maximizing a single measure, agent performance. The paper describes several existing optimization methods, but the passages do not establish that existing methods typically use a single objective. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
    • cites Zhuge et al., 2024 source
  • Premisecitation
    § 1. Introduction · p. 2

    Existing automated agentic workflows typically result in a singular, complex workflow incorporating elements like CoT ensembles and multi-turn discussions.

    “This typically results in a singular, complex workflow incorporating elements like CoT ensembles and multi-turn discussions”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Hu et al., 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shang et al., 2024 source

      The Additional information section describes combining multiple reasoning techniques into a single module, including generating and voting on multiple answers. But neither that passage nor the Workflow Overview (§2.2) says automated agentic workflows typically produce one complex workflow or identifies multi-turn discussions as a typical component. Page numbers are unavailable in our copy. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 1. Introduction · p. 2

    Existing automated agentic workflows often prioritize single-objective optimization, focusing on performance or solution quality.

    “Existing automated agentic workflows often prioritize single-objective optimization, focusing on performance or solution quality.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Hu et al., 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shang et al., 2024 source

      The Introduction describes earlier work that optimizes agent prompts or entire agentic systems, and the discussion of search baselines evaluates those methods by performance improvement (pp. not available in our copy). The paper does not establish that existing automated agentic workflows generally prioritize a single objective, so that broader characterization is unsupported. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Premisecitation
    § Abstract · p. 1

    Existing agentic automation pipelines focus on single-objective performance optimization.

    “focus on single-objective performance optimization”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Hu et al., 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shang et al., 2024 source

      Section 5.2 describes existing approaches that optimize agent prompts or search agent designs, and the comparison of search methods discusses their performance improvements. The paper does not characterize these approaches as single-objective or say they focus exclusively on performance optimization. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 4.3. Niching-based Selection · p. 5

    Running ADAS on ARC benchmark with gpt-3.5-turbo-0125 incurs cost up to $300 USD.

    “running ADAS (Hu et al., 2024b) on ARC benchmark (Chollet, 2019) with gpt-3.5-turbo-0125 incurs cost up to $300 USD.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Hu et al., 2024b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Chollet, 2019 source
  • Classified as probably not a substantial claim. 14
  • May combine two claims. 22
  • The quote only partly states this claim. 1

  • Classified as probably not a substantial claim.

    Premisecitation
    § 2. Related Work · p. 2

    Interaction among multiple LLM-based agents can substantially enhance individual model capabilities.

    “interaction among multiple LLM-based agents can substantially enhance individual model capabilities”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Wang et al., 2024 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1. Introduction · p. 1

    Manually crafted artifacts are often replaced by learnable and optimizable ones.

    “manually crafted artifacts are often replaced by learnable and optimizable ones”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites Tyson & Zysman, 2022 source

      The discussion on p. 257 describes machine learning and AI automating tasks, while the References explain that algorithms can improve through training. Neither passage addresses whether manually crafted artifacts are often replaced by learnable, optimizable ones; searches for that phrase and its key terms found no passage that does. (LLM, conf=0.72)

      Experimental — accuracy not yet measured
    • cites Clune, 2019 source

  • May combine two claims.

    Premisecitation
    § 1. Introduction · p. 2

    Complex workflows are essential for difficult queries, while simpler queries can be efficiently addressed by single-agent I/O.

    “complex workflows are essential for the latter, simpler queries can be efficiently addressed by single-agent I/O”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Feng et al., 2024 source

      The Conclusion and Discussion (section 6; page unavailable) says that Chain-of-Thought and Tree-of-Thought prompting are widely used to enhance reasoning on complex queries. It does not say complex workflows are essential, or that simpler queries can be efficiently handled by single-agent I/O. The paper presents multi-agent LLM selection as future work, not as a tested comparison with single-agent I/O. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 2. Related Work · p. 3

    EvoAgent and EvoPrompt employ simple genetic algorithms to optimize agent profiles and prompts, with a highly constrained level of automation, focusing solely on single-agent prompt optimization and failing to evolve at the workflow level.

    “EvoAgent (Yuan et al., 2024) and EvoPrompt (Guo et al., 2023) employ simple genetic algorithms to optimize agent profiles and prompts, whose, however, level of automation is highly constrained, focusing solely on single-agent prompt optimization and failing to evolve at the workflow level”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Yuan et al., 2024 source

      The Introduction and §3.2 describe evolutionary operators that update agent settings, including roles, skills, and prompts, so the evolutionary-optimization part is supported. But they describe automatically generating multiple specialized agents that collaborate, not focusing solely on single-agent prompt optimization. The paper does not establish the claim's characterization of EvoAgent's automation as highly constrained; it says generation requires no additional human design. The cited passages do not show workflow-level evolution. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites Guo et al., 2023 source

      Sections 3.1 and 3.2 describe EvoPrompt evolving prompts with a genetic algorithm, which supports the prompt-optimization part of the claim. The paper also implements differential evolution (Section 3.3). It does not assess EvoPrompt’s level of automation or explicitly establish a failure to evolve workflows; those characterizations go beyond what this source reports. Page numbers are not available in our copy. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 3. Preliminary · p. 3

    Prior agentic automation pipelines have generally excluded M from their search space, often preselecting a single LLM M to instantiate all agents, which constrains the development of more diverse and capability-rich agentic systems.

    “prior agentic automation pipelines have generally excluded M from their search space (Zhang et al., 2024c; Yuan et al., 2024; Zhuge et al., 2024), often preselecting a single LLM M to instantiate all agents, which constrains the development of more diverse and capability-rich agentic systems.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Zhang et al., 2024c

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Yuan et al., 2024 source

      The Ablation Studies section says EVOAGENT uses an LLM with the same backbone as its initial agent, and the NLP and Multi-Modal Tasks section describes selecting backbone models for experiments. The paper also criticizes prior frameworks for relying on handcrafted agent settings (Agent Generation), but it does not establish that prior pipelines generally exclude LLM choice from their search space or that this specifically limits model diversity and capability. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites Zhuge et al., 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims. The quote only partly states this claim.

    Findingown data
    § 5.2. Performance Analysis · p. 7

    EvoFlow achieves an average score of 82.55 across GSM8K, MATH, MultiArith, HumanEval, MBPP, and ALFWorld, outperforming all listed baselines.

    “| EvoFlow (Ours) | 92 . 90 ↑ 4 . 85 | 57 . 70 ↑ 11 . 41 | 98 . 80 ↑ 1 . 95 | 92 . 85 ↑ 5 . 77 | 84 . 50 ↑ 10 . 34 | 68 . 57 ↑ 29 . 86 | 82 . 55 ↑ 11 . 18 |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3. Cost Analysis & Case Study · p. 7

    Optimizing AFlowQwen on MATH incurs a training cost of 1.22$ and an inference cost of 2.62$.

    “optimizing AFlowQwen on MATH incurs a training cost of 1 . 22$ and an inference cost of 2 . 62$”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.2. Performance Analysis · p. 7

    Even with AFlow optimization, qwen-2.5-72b only shows a 2.58% improvement on MATH, still trailing o1-preview by 3.82%.

    “even with the sophisticated optimization from AFlow, it only shows a 2 . 58% improvement on MATH, still trailing behind the powerful o1-preview by 3 . 82% ↓”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3. Cost Analysis & Case Study · p. 8

    The most high-performing workflows include iterative generation and ensembling, with accuracy 72.57% and cost 0.0037$.

    “The most high-performing workflows include iterative generation and ensembling, but at the cost of higher per-query token consumption (accuracy@ 72 . 57% , cost@ 0 . 0037$ )”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.3. Cost Analysis & Case Study · p. 8

    The population begins with simple and inexpensive workflows, consisting solely of basic I/O and self-refine, with accuracy 38.7% and cost 0.00018$.

    “The population begins with simple and inexpensive workflows, consisting solely of basic I/O and self-refine (accuracy@ 38 . 7% , cost@ 0 . 00018$ )”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning
    § 5.3. Cost Analysis & Case Study · p. 7

    EvoFlow requires only 37.5% of the training cost and 19.5% of the inference cost compared to AFlowQwen on MATH.

    “EvoFlow requires only 37 . 5% of the training cost and 19 . 5% of the inference cost”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 6. Conclusion · p. 8

    Extensive experiments across six benchmarks demonstrate the superior performance of EvoFlow with significantly lower token costs.

    “Extensive experiments across six benchmarks demonstrate the superior performance of EvoFlow with significantly lower token costs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 5.4. Framework Analysis · p. 8

    Removing tag-based retrieval and LLM mutation consistently leads to performance degradation and greater variance.

    “removing tag-based retrieval and LLM mutation consistently leads to performance degradation and greater variance”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.4. Framework Analysis · p. 8

    A larger population increases complexity, with the per-query cost rising from 8e-4 to 2e-3.

    “a larger population also increases complexity, with the perquery cost rising from 8 e -4 to 2 e -3”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.4. Framework Analysis · p. 8

    Increasing the population size N consistently improves performance, with a gain of 3.1% from N=5 to N=25.

    “increasing the population size N consistently improves performance, with a gain of 3 . 1% from N = 5 to N = 25”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § Abstract · p. 1

    The lack of LLM heterogeneity and single-objective optimization limits the potential of existing pipelines to combine weaker models for more customized and cost-effective solutions.

    “limiting their potential to combine weaker models for more customized and cost-effective solutions”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 5.3. Cost Analysis & Case Study · p. 7

    EvoFlow's workflows do not continuously rely on the most expensive qwen-2.5-72b, and instead opt for more economical models such as llama-3.1 or herme-3-70b when appropriate, without compromising performance.

    “EvoFlow 's workflows do not continuously rely on the most expensive qwen-2.5-72b , and instead opt for more economical models such as llama-3.1 or herme-3-70b when appropriate, without compromising performance.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.2. Performance Analysis · p. 7

    Optimizing LLM-heterogeneous workflows is necessary and has potential.

    “This clearly illustrates both the necessity and potential of optimizing LLM-heterogeneous workflows.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given
    § 2. Related Work · p. 3

    Existing automation pipelines are predominantly homogeneous, utilizing a single-source LLM, and lack the integration of heterogeneous LLM agents of varying sizes and sources.

    “these automation pipelines are predominantly homogeneous , i.e. , utilizing a single-source LLM, and lack the integration of heterogeneous LLM agents of varying sizes and sources.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 4.4. Discussion · p. 6

    The evolution process of EvoFlow gradually produces a Pareto set of agentic workflows with varying complexity and superior performance.

    “This iterative process gradually produces a Pareto set of agentic workflows with varying complexity and superior performance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 4.1. Population Initialization · p. 4

    EvoFlow's crossover and mutation can generate novel operators.

    “EvoFlow 's crossover and mutation can generate novel operators.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 1. Introduction · p. 1

    Agentic systems have experienced a rapid transition from manual to automated over the past two years.

    “agentic systems have experienced a rapid transition from manual to automated over the past two years”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § D. Operator Repository · p. 15

    The selection of operators in EvoFlow is highly customizable, allowing users the flexibility to incorporate their desired operators into the operator repository.

    “We respectfully note that the selection of these operators is highly customizable, allowing users the flexibility to incorporate their desired operators into the operator repository of EvoFlow .”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § Algorithm 1 Algorithm workflow of EvoFlow · p. 15

    Pareto optimal solutions ensure that the agents, composed of invoking nodes and operator nodes, operate efficiently while maintaining high utility for the target tasks.

    “These solutions ensure that the agents, composed of invoking nodes and operator nodes, operate efficiently while maintaining high utility for the target tasks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § D. Operator Repository · p. 15

    CoT reasoning enhances the LLM's capability to solve complex problems through intermediate reasoning steps, improving task handling and providing greater transparency in the decision-making process.

    “This approach enhances its capability to solve complex problems through intermediate reasoning steps, improving task handling and providing greater transparency in the decision-making process.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wei et al., 2022 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § G.2. Baseline Setups · p. 20

    AFlow utilizes both gpt-4o-mini and the advanced claude-3.5-sonnet.

    “In (Zhang et al., 2024c), AFlow utilizes both gpt-4o-mini and the advanced claude-3.5-sonnet .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zhang et al., 2024c

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 5.4. Framework Analysis · p. 8

    The performance degradation for too small K is likely because a small K reduces offspring diversity, while for large K it challenges the LLM's ability to aggregate multiple workflows.

    “likely because a small K reduces offspring diversity, while a large K challenges the LLM's ability to aggregate multiple workflows”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5.3. Cost Analysis & Case Study · p. 8

    EvoFlow's query-aware paradigm selects economical workflows for simple queries and sophisticated workflows for complex ones.

    “This highlights EvoFlow 's query-aware paradigm: for simple queries, it intelligently selects economical workflows for rapid completion, while for more complex ones, it leverages sophisticated workflows to address the increased demands.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § Algorithm 1 Algorithm workflow of EvoFlow · p. 15

    The Pareto set provides diverse design options, offering flexibility in adapting workflows to varying operational constraints and objectives.

    “Furthermore, the Pareto set provides diverse design options, offering flexibility in adapting workflows to varying operational constraints and objectives.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § D. Operator Repository · p. 15

    ReAct enables the agent to leverage versatile tools, including code interpreter, web searching, external knowledge database, etc., to handle diverse user demands.

    “Following (Yao et al., 2023b), this operator enables the agent to leverage versatile tools, including code interpreter, web searching, external knowledge database, etc. , to handle diverse user demands.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Yao et al., 2023b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § G.2. Baseline Setups · p. 20

    To ensure fairness in homogeneous settings, AFlow is limited to gpt-4o-mini and MAX ITERATION is set to 20.

    “To ensure fairness in homogeneous settings, we limit AFlow to gpt-4o-mini and set MAX ITERATION=20.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § D. Operator Repository · p. 15

    LLM-Debate allows multiple LLMs to debate, leveraging diverse perspectives to identify better solutions.

    “LLM-Debate (Du et al., 2023) allows multiple LLMs to debate, leveraging diverse perspectives to identify better solutions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Du et al., 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § D. Operator Repository · p. 15

    Focusing on foundational principles enhances the model's reasoning and delivers more accurate solutions.

    “By focusing on foundational principles, the model enhances its reasoning and delivers more accurate solutions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Zheng et al., 2023b

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

33 of 33

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • EvoFlow's MBPP overall cost does not add up

    Major

    The reported training and inference costs sum to 765.15, not the reported overall cost of 565.15, in the table's units.

    Reported 565.15 · recomputed 765.15

  • AFlow's MATH improvement does not match its scores

    Minor

    AFlow's 51.28 minus Vanilla's 46.29 is 4.99 points, not 3.31.

    Reported 3.31 · recomputed 4.99

  • Operator repository says seven types but enumerates eight

    Minor

    The repository introduces seven operator types, yet its numbered list runs through an eighth, ExpertPrompt.

  • EvoFlow's MBPP improvement is understated

    Minor

    EvoFlow's 84.50 minus Vanilla's 71.83 is 12.67 points, not the reported 10.34.

    Reported 10.34 · recomputed 12.67

  • AgentSquare's HumanEval improvement does not match its scores

    Minor

    AgentSquare's 89.08 minus Vanilla's 87.08 is 2.00 points, not 3.00.

    Reported 3 · recomputed 2

  • Benchmark count alternates between six and seven

    Minor

    The abstract says the evaluations cover seven benchmarks, while the experiment setup identifies six.

  • Homogeneous-baseline description conflicts with LLM-Blender setup

    Minor

    Table 2 describes a single base LLM for all baselines, but the specified LLM-Blender baseline uses Qwen and Llama as well as gpt-4o-mini.

  • EvoFlow's GSM8K improvement is understated

    Minor

    The table reports 92.90 for EvoFlow and 87.45 for Vanilla, a difference of 5.45 points rather than 4.85.

    Reported 4.85 · recomputed 5.45

8 of 8

Cites

Flagged first, then by citations

12 of the 89 references couldn’t be matched to a known paper.

Research works printed but not linked: 4

  1. Cetnarowicz, K., Kisiel-Dorohinicki, M., and Nawarecki, E. The application of evolution process in multi-agent world to the prediction system. In Proceedings of the Second International Conference on Multi-Agent Systems, ICMAS , volume 96, pp. 26-32, 1996.
  2. Chen, G., Dong, S., Shu, Y., Zhang, G., Sesay, J., Karlsson, B. F., Fu, J., and Shi, Y. Autoagents: A framework for automatic agent generation. arXiv preprint arXiv:2309.17288 , 2023b. · arXiv
  3. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR) , 2021a.
  4. Latin American Conference on Computational Intelligence (LA-CCI) , pp. 1-6. IEEE, 2023.

Cited by

Sorted by citations

Related

Authors
Zhang, G., Chen, K., Wan, G., Chang, H., Cheng, H., Wang, K., Hu, S., & Bai, L.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2025
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2502.07373
arXiv
2502.07373
OpenAlex
W4407424725
Cited by
0
References
89
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
multi-agent LLM systems, population diversity, crossover and mutation operators, inference cost reduction, open-source LLMs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-24.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-24.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review