The DOI matches its Crossref record; 66 of the 66 references it cites were found in a registry or an index; 6 of the 57 sources its claims cite were checked for support.
Designing agroecological systems across scales: a new analytical framework
Open accessOverview
CoChat Summary
## TL;DR
The paper proposes an analytical framework for agroecological design as a non-linear, coevolving process involving researchers, non-scientific actors, a problem situation, and a design object. Using three long-term tropical horticulture case studies, it shows that design objects can transform horizontally, upward, or downward across organizational scales, with different implications for learning and adoption.
## Key findings
- The framework identifies four interacting components of agroecological design processes: a science consortium, actors in the field, a problem situation, and a design object.
- The paper distinguishes three types of object transformation: horizontal (same organizational scale), upward (relocating an object to a wider system), and downward (isolating a component for reuse in a new context).
- In the three case studies, object transformations were driven by changes in research team membership, non-scientific actors highlighting adoption obstacles, and changed perceptions of the problem situation.
- The case studies showed that design processes were long and unpredictable, with non-linear trajectories including “branching, and dead ends,” and that scale transformations could help overpass or bypass adoption breaks.
## Methods
- The study uses an ex-post reflexive analytical approach to analyze three researcher-led agroecological design processes in tropical horticulture.
- Data came from 3 workshops, individual interviews with lead researchers, and a literature review covering 2000–2020; the abstract also states that focus group discussions were used.
- Analysis was guided by the authors’ analytical framework and addressed the nature and transformation of design objects, the roles of researchers and actors, and how the four components coevolved over time.
## Limitations
- The paper explicitly notes that the case studies were documented by the co-authors themselves and analyzed through a reflexive approach, which can constrain independence of interpretation.
- It studied only three case studies from tropical horticulture and only researcher-led design processes, so broader generalization is limited.
- Evident from the text: the paper is qualitative and ex-post, so it does not provide quantitative causal tests of the framework’s effects.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures



















Verification
Claims the paper makes
6 Valid · 2 Not valid · 29 Not checked · 127 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Over the last few decades, scientific research in agroecology has focused on systems of increasing scale and size, from simple cropping techniques (e.g. pruning and organic fertilisation) to more complex objects such as cropping systems (Debaeke etal. 2009), innovative farms (Bos and Grin 2012; Cerf etal. 2012; Dogliotti etal. 2014), alternative food systems (Wezel etal. 2009), and multifunctional”
What the sources say
- Not matchedcites Debaeke etal. 2009
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Not matchedcites Bos and Grin 2012
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Supportedcites Van Berkel and Verburg 2012 source
The citation appears to identify an example of research on multifunctional landscapes, rather than support every step in the sentence’s list. The Introduction describes rural landscapes’ multifunctional character, and the Conclusions describe this paper’s use of an agent-based model to plan for landscape functionality. Our copy provides no page numbers for these passages. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All three design processes were aimed at reducing the use of agrochemicals in horticultural systems.”
plainn = 3- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For their part, some farmers redesigned their entire cropping systems by gradually adding simple changes or by a direct change of system (Fig.3c, step B).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the most recent period, researchers have been trying to bypass obstacles to the adoption of the ant method by isolating repellent compounds from ants to produce a repellent spray against fruit flies, so taking advantage of the ants' repellent power without using them in the orchards (Fig.3a, step C).”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of this multi-site experiment were encouraging, but farmers pointed out the amount of work required to install and remove the nets every day.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
80 more claims
Create a free account to see all 164 claims and every source verdict.
- Probably not a substantial claim. 58
- It may state two separate claims as one. 39
- The quote only partly states this claim. 1
Citations
Cites
Food in the Anthropocene: the EAT–Lancet Commission on healthy diets from sustainable food systems
Willett, W., Rockström, J., Loken, B., Springmann, M., Lang, T., Vermeulen, S., Garnett, T., Tilman, D., DeClerck, F., Wood, A., et al. (2019) · The Lancet · doi:10.1016/s0140-6736(18)31788-4 · Cited by 10,880
The DSSAT cropping system model
Jones, J. W., Hoogenboom, G., Porter, C. H., Boote, K. J., Batchelor, W. D., Hunt, L. A., Wilkens, P. W., Singh, U., Gijsman, A. J., & Ritchie, J. T. (2003) · European Journal of Agronomy · doi:10.1016/s1161-0301(02)00107-7 · Cited by 4,465
Creativity in the design process: co-evolution of problem–solution
Dorst, K., & Cross, N. (2001) · Design Studies · doi:10.1016/s0142-694x(01)00009-6 · Cited by 2,357
Agroecology as a science, a movement and a practice. A review
Wezel, A., Bellon, S., Doré, T., Francis, C., Vallod, D., & David, C. (2009) · Agronomy for Sustainable Development · doi:10.1051/agro/2009004 · Cited by 1,504
Agroecology and the design of climate change-resilient farming systems
Altieri, M. A., Nicholls, C. I., Henao, A., & Lana, M. A. (2015) · Agronomy for Sustainable Development · doi:10.1007/s13593-015-0285-2 · Cited by 1,472
Participatory Action Research Approaches and Methods
(2007) · Routledge eBooks · doi:10.4324/9780203933671 · Cited by 1,239
Agroecology: The Ecology of Food Systems
Francis, C., Lieblein, G., Gliessman, S., Breland, T. A., Creamer, N., Harwood, R., Salomonsson, L., Helenius, J., Rickerl, D., Salvador, R., et al. (2003) · Journal of Sustainable Agriculture · doi:10.1300/j064v22n03_10 · Cited by 975
Sharrock, S. L. (2016) · CGSPace A Repository of Agricultural Research Outputs (Consultative Group for International Agricultural Research) · Cited by 779
Pohl, C., Rist, S., Zimmermann, A., Fry, P., Gurung, G. S., Schneider, F., Speranza, C. I., Kiteme, B., Boillat, S., Serrano, E., et al. (2010) · Science and Public Policy · doi:10.3152/030234210x496628 · Cited by 555
Designing agroecological transitions; A review
Duru, M., Therond, O., & Fares, M. (2015) · Agronomy for Sustainable Development · doi:10.1007/s13593-015-0318-x · Cited by 542
Transforming food systems with agroecology
Gliessman, S. (2016) · Agroecology and Sustainable Food Systems · doi:10.1080/21683565.2015.1130765 · Cited by 526
Doré, T., Makowski, D., Malézieux, E., Munier-Jolain, N., Tchamitchian, M., & Tittonell, P. (2011) · European Journal of Agronomy · doi:10.1016/j.eja.2011.02.006 · Cited by 474
Green Pesticides: Essential Oils as Biopesticides in Insect-pest Management
Mossa, A.-T. H. (2016) · Journal of Environmental Science and Technology · doi:10.3923/jest.2016.354.378 · Cited by 454
Agroecology as a Transdisciplinary, Participatory, and Action-Oriented Approach
Méndez, V. E., Bacon, C. M., & Cohen, R. (2013) · Agroecology and Sustainable Food Systems · doi:10.1080/10440046.2012.736926 · Cited by 438
Pigford, A.-A. E., Hickey, G. M., & Klerkx, L. (2018) · Agricultural Systems · doi:10.1016/j.agsy.2018.04.007 · Cited by 434
Designing coupled innovations for the sustainability transition of agrifood systems
Meynard, J.-M., Jeuffroy, M.-H., Le Bail, M., Lefèvre, A., Magrini, M.-B., & Michon, C. (2017) · Agricultural Systems · doi:10.1016/j.agsy.2016.08.002 · Cited by 329
Designing cropping systems from nature
Malézieux, E. (2012) · Agronomy for Sustainable Development · doi:10.1007/s13593-011-0027-z · Cited by 295
Self-citationGliessman, S. (2018) · Agroecology and Sustainable Food Systems · doi:10.1080/21683565.2018.1432329 · Cited by 290
Blay-Palmer, A., Santini, G., Dubbeling, M., Renting, H., Taguchi, M., & Giordano, T. (2018) · Sustainability · doi:10.3390/su10051680 · Cited by 253
Le Gal, P.-Y., Dugué, P., Faure, G., & Novak, S. (2011) · Agricultural Systems · doi:10.1016/j.agsy.2011.07.007 · Cited by 230
de Bon, H., Huat, J., Parrot, L., Sinzogan, A., Martin, T., Malézieux, E., & Vayssières, J.-F. (2014) · Agronomy for Sustainable Development · doi:10.1007/s13593-014-0216-7 · Cited by 226
Self-citationAnchoring of innovations: Assessing Dutch efforts to harvest energy from glasshouses
Elzen, B., van Mierlo, B., & Leeuwis, C. (2012) · Environmental Innovation and Societal Transitions · doi:10.1016/j.eist.2012.10.006 · Cited by 221
Jakku, E., & Thorburn, P. J. (2010) · Agricultural Systems · doi:10.1016/j.agsy.2010.08.007 · Cited by 204
Participatory design of agricultural decision support tools: taking account of the use situations
Cerf, M., Jeuffroy, M., Prost, L., & Meynard, J.-M. (2012) · Agronomy for Sustainable Development · doi:10.1007/s13593-012-0091-z · Cited by 188
Integrating Agroecology and Participatory Action Research (PAR): Lessons from Central America
Méndez, V., Caswell, M., Gliessman, S., & Cohen, R. (2017) · Sustainability · doi:10.3390/su9050705 · Cited by 188
Re-design and co-design of farming systems. An overview of methods and practices
Meynard, J.-M., Dedieu, B., & Bos, A. P. (2012) · Farming Systems Research into the 21st Century: The New Dynamic · doi:10.1007/978-94-007-4503-2_18 · Cited by 187
Berthet, E. T., Hickey, G. M., & Klerkx, L. (2018) · Agricultural Systems · doi:10.1016/j.agsy.2018.06.004 · Cited by 163
Mixing Plant Species in Cropping Systems: Concepts, Tools and Models: A Review
Malézieux, E., Crozat, Y., Dupraz, C., Laurans, M., Makowski, D., Ozier-Lafontaine, H., Rapidel, B., de Tourdonnet, S., & Valantin-Morison, M. (2009) · Sustainable Agriculture · doi:10.1007/978-90-481-2666-8_22 · Cited by 161
Self-citationHow to foster agroecological innovations? A comparison of participatory design methods
Berthet, E. T. A., Barnaud, C., Girard, N., Labatut, J., & Martin, G. (2016) · Journal of Environmental Planning and Management · doi:10.1080/09640568.2015.1009627 · Cited by 159
Scale dynamics of grassroots innovations through parallel pathways of transformative change
Hermans, F., Roep, D., & Klerkx, L. (2016) · Ecological Economics · doi:10.1016/j.ecolecon.2016.07.011 · Cited by 155
The influence of landscape on insect pest dynamics: a case study in southeastern France
Ricci, B., Franck, P., Toubon, J., Bouvier, J., Sauphanor, B., & Lavigne, C. (2008) · Landscape Ecology · doi:10.1007/s10980-008-9308-6 · Cited by 145
Innovative design for agriculture in the move towards sustainability: scientific challenges
Prost, L., Berthet, E. T. A., Cerf, M., Jeuffroy, M., Labatut, J., & Meynard, J.-M. (2016) · Research in Engineering Design · doi:10.1007/s00163-016-0233-4 · Cited by 144
Ollivier, G., Magda, D., Mazé, A., Plumecocq, G., & Lamine, C. (2018) · Ecology and Society · doi:10.5751/es-09952-230205 · Cited by 144
Cuéllar-Padilla, M., & Collado, Á. C. (2011) · Journal of Rural Studies · doi:10.1016/j.jrurstud.2011.08.004 · Cited by 141
Fruit fly management research: A systematic review of monitoring and control tactics in the world
Dias, N. P., Zotti, M. J., Montoya, P., et al. (2018) · Crop Protection · doi:10.1016/j.cropro.2018.05.019 · Cited by 132
ROTOR, a tool for generating and evaluating crop rotations for organic farming systems
Bachinger, J., & Zander, P. (2007) · European Journal of Agronomy · doi:10.1016/j.eja.2006.09.002 · Cited by 131
Hay, K. E., & Barab, S. A. (2001) · Journal of the Learning Sciences · doi:10.1207/s15327809jls1003_3 · Cited by 131
A modelling framework to support farmers in designing agricultural production systems
Gal, P. ‐. L., Mérot, A., Moulin, C. H., Navarrete, M., & Wéry, J. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2008.12.013 · Cited by 123
Debaeke, P., Munier-Jolain, N. M., Bertrand, M., Guichard, L., Nolot, J., Faloya, V., & Saulas, P. (2008) · Agronomy for Sustainable Development · doi:10.1051/agro:2008050 · Cited by 121
Klerkx, L., van Bommel, S., Bos, A. P., Holster, H. C., Zwartkruis, J. V., & Aarts, N. (2012) · Agricultural Systems · doi:10.1016/j.agsy.2012.07.006 · Cited by 110
Companion modelling: a participatory approach to support sustainable development.
Étienne, M. (2010) · doi:10.1007/978-94-017-8557-0 · Cited by 99
Van Berkel, D. B., & Verburg, P. H. (2012) · Landscape Ecology · doi:10.1007/s10980-012-9730-7 · Cited by 96
Towards an ontology of design: lessons from C–K design theory and Forcing
Hatchuel, A., Weil, B., & Masson, P. L. (2012) · Research in Engineering Design · doi:10.1007/s00163-012-0144-y · Cited by 95
A theoretical framework for tracking farmers’ innovations to support farming system design
Salembier, C., Segrestin, B., Weil, B., Jeuffroy, M., Cadoux, S., Cros, C., Favrelière, E., Fontaine, L., Gimaret, M., Noilhan, C., et al. (2021) · Agronomy for Sustainable Development · doi:10.1007/s13593-021-00713-z · Cited by 88
Genealogy of design reasoning in agronomy: Lessons for supporting the design of agricultural systems
Salembier, C., Segrestin, B., Berthet, E. T. A., Weil, B., & Meynard, J.-M. (2018) · Agricultural Systems · doi:10.1016/j.agsy.2018.05.005 · Cited by 83
Bellec, F. L., Rajaud, A., Ozier‐Lafontaine, H., Bockstaller, C., & Malézieux, É. (2011) · Agronomy for Sustainable Development · doi:10.1007/s13593-011-0070-9 · Cited by 78
Self-citationPest management: Reconciling farming practices and natural regulations
Brévault, T., & Clouvel, P. (2018) · Crop Protection · doi:10.1016/j.cropro.2018.09.003 · Cited by 69
A holon approach to agroecology
Bland, W. L., & Bell, M. M. (2007) · International Journal of Agricultural Sustainability · doi:10.1080/14735903.2007.9684828 · Cited by 58
From Industrial Agriculture to Diversified Agroecological Systems
Frison, E. (2016) · Indian Journal of Plant Genetic Resources · doi:10.5958/0976-1926.2016.00033.4 · Cited by 52
Gourmelon, F., Chlous, F., Kerbiriou, C., Rouan, M., & Bioret, F. (2012) · Land Use Policy · doi:10.1016/j.landusepol.2012.10.015 · Cited by 49
Designing a research agenda for coupled innovation towards sustainable agrifood systems
Brun, J., Jeuffroy, M., Pénicaud, C., Cerf, M., & Meynard, J.-M. (2021) · Agricultural Systems · doi:10.1016/j.agsy.2021.103143 · Cited by 43
Landscape-scale approaches for enhancing biological pest control in agricultural systems
Petit, S., Deytieux, V., & Cordeau, S. (2021) · Environmental Monitoring and Assessment · doi:10.1007/s10661-020-08812-2 · Cited by 42
Gliessman, S. (2015) · Agroecology and Sustainable Food Systems · doi:10.1080/21683565.2014.965869 · Cited by 39
Soti, V., Thiaw, I., Debaly, Z. M., Sow, A., Diaw, M., Souleymane, F., Diakhate, M., Thiaw, C., & Brévault, T. (2018) · Biological Control · doi:10.1016/j.biocontrol.2018.10.006 · Cited by 38
Brévault, T., Renou, A., Vayssières, J., Amadji, G. L., Assogba-Komlan, F., Diallo, M. D., Bon, H. D., Diarra, K., Hamadoun, A., Huat, J., et al. (2014) · Crop Protection · doi:10.1016/j.cropro.2014.08.017 · Cited by 37
Self-citationMarín, A., Ely, A., & van Zwanenberg, P. (2016) · Current Opinion in Environmental Sustainability · doi:10.1016/j.cosust.2016.09.003 · Cited by 35
Design as a source of renewal in the production of scientific knowledge in crop science
Toffolini, Q., Jeuffroy, M., Meynard, J.-M., Borg, J., Enjalbert, J., Gauffreteau, A., Goldringer, I., Lefèvre, A., Loyce, C., Martin, P., et al. (2020) · Agricultural Systems · doi:10.1016/j.agsy.2020.102939 · Cited by 33
Deffontaines, L., Mottes, C., Della Rossa, P., Lesueur-Jannoyer, M., Cattan, P., & Le Bail, M. (2020) · Agricultural Systems · doi:10.1016/j.agsy.2019.102769 · Cited by 32
Self-citationElSawah, S., Mclucas, A., & Mazanov, J. (2013) · Water Resources Management · doi:10.1007/s11269-013-0357-5 · Cited by 32
Design multiple: How different configurations of participation matter in design practice
Fischer, B., Östlund, B., & Peine, A. (2021) · Design Studies · doi:10.1016/j.destud.2021.101016 · Cited by 31
Designing sustainable agricultural production systems for a changing world: Methods and applications
Dogliotti, S., Rodrı́guez, D., López‐Ridaura, S., Tittonell, P. A., & Rossing, W. A. H. (2014) · Agricultural Systems · doi:10.1016/j.agsy.2014.02.003 · Cited by 30
Colbach, N., Colas, F., Pointurier, O., et al. (2017) · European Journal of Agronomy · doi:10.1016/j.eja.2017.04.005 · Cited by 27
Duru, M. (2013) · Agricultural Systems · doi:10.1016/j.agsy.2012.09.002 · Cited by 24
Agricultural systems and the innovation process
Meynard, J.-M., & Casabianca, F. (2012) · Wageningen Academic Publishers eBooks · doi:10.3920/978-90-8686-726-4_1 · Cited by 6
Berthet, E. T. A. (2013) · theses.fr (ABES) · doi:10.70675/228133e9z9876z48e6zb46fz25f110c501b8 · Cited by 6
Resource over-exploitation and running out
Malézieux, É., & Dabbadie, L. (2019) · CIRAD; FAO eBooks · doi:10.19182/agritrop/00092 · Cited by 3
Research works printed but not linked: 5
- Bos AP, Grin J (2012) Reflexive interactive design as an instrument for dual track governance. In: System Innovations, Knowledge Regimes, and Design Practices towards Transitions for Sustainable Agriculture. INRA, Paris, pp 132–153 https://edepot.wur.nl/242654
- Cerdan C, Bienabe E, David-Benz H, et al (2019) What market dynamics for promoting an agroecological transition? In: The agroecological transition of agricultural systems in the Global South / Cote Francois-Xavier (ed.), Poirier-Magona Emmanuelle (ed.), Perret Sylvain (ed.), Roudier Philippe (ed.), Bruno Rapidel (ed.), Thirion Marie-Cecile (ed.). Ed. Quae, Versailles, pp 271–291 https://agritrop.cirad.fr/593022/1/ID593022.pdf
- Della Rossa P (2020) Conception collective d’organisations territoriales innovantes pour une évolution coordonnée de systèmes de production agricoles. Thesis, Université Paris-Saclay, Cas d’une réduction de la pollution herbicide d’une rivière en Martinique https://www.theses.fr/2020UPASA005.pdf
- Hatchuel A, Weil B (2003) A new approach of innovative design: an introduction to CK theory. In: DS 31: Proceedings of ICED 03, the 14th International Conference on Engineering Design, Stockholm file:///Users/belmin/Downloads/DS31_1794FPC.pdf
- Le Masson P, Weil B, Hatchuel A (2006) Les processus d’innovation, conception innovante et croissance des entreprises. Hermès Lavoisier, Paris
Cited by
Agroecological crop protection for sustainable agriculture
Deguine, J., Aubertot, J., Bellon, S., Côté, F., Lauri, P., Lescourret, F., Ratnadass, A., Scopel, É., Andrieu, N., Bàrberi, P., et al. (2023) · Advances in agronomy · doi:10.1016/bs.agron.2022.11.002 · Cited by 198
Zeng, S., Li, J., & Wanger, T. C. (2023) · iScience · doi:10.1016/j.isci.2023.107510 · Cited by 21
Schaffer, C., Elbakidze, M., & Björklund, J. (2024) · Agroforestry Systems · doi:10.1007/s10457-024-00964-1 · Cited by 23
Boulestreau, Y., Casagrande, M., & Navarrete, M. (2023) · Agricultural Systems · doi:10.1016/j.agsy.2023.103752 · Cited by 31
Kirui, E. C., Kidoido, M. M., Akutse, K. S., Wanyama, R., Boni, S. B., Dubois, T., Dinssa, F. F., & Mutyambai, D. M. (2025) · Sustainability · doi:10.3390/su17031148 · Cited by 9
Metadata
- Authors
- Belmin, R., Malézieux, E., Basset-Mens, C., Martin, T., Mottes, C., Della Rossa, P., Vayssières, J.-F., & Le Bellec, F.
- Affiliations
- Centre de Coopération Internationale en Recherche Agronomique pour le Développement (FR)
- Corresponding author
- Belmin, Raphaël
- Venue
- Agronomy for Sustainable Development
- Published
- 2022-01-11
- Volume
- 42
- Issue
- 1
- ISSN
- 1773-0155 (ISSN-L)
- 1774-0746
- OpenAlex
- W4205792888
- Semantic Scholar
- b734983646ae48096085eb60a69767ea6c6fa06e
- Cited by
- 16
- References
- 64
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 3 · 2023: 3
- FWCI
- 3.36
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Agronomy for Sustainable Development (publishedVersion)
- HAL (Le Centre pour la Communication Scientifique Directe) (submittedVersion)
- Agritrop (Cirad) (submittedVersion)
- Topics
- Social Sciences › Development › Agriculture, Land Use, Rural Development
- Life Sciences › Agricultural and Biological Sciences › General Agricultural and Biological Sciences › Agricultural Innovations and Practices
- Physical Sciences › Environmental Science › Global and Planetary Change › Sustainability and Climate Change Governance
- Keywords
- scale transformation, design process, analytical framework, knowledge flows, problem situations, design objects, coevolution, tropical horticulture, agroecological innovation, multifunctional landscapes, alternative food systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.