Inclusive development and prioritization of review questions in a highly controversial field of regulatory science
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
How to best assess potential health, environmental and other impacts of genetically modified organisms (GMOs) and how to interpret the resulting evidence base have been long-standing controversial issues in the EU. As a response, transparency and inclusiveness became a major focus of regulatory science activities in the GMO impact area. Nevertheless, nearly three decades of controversies resulted in a heavily polarized policy environment, calling for further efforts. Against this backdrop the EU funded project GRACE explored the value of evidence synthesis approaches for GMO impact assessment and developed an evidence synthesis framework with a strong emphasis on openness, stakeholder engagement, transparency, and responsiveness to tackle regulatory science challenges. This framework was tested and implemented in the course of 14 systematic reviews or maps conducted on selected review questions spanning potential health, environmental, and socioeconomic impacts of GMOs. An inclusive development and prioritisation of review questions is of key importance in evidence synthesis as it helps to provide a better link between stakeholder demands and concerns and policy relevant outcomes. This paper, therefore, places a particular focus on the stakeholder involvement strategy developed and experiences gathered during this particular step in the course of the GRACE project. Based on this experience, possible lessons for future engagement exercises in highly controversial fields of regulatory science are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Consistent with prior research (Kohl et al. 2018; Burgard and Bittermann 2023; Schmidt et al. 2021), our study reveals that the majority of SLR tools still depend on possibly outdated methodologies. This includes the use of basic classifiers, which are no longer considered state-of-the-art for text and document classification.”
From Artificial intelligence for literature reviews: opportunities and challenges (2024) · cites this paper as Kohl et al. 2018
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Responsible research and innovation: From science in society to science for society, with society
Owen, R., Macnaghten, P., & Stilgoe, J. (2012) · Science and Public Policy · doi:10.1093/scipol/scs093 · Cited by 1,607
A Framework for Responsible Innovation
Owen, R., Stilgoe, J., Macnaghten, P., Gorman, M., Fisher, E., & Guston, D. (2013) · Responsible Innovation · doi:10.1002/9781118551424.ch2 · Cited by 730
Authority, E. G. F. T. C. O. S. R. E. F. S. (2010) · EFSA Journal · doi:10.2903/j.efsa.2010.1637 · Cited by 693
Séralini, G.-E., Clair, E., Mesnage, R., Gress, S., Defarge, N., Malatesta, M., Hennequin, D., & de Vendômois, J. S. (2014) · Environmental Sciences Europe · doi:10.1186/s12302-014-0014-5 · Cited by 299
Brooks, J., Waylen, K. A., & Mulder, M. B. (2013) · Environmental Evidence · doi:10.1186/2047-2382-2-2 · Cited by 252
Prospects for Technology Assessment in a Framework of Responsible Research and InnovationPreprint
von Schomberg, R. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2439112 · Cited by 142
Schmidt, J. E. U., Braun, C. U., Whitehouse, L. P., & Hilbeck, A. (2008) · Archives of Environmental Contamination and Toxicology · doi:10.1007/s00244-008-9191-9 · Cited by 111
Álvarez-Alfageme, F., Bigler, F., & Romeis, J. (2011) · Transgenic Research · doi:10.1007/s11248-010-9430-5 · Cited by 103
Is the German suspension of MON810 maize cultivation scientifically justified?
Ricroch, A., Bergé, J. B., & Kuntz, M. (2010) · Transgenic Research · doi:10.1007/s11248-009-9297-5 · Cited by 80
Clavisi, O., Bragge, P., Tavender, E. J., Turner, T., & Gruen, R. L. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2012.04.002 · Cited by 67
Research works printed but not linked: 6
- Kohl C, Frampton G, Damgaard CF, Wilhelm R, Schiemann J. Draft Guidelines for Good Review Practice in GMO Impact Assessment. GRACE Project Report. 2013. http://www.grace-fp7.eu/sites/default/files/DRAFT%20Info%20stakeholder%20SR_WP8_GA-final.pdf . Accessed 5 Jan 2018.
- Smets G, Spök A et al. Assessing the Evidence of Health, Environmental and Socio-Economic Impacts of GMO—GRACE Stakeholder Consultation on Good Review Practice in GMO Impact Assessment: Part 1: Overall Process and Review Questions1, GRACE Report. 2013. http://www.grace-fp7.eu/en/content/reports-study-plans-consultation-documents . Accessed 20 Dec 2016.
- Adduci G, Spök A. (eds). GRACE Stakeholder Consultation on Good Review Practice in GMO Impact Assessment: Part 2: Overall Process and Review Questions—Responses to Stakeholder Comments, GRACE Report. 2014. http://www.grace-fp7.eu/en/content/reports-study-plans-consultation-documents . Accessed 20 Dec 2016.
- Adduci G, Spök A. GRACE Stakeholder Consultation on Good Review Practice in GMO Impact Assessment: Part 3: Stakeholder Priorities for Review Questions, GRACE Report. 2014. http://www.grace-fp7.eu/en/content/reports-study-plans-consultation-documents . Accessed 20 Dec 2016.
- Adduci G, Spök A. GRACE Stakeholder Consultation on Good Review Practice in GMO Impact Assessment: Part 4: Draft Protocols for Systematic Reviews and Evidence Maps, GRACE Report. 2014. http://www.grace-fp7.eu/en/content/reports-study-plans-consultation-documents . Accessed 20 Dec 2016.
- GRACE. Presentations. GRACE Final Conference. 2015. http://www.grace-fp7.eu/sites/default/files/Presentations-GFC-Nov2015-final.pdf . Accessed 20 Dec 2016.
Cited by
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246
Estoque, R. C., Togawa, T., Ooba, M., Gomi, K., Nakamura, S., Hijioka, Y., & Kameyama, Y. (2018) · AMBIO · doi:10.1007/s13280-018-1090-3 · Cited by 194
Robledo, S., Grisales Aguirre, A. M., Hughes, M., & Eggers, F. (2023) · Journal of Small Business Management · doi:10.1080/00472778.2021.1955125 · Cited by 80
McIntosh, E. J., Chapman, S., Kearney, S. G., Williams, B. A., Althor, G., Thorn, J., Pressey, R. L., McKinnon, M., & Grenyer, R. (2018) · Environmental Evidence · doi:10.1186/s13750-018-0134-2 · Cited by 87
How does sediment exposure affect corals? A systematic review protocol
Tuttle, L. J., Johnson, C. E., Kolinski, S., Minton, D., & Donahue, M. J. (2020) · Environmental Evidence · doi:10.1186/s13750-020-00200-0 · Cited by 51
Metadata
- Authors
- Spök, A., Racovita, M., Karner, S., Adduci, G., Smets, G., Rüdelsheim, P., Kohl, C., Wilhelm, R., & Schiemann, J.
- Affiliations
- Graz University of Technology (AT)
- University of Klagenfurt (AT)
- Perseus (Belgium) (BE)
- Julius Kühn-Institut (DE)
- Corresponding author
- Spök, Armin
- Venue
- Environmental Evidence
- Published
- 2018-01-12
- Volume
- 7
- Issue
- 1
- ISSN
- 2047-2382 (ISSN-L)
- OpenAlex
- W2783167457
- Cited by
- 45
- References
- 24
- Citations by year
- 2025: 4 · 2024: 7 · 2023: 7 · 2022: 16 · 2021: 6 · 2020: 3 · 2018: 2
- FWCI
- 2.43
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Environmental Evidence (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OpenAgrar (submittedVersion, cc-by)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- Funders
- Seventh Framework Programme (311957)
- Topics
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Genetically Modified Organisms Research
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Environmental Science › Management, Monitoring, Policy and Law › Environmental and Social Impact Assessments
- Keywords
- genetically modified organisms, regulatory science, stakeholder engagement, inclusive development, evidence synthesis, systematic reviews, transparency, health and environmental impacts, socio-economic impacts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Inclusive development and prioritization of review questions in a highly controversial field of regulatory science. https://app.cochat.ai/papers/10.1186/s13750-017-0113-z
- The paper
- Spök, A., Racovita, M., Karner, S., Adduci, G., Smets, G., Rüdelsheim, P., Kohl, C., Wilhelm, R., & Schiemann, J. (2018). Inclusive development and prioritization of review questions in a highly controversial field of regulatory science. Environmental Evidence. https://doi.org/10.1186/s13750-017-0113-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.