Toward good in vitro reporting standards
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index; 17 of the 34 sources its claims cite were checked for support.
Abstract
A good experiment reported badly is worthless. Meaningful contributions to the body of science are made by sharing the full methodology and results so that they can be evaluated and reproduced by peers. Erroneous and incomplete reporting does not do justice to the resources spent on conducting the experiment and the time peers spend reading the article. In theory peer-review should ensure adequate reporting - in practice it does not. Many areas have developed reporting standards and checklists to support the adequate reporting of scientific efforts, but in vitro research still has no generally accepted criteria. It is characterized by a "Wild West" or "anything goes" attitude. Such a culture may undermine trust in the reproducibility of animal-free methods, and thus parallel the "reproducibility crisis" discussed for other life science fields. The increasing data retrieval needs of computational approaches (in extreme as "big data" and artificial intelligence) makes reporting quality even more important so that the scientific community can take full advantage of the results. The first priority of reporting standards is to ensure the completeness and transparency of information provided (data focus). The second tier is a quality of data display that makes information digestible and easy to grasp, compare and further analyze (information focus). This article summarizes a series of initiatives geared towards improving the quality of in vitro work and its reporting. This shall ultimately lead to Good In Vitro Reporting Standards (GIVReSt).
Overview
CoChat Summary
TL;DR
The paper argues that in vitro research needs Good In Vitro Reporting Standards (GIVReSt) to improve reproducibility, evidence retrieval, and data integration. It reviews QA/QC concepts and related initiatives such as GCCP, GIVIMP, OECD GD 211, IV-CAT, and TOP, and proposes a reporting framework.
Key findings
- Poor reporting severely hampers risk-of-bias assessment: citing SYRCLE experience, the paper states that in 80–90% of cases it is not possible to assess the actual risk of bias because crucial details of study design and conduct are not reported.
- Irreproducibility is costly: citing Freedman et al. (2015), the paper reports that the cumulative prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US$28B)/year spent on non-reproducible preclinical research in the United States alone.
- Many reporting guidance efforts exist: the EQUATOR network identified 408 reporting guidelines, and the TOP guidelines have been elected by over 1,000 journals and publishers; the CAAT GIVReSt initiative involves about 90 experts.
- Test-readiness scoring has been proposed: Bal-Price et al. (2018) defined more than 60 scoring items bundled into 13 groups for new approach methods, exemplified for developmental neurotoxicity.
Methods
- Design: conceptual “Food for Thought” article/perspective, not an empirical study; it synthesizes QA/QC principles and reporting initiatives for in vitro work.
- Data or sample: no primary data or sample; it draws on cited guidance documents, workshops, and prior publications, including GCCP, GIVIMP, OECD GD 211, IV-CAT, and TOP.
- Main analysis: narrative synthesis and proposal of GIVReSt, including tables differentiating QA and QC and a chapter structure/responsibilities for the CAAT GIVReSt effort.
Limitations
- Stated: the article “summarizes, probably not even comprehensively, the activities known to and/or involving the authors,” so coverage is not exhaustive and is author-centered.
- Evident: it is a perspective/commentary without a systematic search or empirical validation of the proposed standards.
- Disclosed conflict: Thomas Hartung is the founder of Organome LLC, which aims to commercialize the mini-brains mentioned.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
15 Valid · 2 Not valid · 11 Not checked · 28 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A number of initiatives are converging toward the development of Good In Vitro Reporting Standards (GIVReSt): Good Cell Culture Practices (GCCP) (Coecke et al., 2005)”
§ Toward Good In Vitro Reporting Standards > 1 Introduction
What the sources say
- Not checked — no full text of the sourcecites Coecke et al., 2005 matched to “Guidance on Good Cell Culture Practice: A Report of the Second ECVAM Task Force on Good Cell Culture Practice” (2005) source
- Partly supportedcites Pamies et al., 2017a matched to “Good Cell Culture Practice for stem cells and stem-cell-derived models” (2016) sourceExperimental — accuracy not yet measured
Our copy IS the GCCP initiative (Abstract and §2 'The genesis of GCCP 1.0'/'3 The need for GCCP 2.0'), so the GCCP element of the list is supported, and the section '-Performance of the Study -Reporting of Study Results' explicitly names GIVIMP as another concurrent activity contributing to standardization/harmonization of in vitro information. The paper also gestures at converging reporting efforts (§2: 'A key element of this guidance is the advice on documentation and publication... guidance also has been developed for the publication of journal articles on in vitro experiments (Leist et al., 2010). A CAAT workshop was held in March 2012...'). However, the specific umbrella construct attributed to this source — initiatives 'converging toward the development of Good In Vitro Reporting Standards (GIVReSt)' — appears nowhere in our copy (searched: GIVReSt, reporting standards, TOP, evidence-based toxicology), and neither 'TOP' nor 'evidence-based toxicology' is named as one of the converging initiatives; the only 'evidence-based toxicology'-adjacent text is the Funding acknowledgement of a toxicology chair collaboration. So the framing term and two of the four listed initiatives are not supported by this source. (LLM, conf=0.70)
- Not checked — no full text of the sourcecites Eskes et al., 2017 matched to “Good cell culture practices & in vitro toxicology” (2017) source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“The characteristics of the data, such as the experimental unit, the data type, and the (assumed) distribution, will guide various experimental features, including different levels of replication, number of test concentrations, and experimental repetitions (Lovell and Omori, 2008).”
§ Toward Good In Vitro Reporting Standards > 5.3 Data analysis and representation
What the sources say
- Supportedcites Lovell and Omori, 2008 matched to “Statistical issues in the use of the comet assay” (2008) source
The Experimental unit section explains how identifying the unit affects replication, while Replication and repeat experiments distinguishes levels of replication from separate experiments. Dose-response modelling discusses the number of doses to include, and Within sample distributions addresses assumptions about data distributions. Together, these sections support the citing sentence’s broad account of statistical considerations guiding comet-assay design; the indexed passages provide section labels but no page numbers. (LLM, conf=0.87)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In their experience, in 80-90% of the cases it is not possible to assess the actual risk of bias, because crucial details of the design and conduct of the studies are not reported.”
§ Toward Good In Vitro Reporting Standards > 9 Earlier in vitro reporting standard initiatives
What the sources say
- Partly supportedcites Hooijmans et al., 2014 matched to “SYRCLE’s risk of bias tool for animal studies” (2014) sourceExperimental — accuracy not yet measured
The Results (‘Use of SYRCLE’s RoB tool’; page unavailable) say that risk of bias is judged ‘unclear’ when insufficient details are reported. The Discussion and conclusion says users had to judge many entries unclear because experimental details were poorly reported. Neither passage gives the claimed 80–90% figure or says that proportion of cases could not be assessed. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
In the paper
“Currently, consistency in terminology to report study methods and findings can be low, as is being found in, e.g., the epigenetic literature and especially for mechanistic research with its variety of assays and outcomes that it investigates and attempts to relate into disease progression pathways (Whaley et al., in preparation).”
§ Toward Good In Vitro Reporting Standards > 5.3 Data analysis and representation
What the sources say
- Not matchedcites Whaley et al., in preparation
This doesn’t read as a citation we can look up.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Not matched
None of the sources cited for this claim have been checked yet.
In the paper
“The originally defined concept of performance standards (Hartung et al., 2004;OECD, 2005) is currently undergoing a revival, owing to rapid advances in biotechnology, such as microphysiological systems, bioprinting, and organ-on-a-chip.”
§ Toward Good In Vitro Reporting Standards > 3 Good Cell Culture Practice (GCCP) and its relation to reporting standards
What the sources say
- Not checked — no full text of the sourcecites Hartung et al., 2004 matched to “A Modular Approach to the ECVAM Principles on Test Validity” (2004) source
- Not matchedcites OECD, 2005
An author–year citation the paper’s reference list doesn’t settle. We don’t search these by name.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
8 more claims
Create a free account to see all 56 claims and every source verdict.
- Probably not a substantial claim. 37
- It may state two separate claims as one. 18
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Initiatives fostering a standardization of documentation required for cell models culminated in the definition of GCCP standards (Coecke et al., 2005; Hartung et al., 2019), which were recently modified to meet requirements that have emerged with the advent of stem cell research (Pamies et al., 2017).”
From In Vitro research Reproducibility: Keeping up High Standards (2019) · cites this paper as Hartung et al., 2019
Citations
Cites
Yaşlı Kadınlarda Üreme Sağlığı
Kokanalı, D., & Engin Üstün, Y. (n.d.) · DergiPark (Istanbul University) · Cited by 170,090
Yaşlı Kadınlarda Üreme Sağlığı
Kokanalı, D., & Üstün, Y. E. (2019) · Jinekoloji-Obstetrik ve Neonatoloji Tıp Dergisi · Cited by 169,581
SYRCLE’s risk of bias tool for animal studies
Hooijmans, C. R., Rovers, M. M., de Vries, R. B., Leenaars, M., Ritskes-Hoitinga, M., & Langendam, M. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-43 · Cited by 4,415 · cited 1×
“In these cases, the risks of bias have to be scored as “unclear”; after all, the researchers might have applied measures to avoid bias, such as randomization and blinding, but just not reported them (Hooijmans et al., 2014).”
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209 · cited 1×
“However, the reproducibility crisis (Baker, 2016; Jarvis and Williams, 2016) in science alone should prompt and cement such quality assurance.”
BackgroundMinimum information about a microarray experiment (MIAME)-toward standards for microarray data
Brazma, A., Hingamp, P., Quackenbush, J., Sherlock, G., Spellman, P., Stoeckert, C., Aach, J., Ansorge, W., Ball, C. A., Causton, H. C., et al. (2001) · Nature Genetics · doi:10.1038/ng1201-365 · Cited by 4,109 · cited 1×
“Rennie and Flanagin, 2018 Brazma et al., 2001 Eskes et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
BackgroundPolicy: NIH to balance sex in cell and animal studies
Clayton, J. A., & Collins, F. S. (2014) · Nature · doi:10.1038/509282a · Cited by 1,816 · cited 1×
“Rennie and Flanagin, 2018 Brazma et al., 2001 Eskes et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
BackgroundPolicy: NIH plans to enhance reproducibility
Collins, F. S., & Tabak, L. A. (2014) · Nature · doi:10.1038/505612a · Cited by 1,247 · cited 1×
“Rennie and Flanagin, 2018 Brazma et al., 2001 Eskes et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
BackgroundA call for transparent reporting to optimize the predictive value of preclinical research
Landis, S. C., Amara, S. G., Asadullah, K., Austin, C. P., Blumenstein, R., Bradley, E. W., Crystal, R. G., Darnell, R. B., Ferrante, R. J., Fillit, H., et al. (2012) · Nature · doi:10.1038/nature11556 · Cited by 1,196 · cited 1×
“Rennie and Flanagin, 2018 Brazma et al., 2001 Eskes et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
The economics of reproducibility in preclinical research
Freedman, L. P., Cockburn, I. M., & Simcoe, T. S. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002165 · Cited by 1,055 · cited 1×
“Freedman et al. (2015) have estimated how this translates to wasted money: “the cumulative (total) prevalence of irreproducible preclinical research exceeds 50%, resulting in approximately US$28,000,000,000 (US $28B)/year spent on preclinical research that is not reproducible – in the United States…”
(2018) · doi:10.1001/jama · Cited by 932
Whetzel, P. L., Noy, N. F., Shah, N. H., Alexander, P. R., Nyulas, C., Tudorache, T., & Musen, M. A. (2011) · Nucleic Acids Research · doi:10.1093/nar/gkr469 · Cited by 783
Foster, E. D., & Deardorff, A. (2017) · Journal of the Medical Library Association · doi:10.5195/jmla.2017.88 · Cited by 715 · cited 1×
“…et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
Visualizing samples with box plots
Krzywinski, M., & Altman, N. (2014) · Nature Methods · doi:10.1038/nmeth.2813 · Cited by 576
Reducing Workload in Systematic Review Preparation Using Automated Citation Classification
Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P.-Y. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1929 · Cited by 411 · cited 1×
“There is growing interest in using machine learning approaches to priority rank studies and reduce human burden in screening literature (Cohen et al., 2006) when conducting systematic reviews.”
BackgroundCoecke, S., Balls, M., Bowe, G., Davis, J., Gstraunthaler, G., Hartung, T., Hay, R., Merten, O.-W., Price, A., Schechtman, L., et al. (2005) · Alternatives to Laboratory Animals · doi:10.1177/026119290503300313 · Cited by 371 · cited 4×
“…GCCP includes needs for documentation “Principle 3: Documentation of the information necessary to track the materials and methods used, to permit the repetition of the work, and to enable the target audience to understand and evaluate the work” (Coecke et al., 2005, referred to as GCCP 1.0).”
InfluentialA Modular Approach to the ECVAM Principles on Test Validity
Hartung, T., Bremer, S., Casati, S., Coecke, S., Corvi, R., Fortaner, S., Gribaldo, L., Halder, M., Hoffmann, S., Roi, A. J., et al. (2004) · Alternatives to Laboratory Animals · doi:10.1177/026119290403200503 · Cited by 341 · cited 2×
“The originally defined concept of performance standards (Hartung et al., 2004; OECD, 2005) is currently undergoing a revival, owing to rapid advances in biotechnology, such as microphysiological systems, bioprinting, and organ-on-a-chip.”
Self-citationStatistical issues in the use of the comet assay
Lovell, D. P., & Omori, T. (2008) · Mutagenesis · doi:10.1093/mutage/gen015 · Cited by 297 · cited 1×
“The characteristics of the data, such as the experimental unit, the data type, and the (assumed) distribution, will guide various experimental features, including different levels of replication, number of test concentrations, and experimental repetitions (Lovell and Omori, 2008).”
BackgroundMarx, U. (2016) · ALTEX · doi:10.14573/altex.1603161 · Cited by 295 · cited 2×
“, 2017a, 2018) reopened the guidance development in order to adapt it to technical progress, especially the increasing importance of stem cells and bioengineering technologies that enable the establishment of organotypic cultures (microphysiological systems) (Marx et al., 2016).”
Pamies, D. (2017) · ALTEX · doi:10.14573/altex.1609122 · Cited by 250
Bal-Price, A. (2018) · ALTEX · doi:10.14573/altex.1712081 · Cited by 207 · cited 1×
“Performance standards for complex and multipurpose in vitro systems such as organ-on-a-chip are going beyond “minimum Recently, we tried to define test-readiness criteria (Bal-Price et al., 2018) to support an assessment of suggested tests for consideration in a regulatory context.”
BackgroundSWIFT-Review: a text-mining workbench for systematic review
Howard, B. E., Phillips, J., Miller, K., Tandon, A., Mav, D., Shah, M. R., Holmgren, S., Pelch, K. E., Walker, V., Rooney, A. A., et al. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0263-z · Cited by 196 · cited 2×
“SWIFT-Review (Howard et al., 2016), SysRev15, DistillerSR (Matwin et al.”
Animal testing and its alternatives—the most important omics is economics
Meigs, L. (2018) · ALTEX · doi:10.14573/altex.1807041 · Cited by 184 · cited 1×
“Along with our mantra “the most important omics is economics” (Meigs et al., 2018), this has also to be seen as an economic problem.”
BackgroundSchmidt, B. Z., Lehmann, M., Gutbier, S., Nembo, E., Noel, S., Smirnova, L., Forsby, A., Hescheler, J., Avci, H. X., Hartung, T., et al. (2017) · Archives of Toxicology · doi:10.1007/s00204-016-1805-9 · Cited by 182 · cited 1×
“These were later updated for some aspects (Leist et al., 2012; Schmidt et al., 2017).”
BackgroundSelf-citationFixing problems with cell lines
Lorsch, J. R., Collins, F. S., & Lippincott-Schwartz, J. (2014) · Science · doi:10.1126/science.1259110 · Cited by 179 · cited 1×
“Rennie and Flanagin, 2018 Brazma et al., 2001 Eskes et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
de Vries, R. B. M., Wever, K. E., Avey, M. T., Stephens, M. L., Sena, E. S., & Leenaars, M. (2014) · ILAR Journal · doi:10.1093/ilar/ilu043 · Cited by 165 · cited 1×
“This validity is threatened by certain types of bias, where systematic differences between experimental groups other than the intervention of interest are introduced, either intentionally or unintentionally (de Vries et al., 2014).”
BackgroundGood Cell Culture Practice for stem cells and stem-cell-derived models
Pamies, D. (2016) · ALTEX · doi:10.14573/altex.1607121 · Cited by 160 · cited 4×
“For example, our human BrainSpheres model (“mini-brains”) (Pamies et al., 2017b) can be used to test neurotoxicity, developmental neurotoxicity, de- and remyelination, and to study many different disease aspects such as infection, cancer, trauma, stroke, neurodegeneration, etc.”
BackgroundInfluentialToward an evidence-based toxicology
Hoffmann, S., & Hartung, T. (2006) · Human & Experimental Toxicology · doi:10.1191/0960327106het648oa · Cited by 135 · cited 2×
“The development of evidence-based toxicology (Hoffmann and Hartung, 2006; Griesinger et al., 2009; Hartung, 2009; Stephens et al., 2013, 2016; Hoffmann et al., 2016, 2017), as promoted by the Evidence-based Toxicology Collaboration3, includes the determination of quality, e.g., by quality…”
BackgroundSelf-citationHan, S., Olonisakin, T. F., Pribis, J. P., Zupetic, J., Yoon, J. H., Holleran, K. M., Jeong, K., Shaikh, N., Rubio, D. M., & Lee, J. S. (2017) · PLOS ONE · doi:10.1371/journal.pone.0183591 · Cited by 131 · cited 2×
“…life science checklists (e.g., the Nature Research reporting summary), and results of recent meta-research studying the efficacy of such interventions (e.g., Macleod et al., 2017; Han et al., 2017) to devise a set of minimal expectations that journals could agree to ask their authors to meet.”
Irreproducibility in Preclinical Biomedical Research: Perceptions, Uncertainties, and Knowledge Gaps
Jarvis, M. F., & Williams, M. (2016) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2015.12.001 · Cited by 129 · cited 1×
“However, the reproducibility crisis (Baker, 2016; Jarvis and Williams, 2016) in science alone should prompt and cement such quality assurance.”
Pamies, D. (2018) · ALTEX · doi:10.14573/altex.1710081 · Cited by 127 · cited 2×
“…Good Cell Culture Practices (GCCP) (Coecke et al., 2005), which are currently being expanded to incorporate organotypic cultures, including microphysiological systems, as GCCP 2.0 (Pamies et al., 2017a, 2018; Pamies and Hartung, 2017) define minimum standards for cell culture and its documentation.”
Influential21st Century Cell Culture for 21st Century Toxicology
Pamies, D., & Hartung, T. (2017) · Chemical Research in Toxicology · doi:10.1021/acs.chemrestox.6b00269 · Cited by 118 · cited 2×
“0 (Pamies et al., 2017a, 2018; Pamies and Hartung, 2017) define minimum standards for cell culture and its documentation.”
BackgroundInfluentialSelf-citationECVAM Good Cell Culture Practice Task Force Report 1
Hartung, T., Balls, M., Bardouille, C., Blanck, O., Coecke, S., Gstraunthaler, G., & Lewis, D. (2002) · Alternatives to Laboratory Animals · doi:10.1177/026119290203000404 · Cited by 109
Self-citationValidation and quality control of replacement alternatives - current status and future challenges
Leist, M., Hasiwa, N., Daneshian, M., & Hartung, T. (2012) · Toxicology Research · doi:10.1039/c2tx20011b · Cited by 100 · cited 2×
“The regulatory use of in vitro methods has brought about the concepts that feed into validation (Leist et al., 2012).”
Self-citationA new algorithm for reducing the workload of experts in performing systematic reviews
Matwin, S., Kouznetsov, A., Inkpen, D., Frunza, O., & O'Blenis, P. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.004325 · Cited by 100 · cited 3×
“, 2016), SysRev15, DistillerSR (Matwin et al., 2010), and other software applications are in the process of developing various”
BackgroundMethodAdoption of structured abstracts by general medical journals and format for a structured abstract.
(2005) · PubMed Central · Cited by 93
Food for Thought … Mechanistic Validation
Hartung, T. (2013) · ALTEX · doi:10.14573/altex.2013.2.119 · Cited by 93 · cited 1×
“The purpose of the system and the engineering goals should be defined, and quality assurance should be provided (which will include both correlative and mechanistic validation, Hartung et al., 2013).”
BackgroundA primer on systematic reviews in toxicology
Hoffmann, S., de Vries, R. B. M., Stephens, M. L., Beck, N. B., Dirven, H. A. A. M., Fowle, J. R., Goodman, J. E., Hartung, T., Kimber, I., Lalu, M. M., et al. (2017) · Archives of Toxicology · doi:10.1007/s00204-017-1980-3 · Cited by 85 · cited 3×
“The development of evidence-based toxicology (Hoffmann and Hartung, 2006; Griesinger et al., 2009; Hartung, 2009; Stephens et al., 2013, 2016; Hoffmann et al., 2016, 2017), as promoted by the Evidence-based Toxicology Collaboration3, includes the determination of quality, e.g., by quality…”
BackgroundSelf-citationSamuel, G. O., Hoffmann, S., Wright, R. A., Lalu, M. M., Patlewicz, G., Becker, R. A., DeGeorge, G. L., Fergusson, D., Hartung, T., Lewis, R. J., et al. (2016) · Environment International · doi:10.1016/j.envint.2016.03.010 · Cited by 82 · cited 2×
“A scoping review of quality scoring tools in the field of toxicology was carried out by EBTC (Samuel et al., 2016).”
MethodSelf-citationFood for thought ... on validation
Hartung, T. (2007) · ALTEX · doi:10.14573/altex.2007.2.67 · Cited by 82 · cited 2×
“The concepts associated with validation are necessarily in constant evolution as they must be adapted to the individual new methods (Hartung, 2007), use scenarios and rigor; different names have been attributed to these variants (e.”
MethodSelf-citation3S - Systematic, systemic, and systems biology and toxicology
Smirnova, L. (2018) · ALTEX · doi:10.14573/altex.1804051 · Cited by 72 · cited 1×
“We discussed the concept of performance standard-based validation in this series earlier this year (Smirnova et al., 2018, and Figure 3 therein).”
MethodGolbach, L. A., Portelli, L. A., Savelkoul, H. F. J., Terwel, S. R., Kuster, N., de Vries, R. B. M., & Verburg-van Kemenade, B. M. L. (2016) · Environment International · doi:10.1016/j.envint.2016.01.014 · Cited by 70 · cited 1×
“SYRCLE19, the Systematic Review Centre for Laboratory animal Experimentation, has been conducting and supporting systematic reviews in the field of preclinical research (both animal and in vitro studies, e.g., Golbach et al., 2016) for nearly 10 years.”
MethodFood for thought ... considerations and guidelines for basic test method descriptions in toxicology
Leist, M. (2010) · ALTEX · doi:10.14573/altex.2010.4.309 · Cited by 68 · cited 2×
“Uniform requirements for manuscripts submitted to biomedical journals (also called the Vancouver style) Christiansen, 2008 Leist et al., 2010; this article Bravo et al., 2015 ALTEX 36(1), 2019 13 Organization / Guideline Description References International Congress on Peer Review and Biomedical…”
BackgroundHighlight report: Launch of a large integrated European in vitro toxicology project: EU-ToxRisk
Daneshian, M., Kamp, H., Hengstler, J., Leist, M., & van de Water, B. (2016) · Archives of Toxicology · doi:10.1007/s00204-016-1698-7 · Cited by 65 · cited 1×
“For instance, the test methods of the EU-ToxRisk project (Daneshian et al., 2016) are now being evaluated according to this scoring scheme.”
MethodDeveloping a guideline to standardize the citation of bioresources in journal articles (CoBRA)
Bravo, E., Calzolari, A., Castro, P. D., Mabile, L., Napolitani, F., Rossi, A. M., & Cambon‐Thomsen, A. (2015) · BMC Medicine · doi:10.1186/s12916-015-0266-y · Cited by 64 · cited 1×
“…requirements for manuscripts submitted to biomedical journals (also called the Vancouver style) Christiansen, 2008 Leist et al., 2010; this article Bravo et al., 2015 ALTEX 36(1), 2019 13 Organization / Guideline Description References International Congress on Peer Review and Biomedical…”
BackgroundALTEX - Alternatives to Animal Experimentation
(2026) · ALTEX · doi:10.14573/altex · Cited by 59
Food for thought … on evidence-based toxicology
Hartung, T. (2009) · ALTEX · doi:10.14573/altex.2009.2.75 · Cited by 56 · cited 2×
“The development of evidence-based toxicology (Hoffmann and Hartung, 2006; Griesinger et al., 2009; Hartung, 2009; Stephens et al., 2013, 2016; Hoffmann et al., 2016, 2017), as promoted by the Evidence-based Toxicology Collaboration3, includes the determination of quality, e.g., by quality…”
BackgroundSelf-citationThe Emergence of Systematic Review in Toxicology
Stephens, M. L., Betts, K., Beck, N. B., Cogliano, V., Dickersin, K., Fitzpatrick, S., Freeman, J., Gray, G., Hartung, T., McPartland, J., et al. (2016) · Toxicological Sciences · doi:10.1093/toxsci/kfw059 · Cited by 54 · cited 1×
“The development of evidence-based toxicology (Hoffmann and Hartung, 2006; Griesinger et al., 2009; Hartung, 2009; Stephens et al., 2013, 2016; Hoffmann et al., 2016, 2017), as promoted by the Evidence-based Toxicology Collaboration3, includes the determination of quality, e.g., by quality…”
BackgroundSelf-citationGood cell culture practices & in vitro toxicology
Eskes, C., Boström, A.-C., Bowe, G., Coecke, S., Hartung, T., Hendriks, G., Pamies, D., Piton, A., & Rovida, C. (2017) · Toxicology in Vitro · doi:10.1016/j.tiv.2017.04.022 · Cited by 52 · cited 2×
“The recent OECD guidance document for Good In Vitro Method Practices (GIVIMP)2 (Eskes et al., 2017) sets standards for regulatory tests under Good Laboratory Practices (GLP).”
BackgroundMethodEvidence-based toxicology for the 21st century: Opportunities and challenges
Stephens, M. (2013) · ALTEX · doi:10.14573/altex.2013.1.074 · Cited by 48 · cited 1×
“The development of evidence-based toxicology (Hoffmann and Hartung, 2006; Griesinger et al., 2009; Hartung, 2009; Stephens et al., 2013, 2016; Hoffmann et al., 2016, 2017), as promoted by the Evidence-based Toxicology Collaboration3, includes the determination of quality, e.g., by quality…”
BackgroundGuidance Document for Describing Non-Guideline In Vitro Test Methods
OECD (2017) · OECD series on testing and assessment · doi:10.1787/9789264274730-en · Cited by 47
A FAIR guide for data providers to maximise sharing of human genomic data
Corpas, M., Kovalevskaya, N. V., McMurray, A., & Nielsen, F. G. G. (2018) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005873 · Cited by 44 · cited 1×
“These few examples illustrate that there is an urgent need for reporting standards, focusing on the transfer of information, even when we reach a state where at least the data as such are made available in a findable, accessible, interoperable, and re-usable (FAIR) way (Corpas et al., 2018).”
Toxicology ontology perspectives
Hardy, B. (2012) · ALTEX · doi:10.14573/altex.2012.2.139 · Cited by 43 · cited 1×
“…context, what is more achievable and of higher practical use in evidence retrieval is the definition of universal ontologies (Whetzel et al., 2011; Hardy et al., 2012a,b) to cover the concepts being used by researchers as they describe their research practices and study results, and the…”
BackgroundThree Decades of Peer Review Congresses
Rennie, D., & Flanagin, A. (2018) · JAMA · doi:10.1001/jama.2017.20606 · Cited by 38 · cited 1×
“Rennie and Flanagin, 2018 Brazma et al., 2001 Eskes et al., 2017 Landis et al., 2012; Collins and Tabak, 2014; Clayton and Collins, 2014; Lorsch et al., 2014 O’Brien et al., 2014 Nakayama et al., 2005 Foster and Deardorff, 2017 ahttps://bit.ly/1F0K2O6; bhttps://www.nlm.nih.gov/cde/index.html;…”
BackgroundHardy, B. J., Apic, G., Carthew, P., Clark, D., Cook, D., Dix, I., Escher, S., Hastings, J., Heard, D. J., Jeliazkova, N., et al. (2012) · ALTEX · doi:10.14573/altex.2012.2.129 · Cited by 34 · cited 1×
“…context, what is more achievable and of higher practical use in evidence retrieval is the definition of universal ontologies (Whetzel et al., 2011; Hardy et al., 2012a,b) to cover the concepts being used by researchers as they describe their research practices and study results, and the…”
BackgroundMaking Big Sense From Big Data
Hartung, T. (2018) · Frontiers in Big Data · doi:10.3389/fdata.2018.00005 · Cited by 32 · cited 2×
“Again, there is a lot of cross-talk with reporting standards as the method definition in of curated datasets and literature fuels this approach (Hartung, 2018).”
BackgroundMacleod, M. R., & The NPQIP Collaborative group (2017) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/187245 · Cited by 25
Raising the standard of systematic reviews published in Environment International
Whaley, P., Letcher, R. J., Covaci, A., & Alcock, R. (2016) · Environment International · doi:10.1016/j.envint.2016.08.007 · Cited by 14 · cited 1×
“Over the last years, however, the number of systematic reviews in these latter fields has been steadily increasing (Hoffmann et al., 2017; Whaley et al., 2016); where we have a rudimentary count of them, they are showing exponential growth (Whaley et al., 2016).”
Hoffmann, S., Härtung, T., & Stephens, M. L. (2016) · Advances in experimental medicine and biology · doi:10.1007/978-3-319-33826-2_9 · Cited by 8 · cited 1×
“The development of evidence-based toxicology (Hoffmann and Hartung, 2006; Griesinger et al., 2009; Hartung, 2009; Stephens et al., 2013, 2016; Hoffmann et al., 2016, 2017), as promoted by the Evidence-based Toxicology Collaboration3, includes the determination of quality, e.g., by quality…”
BackgroundSelf-citationEthical and Legal Guidance in Biomedical Publishing
Christiansen, S. L. (2008) · CHEST Journal · doi:10.1378/chest.08-1165 · Cited by 8 · cited 1×
“Uniform requirements for manuscripts submitted to biomedical journals (also called the Vancouver style) Christiansen, 2008 Leist et al., 2010; this article Bravo et al., 2015 ALTEX 36(1), 2019 13 Organization / Guideline Description References International Congress on Peer Review and Biomedical…”
BackgroundIn Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol
de Vries, R. B. M., & Whaley, P. (2018) · Lancaster EPrints (Lancaster University) · doi:10.5281/zenodo.1493497 · Cited by 7
Food for thought … the first ten years
Härtung, T. (2017) · ALTEX · doi:10.14573/altex.1703311 · Cited by 6 · cited 2×
“…Good Cell Culture Practices (GCCP) (Coecke et al., 2005), which are currently being expanded to incorporate organotypic cultures, including microphysiological systems, as GCCP 2.0 (Pamies et al., 2017a, 2018; Pamies and Hartung, 2017) define minimum standards for cell culture and its documentation.”
BackgroundInfluentialSelf-citationIn Vitro Critical Appraisal Tool (IV-CAT): Tool Development Protocol
de Vries, R. B. M., & Whaley, P. (2018) · Lancaster EPrints (Lancaster University) · doi:10.5281/zenodo.1493498 · Cited by 6
Research works printed but not linked: 3
- Griesinger, C., Hoffmann, S., Kinsner-Ovaskainen, A. et al. (2009). Proceedings of the First International Forum To- wards Evidence-Based Toxicology. Conference Centre Spazio Villa Erba, Como, Italy. 15-18 October 2007. Human Exp Toxicol, Spec Issue: Evidence-Based Toxicology (EBT) 28, 83-163.
- Hoffmann, S., Edler, L., Gardner, I. et al. (2008). Points of reference in validation -The report and recommendations of ECVAM Workshop. Altern Lab Anim 36, 343-352.
- Whaley, P., Watford S., Allard P. et al. (in preparation). En- hancing systematic review and evidence mapping with shared ontologies and semantic matching: An example from epigenetic research.
Cited by
Clinton, S. K., Giovannucci, E. L., & Hursting, S. D. (2020) · The Journal of Nutrition · doi:10.1093/jn/nxz268 · Cited by 973
“This effort has also been addressed and supported by the International Council for Laboratory Animal Science (26) and by others seeking to improve reporting standards for in vitro cell-based research (27).”
BackgroundNew approach methodologies in human regulatory toxicology – Not if, but how and when!
Schmeisser, S., Miccoli, A., von Bergen, M., Berggren, E., Braeuning, A., Busch, W., Desaintes, C., Gourmelon, A., Grafström, R., Harrill, J., et al. (2023) · Environment International · doi:10.1016/j.envint.2023.108082 · Cited by 380
“More general approaches toward Good In Vitro Reporting Standards are on the way (Hartung et al. 2019).”
MethodSelf-citationImproving target assessment in biomedical research: the GOT-IT recommendations
Emmerich, C. H., Gamboa, L. M., Hofmann, M. C. J., Bonin-Andresen, M., Arbach, O., Schendel, P., Gerlach, B., Hempel, K., Bespalov, A., Dirnagl, U., et al. (2021) · Nature Reviews Drug Discovery · doi:10.1038/s41573-020-0087-3 · Cited by 226
Marx, U. (2020) · ALTEX · doi:10.14573/altex.2001241 · Cited by 218
In Vitro research Reproducibility: Keeping up High Standards
Hirsch, C., & Schildknecht, S. (2019) · Frontiers in Pharmacology · doi:10.3389/fphar.2019.01484 · Cited by 161
Metadata
- Authors
- Hartung, T.
- Venue
- ALTEX
- Publisher
- ALTEX Edition
- Published
- 2019
- Volume
- 36
- Issue
- 1
- Pages
- 3-17
- ISSN
- 0946-7785 (ISSN-L)
- 1868-596X
- OpenAlex
- W2910963560
- Semantic Scholar
- 7742f7bf3cb2a9632385fbeb46ec177488823357
- Cited by
- 65
- References
- 60
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 5 · 2023: 9 · 2022: 7 · 2021: 11 · 2020: 14 · 2019: 8
- FWCI
- 14.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- ALTEX (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Lancaster EPrints (Lancaster University) (acceptedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- KOPS (University of Konstanz) (submittedVersion, cc-by)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Health Sciences › Veterinary › Small Animals › Animal testing and alternatives
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- in vitro research, reporting guidelines, reproducibility, peer review, reporting quality, data transparency, data presentation, replication crisis
- MeSH terms
- Publishing, Research Design, Reproducibility of Results, Checklist, In Vitro Techniques
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
