The Ontology of Clinical Research (OCRe): An informatics foundation for the science of clinical research
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
To date, the scientific process for generating, interpreting, and applying knowledge has received less informatics attention than operational processes for conducting clinical studies. The activities of these scientific processes — the science of clinical research — are centered on the study protocol, which is the abstract representation of the scientific design of a clinical study. The Ontology of Clinical Research (OCRe) is an OWL 2 model of the entities and relationships of study design protocols for the purpose of computationally supporting the design and analysis of human studies. OCRe’s modeling is independent of any specific study design or clinical domain. It includes a study design typology and a specialized module called ERGO Annotation for capturing the meaning of eligibility criteria. In this paper, we describe the key informatics use cases of each phase of a study’s scientific lifecycle, present OCRe and the principles behind its modeling, and describe applications of OCRe and associated technologies to a range of clinical research use cases. OCRe captures the central semantics that underlies the scientific processes of clinical research and can serve as an informatics foundation for supporting the entire range of knowledge activities that constitute the science of clinical research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maintaining knowledge about temporal intervals
Allen, J. F. (1983) · Communications of the ACM · doi:10.1145/182.358434 · Cited by 7,593 · cited 1×
“, 3 weeks after end of chemotherapy), or a time-interval constraint where the time interval associated with the constrained event is compared with another time interval using Allen’s temporal-interval comparison operators [66].”
SPIRIT 2013 Statement: Defining Standard Protocol Items for Clinical Trials
Chan, A.-W., Tetzlaff, J. M., Altman, D. G., Laupacis, A., Gøtzsche, P. C., Krleža-Jerić, K., Hróbjartsson, A., Mann, H., Dickersin, K., Berlin, J. A., et al. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-3-201302050-00583 · Cited by 7,363
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931 · cited 1×
“The first step of a clinical study’s lifecycle is highly iterative: potential scientific questions are posed and revised many times as batches of prior studies are retrieved and interpreted over time.”
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · cited 1×
“Systematic reviews are, however, very time and labor intensive and current practices are not sufficient to keep up with the flood of new studies [53].”
Martinson, B. C., Anderson, M. S., & de Vries, R. (2005) · Nature · doi:10.1038/435737a · Cited by 1,096 · cited 1×
“In statistical terms, they need to appraise the “internal validity” of each study, with tasks ranging from determining the comparability of comparison groups to the existence and nature of follow-up bias [24], Critical appraisal remains somewhat of an art, as many study quality scales and bias…”
Delgado-Rodríguez, M., & Llorca, J. (2004) · Journal of Epidemiology and Community Health · doi:10.1136/jech.2003.008466 · Cited by 1,064
The scandal of poor medical research.
Altman, D. G. (1994) · BMJ · doi:10.1136/bmj.308.6924.283 · Cited by 897
The ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838
An overview of clinical research: the lay of the land
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07283-5 · Cited by 734 · cited 1×
“The first step of a clinical study’s lifecycle is highly iterative: potential scientific questions are posed and revised many times as batches of prior studies are retrieved and interpreted over time.”
Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-60 · Cited by 628
Achieving a Nationwide Learning Health System
Friedman, C. P., Wong, A. K., & Blumenthal, D. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3001456 · Cited by 539 · cited 1×
“Keywords: ontology, clinical research, eligibility criteria, clinical research informatics, evidence-based medicine, trial registration, epistemology Original word count: 15,083 Current count: 10, 849”
Characteristics of Clinical Trials Registered in ClinicalTrials.gov, 2007-2010
Califf, R. M. (2012) · JAMA · doi:10.1001/jama.2012.3424 · Cited by 516
Reducing Workload in Systematic Review Preparation Using Automated Citation Classification
Cohen, A. M., Hersh, W. R., Peterson, K., & Yen, P.-Y. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1929 · Cited by 411 · cited 1×
“Second, while there have been heuristic and machine-learning approaches to determining the relevance of studies to a search query [96,97] many of the approaches require training a classifier against a hand-coded gold standard, which limits their scalability.”
Which clinical studies provide the best evidence?
Barton, S. (2000) · BMJ · doi:10.1136/bmj.321.7256.255 · Cited by 384
Greenland, S. (2001) · Biostatistics · doi:10.1093/biostatistics/2.4.463 · Cited by 359 · cited 1×
“In statistical terms, they need to appraise the “internal validity” of each study, with tasks ranging from determining the comparability of comparison groups to the existence and nature of follow-up bias [24], Critical appraisal remains somewhat of an art, as many study quality scales and bias instruments have attempt…”
BackgroundSemi-automated screening of biomedical citations for systematic reviews
Wallace, B. C., Trikalinos, T. A., Lau, J., Brodley, C., & Schmid, C. H. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-55 · Cited by 351 · cited 1×
“Second, while there have been heuristic and machine-learning approaches to determining the relevance of studies to a search query [96,97] many of the approaches require training a classifier against a hand-coded gold standard, which limits their scalability.”
Rising, K., Bacchetti, P., & Bero, L. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050217 · Cited by 328
The strengths and limitations of meta-analyses based on aggregate data
Lyman, G. H., & Kuderer, N. M. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-14 · Cited by 250
Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists
Smyth, R. M. D., Kirkham, J. J., Jacoby, A., Altman, D. G., Gamble, C., & Williamson, P. R. (2011) · BMJ · doi:10.1136/bmj.c7153 · Cited by 220
Types of Study in Medical Research
Röhrig, B., Prel, J.-B. D., Wachtlin, D., & Blettner, M. (2009) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2009.0262 · Cited by 192
ExaCT: automatic extraction of clinical trial characteristics from journal publications
Kiritchenko, S., de Bruijn, B., Carini, S., Martin, J., & Sim, I. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-56 · Cited by 188 · cited 1×
“We ourselves developed the ExaCT semi-automated system for acquiring study-design data from full-text papers [73], as well as NLP techniques to analyze and instantiate eligibility criteria into ERGO Annotation [65].”
Self-citationRegistration of observational studies: Is it time?
Williams, R. J., Tse, T., Harlan, W. R., & Zarin, D. A. (2010) · Canadian Medical Association Journal · doi:10.1503/cmaj.092252 · Cited by 152 · cited 1×
“Indeed, transparent, unbiased study registration and reporting of interventional studies is a fundamental ethical duty as codified in the Declaration of Helsinki [45], and is arguably an ethical duty for observational studies as well [46].”
Bias, Spin, and Misreporting: Time for Full Access to Trial Protocols and Results
Chan, A.-W. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050230 · Cited by 144
The automation of systematic reviews
Tsafnat, G., Dunn, A., Glasziou, P., & Coiera, E. (2013) · BMJ · doi:10.1136/bmj.f139 · Cited by 131
The Evolution of Trial Registries and Their Use to Assess the Clinical Trial Enterprise
Dickersin, K. (2012) · JAMA · doi:10.1001/jama.2012.4230 · Cited by 127
Synthesizing Evidence: Shifting the Focus From Individual Studies to the Body of Evidence
Murad, M. H., & Montori, V. M. (2013) · JAMA · doi:10.1001/jama.2013.5616 · Cited by 104
Hartling, L., Bond, K., Santaguida, P. L., Viswanathan, M., & Dryden, D. M. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.010 · Cited by 69
Chung, G. Y.-C. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.12.011 · Cited by 34 · cited 1×
“Examples abound of methods to extract key trial elements such as population description, interventions, and outcomes [95-99] or PICO design features [100] from RCT publications.”
Extracting Clinical Trial Design Information from MEDLINE Abstracts.
Hara, K., & Matsumoto, Y. (2007) · New Generation Computing · doi:10.1007/s00354-007-0017-5 · Cited by 21 · cited 1×
“Examples abound of methods to extract key trial elements such as population description, interventions, and outcomes [95-99] or PICO design features [100] from RCT publications.”
Cited by
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
Improving reference prioritisation with PICO recognition.
Brockmeier, A. J., Ju, M., Przybyła, P., & Ananiadou, S. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0992-8 · Cited by 62
The INTEGRATE project: delivering solutions for efficient multi-centric clinical research and trials
Kondylakis, H., Claerhout, B., Keyur, M., Koumakis, L., van Leeuwen, J., Marias, K., Perez-Rey, D., De Schepper, K., Tsiknakis, M., & Bucur, A. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.05.006 · Cited by 18
Embi, P. J., & Payne, P. R. O. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.10.007 · Cited by 13
Metadata
- Authors
- Sim, I., Tu, S. W., Carini, S., Lehmann, H. P., Pollock, B. H., Peleg, M., & Wittkowski, K. M.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2014-12
- Volume
- 52
- Pages
- 78-91
- ISSN
- 1532-0464 (ISSN-L)
- OpenAlex
- W1971703763
- Cited by
- 68
- References
- 31
- FWCI
- 2.29
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.