Quantifying the complexity of medical research
Open accessThe DOI matches its Crossref record; 26 of the 27 references it cites were found in a registry or an index.
Abstract
Motivation: A crucial phenomenon of our times is the diminishing marginal returns of investments in pharmaceutical research and development. A potential reason is that research into diseases is becoming increasingly complex, and thus more burdensome, for humans to handle. We sought to investigate whether we could measure research complexity by analyzing the published literature. Results: Through the text mining of the publication record of multiple diseases, we have found that the complexity and novelty of disease research has been increasing over the years. Surprisingly, we have also found that research on diseases with higher publication rate does not possess greater complexity or novelty than that on less-studied diseases. We have also shown that the research produced about a disease can be seen as a differentiated area of knowledge within the wider biomedical research. For our analysis, we have conceptualized disease research as a parallel multi-agent search in which each scientific agent (a scientist) follows a search path based on a model of a disease. We have looked at trends in facts published for diseases, measured their diversity and turnover using the entropy measure and found similar patterns across disease areas. Contact: raul.rodriguez-esteban@roche.com
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“assessments in various disciplines suggest that, far from becoming increasingly short and trivial, published studies are getting longer, more complex, and richer in data (e.g., refs. 32 -34).”
From Is science really facing a reproducibility crisis, and do we need it to? (2018) · cites this paper as [32]
Citations
Cites
Nonlinear dynamical analysis of EEG and MEG: Review of an emerging field
Stam, C. J. (2005) · Clinical Neurophysiology · doi:10.1016/j.clinph.2005.06.011 · Cited by 1,417
Larsen, P. O., & von Ins, M. (2010) · Scientometrics · doi:10.1007/s11192-010-0202-z · Cited by 964
Mutations within the Rhodopsin Gene in Patients with Autosomal Dominant Retinitis Pigmentosa
Dryja, T. P., McGee, T. L., Hahn, L. B., Cowley, G. S., Olsson, J. E., Reichel, E., Sandberg, M. A., & Berson, E. L. (1990) · New England Journal of Medicine · doi:10.1056/nejm199011083231903 · Cited by 501
Measures of statistical complexity: Why?
Feldman, D. P., & Crutchfield, J. P. (1998) · Physics Letters A · doi:10.1016/s0375-9601(97)00855-4 · Cited by 436
Adami, C. (2002) · BioEssays · doi:10.1002/bies.10192 · Cited by 395
Mutation spectrum of the rhodopsin gene among patients with autosomal dominant retinitis pigmentosa.
Dryja, T. P., Hahn, L. B., Cowley, G. S., McGee, T. L., & Berson, E. L. (1991) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.88.20.9370 · Cited by 367
ChemSpot: a hybrid system for chemical named entity recognition
Rocktäschel, T., Weidlich, M., & Leser, U. (2012) · Bioinformatics · doi:10.1093/bioinformatics/bts183 · Cited by 269
The Cognitive Science of Science: Explanation, Discovery, and Conceptual Change
Thagard, P. (2012) · The MIT Press eBooks · doi:10.7551/mitpress/9218.001.0001 · Cited by 181
Coden, A., Savova, G., Sominsky, I., Tanenblatt, M., Masanz, J., Schuler, K., Cooper, J., Guan, W., & de Groen, P. C. (2009) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.12.005 · Cited by 177
The Cognitive Science of Science: Explanation, Discovery, and Conceptual Change
Thagard, P. R. (2012) · Cited by 169
A vision and strategy for the virtual physiological human in 2010 and beyond
Hunter, P., Coveney, P. V., de Bono, B., Diaz, V., Fenner, J., Frangi, A. F., Harris, P., Hose, R., Kohl, P., Lawford, P., et al. (2010) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2010.0048 · Cited by 160
Plake, C., Schiemann, T., Pankalla, M., Hakenberg, J., & Leser, U. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl408 · Cited by 137
Fractal and Complexity Measures of Heart Rate Variability
Perkiömäki, J. S., Mäkikallio, T. H., & Huikuri, H. V. (2005) · Clinical and Experimental Hypertension · doi:10.1081/ceh-48742 · Cited by 127
The GNAT library for local and remote gene mention normalization
Hakenberg, J., Gerner, M., Haeussler, M., Solt, I., Plake, C., Schroeder, M., Gonzalez‐Hernandez, G., Nenadić, G., & Bergman, C. (2011) · Bioinformatics · doi:10.1093/bioinformatics/btr455 · Cited by 90
GeneView: a comprehensive semantic search engine for PubMed
Thomas, P., Starlinger, J., Vowinkel, A., Arzt, S., & Leser, U. (2012) · Nucleic Acids Research · doi:10.1093/nar/gks563 · Cited by 70
Life cycles of successful genes
Hoffmann, R., & Valencia, A. (2003) · Trends in Genetics · doi:10.1016/s0168-9525(02)00014-8 · Cited by 69 · cited 1×
“A number of works have been devoted to analyzing the growth and evolution of publications on gene and protein interactions outside the context of disease by analyzing the accumulated published record (Arbesman and Christakis, 2011; Cokol et al., 2005; Cokol and Rodriguez-Esteban, 2006; He and Zhang, 2009; Hoffmann and…”
BackgroundTemporal patterns of genes in scientific publications
Pfeiffer, T., & Hoffmann, R. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0701315104 · Cited by 64 · cited 1×
“A number of works have been devoted to analyzing the growth and evolution of publications on gene and protein interactions outside the context of disease by analyzing the accumulated published record (Arbesman and Christakis, 2011; Cokol et al., 2005; Cokol and Rodriguez-Esteban, 2006; He and Zhang, 2009; Hoffmann and…”
BackgroundEmergent behavior of growing knowledge about molecular interactions
Cokol, M., Iossifov, I. H., Weinreb, C., & Rzhetsky, A. (2005) · Nature Biotechnology · doi:10.1038/nbt1005-1243 · Cited by 62
de Bono, B., Grenon, P., & Sammut, S. (2012) · Human Mutation · doi:10.1002/humu.22065 · Cited by 26
Roy, S., Heinrich, K. E., Phan, V., Berry, M. W., & Homayouni, R. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-s10-s19 · Cited by 22
Visualizing evolution and impact of biomedical fields
Cokol, M., & Rodriguez‐Esteban, R. (2008) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2008.05.002 · Cited by 19
Self-citationOn the Growth of Scientific Knowledge: Yeast Biology as a Case Study
He, X., & Zhang, J. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000320 · Cited by 18
Immunological network signatures of cancer progression and survival
Clancy, T., Pedicini, M., Castiglione, F., Santoni, D., Nygaard, V., Lavelle, T. J., Benson, M., & Hovig, E. J. (2011) · BMC Medical Genomics · doi:10.1186/1755-8794-4-28 · Cited by 17
Eurekometrics: Analyzing the Nature of Discovery
Samuel Arbesman, C., & Christakis, N. A. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002072 · Cited by 16
Fluorescein Angiography During Induced Ocular Hypertension in Retinitis Pigmentosa
Best, M., Galin, M. A., Blumenthal, M., & Toyofuku, H. (1971) · American Journal of Ophthalmology · doi:10.1016/0002-9394(71)90967-6 · Cited by 9
Identifying and classifying biomedical perturbations in text
Rodriguez‐Esteban, R., Roberts‐Miller, P., & Crawford, M. (2008) · Nucleic Acids Research · doi:10.1093/nar/gkn986 · Cited by 6
Self-citation
Cited by
Is science really facing a reproducibility crisis, and do we need it to?
Fanelli, D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708272114 · Cited by 416
Rodriguez-Esteban, R., & Jiang, X. (2017) · BMC Medical Genomics · doi:10.1186/s12920-017-0293-y · Cited by 94
Self-citationA Drug-Centric View of Drug Development: How Drugs Spread from Disease to Disease
Rodriguez‐Esteban, R. (2016) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1004852 · Cited by 23
Self-citationIyappan, A., Bagewadi, S., Raschka, T., Hofmann‐Apitius, M., & Senger, P. (2016) · Journal of Biomedical Semantics · doi:10.1186/s13326-016-0079-8 · Cited by 21
A new approach and gold standard toward author disambiguation in MEDLINE
Vishnyakova, D., Rodriguez‐Esteban, R., & Rinaldi, F. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz028 · Cited by 15
Self-citation
Metadata
- Authors
- Rodriguez-Esteban, R., & Loging, W. T.
- Affiliations
- Computational Biology, Boehringer Ingelheim Pharmaceuticals, Inc., Ridgefield, CT 06877, USA
- Boehringer Ingelheim (United States) (US)
- Venue
- Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-08-31
- Volume
- 29
- Issue
- 22
- Pages
- 2918-2924
- ISSN
- 1367-4803 (ISSN-L)
- 1367-4811
- OpenAlex
- W2103403573
- Cited by
- 19
- References
- 27
- Citations by year
- 2026: 1 · 2024: 2 · 2022: 4 · 2019: 1 · 2018: 2 · 2017: 3 · 2016: 4 · 2015: 2
- FWCI
- 2.21
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Bioinformatics (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Medicine › Health Informatics › Biomedical Text Mining and Ontologies
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Interdisciplinary Research and Collaboration
- Keywords
- research complexity, disease research, novelty, publication trends, literature mining, entropy measures, pharmaceutical R&D, knowledge diversity
- MeSH terms
- Disease, Humans, Models, Biological, Publications, Biomedical Research, Data Mining
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.