Predicting the impact of scientific concepts using full-text features
Abstract
New scientific concepts, interpreted broadly, are continuously introduced in the literature, but relatively few concepts have a long‐term impact on society. The identification of such concepts is a challenging prediction task that would help multiple parties—including researchers and the general public—focus their attention within the vast scientific literature. In this paper we present a system that predicts the future impact of a scientific concept, represented as a technical term, based on the information available from recently published research articles. We analyze the usefulness of rich features derived from the full text of the articles through a variety of approaches, including rhetorical sentence analysis, information extraction, and time‐series analysis. The results from two large‐scale experiments with 3.8 million full‐text articles and 48 million metadata records support the conclusion that full‐text features are significantly more useful for prediction than metadata‐only features and that the most accurate predictions result from combining the metadata and full‐text features. Surprisingly, these results hold even when the metadata features are available for a much larger number of documents than are available for the full‐text features.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Full text features drawn from academic papers are more informative in calculating academic impact.
“This would support the view taken by McKeown et al. that full text features drawn from academic papers are more informative in calculating academic impact [20].”
From Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements (2017) · cites this paper as [20]
Cites
A new look at the statistical model identification
Akaike, H. (1974) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1974.1100705 · Cited by 51,879 · see citing works
Collective dynamics of ‘small-world’ networks
Watts, D. J., & Strogatz, S. H. (1998) · Nature · doi:10.1038/30918 · Cited by 43,060 · see citing works
Model selection and multimodel inference: a practical information-theoretic approach.
Guthery, F. S., Burnham, K. P., & Anderson, D. R. (2003) · The Journal of Wildlife Management · doi:10.2307/3802723 · Cited by 41,495 · see citing works
Centrality in social networks conceptual clarification
Freeman, L. C. (1978) · Social Networks · doi:10.1016/0378-8733(78)90021-7 · Cited by 17,062 · see citing works
An Analysis of Transformations
Box, G. E. P., & Cox, D. R. (1964) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1964.tb00553.x · Cited by 15,059 · see citing works
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,953 · see citing works
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669 · see citing works
Multimodel Inference: Understanding AIC and BIC in Model Selection
Burnham, K. P., & Anderson, D. R. (2004) · Sociological Methods & Research · doi:10.1177/0049124104268644 · Cited by 11,648 · see citing works
Science in Action: How to Follow Scientists and Engineers through Society
Bijker, W. E., & Latour, B. (1988) · Technology and Culture · doi:10.2307/3105094 · Cited by 10,768 · see citing works
A set of measures of centrality based upon betweenness
Freeman, L. (1977) · Sociometry · doi:10.2307/3033543 · Cited by 10,557 · see citing works
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026683 · Cited by 8,134 · see citing works
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,865 · see citing works
Newman, M. (2010) · doi:10.1093/acprof:oso/9780199206650.001.0001 · Cited by 6,595 · see citing works
RELATIONS BETWEEN TWO SETS OF VARIATES
Hotelling, H. (1936) · Biometrika · doi:10.1093/biomet/28.3-4.321 · Cited by 5,894 · see citing works
Co‐citation in the scientific literature: A new measure of the relationship between two documents
Small, H. (1973) · Journal of the American Society for Information Science · doi:10.1002/asi.4630240406 · Cited by 5,345 · see citing works
Cetina, K. K. (1999) · Harvard University Press eBooks · doi:10.4159/9780674039681 · Cited by 3,950 · see citing works
Epistemic cultures: how the sciences make knowledge
(1999) · Choice Reviews Online · doi:10.5860/choice.37-0887 · Cited by 3,347 · see citing works
Newman, M. (2002) · Physical review. E, Statistical, nonlinear, and soft matter physics · doi:10.1103/physreve.67.026126 · Cited by 3,130 · see citing works
What do citation counts measure? A review of studies on citing behavior
Bornmann, L., & Daniel, H. (2008) · Journal of Documentation · doi:10.1108/00220410810844150 · Cited by 1,607 · cited 2× · see citing works
“Another approach commonly used within the domain of science and technology policy is an expert panel, such as the Delphi method (Bornmann & Daniel, 2008).”
BackgroundMethodHilborn, R. C. (2000) · doi:10.1093/acprof:oso/9780198507239.001.0001 · Cited by 1,379 · see citing works
Eysenbach, G. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.2012 · Cited by 1,088 · see citing works
Does the h index have predictive power?
Hirsch, J. E. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0707962104 · Cited by 980 · see citing works
Chaos and nonlinear dynamics: an introduction for scientists and engineers
Hilborn, R. C. (1994) · Choice Reviews Online · doi:10.5860/choice.32-0359 · Cited by 917 · see citing works
Beamtimes and Lifetimes: The World of High Energy Physicists
Traweek, S., & Kernan, A. (1990) · Physics Today · doi:10.1063/1.2810632 · Cited by 905 · see citing works
Alfredo Cuzzocrea, J. A., Cuzzocrea, A., Allan, J., Paton, N. W., Srivastava, D., Agrawal, R., Broder, A., Zaki, M. J., Selçuk Candan, K., Labrinidis, A., et al. (2006) · doi:10.1145/1183614 · Cited by 756 · see citing works
Characteristic-Based Clustering for Time Series Data
Wang, X., Smith‐Miles, K., & Hyndman, R. J. (2006) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-005-0039-x · Cited by 695 · see citing works
Some Results on the Function and Quality of Citations
Moravcsik, M. J., & Murugesan, P. (1975) · Social Studies of Science · doi:10.1177/030631277500500106 · Cited by 693 · see citing works
Scientific Output and Recognition: A Study in the Operation of the Reward System in Science
Cole, S., & Cole, J. R. (1967) · American Sociological Review · doi:10.2307/2091085 · Cited by 655 · see citing works
Callaham, M. (2002) · JAMA · doi:10.1001/jama.287.21.2847 · Cited by 644 · see citing works
A Practical Guide to Heavy Tails: Statistical Techniques and Applications
Lenth, R. V., Alder, R. J., Feldman, R. E., & Taqqu, M. S. (1999) · Journal of the American Statistical Association · doi:10.2307/2670194 · Cited by 639 · see citing works
Citation indexes for science. A new dimension in documentation through association of ideas†
Garfield, E. (2006) · International Journal of Epidemiology · doi:10.1093/ije/dyl189 · Cited by 510 · see citing works
Studying the history of ideas using topic models
Hall, D., Jurafsky, D. S., & Manning, C. D. (2008) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1613715.1613763 · Cited by 459 · see citing works
PageRank for ranking authors in co‐citation networks
Ding, Y., Yan, E., Frazho, A. E., & Caverlee, J. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21171 · Cited by 407 · see citing works
Earlier Web usage statistics as predictors of later citation impact
Brody, T., Harnad, S. R., & Carr, L. A. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20373 · Cited by 403 · see citing works
Bornmann, L., & Daniel, H. (2008) · EMBO Reports · doi:10.1038/embor.2008.233 · Cited by 396 · see citing works
POWER OF THE NEURAL NETWORK LINEARITY TEST
Teräsvirta, T., Lin, C., & Granger, C. W. J. (1993) · Journal of Time Series Analysis · doi:10.1111/j.1467-9892.1993.tb00139.x · Cited by 381 · see citing works
Content Analysis of References: Adjunct or Alternative to Citation Counting?
Chubin, D. E., & Moitra, S. D. (1975) · Social Studies of Science · doi:10.1177/030631277500500403 · Cited by 357 · see citing works
Self-similarity and heavy tails: structural modeling of network traffic
Willinger, W., Paxson, V., & Taqqu, M. S. (1998) · Cited by 340 · see citing works
Nobel Laureates in Science: Patterns of Productivity, Collaboration, and Authorship
Zuckerman, H. (1967) · American Sociological Review · doi:10.2307/2091086 · Cited by 332 · see citing works
Predictive effects of structural variation on citation counts
Chen, C. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21694 · Cited by 317 · see citing works
Let's make science metrics more scientific
Lane, J. I. (2010) · Nature · doi:10.1038/464488a · Cited by 308 · see citing works
World citation and collaboration networks: uncovering the role of geography in science
Pan, R. K., Kaski, K., & Fortunato, S. (2012) · Scientific Reports · doi:10.1038/srep00902 · Cited by 304 · see citing works
Gehrke, J. E., Ginsparg, P., & Kleinberg, J. M. (2003) · ACM SIGKDD Explorations Newsletter · doi:10.1145/980972.980992 · Cited by 285 · see citing works
Quantitative Measures of Communication in Science: A Critical Review
Edge, D. O. (1979) · History of Science · doi:10.1177/007327537901700202 · Cited by 283 · see citing works
TRAWEEK, S. (1992) · Harvard University Press eBooks · doi:10.2307/j.ctv260711b · Cited by 276 · see citing works
Boyack, K. W., Newman, D. J., Duhon, R. J., Klavans, R., Patek, M., Biberstine, J. R., Schijvenaars, B. J. A., Skupin, A., Ma, N., & Börner, K. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018029 · Cited by 266 · see citing works
Measuring academic influence: Not all citations are equal
Zhu, X., Turney, P. D., Lemire, D., & Vellino, A. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23179 · Cited by 256 · see citing works
Acuna, D. E., Allesina, S., & Kording, K. P. (2012) · Nature · doi:10.1038/489201a · Cited by 249 · see citing works
Rohde, H., Elsner, M., & Clarke, A. V. (2014) · The Association for Computational Linguistics · Cited by 245 · see citing works
Public Knowledge: An essay Concerning the Social Dimension of Science
(1969) · doi:10.1063/1.3035233 · Cited by 229 · see citing works
Some Correlates of a Citation Measure of Productivity in Science
Bayer, A. E., & Folger, J. K. (1966) · Sociology of Education · doi:10.2307/2111920 · Cited by 208 · see citing works
A Review of Relation Extraction
Yang, Y., Xun, H., & You, H. (2013) · Shuju fenxi yu zhishi faxian · doi:10.11925/infotech.1003-3513.2013.11.05 · Cited by 193 · see citing works
Public Knowledge: An Essay concerning the Social Dimension of Science
Ziman, J. M. (1969) · CERN Document Server (European Organization for Nuclear Research) · Cited by 189 · see citing works
Batagelj, V., & Cerinšek, M. (2013) · Scientometrics · doi:10.1007/s11192-012-0940-1 · Cited by 186 · see citing works
Yan, R., Tang, J., Liu, X., Shan, D., & Li, X. (2011) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2063576.2063757 · Cited by 179 · see citing works
Science Studies: Bibliometric and Content Analysis
Spiegel-Rösing, I. (1977) · Social Studies of Science · doi:10.1177/030631277700700111 · Cited by 175 · see citing works
Predicting scientific success based on coauthorship networks
Sarigöl, E., Pfitzner, R., Scholtes, I., Garas, A., & Schweitzer, F. (2014) · EPJ Data Science · doi:10.1140/epjds/s13688-014-0009-x · Cited by 174 · see citing works
Textual Data Mining to Support Science and Technology Management
Losiewicz, P., Oard, D. W., & Kostoff, R. N. (2000) · Journal of Intelligent Information Systems · doi:10.1023/a:1008777222412 · Cited by 170 · see citing works
Characteristics Associated with Citation Rate of the Medical Literature
Kulkarni, A. V., Busse, J. W., & Shams, I. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000403 · Cited by 165 · see citing works
Sentiment Analysis of Citations using Sentence Structure-Based Features
Athar, A. (2011) · Cited by 158 · see citing works
Measuring the Results of Science Investments
Lane, J. I., & Bertuzzi, S. (2011) · Science · doi:10.1126/science.1201865 · Cited by 156 · see citing works
Lokker, C., McKibbon, K. A., McKinlay, R. J., Wilczynski, N. L., & Haynes, R. B. (2008) · BMJ · doi:10.1136/bmj.39482.526713.be · Cited by 152 · see citing works
Sun, X., Kaur, J., Milojević, S., Flammini, A., & Menczer, F. (2013) · Scientific Reports · doi:10.1038/srep01069 · Cited by 132 · see citing works
The NEGOPY network analysis program
Richards, W. D., & Rice, R. E. (1981) · Social Networks · doi:10.1016/0378-8733(81)90017-4 · Cited by 124 · see citing works
Comparison of the results of bibliographic coupling and analytic subject indexing
Kessler, M. M. (1965) · American Documentation · doi:10.1002/asi.5090160309 · Cited by 121 · see citing works
Scientometric analysis of the CHI proceedings
Bartneck, C., & Hu, J. (2009) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518810 · Cited by 120 · see citing works
Interpreting maps of science using citation context sentiments: a preliminary investigation
Small, H. G. (2011) · Scientometrics · doi:10.1007/s11192-011-0349-2 · Cited by 112 · see citing works
Google Scholar's ranking algorithm: The impact of citation counts (An empirical study)
Beel, J., & Gipp, B. (2009) · International Conference on Research Challenges in Information Science (RCIS) · doi:10.1109/rcis.2009.5089308 · Cited by 112 · see citing works
The Negational Reference: or the Art of Dissembling
MacRoberts, M. H., & MacRoberts, B. R. (1984) · Social Studies of Science · doi:10.1177/030631284014001006 · Cited by 104 · see citing works
Predicting publication success for biologists
Laurance, W. F., Useche, D. C., Laurance, S. G., & Bradshaw, C. J. A. (2013) · BioScience · doi:10.1525/bio.2013.63.10.9 · Cited by 100 · see citing works
A hybrid approach to extract protein–protein interactions
Bui, Q.-C., Katrenko, S., & Sloot, P. M. A. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq620 · Cited by 100 · see citing works
‘In this paper we report …'’: Speech acts and scientific facts
Myers, G. (1992) · Journal of Pragmatics · doi:10.1016/0378-2166(92)90013-2 · Cited by 99 · see citing works
Velden, T., Haque, A.-U., & Lagoze, C. (2010) · Scientometrics · doi:10.1007/s11192-010-0224-6 · Cited by 97 · see citing works
Entitymetrics: Measuring the Impact of Entities
Ding, Y., Song, M., Han, J., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0071416 · Cited by 93 · see citing works
Estimating Number of Citations Using Author Reputation
Castillo, C., Donato, D., & Gionis, A. (2007) · Springer eBooks · doi:10.1007/978-3-540-75530-2_10 · Cited by 84 · see citing works
Characteristics of a literature as predictors of relatedness between cited and citing works
Bonzi, S. (1982) · Journal of the American Society for Information Science · doi:10.1002/asi.4630330404 · Cited by 79 · see citing works
Louis, A. P., & Nenkova, A. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00232 · Cited by 74 · see citing works
The Structure of Scientific Articles: Applications to Citation Indexing and Summarization
Teufel, S. (2010) · Cited by 69 · see citing works
Self-citationPredicting author h-index using characteristics of the co-author network
McCarty, C., Jawitz, J. W., Hopkins, A. L., & Goldman, A. (2012) · Scientometrics · doi:10.1007/s11192-012-0933-0 · Cited by 69 · see citing works
Six degrees of jonathan grudin
Horn, D. B., Finholt, T. A., Birnholtz, J. P., et al. (2004) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1031607.1031707 · Cited by 69 · see citing works
Will This Paper Increase Your h-index? Scientific Impact PredictionPreprint
Dong, Y., Johnson, R. A., & Chawla, N. V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.4754 · Cited by 67 · see citing works
Predicting citation count ofBioinformaticspapers within four years of publication
Ibáñez, A., Larrañaga, P., & Bielza, C. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp585 · Cited by 57 · see citing works
Models for predicting and explaining citation count of biomedical articles.
Fu, L. D., & Aliferis, C. (2008) · Europe PMC (PubMed Central) · Cited by 55 · see citing works
Predicting a Scientific Community’s Response to an Article
Yogatama, D., Heilman, M., O’Connor, B. T., Dyer, C., Routledge, B., & Smith, N. A. (2011) · KiltHub Repository · Cited by 51 · see citing works
Concept-based analysis of scientific literature
Tsai, C.-T., Kundu, G., & Roth, D. (2013) · Proceedings of the ... ACM International Conference on Information & Knowledge Management · doi:10.1145/2505515.2505613 · Cited by 51 · see citing works
On time-varying collaboration networksPreprint
Viana, M. P., Amancio, D. R., Paulo, S. C. S., Costa, L. D. F., & Paulo, S. C. S. (2013) · Cited by 46 · see citing works
How relevant is the predictive power of the h-index? A case study of the time-dependent Hirsch index
Schreiber, M. (2013) · RePEc: Research Papers in Economics · Cited by 45 · see citing works
Proceedings of the ACL 2011 Student Session
Petrović, S., Pitler, E., & Selfridge, E. O. (2011) · Cited by 41 · see citing works
How relevant is the predictive power of the h-index? A case study of the time-dependent Hirsch index
Schreiber, M. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.01.001 · Cited by 40 · see citing works
On time-varying collaboration networks
Viana, M. P., Amancio, D. R., & Costa, L. D. F. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2012.12.005 · Cited by 38 · see citing works
Learning Dictionaries for Named Entity Recognition using Minimal Supervision
Neelakantan, A., & Collins, M. (2014) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/v1/e14-1048 · Cited by 37 · see citing works
Self-citationFuture Influence Ranking of Scientific LiteraturePreprint
Wang, S., Xie, S., Zhang, X., Li, Z., Yu, P. S., & Shu, X. (2016) · Cited by 31 · see citing works
Bibliometric indicators of young authors in astrophysics: Can later stars be predicted?
Havemann, F., & Larsen, B. (2014) · Scientometrics · doi:10.1007/s11192-014-1476-3 · Cited by 30 · see citing works
Tan, C., & Lee, L. (2014) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/v1/p14-2066 · Cited by 27 · see citing works
Hönekopp, J., & Khan, J. (2011) · Scientometrics · doi:10.1007/s11192-011-0551-2 · Cited by 26 · see citing works
Commentary: The case for caution in predicting scientists’ future impact
Penner, O., Petersen, A. M., Pan, R. K., & Fortunato, S. (2013) · IMT Institutional Repository (IMT School for Advanced Studies Lucca) · Cited by 26 · see citing works
Commentary: The case for caution in predicting scientists’ future impact
Penner, O., Petersen, A. M., Pan, R. K., & Fortunato, S. (2013) · Physics Today · doi:10.1063/pt.3.1928 · Cited by 25 · see citing works
Future Influence Ranking of Scientific Literature
Wang, S., Xie, S., Zhang, X., et al. (2014) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611973440.86 · Cited by 24 · see citing works
Birnholtz, J. P., Guha, S., Yuan, Y. C., Gay, G. K., & Heller, C. A. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22807 · Cited by 24 · see citing works
Velden, T., & Lagoze, C. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22929 · Cited by 22 · see citing works
Citation prediction using time series approach KDD Cup 2003 (task 1)
Manjunatha, J. N., Sivaramakrishnan, K. R., Pandey, R. K., et al. (2003) · ACM SIGKDD Explorations Newsletter · doi:10.1145/980972.980993 · Cited by 22 · see citing works
Experimenting with the partnership ability φ-index on a million computer scientists
Cabanac, G. (2012) · Scientometrics · doi:10.1007/s11192-012-0862-y · Cited by 15 · see citing works
Following bibliometric footprints
Guha, S., Steinhardt, S. B., Ahmed, S. I., et al. (2013) · ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL) · doi:10.1145/2467696.2467732 · Cited by 14 · see citing works
Eurekometrics: Analyzing the Nature of Discovery
Arbesman, S., & Christakis, N. A. (2011) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1002072 · Cited by 12 · see citing works
A Dynamic Network Approach to Breakthrough InnovationPreprint
Funk, R. J., & Owen-Smith, J. D. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2189528 · Cited by 4 · see citing works
Large-scale author coreference via hierarchical entity representations
Wick, M., Kobren, A., & McCallum, A. (2013) · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Ravenscroft, J., Liakata, M., Clare, A., & Duma, D. (2017) · PLOS ONE · doi:10.1371/journal.pone.0173152 · Cited by 202 · see citing works
- Authors
- McKeown, K., Daume, H., Chaturvedi, S., Paparrizos, J., Thadani, K., Barrio, P., Biran, O., Bothe, S., Collins, M., Fleischmann, K. R., Gravano, L., Jha, R., King, B., McInerney, K., Moon, T., Neelakantan, A., O'Seaghdha, D., Radev, D., Templeton, C., & Teufel, S.
- Affiliations
- Columbia University (US)
- Computer Science University of Maryland College Park MD 20742 USA
- University of Maryland, College Park (US)
- The University of Texas at Austin (US)
- University of Michigan (US)
- Ubiquiti, Inc. 303 Detroit Street Suite #202 Ann Arbor MI 48109 USA
- Rutgers, The State University of New Jersey (US)
- IBM (United States) (US)
- University of Massachusetts Amherst (US)
- University of Cambridge (GB)
- Corresponding author
- McKeown, Kathy
- Chaturvedi, Snigdha
- Paparrizos, John
- Thadani, Kapil
- Barrio, Pablo
- Biran, Or
- Bothe, Suvarna
- Collins, Michael
- Fleischmann, Kenneth R.
- Gravano, Luis
- Jha, Rahul
- King, Ben
- McInerney, Kevin
- Moon, Taesun
- Neelakantan, Arvind
- O'Seaghdha, Diarmuid
- Radev, Dragomir
- Templeton, Clay
- Teufel, Simone
- ORCID
- Venue
- Journal of the Association for Information Science and Technology
- Publisher
- Wiley
- Published
- 2016-01-11
- Volume
- 67
- Issue
- 11
- Pages
- 2684-2696
- ISSN
- 2330-1635 (ISSN-L)
- 2330-1643
- OpenAlex
- W2479682262
- Cited by
- 73
- References
- 107
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 9 · 2023: 6 · 2022: 5 · 2021: 9 · 2020: 9 · 2019: 5 · 2018: 10 · 2017: 8 · 2016: 4
- FWCI
- 8.38
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the Association for Information Science and Technology (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Deep Blue (University of Michigan) (submittedVersion)
- Funders
- Department of Interior National Business Center (DoI/NBC) (D11PC20153)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Keywords
- technical terms, information extraction, time series analysis, bibliometric analysis, scientific publications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review