Cited by 73scientometrics and bibliometrics researchAdvanced Text Analysis TechniquesWeb visibility and informetrics

Predicting the impact of scientific concepts using full-text features

McKeown, Kathy, Daume, Hal, Chaturvedi, Snigdha, Paparrizos, John, Thadani, Kapil, Barrio, Pablo, Biran, Or, Bothe, Suvarna +12 more·2016·Journal of the Association for Information Science and Technology

Abstract

New scientific concepts, interpreted broadly, are continuously introduced in the literature, but relatively few concepts have a long‐term impact on society. The identification of such concepts is a challenging prediction task that would help multiple parties—including researchers and the general public—focus their attention within the vast scientific literature. In this paper we present a system that predicts the future impact of a scientific concept, represented as a technical term, based on the information available from recently published research articles. We analyze the usefulness of rich features derived from the full text of the articles through a variety of approaches, including rhetorical sentence analysis, information extraction, and time‐series analysis. The results from two large‐scale experiments with 3.8 million full‐text articles and 48 million metadata records support the conclusion that full‐text features are significantly more useful for prediction than metadata‐only features and that the most accurate predictions result from combining the metadata and full‐text features. Surprisingly, these results hold even when the metadata features are available for a much larger number of documents than are available for the full‐text features.

Integrity signals

Registry

Crossref · 10.1002/asi.23612

Cites

107 of 107 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
McKeown, K., Daume, H., Chaturvedi, S., Paparrizos, J., Thadani, K., Barrio, P., Biran, O., Bothe, S., Collins, M., Fleischmann, K. R., Gravano, L., Jha, R., King, B., McInerney, K., Moon, T., Neelakantan, A., O'Seaghdha, D., Radev, D., Templeton, C., & Teufel, S.
Affiliations
  • Columbia University (US)
  • Computer Science University of Maryland College Park MD 20742 USA
  • University of Maryland, College Park (US)
  • The University of Texas at Austin (US)
  • University of Michigan (US)
  • Ubiquiti, Inc. 303 Detroit Street Suite #202 Ann Arbor MI 48109 USA
  • Rutgers, The State University of New Jersey (US)
  • IBM (United States) (US)
  • University of Massachusetts Amherst (US)
  • University of Cambridge (GB)
Corresponding author
  • McKeown, Kathy
  • Chaturvedi, Snigdha
  • Paparrizos, John
  • Thadani, Kapil
  • Barrio, Pablo
  • Biran, Or
  • Bothe, Suvarna
  • Collins, Michael
  • Fleischmann, Kenneth R.
  • Gravano, Luis
  • Jha, Rahul
  • King, Ben
  • McInerney, Kevin
  • Moon, Taesun
  • Neelakantan, Arvind
  • O'Seaghdha, Diarmuid
  • Radev, Dragomir
  • Templeton, Clay
  • Teufel, Simone
Venue
Journal of the Association for Information Science and Technology
Publisher
Wiley
Published
2016-01-11
Volume
67
Issue
11
Pages
2684-2696
ISSN
  • 2330-1635 (ISSN-L)
  • 2330-1643
OpenAlex
W2479682262
Cited by
73
References
107
Citations by year
2026: 2 · 2025: 6 · 2024: 9 · 2023: 6 · 2022: 5 · 2021: 9 · 2020: 9 · 2019: 5 · 2018: 10 · 2017: 8 · 2016: 4
FWCI
8.38
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of the Association for Information Science and Technology (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Deep Blue (University of Michigan) (submittedVersion)
Funders
Department of Interior National Business Center (DoI/NBC) (D11PC20153)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Text Analysis Techniques
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Keywords
technical terms, information extraction, time series analysis, bibliometric analysis, scientific publications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review