Content Analysis of References: Adjunct or Alternative to Citation Counting?
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Chubin and Moitra (1975) found only very low percentages of negational citations (6 percent and 4 percent).
“Chubin and Moitra (1975) found only very low percentages of negational citations (6 percent and 4 percent).”
From What do citation counts measure? A review of studies on citing behavior (2008) · cites this paper as Chubin and Moitra (1975)
In theoretical high energy physics, papers cited as partial negational were highly cited immediately after publication and then declined rapidly, whereas papers cited as affirmative essential did not show that course.
“For works that had been cited in the form of partial negational citations, it was noteworthy that they were highly cited immediately after publication, and then there was a rapid rate of decay (decline in citations to these sources). Papers that had been cited in the form of affirmative essential citations did not show that kind of course.”
From What do citation counts measure? A review of studies on citing behavior (2008) · cites this paper as Chubin and Moitra (1975)
About one-fifth of the citations were found to be affirmative perfunctory citations.
“About one-fifth of the citations were found to be affirmative perfunctory citations.”
From What do citation counts measure? A review of studies on citing behavior (2008) · cites this paper as Chubin and Moitra (1975)
Chubin and Moitra (1975) reached similar results: citations were most frequently affirmative basic essential (27 percent and 13 percent) or affirmative subsidiary essential (13 percent and 34 percent), and affirmative additional supplementary citations (32 percent) were also made.
“Chubin and Moitra reached similar results: the citations made by the scientists in letters and full-length articles were most frequently affirmative basic essential (27 percent and 13 percent) or affirmative subsidiary essential (13 percent and 34 percent) citations, and affirmative additional supplementary citations (32 percent) were also made”
From What do citation counts measure? A review of studies on citing behavior (2008) · cites this paper as Chubin and Moitra (1975)
Most published content analyses provide no operational definition for the judgment tasks and report no reliability coefficient for the decisions made by judges.
“Most published content analyses provide no operational definition for the judgment tasks and report no reliability coefficient for the decisions made by judges (e.g. Chubin and Moitra, 1975; Spiegel-Ro ¨sing, 1977).”
From What do citation counts measure? A review of studies on citing behavior (2008) · cites this paper as Chubin and Moitra, 1975
Cites
Still being checked.
Cited by
Sorted by citations
What do citation counts measure? A review of studies on citing behavior
Bornmann, L., & Daniel, H. (2008) · Journal of Documentation · doi:10.1108/00220410810844150 · Cited by 1,607 · see citing works
Mizruchi, M. S., & Fein, L. C. (1999) · Administrative Science Quarterly · doi:10.2307/2667051 · Cited by 1,113 · see citing works
Is citation analysis a legitimate evaluation tool?
Garfield, E. (1979) · Scientometrics · doi:10.1007/bf02019306 · Cited by 911 · see citing works
O papel da informetria e da cienciometria e sua perspectiva nacional e internacional
Macías-Chapula, C. A. (1998) · Ciência da Informação · doi:10.1590/s0100-19651998000200005 · Cited by 580 · see citing works
Martin, B. R., & Irvine, J. M. (1983) · Research Policy · doi:10.1016/0048-7333(83)90005-7 · Cited by 529 · see citing works
Automatic classification of citation function
Teufel, S., Siddharthan, A., & Tidhar, D. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610091 · Cited by 437 · see citing works
Leydesdorff, L. (1998) · Scientometrics · doi:10.1007/bf02458391 · Cited by 347 · see citing works
Nicolaisen, J. (2007) · Annual Review of Information Science and Technology · doi:10.1002/aris.2007.1440410120 · Cited by 295 · see citing works
Morris, S. A., & Martens, B. V. D. V. (2008) · Annual Review of Information Science and Technology · doi:10.1002/aris.2008.1440420113 · Cited by 286 · see citing works
Explaining Creativity: The Attributional Perspective
Kasof, J. (1995) · Creativity Research Journal · doi:10.1207/s15326934crj0804_1 · Cited by 261 · see citing works
Private acts and public objects: An investigation of citer motivations
Brooks, T. A. (1985) · Journal of the American Society for Information Science · doi:10.1002/asi.4630360402 · Cited by 256 · see citing works
An interview-based study of the functions of citations in academic writing across two disciplines
Harwood, N. (2009) · Journal of Pragmatics · doi:10.1016/j.pragma.2008.06.001 · Cited by 255 · see citing works
Measuring academic influence: Not all citations are equal
Zhu, X., Turney, P. D., Lemire, D., & Vellino, A. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23179 · Cited by 256 · see citing works
What is project governance and what are its origins?
Ahola, T., Ruuska, I., Artto, K. A., & Kujala, J. (2013) · International Journal of Project Management · doi:10.1016/j.ijproman.2013.09.005 · Cited by 263 · see citing works
Citation Analysis and Discourse Analysis
Swales, J. M. (1986) · Applied Linguistics · doi:10.1093/applin/7.1.39 · Cited by 240 · see citing works
CO‐CITATION CONTEXT ANALYSIS AND THE STRUCTURE OF PARADIGMS
Small, H. G. (1980) · Journal of Documentation · doi:10.1108/eb026695 · Cited by 236 · see citing works
Moore, S., Shiell, A., Hawe, P., & Haines, V. A. (2005) · American Journal of Public Health · doi:10.2105/ajph.2004.046094 · Cited by 216 · see citing works
What is Special about Patent Citations? Differences between Scientific and Patent Citations
Meyer, M. S. (2000) · Scientometrics · doi:10.1023/a:1005613325648 · Cited by 218 · see citing works
Measuring the Evolution of a Scientific Field through Citation Frames
Jurgens, D., Kumar, S., Hoover, R., McFarland, D., & Jurafsky, D. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00028 · Cited by 212 · see citing works
An indicator of cross-disciplinary research
Porter, A. L., & Chubin, D. E. (1985) · Scientometrics · doi:10.1007/bf02016934 · Cited by 205 · see citing works
Self-citationFactors affecting citation rates of research articles
Onodera, N., & Yoshikane, F. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23209 · Cited by 202 · see citing works
State of the Field The Conceptualization of Scientific Specialties
Chubin, D. E. (1976) · Sociological Quarterly · doi:10.1111/j.1533-8525.1976.tb01715.x · Cited by 201 · see citing works
Self-citationContent‐based citation analysis: The next generation of citation analysis
Ding, Y., Zhang, G., Chambers, T., Song, M., Wang, X., & Zhai, C. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23256 · Cited by 252 · see citing works
PROGRESS IN DOCUMENTATION THE COMPLEXITIES OF CITATION PRACTICE: A REVIEW OF CITATION STUDIES
Liu, M. (1993) · Journal of Documentation · doi:10.1108/eb026920 · Cited by 187 · see citing works
THE NEED FOR A THEORY OF CITING
Cronin, B. (1981) · Journal of Documentation · doi:10.1108/eb026703 · Cited by 181 · see citing works
Wouters, P. F. (1999) · UvA-DARE (University of Amsterdam) · Cited by 180 · see citing works
Zhang, G., Ding, Y., & Milojević, S. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22850 · Cited by 169 · see citing works
Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature
Tahamtan, I., & Bornmann, L. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.01.002 · Cited by 149 · see citing works
Citation analysis and the complex problem of intellectual influence
Zuckerman, H. (1987) · Scientometrics · doi:10.1007/bf02016675 · Cited by 149 · see citing works
Baird, L. M., & Oppenheim, C. (1994) · Journal of Information Science · doi:10.1177/016555159402000102 · Cited by 149 · see citing works
Motivations for citation: A comparison of self citation and citation to others
Bonzi, S., & Snyder, H. W. (1991) · Scientometrics · doi:10.1007/bf02017571 · Cited by 145 · see citing works
Citations: Indicators of Quality? The Impact Fallacy
Loet Leydesdorff, A., Bornmann, L., Comins, J. A., & Milojević, S. (2016) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2016.00001 · Cited by 142 · see citing works
The distribution of references across texts: Some implications for citation analysis
Ding, Y., Liu, X., Guo, C., & Cronin, B. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.03.003 · Cited by 137 · see citing works
Invited Review Article: Bibliometrics and Evaluation of Research Performance
Luukkonen, T. (1990) · Annals of Medicine · doi:10.3109/07853899009147259 · Cited by 137 · see citing works
A quasi-quantitative citation model
Vinkler, P. (1987) · Scientometrics · doi:10.1007/bf02016689 · Cited by 132 · see citing works
M endeley readership altmetrics for medical articles: An analysis of 45 fields
Thelwall, M. A., & Wilson, P. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23501 · Cited by 127 · see citing works
Wang, P., & White, M. D. (1999) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(1999)50:2 · Cited by 126 · see citing works
Dimensions of Citation Analysis
Loet Leydesdorff, A., & Amsterdamska, O. (1990) · Science Technology & Human Values · doi:10.1177/016224399001500303 · Cited by 125 · see citing works
(2013) · Cited by 121 · see citing works
An annotation scheme for citation function
Teufel, S., Siddharthan, A., & Tidhar, D. (2006) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.3115/1654595.1654612 · Cited by 121 · see citing works
Highly cited old papers and the reasons why they continue to be cited
Oppenheim, C., & Renn, S. P. (1978) · Journal of the American Society for Information Science · doi:10.1002/asi.4630290504 · Cited by 200 · see citing works
Citation context analysis and aging patterns of journal articles in molecular genetics
McCain, K. W., & Turner, K. (1989) · Scientometrics · doi:10.1007/bf02017729 · Cited by 118 · see citing works
Citation context analysis of a co-citation cluster: Recombinant-DNA
Small, H. G., & Greenlee, E. (1980) · Scientometrics · doi:10.1007/bf02016349 · Cited by 118 · see citing works
How much is too much? The difference between research influence and self-citation excess
Szomszor, M., Pendlebury, D., & Adams, J. R. (2020) · Scientometrics · doi:10.1007/s11192-020-03417-5 · Cited by 124 · see citing works
Is citation analysis a legitimate evaluation tool?
Chubin, D. E., & Garfield, E. (1980) · Scientometrics · doi:10.1007/bf02016602 · Cited by 114 · see citing works
Self-citationInterpreting maps of science using citation context sentiments: a preliminary investigation
Small, H. G. (2011) · Scientometrics · doi:10.1007/s11192-011-0349-2 · Cited by 112 · see citing works
NLP-driven citation analysis for scientometrics
Jha, R. K., JBARA, A.-A., Qazvinian, V., & Radev, D. (2016) · Natural Language Engineering · doi:10.1017/s1351324915000443 · Cited by 108 · see citing works
How Are Academic Age, Productivity and Collaboration Related to Citing Behavior of Researchers?
Milojević, S. (2012) · PLoS ONE · doi:10.1371/journal.pone.0049176 · Cited by 106 · see citing works
A classification of citation roles for the social sciences and related fields
Peritz, B. C. (1983) · Scientometrics · doi:10.1007/bf02147226 · Cited by 105 · see citing works
The Negational Reference: or the Art of Dissembling
MacRoberts, M. H., & MacRoberts, B. R. (1984) · Social Studies of Science · doi:10.1177/030631284014001006 · Cited by 104 · see citing works
An appraisal of athlete development models through citation network analysis
Bruner, M. W., Erickson, K. T., Wilson, B., & Côté, J. (2009) · Psychology of sport and exercise · doi:10.1016/j.psychsport.2009.05.008 · Cited by 104 · see citing works
Unraveling scientific impact: Citation types in marketing journals
Stremersch, S., Camacho, N., Vanneste, S., & Verniers, I. (2015) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2014.09.004 · Cited by 102 · see citing works
Interpreting social science link analysis research: A theoretical framework
Thelwall, M. A. (2005) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20253 · Cited by 100 · see citing works
Citations and the evaluation of individual scientists
Cole, S. (1989) · Trends in Biochemical Sciences · doi:10.1016/0968-0004(89)90078-9 · Cited by 101 · see citing works
Measuring the Quality of Academic Journals: The Case of Economics
Beed, C., & Beed, C. (1996) · Journal of Post Keynesian Economics · doi:10.1080/01603477.1996.11490078 · Cited by 98 · see citing works
White, M. D., & Wang, P. (1997) · The Library Quarterly · doi:10.1086/629929 · Cited by 97 · see citing works
Lawani, S. M., & Bayer, A. E. (1983) · Journal of the American Society for Information Science · doi:10.1002/asi.4630340109 · Cited by 95 · see citing works
Variation of the nature of citation measures with journals and scientific specialties
Murugesan, P., & Moravcsik, M. J. (1978) · Journal of the American Society for Information Science · doi:10.1002/asi.4630290307 · Cited by 90 · see citing works
Referential Behavior in Scientific Writing: aDiachronic Study (1810–1995)
Salager‐Meyer, F. (1999) · English for Specific Purposes · doi:10.1016/s0889-4906(97)00042-2 · Cited by 83 · see citing works
The design of an online concordancing program for teaching about reporting verbs
Bloch, J. (2009) · Language learning & technology · doi:10.64152/10125/44168 · Cited by 81 · see citing works
Krampen, G., Becker, R., Wahner, U., & Montada, L. (2007) · Scientometrics · doi:10.1007/s11192-007-1659-2 · Cited by 80 · see citing works
Characteristics of a literature as predictors of relatedness between cited and citing works
Bonzi, S. (1982) · Journal of the American Society for Information Science · doi:10.1002/asi.4630330404 · Cited by 79 · see citing works
Citation Context Analysis as a Method for Conducting Rigorous and Impactful Literature Reviews
Anderson, M. H., & Lemken, R. K. (2020) · Organizational Research Methods · doi:10.1177/1094428120969905 · Cited by 83 · see citing works
Differences in Impact of Scientific Publications: Some Indices Derived from a Citation Analysis
Dieks, D., & Chang, H. (1976) · Social Studies of Science · doi:10.1177/030631277600600204 · Cited by 77 · see citing works
Predicting the impact of scientific concepts using full-text features
McKeown, K., Daume, H., Chaturvedi, S., Paparrizos, J., Thadani, K., Barrio, P., Biran, O., Bothe, S., Collins, M., Fleischmann, K. R., et al. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23612 · Cited by 73 · see citing works
The persistence of error: a study of retracted articles on the Internet and in personal libraries
Davis, P. M. (2012) · Journal of the Medical Library Association : JMLA · doi:10.3163/1536-5050.100.3.008 · Cited by 74 · see citing works
How is research blogged? A content analysis approach
Shema, H., Bar‐Ilan, J., & Thelwall, M. A. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23239 · Cited by 73 · see citing works
Iqbal, S., Hassan, S., Aljohani, N. R., Alelyani, S., Nawaz, R., & Bornmann, L. (2021) · Scientometrics · doi:10.1007/s11192-021-04055-1 · Cited by 77 · see citing works
Cronin, B. (1991) · Journal of Documentation · doi:10.1108/eb026878 · Cited by 71 · see citing works
The Law of Exponential Growth: Evidence, Implications and Forecasts.
Tague, J. M., Beheshti, J., & Rees-Potter, L. (1981) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 71 · see citing works
Hassan, S., Safder, I., Akram, A., & Kamiran, F. (2018) · Scientometrics · doi:10.1007/s11192-018-2767-x · Cited by 70 · see citing works
Citation Analysis: Queries and Caveats
Porter, A. L. (1977) · Social Studies of Science · doi:10.1177/030631277700700207 · Cited by 70 · see citing works
The knowledge pool: Measurement challenges in evaluating fundamental research programs
Cozzens, S. E. (1997) · Evaluation and Program Planning · doi:10.1016/s0149-7189(96)00038-9 · Cited by 68 · see citing works
Deep context of citations using machine-learning models in scholarly full-text articles
Hassan, S., Imran, M., Iqbal, S., Aljohani, N. R., & Nawaz, R. (2018) · Scientometrics · doi:10.1007/s11192-018-2944-y · Cited by 65 · see citing works
Scientific Text Sentiment Analysis using Machine Learning Techniques
Raza, H., Faizan, M., Hamza, A., Mushtaq, A., & Akhtar, N. (2019) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2019.0101222 · Cited by 67 · see citing works
Can A mazon.com reviews help to assess the wider impacts of books?
Kousha, K., & Thelwall, M. A. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23404 · Cited by 61 · see citing works
Hou, W., Li, M., & Niu, D. (2011) · BioEssays · doi:10.1002/bies.201100067 · Cited by 62 · see citing works
Wouters, P. F. (1998) · Scientometrics · doi:10.1007/bf02457980 · Cited by 62 · see citing works
Ahmed, T., Johnson, B., Oppenheim, C., & Peck, C. (2004) · Scientometrics · doi:10.1023/b:scie.0000041645.60907.57 · Cited by 58 · see citing works
McLaren, C. D., & Bruner, M. W. (2022) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1989705 · Cited by 61 · see citing works
National research impact indicators from Mendeley readers
Fairclough, R., & Thelwall, M. A. (2015) · RePEc: Research Papers in Economics · Cited by 57 · see citing works
On the objectives of citation analysis: Problems of theory and method
Peritz, B. C. (1992) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199207)43:6 · Cited by 56 · see citing works
Using Social Network Analysis to Analyze Relationships Among IS Journals
Polites, G. L., & Watson, R. T. (2009) · Journal of the Association for Information Systems · doi:10.17705/1jais.00206 · Cited by 54 · see citing works
Norms and functions in citation: The view of journal editors and referees in psychology
Cronin, B. (1982) · Social Science Information Studies · doi:10.1016/0143-6236(82)90001-1 · Cited by 54 · see citing works
Martin, B. R., & Irvine, J. M. (1993) · Research Policy · doi:10.1016/0048-7333(93)90049-n · Cited by 53 · see citing works
Whitehand, J. W. R. (1985) · Transactions of the Institute of British Geographers · doi:10.2307/621825 · Cited by 53 · see citing works
How socially connected are citers to those that they cite?
Johnson, B., & Oppenheim, C. (2007) · Journal of Documentation · doi:10.1108/00220410710827727 · Cited by 52 · see citing works
Bordignon, F. (2020) · Scientometrics · doi:10.1007/s11192-020-03536-z · Cited by 51 · see citing works
Towards a Generic and Flexible Citation Classifier Based on a Faceted Classification Scheme
Jochim, C., & Schütze, H. (2012) · Cited by 51 · see citing works
The linguistic patterns and rhetorical structure of citation context: An approach using n-grams
Bertin, M., Atanassova, I., Sugimoto, C. R., & Lariviere, V. (2016) · Scientometrics · doi:10.1007/s11192-016-2134-8 · Cited by 50 · see citing works
The multilayered nature of reference selection
Camacho‐Miñano, M., & Nickel, M. N. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21018 · Cited by 49 · see citing works
Research on research evaluation
Hemlin, S. (1996) · Social Epistemology · doi:10.1080/02691729608578815 · Cited by 48 · see citing works
A TEST OF THE ORTEGA HYPOTHESIS IN CRIMINOLOGY
GREEN, G. S. (1981) · Criminology · doi:10.1111/j.1745-9125.1981.tb00403.x · Cited by 48 · see citing works
Reinstein, A., Hasselback, J. R., Riley, M. E., & Sinason, D. H. (2011) · Issues in Accounting Education · doi:10.2308/iace.2011.26.1.99 · Cited by 48 · see citing works
Some aspects of citation behavior: A pilot study in business administration
Prabha, C. G. (1983) · Journal of the American Society for Information Science · doi:10.1002/asi.4630340305 · Cited by 48 · see citing works
Literature review writing: how information is selected and transformed
Jaidka, K., Khoo, C. S. G., & Na, J. (2013) · Aslib Journal of Information Management · doi:10.1108/00012531311330665 · Cited by 46 · see citing works
Citations, contexts, and humanistic discourse: Toward automatic extraction and classification
Sula, C. A., & Miller, M. (2014) · Literary and Linguistic Computing · doi:10.1093/llc/fqu019 · Cited by 46 · see citing works
The classification of citing motivations: a meta-synthesis
Lyu, D., Ruan, X., Xie, J., & Cheng, Y. (2021) · Scientometrics · doi:10.1007/s11192-021-03908-z · Cited by 46 · see citing works
National research impact indicators from Mendeley readers
Fairclough, R., & Thelwall, M. (2015) · Journal of Informetrics · doi:10.1016/j.joi.2015.08.003 · Cited by 45 · see citing works
Hyperlinks as a data source for science mapping
Harries, G., Wilkinson, D., Price, L., Fairclough, R., & Thelwall, M. A. (2004) · Journal of Information Science · doi:10.1177/0165551504046736 · Cited by 44 · see citing works
- Authors
- Chubin, D. E., & Moitra, S. D.
- ORCID
- Chubin, Daryl E.
- Venue
- Social Studies of Science
- Publisher
- SAGE Publications
- Published
- 1975-11
- Volume
- 5
- Issue
- 4
- Pages
- 423-441
- ISSN
- 0036-8539 (ISSN-L)
- 0306-3127
- 1460-3659
- OpenAlex
- W1980282589
- Cited by
- 357
- References
- 0
- Citations by year
- 2026: 7 · 2025: 7 · 2024: 8 · 2023: 8 · 2022: 7 · 2021: 19 · 2020: 13 · 2019: 12 · 2018: 15 · 2017: 11 · 2016: 13 · 2015: 13 · 2014: 16 · 2013: 18 · 2012: 10
- FWCI
- 12.48
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Social Studies of Science (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- citation analysis, citation counting, bibliometric analysis, research evaluation, bibliographic references
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review