CoChat
Sign Up
Cited by 102scientometrics and bibliometrics researchPsychology Research and BibliometricsAcademic Writing and Publishing

Unraveling scientific impact: Citation types in marketing journals

Open access
Stremersch, Stefan, Camacho, Nuno, Vanneste, Sofie, Verniers, Isabel·2015·International Journal of Research in Marketing
DOI 10.1016/j.ijresmar.2014.09.004
Open this paper in CoChat — free account

Abstract

The number of citations a paper receives is the most commonly used measure of scientific impact. In this paper, we study not only the number but also the type of citations that 659 marketing articles generated. We discern five citation types: application, affirmation, negation, review and perfunctory mention (i.e., citing an article only indirectly without really using it). Prior literature in scientometrics recognizes that the former three types, on average, signal a higher level of scientific indebtedness than the latter two types. In our sample, these three types of citation represent only 15% of all citations. We also find different determinants of citation behavior across citation types. Across the 49 determinants we included, only 13 have the same effect across all citation types, of which only 5 are statistically significant across all citation types. For instance, we find a significant inverted U-effect of challenging commonly held beliefs on citation counts, but only for three of the citation types: affirmation, review and perfunctory mention. Our results encourage scientific stakeholders to move beyond mere citation counts to assess a paper's or a scholar's scientific contribution, as well as to devote greater attention to the citation process itself.

AI snapshot

Written from the paper’s full text.

TL;DR

Stremersch et al. content-code 24,632 citations to 659 marketing articles and show that most citations are review or perfunctory rather than application, affirmation, or negation. They find that citation determinants differ substantially across citation types, and that challenging commonly held beliefs has an inverted-U relationship with total citations and with affirmation, review, and perfunctory citations.

Key findings

  • Across 24,632 coded citations, 52.9% were review citations, 31.8% perfunctory mentions, 10.1% application, 4.6% affirmation, and 0.6% negation; only about 15% of all citations were application, affirmation, or negation.
  • Challenging commonly held beliefs had an inverted-U effect on total citations: the highest expected number of citations occurred at a moderately high level (CCHB = 4 on the 7-point scale), and such a paper received 6.3 more citations than a paper with CCHB = 1, of which about 4.2 were review citations and 1.6 were perfunctory mentions. The inverted-U was statistically significant for affirmation, review, and perfunctory citations, but not for application or negation.
  • Across 49 determinants, only 13 had the same effect across all citation types, and only 5 were statistically significant across all citation types. Editorial board membership increased perfunctory mentions but not total citations; reference intensity increased total citations entirely through perfunctory mentions; number of co-authors increased application citations but not total citations or other citation types.
  • Journal-level differences were reported: IJRM had a lower share of perfunctory citations (27.6%) than JCR (32.0%), JM (31.4%), MKS (35.0%), and JMR (30.5%); JMR had a higher share of application citations (12.5%) than other journals.

Methods

  • Random sample of 659 cited articles balanced across years, journals, and volumes from 1990–2007 in IJRM, JCR, JM, JMR, and MKS; editorials, book reviews, and articles without marketing theory were excluded (7% of the full sample).
  • Citation types were content-coded by trained research assistants using five types: application, affirmation, negation, review, and perfunctory mention; 24,632 of 29,320 citing papers were coded. Challenging commonly held beliefs was measured with an 8-item scale based on Davis (1971), with ICC improving from .69 to .77 after a Delphi procedure.
  • Negative binomial models with quasi-maximum likelihood and Cameron–Trivedi overdispersion correction were estimated for total citations and for each citation type, using universalist, social constructivist, and presentation determinants.

Limitations

  • Stated: the observation window ends in 2007; results are conditional on acceptance, creating unavoidable survival bias; citation-type classification may mask nuances, especially within heterogeneous review citations; purely methodological articles without marketing theory were excluded; negation citations are rare (0.6%, N = 149), limiting analysis of their drivers; citation types remain citation counts and are an imperfect measure of impact, missing diffusion through textbooks, media, teaching, or practice.
  • Evident from the text: 4,688 citing papers could not be coded because of subscription access limits or non-English publication, though the authors report no particular pattern besides lower-tier or non-marketing sources.
  • Evident from the text: the challenging-commonly-held-beliefs median never equaled 7, so Figure 1 displays effects only over a 1-to-6 range.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.ijresmar.2014.09.004

Cites

97 of 97 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 34 cited sources checked · 0 supported · 0 flagged

  • Findingown data
    § a b s t r a c t · p. 3

    The effect of the number of co-authors on total citations and other citation types is insignificant.

    “The number of co-authors has a positive effect on application citations, while its effect on total citations and other citation types is insigni fi cant.”
    Quote in the paperSubstantial claim
  • Findingown data
    § 2.3. Determinants of citation behavior · p. 4

    The paper's survey of 59 respondents from 28 universities revealed that in 69% of cases the respondent's school includes the Davis (1971) article in doctoral education.

    “This survey revealed that, in 69% of the cases, the respondent's school includes the Davis (1971) article in their doctoral education.”
    Quote in the paperNot a substantial claim
  • Findingown data
    § a b s t r a c t · p. 3

    The highest number of citations occurs for a paper with a moderately high level of challenging commonly held beliefs.

    “The highest number of citations occurs for a paper with a moderately high level of challenging commonly held beliefs.”
    Quote in the paperSubstantial claim
  • Findingown data
    § a b s t r a c t · p. 3

    There is an inverted-U relationship between challenging commonly held beliefs and citation counts.

    “We fi nd an inverted-U relationship between challenging commonly held beliefs and citation counts and between challenging commonly held beliefs and three of the citation types: af fi rmation, review and perfunctory mention.”
    Quote in the paperSubstantial claim
  • Findingown data
    § a b s t r a c t · p. 2

    The paper finds different determinants of citation behavior across citation types.

    “We also fi nd different determinants of citation behavior across citation types.”
    Quote in the paperSubstantial claim
  • May combine two claims. 36
  • Classified as probably not a substantial claim. 30
  • The quote only partly states this claim. 4

  • May combine two claims.

    Findingown data
    § 3.2.2. Citation types · p. 5

    The missing citations did not show any particular pattern besides often being from lower tier journals and/or fields outside marketing.

    “Two of the authors examined these missing citations and did not fi nd any particular pattern besides the fact that these missing references are often from lower tier journals and/or fi elds outside marketing.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingown data
    § 3.2.2. Citation types · p. 5

    In 5% of cases for the 10 articles with highest missing citations, the citing paper was not written in English.

    “In 5% of all cases, the citing paper was not written in English.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 3.2.2. Citation types · p. 5

    The RAs were able to code citation types for 24,632 of 29,320 citing papers.

    “From the total of 29,320 citing papers to the 659 cited articles in our sample, our RAs were able to code the citation types for 24,632 citing papers.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data
    § 3.1. Data collection procedure · p. 5

    The final sample contained 88 articles from IJRM, 141 from JCR, 143 from JMR and MKS, and 144 from JM.

    “The fi nal sample contained 88 articles from IJRM, 141 articles from JCR, 143 articles from JMR and MKS and 144 articles from JM.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 3.3. Descriptives · p. 8

    For articles in MKS, the relative frequency of perfunctory citations was higher (35.0%) than all other journals, with differences statistically significant at the 1% level.

    “For articles in MKS the relative frequency of perfunctory citations was higher (35.0%) than all other journals, with these differences statistically signi fi cant at the 1% level.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.2. Model fit · p. 8

    The LRI of the full model compared to the null model is .13, representing a satisfactory fit.

    “The LRI of our full model compared to this null model is .13, which represents a satisfactory fi t.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.3. Descriptives · p. 8

    The relative frequency of review citations to JM articles (53.9%) was significantly higher than to JCR (52.1%) and JMR (51.3%), but not MKS (52.6%).

    “The relative frequency of review citations to JM articles (53.9%) was statistically signi fi cantly higher (at the 5% level) than to articles in JCR (52.1%; p =.04) and JMR (51.3%; p b .01), but not in MKS (52.6%; p =.17).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.4. Determinants of citation types · p. 10

    Of these 13 effects, only 5 are statistically significant across all citation types and 8 are statistically significant for some citation types but insignificant for others.

    “Of these 13 effects, only 5 are statistically signi fi cant across all citation types and 8 are statistically significant for some citation types, some at different signi fi cance levels, but statistically insigni fi cant for other citation types.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingreasoning
    § 4.4. Determinants of citation types · p. 11

    The more references an article contains, the more citations the article receives.

    “The more references an article contains the more citations the article receives.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 3.3. Descriptives · p. 7

    The average number of citations per article was 44.49 (range 0-662).

    “| Citations | Average [range] | 44.49 [0, 662] |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.3. Descriptives · p. 8

    There were no statistically significant differences in negation citation frequency between JCR and JM, or among IJRM and JMR, IJRM and MKS, and JMR and MKS.

    “There were no statistically signi fi cant differences between JCR and JM ( p =.99) or among IJRM and JMR ( p =.43), IJRM and MKS ( p =.49), and JMR and MKS ( p =.91).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 3.3. Descriptives · p. 7

    The average number of perfunctory citations per article was 11.88 (range 0-197).

    “| Citation functions: Perfunctory | Average [range] | 11.88 [0; 197] |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.3. Descriptives · p. 8

    The difference in perfunctory citation frequency between IJRM and JCR, JM, MKS was statistically significant at the 5% level, and between IJRM and JMR at the 10% level.

    “Pairwise z-tests show that these differences were statistically signi fi cant, at the 5% level, for JCR (which had 32.0% of perfunctory cites, p b .01), JM (31.4%, p = .014) and MKS (35.0%, p b .01) and, at the 10% level, for JMR (30.5%, p =.07).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.3. Determinants of citation counts · p. 10

    From these 6.3 additional citations, approximately two thirds are review citations (4.2 additional review citations) and approximately one quarter are perfunctory mentions (1.6 additional perfunctory mentions).

    “From these 6.3 additional citations, approximately two thirds are review citations (4.2 additional review citations) and approximately one quarter are perfunctory mentions (1.6 additional perfunctory mentions).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 3.3. Descriptives · p. 7

    The average number of negation citations per article was 0.23 (range 0-4).

    “| Citation functions: Negation | Average [range] | .23 [0; 4] |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.2. Model fit · p. 8

    The AIC of the full model is 8.77, while the AIC of the null model is 9.90, indicating the full model fits better.

    “The AIC of our full model is 8.77, while the AIC of the null model is 9.90, which indicates that our full model fi ts better.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data
    § 3.3. Descriptives · p. 7

    The average number of affirmation citations per article was 1.71 (range 0-37).

    “| Citation functions: Af fi rmation | Average [range] | 1.71 [0; 37] |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.3. Descriptives · p. 8

    Differences in review citation frequency between JMR and MKS, JCR and MKS, and JMR and JCR were statistically insignificant.

    “The differences between JMR and MKS ( p = .19), JCR and MKS ( p = .61), and JMR and JCR ( p =.39) were statistically insigni fi cant.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.4. Determinants of citation types · p. 10

    Wald tests confirm that among the remaining 36 effects, 19 are significantly different across citation types at the 5% level, and three additional at the 10% level.

    “Wald tests confi rm that, among the remaining 36 effects (i.e. among those which do not have the same sign across all citation types), 19 effects are signi fi cantly different across citation types at the 5% level. Three additional effects are signi fi cantly different at the 10% level.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.3. Determinants of citation counts · p. 9

    There is an inverted-U relationship between the degree of challenging commonly held beliefs and citations: a positive linear effect (φ1=.23, p<.05) and a negative quadratic effect (φ2=-.03, p<.05).

    “We fi nd an inverted-U relationship between the degree of challenging commonly held beliefs and citations.”
    • Also stated in: § 4.3. Determinants of citation counts · p. 9 · hedged
      “Articles that challenge commonly held beliefs may be more in fl uential ( φ 1 =.23, p b 0.05), as measured by citations, but this does not apply to articles that challenge commonly held beliefs to a very high degree, as we fi nd a negative quadratic effect ( φ 2= -.03; p b .05).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.5.3. Robustness: independent variables · p. 11

    There were no statistically significant differences in citation counts or in any citation type between contingency and non-contingency papers.

    “We did not fi nd statistically signi fi cant differences for citation counts, nor for any of the citation types, possibly due to the small sample size.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5. Conclusion · p. 12

    Editorial review board members receive more perfunctory citations than non-editorial board members, while for other types of citation they are cited equally frequently.

    “Editorial review board members receive more perfunctory citations than non-editorial board members, while for other types of citation they are cited equally frequently.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5. Conclusion · p. 12

    In the sample, 31.8% of all citations are perfunctory mentions and 52.9% are review citations.

    “In our sample, 31.8% of all citations are perfunctory mentions and 52.9% are review citations.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data
    § 5. Conclusion · p. 12

    Articles that challenge commonly held beliefs are more influential, as measured by citations.

    “we demonstrated that articles that challenge commonly held beliefs are indeed, as hypothesized by Murray Davis, more in fl uential, as measured by citations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Content analyses of the citations that papers receive are rare.

  • May combine two claims.

    Findingown data
    § 4.5.2. Robustness: functional form · p. 11

    The non-linear effect of challenging commonly held beliefs weakens over time, but articles with a low or very high level of challenging commonly held beliefs still receive fewer citations than articles with a moderately high level, even almost two decades after publication.

    “Even though this non-linear effect weakens over time (i.e. the effect of CCHBij becomes less positive and the effect of CCHBij 2 becomes signi fi cantly less negative), articles with a low or very high level of challenging commonly held beliefs still receive fewer citations than articles with a moderately high level of challenging commonly held beliefs, even almost two decades after being publishe”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5. Conclusion · p. 12

    Articles with more references receive more perfunctory mentions, but not more citations of other types.

    “Also articles with more references receive more perfunctory mentions, but not more citations of other types, con fi rming the ' you cite me, I'll cite you ' citation gaming, mentioned in prior literature.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § a b s t r a c t · p. 2

    Prior literature in scientometrics recognizes that application, affirmation, and negation citations, on average, signal a higher level of scientific indebtedness than review and perfunctory mentions.

    “Prior literature in scientometrics recognizes that the former three types, on average, signal a higher level of scienti fi c indebtedness than the latter two types.”
    • Also stated in: § a b s t r a c t · p. 3 · plain
      “They view af fi rmation, application, and negation citations as signaling a higher level of intellectual indebtedness towards the cited article than review cites and perfunctory mentions.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Chubin & Moitra, 1975

      Not checked in the free run.

    • cites Frost, 1979 source

      Not checked in the free run.

    • cites Moravcsik & Murugesan, 1975

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § a b s t r a c t · p. 3

    Scientific influence is a complex construct that depends not only on the number of citations a paper generates, but also on the types of citation it generates and on other indicators of scholarly impact.

    “Finally, scienti fi c in fl uence is a complex construct that depends not only on the number of citations a paper generates, but also on the types of citation it generates and on other indicators of scholarly impact.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 4

    Papers that challenge commonly held beliefs deny an aspect of the assumption-ground of their audience.

    “Papers that challenge commonly held beliefs deny an aspect of the assumption-ground of its audience (Davis, 1971).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Davis, 1971 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductnone given
    § a b s t r a c t · p. 3

    The sample consists of 659 cited articles randomly drawn from five major marketing journals, balanced across years, journals and volumes over 1990-2007.

    “Our sample consists of 659 cited articles randomly drawn from fi ve major marketing journals, International Journal of Research in Marketing (IJRM), Journal of Consumer Research (JCR), Journal of Marketing (JM), Journal of Marketing Research (JMR), and Marketing Science (MKS), balanced across years, journals and volumes over the time period 1990 -2007.”
    • Also stated in: § 3.1. Data collection procedure · p. 5 · plain
      “We randomly sampled 659 articles balanced across years, journals and volumes over the time period 1990 -2007.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation
    § 3.2.3. Determinants of citation behavior · p. 5

    The paper develops a measurement instrument containing 8 reflective items, based on Davis (1971), to measure the degree to which a paper challenges commonly held beliefs.

    “We also develop a measurement instrument containing 8 re fl ective items, based on Davis (1971) to measure the degree to which a paper challenges commonly held beliefs.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Davis, 1971 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 5

    Confirmatory bias may lead scholars to depart from scientific ideals such as objectivity and neutrality, and trigger resistance.

    “For instance, con fi rmatory bias -one's tendency to overweight con fi rmatory and underweight discon fi rmatory evidence -may lead scholars to depart from scienti fi c ideals such as objectivity and neutrality, and trigger resistance (Mahoney, 1979).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Mahoney, 1979 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 4

    The work of Davis (1971) has been very influential.

    “The work of Davis (1971) has been very infl uential (Bartunek et al., 2006) and is often included in doctoral education programs in marketing.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Bartunek et al., 2006 source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § a b s t r a c t · p. 2

    The number of citations a paper receives is the most commonly used measure of scientific impact.

    “The number of citations a paper receives is the most commonly used measure of scienti fi c impact.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites Egghe, 2006 source

      Not checked in the free run.

    • cites Hirsch, 2005, 2007

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Radicchi, Fortunato, & Castellano, 2008 source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 4.1. Model specification · p. 8

    When the mean-variance equality assumption is violated, overdispersion occurs, with consequences similar to heteroscedasticity in linear regression.

    “When the meanvariance equality assumption is violated, the observed variance of the dependent variable is typically higher than the variance estimated using the Poisson model, a problem known as overdispersion and whoseconsequences are similar to those of heteroscedasticity in the linear regression model (consistent parameter estimates but inconsistent standard errors leading to invalid hypothesis”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Cameron & Trivedi, 1990

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning
    § 3.2.2. Citation types · p. 5

    Application, affirmation and negation citation are the strongest types of citation and do not co-occur, given their definitions.

    “Application, af fi rmation and negation citation are the strongest types of citation and do not co-occur, given their de fi nitions.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.3. Descriptives · p. 7

    Perfunctory mentions in Physics, Sociology and Demography are even higher (up to 40% of all citations).

    “These authors also mention that the proportions of perfunctory mentions in other fi elds, namely Physics, Sociology and Demography, are even higher (up to 40% of all citations).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 4.4. Determinants of citation types · p. 11

    This may signal that reference intensity increases citation counts, not because of programmatic development of a research area, but more by a social constructivist logic, such as 'you cite me, I cite you' behavior.

    “This may signal that reference intensity increases citation counts, not because of programmatic development of a research area, but more by a social constructivist logic, such as ' you cite me, I cite you ' behavior.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.3. Descriptives · p. 7

    Peritz (1983) found only 3.7% of perfunctory mentions among citations to 106 articles in five journals in different fields.

    “The only exception to these high proportions of perfunctory citations comes from a study by Peritz (1983) who found only 3.7% of perfunctory mentions among the citations to 106 articles published in fi ve journals in different fi elds.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Peritz (1983) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 4

    Title length, use of attention grabbers, expositional clarity, readability, number of tables and figures, footnotes, keywords, appendices, and equations are presentation determinants of citations.

    “Think of title length, the use of attention grabbers in the title and expositional clarity, such as readability, number of tables and fi gures, number of footnotes, number of keywords, number of appendices and number of equations (Stremersch et al., 2007).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stremersch et al., 2007 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.3. Descriptives · p. 7

    Moravcsik and Murugesan (1975) found that perfunctory mentions represented 41% of all citations in Physical Review articles published between 1968 and 1972.

    “Moravcsik and Murugesan (1975) analyzed 2,286 references in 30 articles in Physical Review published between 1968 and 1972 and found that perfunctory mentions represented 41% of all citations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Moravcsik and Murugesan (1975)

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data
    § 4.4. Determinants of citation types · p. 11

    This effect suggests that authors may aim to please potential reviewers by citing their work or that editorial board members suggest authors to cite their own work in reviews they write.

    “Given that we control for many other variables, such as publication record, this effect suggests that authors may aim to please potential reviewers by citing their work or that editorial board members suggest authors to cite their own work in reviews they write.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 4

    Social constructivist determinants of citations may include the author's publication history, the author's affiliation, and a scholar's visibility and personal promotion.

    “Other social constructivist determinants of citations may include the author's publication history and the author's af fi liation (Judge et al., 2007) and a scholar's visibility and personal promotion (Stremersch et al., 2007).”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Judge et al., 2007 source

      Not checked in the free run.

    • cites Stremersch et al., 2007 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.3. Descriptives · p. 7

    Cano (1989) found that engineering scholars classified 25.8% of their own citations as perfunctory, and such citations had the lowest utility level.

    “Cano (1989) asked 21 engineering scholars to classify their citations, in two of their recent papers, according to citation type and to indicate the degree to which a citation was indispensable for their paper. Scholars classi fi ed 25.8% of their own citations as perfunctory, and they admitted that such citations had the lowest ' utility ' level.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Cano (1989) source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 4.3. Determinants of citation counts · p. 10

    These differences may be driven by variation in subject area popularity or in citation practices across subject areas.

    “These differences may be driven by variation in subject area popularity or in citation practices across subject areas.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 4

    Scientists that have accrued more citations will get even more.

    “Therefore, paraphrasing the Gospel of Matthew, Merton (1968) argued that scientists that have accrued more citations will get even more.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Merton, 1968 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.3. Descriptives · p. 6

    In Krampen et al. (2007), perfunctory mentions account for about 25% of citations in a study of 3,528 citations to 90 journal articles in five psychology journals.

    “Krampen, Becker, Wahner, and Montada (2007) examine3,528 citations to 90 journal articles in fi ve psychology journals and fi nd that perfunctory mentions account for about 25% of citations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Krampen, Becker, Wahner, and Montada (2007)

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data
    § 3.3. Descriptives · p. 7

    The results for the marketing journals studied suggest a high frequency of perfunctory mentions, which may make inference on impact based solely on number of citation counts problematic.

    “The results we obtain for the marketing journals we study suggest a high frequency of perfunctory mentions, which may make inference on impact based solely on number of citation counts problematic.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Premisecitation
    § 2.3. Determinants of citation behavior · p. 4

    The manner in which science is presented drives citations.

    “In the third perspective, the manner in which science is presented drives citations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Stremersch et al., 2007 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data
    § Appendix. Measuring Challenging Commonly Held Beliefs · p. 14

    The Cronbach's alpha was .99, the composite reliability was .99 and the variance extracted was .94, showing high reliability.

    “The Cronbach ' s alpha was .99, the composite reliability was .99 and the variance extracted was .94, showing high reliability.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § 5. Conclusion · p. 13

    Citation counts are only one possible and imperfect measure of an article's impact.

    “Citation counts are only one possible and imperfect measure of an article's impact (Lehmann et al., 2011).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Lehmann et al., 2011 source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data
    § Appendix. Measuring Challenging Commonly Held Beliefs · p. 14

    All factor loadings were statistically significant (p < .01) and the lowest factor loading equaled .96, showing high convergent validity.

    “All factor loadings were statistically signi fi cant (p b .01) and the lowest factor loading equaled .96, showing high convergent validity.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 5. Conclusion · p. 13

    Some articles may diffuse through knowledge vehicles other than scholarly cites, such as direct practitioner use, textbooks or media, or classroom instruction.

    “In particular, some articles may diffuse through knowledge vehicles other than scholarly cites (e.g. articles whose content is used directly by practitioners, featured in textbooks or media, or used by classroom instructors).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data
    § 4.5.3. Robustness: independent variables · p. 12

    Controlling for article age with Qij and ln(Qij) instead of Qij and Q2ij leaves the results highly similar, with no sign reversals in significant parameters and rare changes in significance.

    “Instead of Q ij and Q 2 ij, we included Qij and ln(Qij). The results remain highly similar, with no sign reversals in signi fi cant parameters and rare changes in signi fi cance (e.g., in the time controls).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data
    § 4.5.1. Robustness: sampling · p. 11

    Excluding articles whose residual lay outside three standard deviations of the mean reduced the final sample from 659 to 647, and the results were similar across all models.

    “Next, we excluded articles for which this difference did not lie within three standard deviations of the mean residual. This exclusion reduced the fi nal sample size from 659 to 647. The results are similar across all models.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data
    § Appendix. Measuring Challenging Commonly Held Beliefs · p. 14

    A single factor best represented the data structure of the measurement instrument, explaining 95.17% of the total variance.

    “We found that one factor represented the data structure the best. This single factor explained 95.17% of the total variance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data
    § 4.5.3. Robustness: independent variables · p. 12

    Running all models with the independent variables standardized yielded similar results.

    “Third, we also ran all models with the independent variables standardized, which yielded similar results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Study design and conductown data
    § Appendix. Measuring Challenging Commonly Held Beliefs · p. 13

    The mean ICC over all items in the training sample of 9 articles was .83.

    “The mean ICC over all the items was equal to .83, which is satisfactory.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data
    § 4.5.1. Robustness: sampling · p. 11

    Estimating three additional, more parsimonious models for the negation equation (17, 39 and 45 parameters) produced no meaningful differences from the full model, only some shifts in significance due to reduced power.

    “We compared the results of these three models with our full model and found no meaningful differences, only some shifts in signi fi cance due to the reduced power in the full model.”
    Quote in the paperNot a substantial claim
59 of 59

  • The paper relies on this without support.

    The results are conditional upon acceptance of an article, so the sample of published articles is subject to survival bias.

    “Second, the results in this study are conditional upon acceptance of an article. Survival bias is unavoidable in citation research of a certain magnitude, given the con fi dential nature of the review process.”

  • The paper relies on this without support.

    Content analyses of the citations that papers receive are rare.

    “Yet, content analyses of the citations papers receive are rare.”

  • The paper relies on this without support.

    Different types of citation may signal different degrees of scientific merit and impact.

    “Different types of citation may signal different degrees of scienti fi c merit and impact.”
3 of 3

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Two references assign different years to the same journal issue

    Minor

    The references place these articles in Marketing Science volume 23, issue 4, on adjacent page ranges, but date one to 1997 and the other to 2004.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. Garfield, 1990, How ISI selects journals for coverage: Quantitative and qualitative considerations, Current Contents, 22, 5
  2. Staelin, 2005, Influencing the practice through big new ideas. In: Marketing renaissance: Opportunities and imperatives for improving marketing thought, practice, and infrastructure, Journal of Marketing, 69, 19

Cited by

Sorted by citations

Related

Authors
Stremersch, S., Camacho, N., Vanneste, S., & Verniers, I.
Affiliations
  • IESE Business School (ES)
  • Erasmus University Rotterdam (NL)
  • Ghent University (BE)
Corresponding author
Stremersch, Stefan
ORCID
Venue
International Journal of Research in Marketing
Publisher
Elsevier BV
Published
2015-03
Volume
32
Issue
1
Pages
64-77
ISSN
0167-8116 (ISSN-L)
DOI
10.1016/j.ijresmar.2014.09.004
OpenAlex
W3021505903
Cited by
102
References
93
Citations by year
2026: 6 · 2025: 8 · 2024: 13 · 2023: 11 · 2022: 14 · 2021: 16 · 2020: 10 · 2019: 10 · 2018: 9 · 2017: 3 · 2016: 2
FWCI
1.63
Type
journal article
Language
en
Update notices
erratum (2015-06-01)
Open access
Licence
Locations
  • International Journal of Research in Marketing (publishedVersion, cc-by-nc-nd)
  • EUR Research Repository (Erasmus University Rotterdam) (acceptedVersion, other-oa)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • RePub (Erasmus University Rotterdam) (submittedVersion, other-oa)
  • Ghent University Academic Bibliography (Ghent University) (submittedVersion)
  • RePub (Erasmus University Rotterdam) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
Keywords
citation types, scientific impact, citation analysis, market analysis, citation behavior, inverted U-shaped relationship, citation counts

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review