CoChat
Sign Up
Cited by 125Interdisciplinary Research and Collaborationscientometrics and bibliometrics researchResearch, Science, and Academia

Linking modes of research to their scientific and societal outcomes. Evidence from 81 sustainability-oriented research projects

Open access
Newig, Jens, Jahn, Stephanie, Lang, Daniel J., Kahle, Judith, Bergmann, Matthias·2019·Environmental Science & Policy
DOI 10.1016/j.envsci.2019.08.008
Open this paper in CoChat — free accountOpen PDF

Abstract

Sustainability-oriented research has increasingly adopted “new” modes of research promoted under labels such as ‘post-normal science’, ‘mode 2 knowledge production’ or ‘transdisciplinarity’, aiming to address societally relevant problems and to produce ‘socially robust’ knowledge by involving relevant scientific disciplines and non-academic actors into the research. We present the results of a comparative quantitative analysis of 81 completed sustainability-oriented research projects, coupled with an in-depth study of six projects, to empirically investigate the assumed connections between research modes and societal and academic project outcomes. Statistical analysis suggests that contributions from practitioners in early phases of research projects positively influence certain societal and practice-relevant outcomes. By contrast, including non-academic actors and practitioner knowledge into research negatively impacts academic outputs and citations, indicating a trade-off between academic and societal impacts. Yet projects which apply structured methods of knowledge integration score generally higher on academic outputs and citations. Moreover, the funding context affects both research mode and research outcomes. Finally, practitioner involvement negatively affects completing of PhD projects. Findings from the in-depth study reinforce a trade-off between the societal and academic impact of interdisciplinary and transdisciplinary sustainability-oriented research. We find that projects which had a double research objective on academic and societal outcomes but which did not specify how to realize both, neglected either the academic or the societal impact during the research process. Moreover, we find that a well-designed combination of disciplinary as well as inter- and transdisciplinary project phases helped projects to meet both demands.

AI snapshot

Written from the paper’s full text.

TL;DR

A mixed-methods study of 81 completed German sustainability-oriented research projects finds a trade-off: early practitioner involvement improves some societal impacts but reduces academic publications and citations, while structured knowledge integration predicts higher academic output. In-depth cases show this trade-off can be mitigated by project design combining disciplinary, inter-, and transdisciplinary phases.

Key findings

  • Early practitioner involvement significantly positively affected only some societal outcomes: impact depth, media impact, and total impact; it did not generally improve all societal outcomes. In the best model for impact depth, R² was 0.30 (adjusted R² 0.25), and research mode alone explained 17% of impact-depth variance. Dissemination positively affected societal impacts, while funding sum showed a slightly negative effect.
  • Practitioner involvement negatively affected academic outcomes: in the publication model, Practitioner_decision had β = −.31 and Practitioner_knowledge β = −.33*, while Knowledge_integration had β = .30. Citations showed a similar but weaker pattern. In-depth cases included the Interdisciplinary Project with 119 publications and the Transdisciplinary Project with 28, providing counter-evidence to a strict trade-off.
  • Funding context mattered: BMBF mission-oriented projects (n = 58) scored higher on depth of reported societal impact than DFG bottom-up projects (n = 23) (mean 2.5 vs. 1.3), but DFG projects scored higher on publications per million euro (mean 30.3 vs. 16.5) and citations per million euro (mean 282.8 vs. 238.2). Goal achievement did not differ significantly (t-test p = 0.06; Bonferroni p = 0.61).
  • PhD completion: across 42 projects with reliable information, 80 of 106 planned dissertations were completed; 24 projects completed the planned number and 18 did not. Practitioner_decision had a significant negative relationship with completed PhD quota (β = −.34*), supporting a trend for Hypothesis 5.

Methods

  • Large-N study: comparative analysis of 81 completed sustainability-oriented research projects funded by DFG or BMBF between 2000 and 2012, with a substantial social science share and at least two disciplines. Data came from project proposals, reports, outcome lists, and questionnaires completed by project heads; analysis used OLS regression.
  • In-depth study: six cases selected for diversity in project type, initiation, disciplinary focus, transdisciplinarity, and funding context. Materials included proposals, interim/final reports, publications, media responses, and 35 semi-structured interviews; analysis used structured content analysis and causal process tracing with counterfactual analysis.
  • Outcome measures included societal impact indices (depth, scale, sectors, media, total), publications per million euro, log citations per million euro, self-reported goal achievement, and quota of completed PhD projects.

Limitations

  • The paper states the study is restricted to sustainability-oriented projects in Germany; funding structures and research modes may differ in other national or international contexts. Many key variables rely on self-reported assessments, which may introduce a bias toward “success,” though the authors argue it should not distort the dataset. Confidentiality prevents assessing or comparing individual projects; the aim was general patterns across cases.
  • The paper also states difficulty in clearly distinguishing applied research involving practitioners from transdisciplinary research, and that the notion of “empowering” participants fit poorly where practitioners collaborated equally or led projects. The study was restricted to research projects typically lasting a few years, whereas trustful academic-practitioner relationships may need more time.
  • Evident from the text: the large-N study did not investigate effects on the innovative character of project outputs and outcomes for Hypothesis 3a, and the PhD analysis used only the 42 projects that provided reliable information.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.envsci.2019.08.008

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 4 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Citations show essentially the same pattern as publications but with lower effect sizes and statistical significance for all independent variables.

    “Turning to Citations (Table 5), we find essentially the same pattern but with lower effect sizes and statistical significance for all independent variables.”
    Quote in the paperNot a substantial claim
  • Findingreasoning

    The trust practitioners developed in applicability of results motivated them to further apply results after project completion.

    “The trust that the practitioners, in turn, developed in the applicability of the results ultimately motivated them to further apply the results after project completion.”
    Quote in the paperNot a substantial claim
  • Findingown data

    The degree to which a project aimed at producing applied knowledge had no discernible effect on societal impacts.

    “whereas the degree to which a project aimed at producing applied knowledge had no discernible effect”
    Quote in the paperNot a substantial claim
  • Findingown data

    Only Impact_media was found to be significantly explained by research-mode variables next to Impact_depth.

    “next to Impact_depth, only Impact_media was found to be significantly explained by researchmode variables”
    Quote in the paperSubstantial claim
  • Findingown data

    Projects which apply structured methods of knowledge integration score generally higher on academic outputs and citations.

    “Yet projects which apply structured methods of knowledge integration score generally higher on academic outputs and citations.”
    • Also stated in: § 4.3. Hypotheses 3a and 3b: Non-academic actor involvement and academic research outcomes · plain
      “By contrast, the use of structured methods to integrate non-academic and academic knowledge is a highly significant positive predictor for academic outcomes.”
    • Also stated in: § 4.3. Hypotheses 3a and 3b: Non-academic actor involvement and academic research outcomes · plain
      “we still find significant negative effects of Practitioner_knowledge and Practitioner_decision, as well as a significant positive effect of Knowledge_integration.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 18
  • The quote only partly states this claim. 2
  • May combine two claims. 25

  • Probably not a substantial claim: may not pass S1 (asserted). The quote only partly states this claim.

    Findingown data

    Non-academic actor involvement generally does not improve societal outcomes.

    “Taken together, from our regression analysis of 81 cases, we cannot confirm that non-academic actor involvement generally improves societal outcomes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Early involvement of practitioners in research projects positively influences certain societal and practice-relevant outcomes, specifically impact depth, media impact, and total impact, but not impact scale or impact sectors.

    “Statistical analysis suggests that contributions from practitioners in early phases of research projects positively influence certain societal and practice-relevant outcomes.”
    • Also stated in: § 4.1. Hypothesis 1: Non-academic actor involvement and societal project outcomes · plain
      “practitioner involvement in early project phases -identification of research problems and formulation research questions -has shown to have a significant effect, but only on the depth of impact and on media impact, as well as on the total impact measure.”
    • Also stated in: § 4.1. Hypothesis 1: Non-academic actor involvement and societal project outcomes · plain
      “Including the control variables (1 h), only Early_involvement has a significant effect.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the comprehensive model, practitioner knowledge and practitioner decision have significant negative effects on publications, while knowledge integration has a significant positive effect.

    “In the more comprehensive model 6b, which includes the two controls Goal_applied and the variable Practitioner_involvement (measuring whether or not practitioners were involved in the project at all), we still find significant negative effects of Practitioner_knowledge and Practitioner_decision, as well as a significant positive effect of Knowledge_integration.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Early involvement, practitioner values, practitioner knowledge, and practitioner decision are individually associated with publications, with significant negative effects.

    “The results of models 1-5 show that most variables individually are associated with Publications, showing significant negative effects for Early_involvement, Practitioner_values, Practitio-ner_knowledge and Practitioner_decision.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    The large-N study could not find significant relationships between knowledge integration methods and societal project outcomes.

    “Analyzing only those 59 cases with at least some degree of practitioner involvement, we could not find significant relationships between this variable and societal project outcomes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    The funding context affects both research mode and research outcomes.

    “Moreover, the funding context affects both research mode and research outcomes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The in-depth study found that methods of knowledge integration provided stage and tools for transdisciplinary collaboration which bolstered practically relevant outputs but not necessarily societal impact.

    “These methods provided the stage and the tools for transdisciplinary collaboration which in turn essentially bolstered the practically relevant outputs -but not necessarily the societal impactof all four projects.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Projects funded in mission-oriented contexts score higher on impacts and lower on publications and citations, while the reverse is true for projects funded in bottom-up contexts.

    “This pattern is replicated in the effect of the funding context: Whereas projects funded in mission-oriented contexts score higher on impacts and lower on publications and citations, the reverse is true for projects funded in bottom-up contexts.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported assessments, despite potential bias toward success, do not distort the dataset.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In the Transdisciplinary Project, one researcher reported that preparing and implementing transdisciplinary workshops, including choosing methods of knowledge integration, required an enormous amount of time.

    “Despite the strong academic output, one researcher from the Transdisciplinary Project mentioned the enormous amount of time needed to prepare and implement the transdisciplinary workshops, including the choice of the methods of knowledge integration to be used.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The effect of practitioner involvement on societal outcomes is only on the depth of impact and on media response, but not on impacts measured through diversity of sectors and geographical scales.

    “on the other hand, the effect is only on the depth of impact and on media response, but not on impacts measured through diversity of sectors and geographical scales.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported assessments, despite potential bias toward success, do not distort the dataset.

  • May combine two claims.

    Findingreasoning

    In the case of the two subprojects in a collaborative research center, the DFG also shaped the projects' goals and stipulated their research modes.

    “However, in the case of the two subprojects in a collaborative research center, the DFG also shaped the projects' goals and stipulated their research modes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Model 6b has an R2 of 0.35 (adjusted R2 of 0.31).

    “With an R 2 of 0.35 (adjusted R 2 of 0.31), the model shows a reasonable overall fit.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Involving practitioner knowledge not only does not foster societal outcomes, it also negatively affects academic publication outputs and citations.

    “The second concerns our finding that involving practitioner knowledge not only does not foster societal outcomes, it also negatively affects academic publication outputs and citations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported assessments, despite potential bias toward success, do not distort the dataset.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    Projects which aim at local solutions, involving local practitioners, do not exhibit global impacts.

    “However, considering the necessarily limited ambitions of projects, it may seem little surprising that projects which aim, for example, at local solutions, involving local practitioners, do not exhibit global impacts.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Projects have necessarily limited ambitions, so projects aiming at local solutions are not expected to exhibit global impacts.

  • May combine two claims.

    Findingreasoning

    With these strategies, both projects reached strong academic outputs in theoretical concepts, methodical innovations, data sets, and considerable numbers of academic publications (Interdisciplinary Project: 119; Transdisciplinary Project: 28).

    “With these strategies, both projects reached strong academic outputs in the form of theoretical concepts, methodical innovations (in support of hypothesis 3a), data sets, as well as considerable numbers of academic publications”
    • Also stated in: § 4.3. Hypotheses 3a and 3b: Non-academic actor involvement and academic research outcomes · plain
      “of theoretical concepts, methodical innovations (in support of hypothesis 3a), data sets, as well as considerable numbers of academic publications (Interdisciplinary Project: 119; Transdisciplinary Project: 28), thus prociding counter-evidence for hypothesis 3b.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Another practitioner with close ties to academia provided funding to conduct a second round of the survey developed in the project.

    “Another practitioner with close ties to academia provided funding to conduct a second round of the survey developed in the project.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Of the 42 projects with reliable PhD information, 24 succeeded in producing the planned number of completed PhDs and 18 did not.

    “While 24 projects succeeded in producing the planned number of completed PhDs, 18 projects did not.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The 42 projects that provided reliable information on PhD dissertations are representative of the 81 projects for the relationship between practitioner involvement and completed PhD theses.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    We found it difficult to clearly distinguish between applied research that involves practitioners and transdisciplinary research.

    “On a conceptual level, we found it difficult to clearly distinguish between applied research that involves practitioners and transdisciplinary research.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The same researcher reported initial difficulties finding a journal to publish the transdisciplinary research results.

    “Additionally, the same researcher reported initial difficulties to find a journal to publish the transdisciplinary research results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The same researcher stressed that the workshops provided limited output for the next research steps in the project.

    “At the same time, the researcher stressed the limited output that the workshops provided for the next research steps in the project.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The DFG's funding criteria generally focus on strong scientific rigor.

    “By contrast, the DFG's funding criteria generally focus on strong scientific rigor.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    A follow-up project partially involved the same actors and continued the transdisciplinary research on the topic.

    “For instance, a follow-up project partially involved the same actors and continued the transdisciplinary research on the topic.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    Both the Interdisciplinary and Transdisciplinary Projects started publishing in an early phase and continued to do so throughout the research process.

    “Both projects started publishing in an early phase and continued to do so throughout the research process.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the in-depth study, three of six projects aimed for and successfully completed doctoral dissertations or habilitations: the Disciplinary Hinge, Interdisciplinary and Intervening Projects, with the latter two involving practitioners as research partners.

    “In the in-depth study, three of six projects aimed for and successfully completed doctoral dissertations or habilitations: The Disciplinary Hinge, the Interdisciplinary and the Intervening Project, with the latter two involving practitioners as research partners.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Societal outcomes could be explained by early involvement of practitioners, but not by other aspects of involvement such as decision-making and value or knowledge contributions.

    “On the one hand, societal outcomes could be explained by early involvement of practitioners, but not by other aspects of involvement such as decision-making and value or knowledge contributions;”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported assessments, despite potential bias toward success, do not distort the dataset.

  • May combine two claims.

    Findingown data

    We do not find a generally positive influence of practitioner involvement on societal outcomes but rather highly selective links.

    “First, we do not find a generally positive influence of practitioner involvement on societal outcomes but rather highly selective links.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Self-reported assessments, despite potential bias toward success, do not distort the dataset.

  • May combine two claims.

    Findingown data

    There are differences across sub-categories, with DFG individual grants showing higher publications and citations than sub-projects in collaborative research centers.

    “Here again, we find differences across the sub-categories, with DFG individual grants showing higher publications and citations than sub-projects in collaborative research centers, and, most notably, among the different BMBF funding lines.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    In the in-depth study, there is a causal relation between intense involvement of non-academic actors and impact depth and impact scale.

    “Summarizing, the analysis of the Transdisciplinary Project suggests a causal relation between the intense involvement of non-academic actors and impact depth, as well as impact scale.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    For reasons of confidentiality, the research does not allow to assess or compare individual projects.

    “For reasons of confidentiality, our research also does not allow to assess or compare individual projects.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Many key variables rely on self-reported assessments.

    “Many key variables rely on self-reported assessments.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    In both projects, effectively distributed tasks among researchers and a targeted combination of disciplinary, interdisciplinary and transdisciplinary research phases made early and continued publishing possible.

    “This became possible due to effectively distributed tasks among the involved researchers and a targeted combination of disciplinary, interdisciplinary and transdisciplinary research phases along the research process.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    The building of trustful academic-practitioner relationships and networks is likely to need more time.

    “However, the building of trustful academicpractitioner relationships and networks, which is so often advocated with regards to transdisciplinary research, is likely to need more time.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    In the Interdisciplinary Project, clear division of tasks among the research team and purposeful combination of disciplinary, inter- and transdisciplinary project phases enabled young researchers to conduct their research and fulfil other project duties.

    “Like for hypothesis 3, we found evidence for the Interdisciplinary Project that clear division of tasks among the research team, as well as purposeful combination of disciplinary, inter-and transdisciplinary project phases enabled the young researchers to conduct their research but also to fulfil other duties within the project, e.g.helping to prepare and implement transdisciplinary workshops.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study was restricted to research projects typically lasting a few years only.

    “Finally, our study was restricted to study research projects, typically lasting a few years only.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    While self-reported achievements relative to stated goals are consistently high and show no significant variation, there is a strong trade-off between societal and academic outcomes across the different funding formats and lines.

    “From these findings, we may conclude that while self-reported achievements relative to stated goals are consistently high and show no significant variation, we find a strong trade-off between societal and academic outcomes across the different funding formats and lines.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The large-N study did not investigate effects on innovative character of project outputs and outcomes (Hypothesis 3a).

    “In the large-N study we did not investigate the effects on the innovative character of the project outputs and outcomes (Hypothesis 3a).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Societal impact was measured through self-reporting by project heads.

    “For quantitative analysis, societal impact was operationalized through four dependent variables, which relied on self-reporting by project heads (for a similar approach see De Jong et al., 2016)”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites De Jong et al., 2016 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    It appears plausible that the structured methods of knowledge integration employed in the studied projects also served to integrate different kinds of academic knowledge, hence the positive effect on academic outcomes.

    “On the one hand it appears plausible that the structured methods of knowledge integration employed in the studied projects also served to integrate different kinds of academic knowledge (regardless of their function to also integrate non-academic knowledge), hence the positive effect on academic outcomes.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    This division allowed researchers to make essential research contributions to the specific project.

    “This allowed the researchers to make essential research contributions to the specific project.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The in-depth study demonstrates that strong academic outputs and outcomes occur partly despite and partly precisely because of involving practitioners.

    “To sum up, the in-depth study demonstrates that strong academic outputs and outcomes occur partly despite and partly precisely because of involving practitioners.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    Funding structures and actual employment of research modes will differ in other national or international contexts.

    “Funding structures and actual employment of research modes will differ in other national or international contexts.”
    Quote in the paperNot a substantial claim
41 of 41

  • The paper relies on this without support.

    The funding context (DFG versus BMBF) impacts the research modes chosen by funded projects.

    “The two institutions were chosen because they constitute the most important public research funding bodies in Germany and because they differ substantially in their funding structures, which arguably impact on the research modes chosen by the funded projects”

  • The paper relies on this without support.

    The 42 projects that provided reliable information on PhD dissertations are representative of the 81 projects for the relationship between practitioner involvement and completed PhD theses.

    “From our sample of 81 projects, 42 provided reliable information on PhD dissertations 5 .These 42 projects were used to quantitatively test the effect of involvement variables on the share of completed PhD theses per project.”

  • The paper relies on this without support.

    Self-reported assessments, despite potential bias toward success, do not distort the dataset.

    “Many key variables rely on self-reported assessments.While this may introduce a bias towards 'success', it should not distort the dataset.”

  • The paper relies on this without support.

    Projects have necessarily limited ambitions, so projects aiming at local solutions are not expected to exhibit global impacts.

    “However, considering the necessarily limited ambitions of projects, it may seem little surprising that projects which aim, for example, at local solutions, involving local practitioners, do not exhibit global impacts.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • There are only a few places (in Austria, Germany, Switzerland, and South Africa) where transdisciplinarity has been institutionalized or research funding explicitly supports transdisciplinary processes, such as the German FONA or Swedish Mistra programs.

    “But there are only a few places (in Austria, Germany, Switzerland, and South Africa) where transdisciplinarity has been institutionalized thus far or research funding explicitly supports transdisciplinary processs such as the German FONA (Newig et al. 2019) or the Swedish Mistra (Polk 2014) program.”

    From Transdisciplinarity: science for and with society in light of the university’s roles and functions (2020) · cites this paper as Newig et al. 2019

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • DFG publication mean disagrees with its two subgroups

    Minor

    Weighting the two DFG subgroup publication means by their project counts gives about 30.8 publications per million euros, not the reported DFG mean of 30.3. Rounding the subgroup means to one decimal cannot account for the difference.

  • Significance mark on PhD-model funding coefficient conflicts with model fit

    Minor

    Table 7 marks the funding-sum coefficient in model 1 as significant at p < .05. With 42 observations, two predictors, and an overall R2 of .03, no individual predictor in that model can reach that significance threshold; its reported overall F-value is also only 0.57.

  • BMBF funding-line counts sum to 57, not 58

    Minor

    The seven BMBF funding lines listed in Table 6 contain 4, 1, 11, 1, 21, 18, and 1 projects, totaling 57. The BMBF total is reported as 58, with no eighth category shown.

    Reported 58 · recomputed 57

3 of 3

The citation lists haven’t been fetched yet

References 27 · Cited by 125

Authors
Newig, J., Jahn, S., Lang, D. J., Kahle, J., & Bergmann, M.
Affiliations
Leuphana University of Lüneburg (DE)
Corresponding author
Newig, Jens
ORCID
Venue
Environmental Science & Policy
Publisher
Elsevier BV
Published
2019-11
Volume
101
Pages
147-155
ISSN
1462-9011 (ISSN-L)
DOI
10.1016/j.envsci.2019.08.008
OpenAlex
W2971327034
Semantic Scholar
d10413d25b5fd62cbe010cf44ce997e97ad8be71
Cited by
125
References
27
Citations by year
2026: 15 · 2025: 12 · 2024: 19 · 2023: 16 · 2022: 26 · 2021: 17 · 2020: 18 · 2019: 2
FWCI
18.66
Type
journal article
Language
en
Open access
Licence
Locations
  • Environmental Science & Policy (publishedVersion, cc-by-nc-nd)
  • Multilingual Matters (Channel View Publications) (publishedVersion, cc-by-nc-nd)
Funders
  • Deutsche Forschungsgemeinschaft (LA3024/1-1, NE1207/4-1)
  • Bundesministerium für Bildung und Forschung
Topics
  • Social Sciences › Decision Sciences › Information Systems and Management › Interdisciplinary Research and Collaboration
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
Keywords
post-normal science, Mode 2 knowledge production, transdisciplinarity, interdisciplinary research, practitioner engagement, knowledge integration, research impact, social impact, PhD completion, research project results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review