Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 56 of the 56 references it cites were found in a registry or an index.

Regional and institutional trends in assessment for academic promotion

Open access
Lim, B. H., D’Ippoliti, C., Dominik, M., Hernández-Mondragón, A. C., Vermeir, K., Chong, K. K., Hussein, H., Morales-Salgado, V. S., Cloete, K. J., Kimengsi, J. N., Balboa, L., Mondello, S., dela Cruz, T. E., Lopez-Verges, S., Sidi Zakari, I., Simonyan, A., Palomo, I., Režek Jambrak, A., Germo Nzweundji, J., Molnar, A., Saktiawati, A. M. I., Elagroudy, S., Kumar, P., Enany, S., Narita, V., Backes, M., Siciliano, V., Egamberdieva, D., Flores Bueso, Y. · 2025 · Nature

Published version of a preprint

Abstract

The assessment of research performance is widely seen as a vital tool in upholding the highest standards of quality, with selection and competition believed to drive progress. Academic institutions need to take critical decisions on hiring and promotion, while facing external pressure by also being subject to research assessment 1–4 . Here we present an outlook on research assessment for career progression with specific focus on promotion to full professorship, based on 314 policies from 190 academic institutions and 218 policies from 58 government agencies, covering 32 countries in the Global North and 89 countries in the Global South. We investigated how frequently various promotion criteria are mentioned and carried out a statistical analysis to infer commonalities and differences across policies. Although quantitative methods of assessment remain popular, in agreement with what is found in more geographically restricted studies 5–9 , they are not omnipresent. We find differences between the Global North and the Global South as well as between institutional and national policies, but less so between disciplines. A preference for bibliometric indicators is more marked in upper-middle-income countries. Although we see some variation, many promotion policies are based on the assumption of specific career paths that become normative rather than embracing diversity. In turn, this restricts opportunities for researchers. These results challenge current practice and have strategic implications for researchers, research managers and national governments.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41586-024-08422-9

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Sample of promotion policies. Map showing the geographical distribution of the data used for analysis. Blue colour shades (on logarithmic scale) indicate the number of estimated active researchers in each of the 121 countries from which we sourced policies36–42. Other countries and territories are shown in grey. The area of the semicircles is proportional to, respectively, the number of institutions or agencies (yellow) and the number of policies (orange) from a given country. The map is based on geodata openly available at the Natural Earth repository, using the 50-m land polygon GeoJSON dataset (https://www.naturalearthdata.com/). The plot was generated with geopandas and matplolib wedge shape patch, as per the annotated code shared in the Code availability section. The authors do not endorse any position over any disputed area or contested border. The number of researchers was obtained from the UNESCO Institute for Statistics and UNESCO Science Reports (Supplementary Information section 1.7).
Fig. 1 Sample of promotion policies. Map showing the geographical distribution of the data used for analysis. Blue colour shades (on logarithmic scale) indicate the number of estimated active researchers in each of the 121 countries from which we sourced policies36–42. Other countries and territories are shown in grey. The area of the semicircles is proportional to, respectively, the number of institutions or agencies (yellow) and the number of policies (orange) from a given country. The map is based on geodata openly available at the Natural Earth repository, using the 50-m land polygon GeoJSON dataset (https://www.naturalearthdata.com/). The plot was generated with geopandas and matplolib wedge shape patch, as per the annotated code shared in the Code availability section. The authors do not endorse any position over any disputed area or contested border. The number of researchers was obtained from the UNESCO Institute for Statistics and UNESCO Science Reports (Supplementary Information section 1.7).CC BY · PubMed Central
Fig. 2 Trends in research assessment. a,b, Frequency with which each of the 30 assessment criteria is estimated to affect researchers in the 121 countries surveyed. a shows 19 criteria along with the general category of research outputs, which is expanded into 7 quantitative (top) and 4 qualitative (bottom) measures shown in b (n = 532). RC, recognition; GTs, general traits.
Fig. 2 Trends in research assessment. a,b, Frequency with which each of the 30 assessment criteria is estimated to affect researchers in the 121 countries surveyed. a shows 19 criteria along with the general category of research outputs, which is expanded into 7 quantitative (top) and 4 qualitative (bottom) measures shown in b (n = 532). RC, recognition; GTs, general traits.CC BY · PubMed Central
Fig. 3 Frequencies by policy type and global regions. a, Spider plots showing the frequency of 30 criteria among institutional (grey) and national (red) policies, grouped into four spider plots according to the class of the criteria: research outputs; career development; teaching and services; and general traits and recognition. b, Bar plots comparing the frequency of qualitative and quantitative criteria for assessing research outputs within national (right) and institutional (left) policies, distinguishing between Global South (green) and Global North (grey). Institutional n = 314 (North, 141; South, 173); national n = 218 (North, 24; South, 194).
Fig. 3 Frequencies by policy type and global regions. a, Spider plots showing the frequency of 30 criteria among institutional (grey) and national (red) policies, grouped into four spider plots according to the class of the criteria: research outputs; career development; teaching and services; and general traits and recognition. b, Bar plots comparing the frequency of qualitative and quantitative criteria for assessing research outputs within national (right) and institutional (left) policies, distinguishing between Global South (green) and Global North (grey). Institutional n = 314 (North, 141; South, 173); national n = 218 (North, 24; South, 194).CC BY · PubMed Central
Fig. 4 Factor analysis of the assessment criteria for promotion to professorship. The heat map shows the factor loadings (a measure of correlation; Methods) of each assessment criterion in the 532 policies on each of the 4 latent factors (factors 1–4) predicted after principal factor analysis and rotated with the oblimin oblique method. ‘Uniqueness’ is the fraction of the variance that a given criterion does not share with others. Blanks denote loading <0.3 in absolute value, and other values are highlighted with a colour scale; all loadings are shown in Supplementary Table 5. We assigned factor interpretation labels to the four factors, to describe the set of criteria they cover.
Fig. 4 Factor analysis of the assessment criteria for promotion to professorship. The heat map shows the factor loadings (a measure of correlation; Methods) of each assessment criterion in the 532 policies on each of the 4 latent factors (factors 1–4) predicted after principal factor analysis and rotated with the oblimin oblique method. ‘Uniqueness’ is the fraction of the variance that a given criterion does not share with others. Blanks denote loading <0.3 in absolute value, and other values are highlighted with a colour scale; all loadings are shown in Supplementary Table 5. We assigned factor interpretation labels to the four factors, to describe the set of criteria they cover.CC BY · PubMed Central
Fig. 5 Coefficients of the regression analyses. Relation between the four predicted factors and policy or country characteristics noted in this study. For the categorical variables, the relation is measured in terms of deviation from a reference category shown in brackets in the figure legend. From top to bottom, categories are: region and policy scope (dark blue), income level (blue), continents (cyan), tracks (orange), disciplines (salmon), and exclusive to full professor (red). Variables with a statistically significant coefficient are indicated; ***P < 0.01, **P < 0.05, *P < 0.1; two-sided t-tests of difference from zero. Exact P values for each variable are provided in Supplementary Table 7. The number of policies within each category can be found in Extended Data Table 1. The length of the bars denotes the size of the coefficient, and the length of the lines denotes the 95% confidence intervals based on robust standard errors. Sample size (n) = 531. The values for each coefficient, their standard errors and statistical significance are reported in Supplementary Table 7. GN, Global North; GS, Global South; sci., science; Prof., professor.
Fig. 5 Coefficients of the regression analyses. Relation between the four predicted factors and policy or country characteristics noted in this study. For the categorical variables, the relation is measured in terms of deviation from a reference category shown in brackets in the figure legend. From top to bottom, categories are: region and policy scope (dark blue), income level (blue), continents (cyan), tracks (orange), disciplines (salmon), and exclusive to full professor (red). Variables with a statistically significant coefficient are indicated; ***P < 0.01, **P < 0.05, *P < 0.1; two-sided t-tests of difference from zero. Exact P values for each variable are provided in Supplementary Table 7. The number of policies within each category can be found in Extended Data Table 1. The length of the bars denotes the size of the coefficient, and the length of the lines denotes the 95% confidence intervals based on robust standard errors. Sample size (n) = 531. The values for each coefficient, their standard errors and statistical significance are reported in Supplementary Table 7. GN, Global North; GS, Global South; sci., science; Prof., professor.CC BY · PubMed Central
Extended Data Fig. 1 Approach to assessment of research outputs by policy scope. Application of quantitative vs qualitative criteria for the 314 institutional and 218 national policies in our dataset. Result shown is the percentage of policies mentioning each type of assessment. Number of policies (n) for each criterion is presented above each corresponding bar.
Extended Data Fig. 1 Approach to assessment of research outputs by policy scope. Application of quantitative vs qualitative criteria for the 314 institutional and 218 national policies in our dataset. Result shown is the percentage of policies mentioning each type of assessment. Number of policies (n) for each criterion is presented above each corresponding bar.CC BY · PubMed Central
Extended Data Fig. 2 Scree plot for the factor analysis. Showing the factors obtained, ordered by eigenvalue - denoting the level of variability captured by the factors. In the red box are the Eigenvalues for the four factors used.
Extended Data Fig. 2 Scree plot for the factor analysis. Showing the factors obtained, ordered by eigenvalue - denoting the level of variability captured by the factors. In the red box are the Eigenvalues for the four factors used.CC BY · PubMed Central
Extended Data Fig. 3 Loading plots for each pair of factors. Each scatter plot in the grid represents a pairwise comparison between factors: Factor 1: Output Metrics, Factor 2: Visibility and Engagement, Factor 3: Career Development, and Factor 4: Outcomes & Impact. Data points are colour-coded to differentiate between quantitative outputs (blue), qualitative outputs (orange), career development criteria (yellow), services (purple), and recognition (red). Ellipses represent the concentration and dispersion of data points associated with each group, indicating the variance and co-relationship strength between factors. Ellipses were overlaid using matplotlib, ellipse function, where the width and height of the ellipse were set to reflect the standard deviation for each group along the X and Y axes, respectively. N = 532.
Extended Data Fig. 3 Loading plots for each pair of factors. Each scatter plot in the grid represents a pairwise comparison between factors: Factor 1: Output Metrics, Factor 2: Visibility and Engagement, Factor 3: Career Development, and Factor 4: Outcomes & Impact. Data points are colour-coded to differentiate between quantitative outputs (blue), qualitative outputs (orange), career development criteria (yellow), services (purple), and recognition (red). Ellipses represent the concentration and dispersion of data points associated with each group, indicating the variance and co-relationship strength between factors. Ellipses were overlaid using matplotlib, ellipse function, where the width and height of the ellipse were set to reflect the standard deviation for each group along the X and Y axes, respectively. N = 532.CC BY · PubMed Central
Extended Data Fig. 4 Single variable comparison between Global North and South. Violin plots showing median, quartiles, and the distributions of the families of criteria described by the four latent factors resulting from the principal factor analysis. Here, factor strength is the overall impact of the measured factor in explaining the observed data (factor score for each policy). Note: The correlation among criteria, the factor loadings and their variance, the factor scores and their graphical representation through kernel density and boxplots are all based on the weighted sample. N for Global North = 165 and for Global South = 367.
Extended Data Fig. 4 Single variable comparison between Global North and South. Violin plots showing median, quartiles, and the distributions of the families of criteria described by the four latent factors resulting from the principal factor analysis. Here, factor strength is the overall impact of the measured factor in explaining the observed data (factor score for each policy). Note: The correlation among criteria, the factor loadings and their variance, the factor scores and their graphical representation through kernel density and boxplots are all based on the weighted sample. N for Global North = 165 and for Global South = 367.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 54 references

Research works printed but not linked: 7

  1. Fauzi, M. A., Tan, C. N. L., Daud, M. & Awalludin, M. M. N. University rankings: a review of methodological flaws. Issues Educ. Res. 30, 79–96 (2020).
  2. INORMS Research Evaluation Group. The SCOPE Framework: A Five-Stage Process for Evaluating Research Responsibly (Emerald Publishing, 2021); https://inorms.net/wp-content/uploads/2022/03/21655-scope-guide-v10.pdf.
  3. Towards a Reform of the Research Assessment System: Scoping Report (European Commission, Directorate-General for Research and Innovation, 2021); https://data.europa.eu/doi/10.2777/707440. · doi:10.2777/707440
  4. Aubert Bonn, N. & Bouter, L. in Handbook of Bioethical Decisions Vol. II. Collaborative Bioethics, Vol. 3 (eds Valdés, E. & Lecaros, J. A.) (Springer, 2023).
  5. D’Ippoliti, C. Democratizing the Economics Debate: Pluralism and Research Evaluation (Routledge, 2022).
  6. Harman, H. H. Modern Factor Analysis 3rd edn (Univ. Chicago Press, 1976).
  7. Huber, P. J. The behavior of maximum likelihood estimates under nonstandard conditions. In Proc. 5th Berkeley Symposium on Mathematical Statistics and Probability Vol. 1, 221–233 (Univ. California Press, 1967).

Cited by

See all 54

Metadata

Authors
Lim, B. H., D’Ippoliti, C., Dominik, M., Hernández-Mondragón, A. C., Vermeir, K., Chong, K. K., Hussein, H., Morales-Salgado, V. S., Cloete, K. J., Kimengsi, J. N., Balboa, L., Mondello, S., dela Cruz, T. E., Lopez-Verges, S., Sidi Zakari, I., Simonyan, A., Palomo, I., Režek Jambrak, A., Germo Nzweundji, J., Molnar, A., Saktiawati, A. M. I., Elagroudy, S., Kumar, P., Enany, S., Narita, V., Backes, M., Siciliano, V., Egamberdieva, D., & Flores Bueso, Y.
Affiliations
  • Universiti Tunku Abdul Rahman Sungai Long Campus (MY)
  • Accademia Nazionale dei Lincei (IT)
  • University of St Andrews (GB)
  • Centre National de la Recherche Scientifique (FR)
  • University of Oxford (GB)
  • University of South Africa (ZA)
  • University of Bamenda (CM)
  • Academia Nacional de Medicina (AR)
  • University of Messina (IT)
  • University of Sussex (GB)
  • Institut polytechnique de Grenoble (FR)
  • Swinburne University of Technology (AU)
  • Universitas Gadjah Mada (ID)
  • Ain Shams University (EG)
  • University of the Witwatersrand (ZA)
  • Suez Canal University (EG)
  • University of Namibia (NA)
  • Italian Institute of Technology (IT)
  • Tashkent Institute of Irrigation and Agricultural Mechanization Engineers (UZ)
Venue
Nature
Published
2025-01-22
Volume
638
Issue
8050
Pages
459-468
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
OpenAlex
W4406732442
Cited by
53
References
54
Citations by year
2026: 31 · 2025: 21
FWCI
33.26
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • HAL (submittedVersion, cc-by)
  • St Andrews Research Repository (St Andrews Research Repository) (submittedVersion, cc-by)
  • Conicet (submittedVersion, cc-by)
  • HAL (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
Funders
  • Breakthrough Cancer Research (18/SP/3522)
  • Research Ireland
Topics
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
  • Social Sciences › Education › Higher Education Governance and Development
Keywords
research evaluation, academic promotion, full professorship, promotion policy, bibliometric indicators, institutional policy, career development, diversity in academia
MeSH terms
Academies and Institutes, Career Mobility, Faculty, Humans, Research Personnel, Universities, Bibliometrics, Organizational Policy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Regional and institutional trends in assessment for academic promotion. https://app.cochat.ai/papers/10.1038/s41586-024-08422-9
The paper
Lim, B. H., D’Ippoliti, C., Dominik, M., Hernández-Mondragón, A. C., Vermeir, K., Chong, K. K., Hussein, H., Morales-Salgado, V. S., Cloete, K. J., Kimengsi, J. N., Balboa, L., Mondello, S., dela Cruz, T. E., Lopez-Verges, S., Sidi Zakari, I., Simonyan, A., Palomo, I., Režek Jambrak, A., Germo Nzweundji, J., . . . Flores Bueso, Y. (2025). Regional and institutional trends in assessment for academic promotion. Nature. https://doi.org/10.1038/s41586-024-08422-9

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.