When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings
Open accessAbstract
“Not everything that can be counted counts, and not everything that counts can be counted.” —Albert Einstein Has university scholarship gone astray? Do our academic assessment systems reward scholarship that addresses the questions that matter most to society? Using international business as an example, we highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. We call for an immediate examination of existing ranking systems, not only as a legitimate scholarly question vis-a-vis performance—a conceptual lens with deep roots in management research—but also because the very health and vibrancy of the field are at stake. Indeed, in light of the data presented here, which suggest that current systems are dysfunctional and potentially cause more harm than good, a temporary moratorium on rankings may be appropriate until more valid and reliable ways to assess scholarly contributions can be developed. The worldwide community of scholars, along with the global network of institutions interacting with and supporting management scholarship (such as the Academy of Management, AACSB, and Thomson Reuters Scientific) are invited to innovate and design more reliable and valid ways to assess scholarly contributions that truly promote the advancement of relevant 21st century knowledge, and likewise recognize those individuals and institutions that best fulfill the university's fundamental purpose.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The influence of management journals in the 1980s and 1990s
Podsakoff, P. M., MacKenzie, S. B., Bachrach, D. G., & Podsakoff, N. P. (2005) · Strategic Management Journal · doi:10.1002/smj.454 · Cited by 912 · cited 1× · see citing works
“"How does one measure the quality of scholarship, and, by extension, the accomplishments of individual scholars?" Management, like other disciplines, has identified a set of top journals to serve as proxies for quality (see, among many others, Singh et al., 2007; Franke, Edlund, & Oster, 1990; Johnson & Podsakoff, 199…”
BackgroundWhat makes management research interesting and why does it matter?
Bartunek, J. M., Rynes, S. L., & Ireland, R. D. (2006) · Academy of Management Journal · doi:10.5465/amj.2006.20785494 · Cited by 432 · see citing works
Lost in publication: how measurement harms science
Lawrence, P. (2008) · Ethics in Science and Environmental Politics · doi:10.3354/esep00079 · Cited by 127 · see citing works
Flagged first, then by citations
Research works printed but not linked: 71
- Adler, N. J. 2008a. Global business as an agent of world benefit: New international business perspectives leading positive change. In A. Scherer & Guido Palazzo (Eds.), Handbook of Research on Corporate Citizenship: 374 -401. Cheltenham, UK: Edward Elgar.
- Adler, N. J. 2008b. International Business Scholarship: Contrib- uting to a Broader Definition of Success. In Jean Boddewyn (Ed.), International Business Scholarship: AIB Fellows on the First 50 Years and Beyond (Volume 13 of Research in Global Strategic Management: 333-343). UK: Emerald Group.
- Adler, N. J. 2006. The arts and leadership: Now that we can do anything, what will we do? Academy of Management Learning and Education, 5(4): 466 -499.
- Adler, N. J., & Bartholomew, S. 1992. Academic and professional communities of discourse: Generating knowledge on tran- snational human resource management. Journal of Interna- tional Business Studies, 23(3): 551-569.
- Becker, H. S., & Geer, B. 1958. The fate of idealism in medical school. American Sociological Review, 23(1): 50 -56.
- Bennis, W., & O'Toole, J. 2005. How business schools lost their way. Harvard Business Review, 83(5): 96 -106.
- Boyacigiller, N., & Adler, N. J. 1991. The parochial dinosaur: The organizational sciences in a global context. Academy of Management Review, 16(2): 262-290.
- Buckley, P. 2002. Is the international business agenda running out of steam? Journal of International Business Studies, 33(2): 365-373.
- Buckley, P. J., & Lessard, D. R. 2005. Regaining the edge for international business research. Journal of International Business Studies, 36(6): 595-599.
- Buckridge, C. 2008. New research head puts focus on tackling society's problems. The University of Melbourne Voice, 2(4): 17 March-14 April 2008.
- Butler, K. C. 2006. Finance and the search for the big question in international business. Academy of International Business Insights, 6(3): 3-9.
- Cameron, K. S., Dutton, J. E., & Quinn, R. E. (Eds.). 2003. Positive organizational scholarship. San Francisco: Berrett-Kohler.
- Chan, K. C., Fung, H., & Lai, P. 2006. International business research: Trends and school rankings. International Busi- ness Review, 15(4): 317-338.
- Detsky, A. S., & Baerlocher, M. O. 2007. Academic mentoring- How to give it and how to get it. Journal of the American Medical Association, 297(19): 2134 -2136.
- DiMaggio, P. J., & Powell, W. W. 1983. The iron cage revisited: Institutional isomorphism and collective rationality in or- ganizational fields. American Sociological Review, 48(2): 147-160.
- Donaldson, L. 2005. For positive management theories while retaining science. Academy of Management Learning and Education, 4(1): 109 -113.
- Dubois, F. L., & Reeb, D. M. 2000. Ranking the international business journals. Journal of International Business Stud- ies, 31(4): 689 -704.
- Franke, R., Edlund, T., & Oster, R. 1990. The development of strategic management: Journal quality and article impact. Strategic Management Journal, 11(3): 243-253.
- Gauffriau, M., & Larsen, P. O. 2005. Counting methods are deci- sive for rankings based on publication and citation studies, Scientometrics, 64(1): 85-93.
- Ghoshal, S. 2005. Bad management theories are destroying good management practices. Academy of Management Learning and Education, 4(1): 75-91.
- Griffith, D. A., Cavusgil, S. T., & Xu, S. 2008. Emerging themes in international business research. Journal of International Business Studies, 39(7): 1220 -1235.
- Gulati, R. 2007. Tent poles, tribalism, and boundary spanning: The rigor-relevance debate in management research. Academy of Management Journal, 50(4): 775-782.
- Hardy, C., & Maguire, S. 2008. Institutional entrepreneurship. In R. Greenwood, C. Oliver, K. Sahlin, & R. Suddaby (Eds.), The Sage handbook of organizational institutionalism: 198 -217.
- Harzing, A. W. K. 2005. Australian research output in economics & business: High volume, low impact? Australian Journal of Management, 30(2): 183-200.
- Harzing, A. W. K. 2008a. Journal quality list. 31st ed., 31 May 2008, downloaded from www.harzing.com 1 August 2008.
- Harzing, A. W. K., & van der Wal, R. 2008a. Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1): 62-71.
- Harzing, A. W. K., & van der Wal, R. 2008b. Comparing the Google Scholar H-index with the ISI Journal Impact Factor, harzing.com white paper, first version, 3 July 2008, http:// www.harzing.com/h_indexjournals.htm.
- Inkpen, A. C., & Beamish, P. W. 1994. An analysis of twenty-five years of research in the Journal of International Business Studies. Journal of International Business Studies, 25(4): 703- 713.
- Johnson, J., & Podsakoff, P. 1994. Journals of influence in the field of management: An analysis using Salancik's index in a dependency network. Academy of Management Journal, 37(5): 1392-1407.
- Kerr, S. 1975. On the folly of rewarding A, while hoping for B. Academy of Management Journal, 18(4): 769 -783.
- Kirkman, B., & Law, K. 2005. International management research in AMJ: Our past, present and future. Academy of Manage- ment Journal, 48(3): 377-386.
- Khurana, R. 2007. From higher aims to hired hands: The social transformation of business schools and the unfulfilled promise of management as a profession. Princeton, NJ: Princeton University Press.
- Kumar, V., & Kundu, S. K. 2004. Ranking the international busi- ness schools: Faculty publication as the measure. Manage- ment International Review, 44: 213-228. 2009
- Adler and Harzing Lanchester, J. 2008. Log on. Tune out. Review of Lee Siegel's Against the Machine: Being Human in the Age of the Elec- tronic Mob. New York Times, February 3.
- MacDonald, S., & Kam, J. 2007. Aardvark et al.: Quality journals and gamesmanship in management studies. Journal of Information Science, 33(6): 702-717.
- Macri, J., & Sinha, D. 2006. Rankings methodology for interna- tional comparisons of institutions and individuals: An application to economics in Australia and New Zealand. Journal of Economic Surveys, 20(1): 111-156.
- Mangematin, V., & Baden-Fuller, C. 2008. Global contests in the production of business knowledge: Regional centres and individual business schools. Long Range Planning, 41(1): 117-139.
- March, J. G. 1996. Continuity and change in theories of organi- zational action. Administrative Science Quarterly, 41(2): 278 -287.
- Merton, R. K. 1936. The unanticipated consequences of purpo- sive social action. American Sociological Review, 1(6): 894 - 904.
- McGahan, A. M. 2007. Academic research that matters to man- agers: On zebras, dogs, lemmings, hammers, and turnips. Academy of Management Journal, 50(4): 749 -753.
- McGrath, R. G. 2007. No longer a stepchild: How the manage- ment field can come into its own. Academy of Management Journal, 50(6): 1365-1378.
- Morgan, H., Harkins, P., & Goldsmith, M. (Eds.) 2005. The art and practice of leadership coaching: 131-137. NY: John Wiley & Sons.
- Morrison, A. J. & Inkpen, A. C. 1991. An analysis of significant contributions to the international business literature. Jour- nal of International Business Studies, 22(1): 143-153.
- Peng, M. W. 2004. Identifying the big question in international business research. Journal of International Business Stud- ies, 35(2): 99 -108.
- Peng, M. W., & Zhou, J. Q. 2006. Most cited articles and authors in global strategy research. Journal of International Man- agement, 12(4): 490 -508.
- Pfeffer, J. 2005. Why do bad management theories persist? A comment on Ghoshal. Academy of Management Learning and Education, 4(1): 96 -100.
- Pfeffer, J., & Fong, C. 2002. The end of business schools? Less success than meets the eye. Academy of Management Learning and Education, 1(1): 78 -95.
- Rynes, S. L. 2007a. Academy of Management Journal editor's forum on citations: Editor's forward. Academy of Manage- ment Journal, 50(3): 489 -490.
- Rynes, S. L. 2007b. Academy of Management Journal editor's forum on research with relevance to practice. Editor's fore- word: Carrying Sumantra Ghoshal's Torch: Creating More Positive, Relevant, and Ecologically Valid Research. Acad- emy of Management Journal, 50(4): 745-747.
- Scott, W. R. 1987. The adolescence of institutional theory. Ad- ministrative Science Quarterly, 32: 493-511.
- Sauder, M. 2008. Interlopers and field change: The entry of U.S. news into the field of legal education. Administrative Sci- ence Quarterly, 53: 209 -234.
- Sauder, M., & W. Espeland, W. 2006. Strength in numbers? A comparison of law and business school rankings. Indiana Law Journal, 81: 205-227.
- Segalla, M. 2008a. Editorial: Publishing in the right place or publishing the right thing. European Journal of Interna- tional Management, 2: 122-127.
- Segalla, M. 2008b. Publishing in the right place or publishing the right thing: Journal targeting and citations strategies for promotion and tenure committees. Unpublished working paper (expanded version of Segalla 2008a). HEC School of Management, Paris, January 20.
- Singh, G., Haddad, K. M., & Chow, C. W. 2007. Are articles in "top" management journals necessarily of higher quality? Journal of Management Inquiry, 16(4): 319 -331.
- Spreitzer, G., & Sonenshein, S. 2003. Positive Deviance and Extraordinary Organizing in Cameron, K. S., Dutton, J. E., & Quinn, R. E. (Eds.), Positive organizational scholarship: 207- 224. San Francisco: Berrett-Kohler.
- Starbuck, W. H. 2005. How much better are the most-prestigious journals? The statistics of academic publication. Organiza- tion Science, 16(2): 180 -200.
- Starbuck, W. H. 2007. Living in mythical spaces. Organizational Studies, 28(1): 21-25
- Swanson, E. 2004. Publishing in the majors: A comparison of accounting, finance, management, and marketing. Contem- porary Accounting Research, 21(1): 223-255.
- Testa, J. nd. The Thomson Scientific journal selection process. http://www.thomsonreuters.com/business_units/scientific/ free/essays/journalselection/, accessed 15 October 2008.
- Tsui, A. S. 2007. From homogenization to pluralism: Interna- tional research in the academy and beyond. Academy of Management Journal, 50 (6): 1353-1364.
- Tung, R. L., & Witteloostuijn, A. van 2008. What makes a study sufficiently international? Journal of International Business Studies, 39(2): 180 -183.
- Tushman, M., & O'Reilly, C. III 2007. Research and relevance: Implications of Pasteur's quadrant for doctoral programs and faculty development. Academy of Management Jour- nal, 50(4): 769 -774.
- Van Fleet, D., McWilliams, A., & Siegel, D. 2000. A theoretical and empirical analysis of journal rankings: The case of formal lists. Journal of Management, 26(5): 839 -861.
- Vermeulen, F. 2005. On rigor and relevance: Fostering dialectic progress in management research. Academy of Manage- ment Journal, 48(6): 978 -982.
- Vermeulen, F. 2007. I shall not remain insignificant: Adding a second loop to matter more. Academy of Management Jour- nal, 50(4): 754 -761.
- Werner, S., & Brouthers, L. E. 2002. How international is man- agement? Journal of International Business Studies, 33(3): 583-591.
- Westney, E. D. 2005. Institutional theory and the multinational corporation. In Sumantra Ghoshal and D. Eleanor Westney (Eds.), Organization theory and the multinational corpora- tion: 47-67. NY: Palgrave MacMillan.
- Xu, S., Yalcinkaya, G., & Seggie, S. H. 2008. Prolific authors and institutions in leading international business journals. Asia Pacific Journal of Management, 25, published on-line No- vember 2007.
- Zemsky, R. 2008. The rain man cometh-Again. Academy of Management Perspectives, 22(1): 5-14.
- Zucker, L. G. 1987. Institutional theories of organization. Annual Review of Sociology, 13: 443-464.
Cited by
Sorted by citations
Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison
Harzing, A.-W., & Alakangas, S. (2016) · Scientometrics · doi:10.1007/s11192-015-1798-9 · Cited by 1,784 · see citing works
Self-citationDeep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441 · see citing works
Intended and unintended consequences of a publish‐or‐perish culture: A worldwide survey
van Dalen, H. P., & Henkens, K. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22636 · Cited by 355 · see citing works
Mingers, J., & Yang, L. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2016.07.058 · Cited by 184 · see citing works
How the publish-or-perish principle divides a science: the case of economists
van Dalen, H. P. (2021) · Scientometrics · doi:10.1007/s11192-020-03786-x · Cited by 128 · see citing works
Tian, M., Su, Y., & Ru, X. (2016) · Publications · doi:10.3390/publications4020009 · Cited by 129 · see citing works
One in four citations in marine biology papers is inappropriate
Todd, P., Guest, Jr., Lu, J., & Chou, L. (2010) · Marine Ecology Progress Series · doi:10.3354/meps08587 · Cited by 60 · see citing works
Characterisation of academic journals in the digital age
Gu, X., & Blackmore, K. (2017) · Scientometrics · doi:10.1007/s11192-016-2219-4 · Cited by 29 · see citing works
Examining the predictive validity of NIH peer review scores
Lindner, M. D., & Nakamura, R. K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126938 · Cited by 24 · see citing works
Research on Impact Evaluation of Open Access Journals
Wei, M. (2020) · Scientometrics · doi:10.1007/s11192-019-03306-6 · Cited by 23 · see citing works
- Authors
- Adler, N. J., & Harzing, A.-W.
- Affiliations
- McGill University (CA)
- Venue
- Academy of Management Learning & Education
- Publisher
- Academy of Management
- Published
- 2009-03
- Volume
- 8
- Issue
- 1
- Pages
- 72-95
- ISSN
- 1537-260X (ISSN-L)
- 1944-9585
- OpenAlex
- W2135547856
- Cited by
- 853
- References
- 3
- Citations by year
- 2026: 20 · 2025: 35 · 2024: 30 · 2023: 30 · 2022: 37 · 2021: 48 · 2020: 54 · 2019: 65 · 2018: 46 · 2017: 57 · 2016: 64 · 2015: 56 · 2014: 53 · 2013: 62 · 2012: 68
- FWCI
- 80.41
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- Locations
- Academy of Management Learning and Education (publishedVersion)
- eScholarship@McGill (McGill) (submittedVersion, other-oa)
- eScholarship@McGill (McGill) (submittedVersion)
- Minerva Access (University of Melbourne) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Political Science and International Relations › Higher Education Governance and Development
- Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
- Keywords
- academic rankings, university rankings, management research, research evaluation, international business research, university mission
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review