Cited by 853scientometrics and bibliometrics researchHigher Education Governance and DevelopmentResearch, Science, and Academia

When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings

Open access
Adler, Nancy J., Harzing, Anne-Wil·2009·Academy of Management Learning & Education

Abstract

“Not everything that can be counted counts, and not everything that counts can be counted.” —Albert Einstein Has university scholarship gone astray? Do our academic assessment systems reward scholarship that addresses the questions that matter most to society? Using international business as an example, we highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. We call for an immediate examination of existing ranking systems, not only as a legitimate scholarly question vis-a-vis performance—a conceptual lens with deep roots in management research—but also because the very health and vibrancy of the field are at stake. Indeed, in light of the data presented here, which suggest that current systems are dysfunctional and potentially cause more harm than good, a temporary moratorium on rankings may be appropriate until more valid and reliable ways to assess scholarly contributions can be developed. The worldwide community of scholars, along with the global network of institutions interacting with and supporting management scholarship (such as the Academy of Management, AACSB, and Thomson Reuters Scientific) are invited to innovate and design more reliable and valid ways to assess scholarly contributions that truly promote the advancement of relevant 21st century knowledge, and likewise recognize those individuals and institutions that best fulfill the university's fundamental purpose.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.5465/amle.2009.37012181

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 71

  1. Adler, N. J. 2008a. Global business as an agent of world benefit: New international business perspectives leading positive change. In A. Scherer & Guido Palazzo (Eds.), Handbook of Research on Corporate Citizenship: 374 -401. Cheltenham, UK: Edward Elgar.
  2. Adler, N. J. 2008b. International Business Scholarship: Contrib- uting to a Broader Definition of Success. In Jean Boddewyn (Ed.), International Business Scholarship: AIB Fellows on the First 50 Years and Beyond (Volume 13 of Research in Global Strategic Management: 333-343). UK: Emerald Group.
  3. Adler, N. J. 2006. The arts and leadership: Now that we can do anything, what will we do? Academy of Management Learning and Education, 5(4): 466 -499.
  4. Adler, N. J., & Bartholomew, S. 1992. Academic and professional communities of discourse: Generating knowledge on tran- snational human resource management. Journal of Interna- tional Business Studies, 23(3): 551-569.
  5. Becker, H. S., & Geer, B. 1958. The fate of idealism in medical school. American Sociological Review, 23(1): 50 -56.
  6. Bennis, W., & O'Toole, J. 2005. How business schools lost their way. Harvard Business Review, 83(5): 96 -106.
  7. Boyacigiller, N., & Adler, N. J. 1991. The parochial dinosaur: The organizational sciences in a global context. Academy of Management Review, 16(2): 262-290.
  8. Buckley, P. 2002. Is the international business agenda running out of steam? Journal of International Business Studies, 33(2): 365-373.
  9. Buckley, P. J., & Lessard, D. R. 2005. Regaining the edge for international business research. Journal of International Business Studies, 36(6): 595-599.
  10. Buckridge, C. 2008. New research head puts focus on tackling society's problems. The University of Melbourne Voice, 2(4): 17 March-14 April 2008.
  11. Butler, K. C. 2006. Finance and the search for the big question in international business. Academy of International Business Insights, 6(3): 3-9.
  12. Cameron, K. S., Dutton, J. E., & Quinn, R. E. (Eds.). 2003. Positive organizational scholarship. San Francisco: Berrett-Kohler.
  13. Chan, K. C., Fung, H., & Lai, P. 2006. International business research: Trends and school rankings. International Busi- ness Review, 15(4): 317-338.
  14. Detsky, A. S., & Baerlocher, M. O. 2007. Academic mentoring- How to give it and how to get it. Journal of the American Medical Association, 297(19): 2134 -2136.
  15. DiMaggio, P. J., & Powell, W. W. 1983. The iron cage revisited: Institutional isomorphism and collective rationality in or- ganizational fields. American Sociological Review, 48(2): 147-160.
  16. Donaldson, L. 2005. For positive management theories while retaining science. Academy of Management Learning and Education, 4(1): 109 -113.
  17. Dubois, F. L., & Reeb, D. M. 2000. Ranking the international business journals. Journal of International Business Stud- ies, 31(4): 689 -704.
  18. Franke, R., Edlund, T., & Oster, R. 1990. The development of strategic management: Journal quality and article impact. Strategic Management Journal, 11(3): 243-253.
  19. Gauffriau, M., & Larsen, P. O. 2005. Counting methods are deci- sive for rankings based on publication and citation studies, Scientometrics, 64(1): 85-93.
  20. Ghoshal, S. 2005. Bad management theories are destroying good management practices. Academy of Management Learning and Education, 4(1): 75-91.
  21. Griffith, D. A., Cavusgil, S. T., & Xu, S. 2008. Emerging themes in international business research. Journal of International Business Studies, 39(7): 1220 -1235.
  22. Gulati, R. 2007. Tent poles, tribalism, and boundary spanning: The rigor-relevance debate in management research. Academy of Management Journal, 50(4): 775-782.
  23. Hardy, C., & Maguire, S. 2008. Institutional entrepreneurship. In R. Greenwood, C. Oliver, K. Sahlin, & R. Suddaby (Eds.), The Sage handbook of organizational institutionalism: 198 -217.
  24. Harzing, A. W. K. 2005. Australian research output in economics & business: High volume, low impact? Australian Journal of Management, 30(2): 183-200.
  25. Harzing, A. W. K. 2008a. Journal quality list. 31st ed., 31 May 2008, downloaded from www.harzing.com 1 August 2008.
  26. Harzing, A. W. K., & van der Wal, R. 2008a. Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics, 8(1): 62-71.
  27. Harzing, A. W. K., & van der Wal, R. 2008b. Comparing the Google Scholar H-index with the ISI Journal Impact Factor, harzing.com white paper, first version, 3 July 2008, http:// www.harzing.com/h_indexjournals.htm.
  28. Inkpen, A. C., & Beamish, P. W. 1994. An analysis of twenty-five years of research in the Journal of International Business Studies. Journal of International Business Studies, 25(4): 703- 713.
  29. Johnson, J., & Podsakoff, P. 1994. Journals of influence in the field of management: An analysis using Salancik's index in a dependency network. Academy of Management Journal, 37(5): 1392-1407.
  30. Kerr, S. 1975. On the folly of rewarding A, while hoping for B. Academy of Management Journal, 18(4): 769 -783.
  31. Kirkman, B., & Law, K. 2005. International management research in AMJ: Our past, present and future. Academy of Manage- ment Journal, 48(3): 377-386.
  32. Khurana, R. 2007. From higher aims to hired hands: The social transformation of business schools and the unfulfilled promise of management as a profession. Princeton, NJ: Princeton University Press.
  33. Kumar, V., & Kundu, S. K. 2004. Ranking the international busi- ness schools: Faculty publication as the measure. Manage- ment International Review, 44: 213-228. 2009
  34. Adler and Harzing Lanchester, J. 2008. Log on. Tune out. Review of Lee Siegel's Against the Machine: Being Human in the Age of the Elec- tronic Mob. New York Times, February 3.
  35. MacDonald, S., & Kam, J. 2007. Aardvark et al.: Quality journals and gamesmanship in management studies. Journal of Information Science, 33(6): 702-717.
  36. Macri, J., & Sinha, D. 2006. Rankings methodology for interna- tional comparisons of institutions and individuals: An application to economics in Australia and New Zealand. Journal of Economic Surveys, 20(1): 111-156.
  37. Mangematin, V., & Baden-Fuller, C. 2008. Global contests in the production of business knowledge: Regional centres and individual business schools. Long Range Planning, 41(1): 117-139.
  38. March, J. G. 1996. Continuity and change in theories of organi- zational action. Administrative Science Quarterly, 41(2): 278 -287.
  39. Merton, R. K. 1936. The unanticipated consequences of purpo- sive social action. American Sociological Review, 1(6): 894 - 904.
  40. McGahan, A. M. 2007. Academic research that matters to man- agers: On zebras, dogs, lemmings, hammers, and turnips. Academy of Management Journal, 50(4): 749 -753.
  41. McGrath, R. G. 2007. No longer a stepchild: How the manage- ment field can come into its own. Academy of Management Journal, 50(6): 1365-1378.
  42. Morgan, H., Harkins, P., & Goldsmith, M. (Eds.) 2005. The art and practice of leadership coaching: 131-137. NY: John Wiley & Sons.
  43. Morrison, A. J. & Inkpen, A. C. 1991. An analysis of significant contributions to the international business literature. Jour- nal of International Business Studies, 22(1): 143-153.
  44. Peng, M. W. 2004. Identifying the big question in international business research. Journal of International Business Stud- ies, 35(2): 99 -108.
  45. Peng, M. W., & Zhou, J. Q. 2006. Most cited articles and authors in global strategy research. Journal of International Man- agement, 12(4): 490 -508.
  46. Pfeffer, J. 2005. Why do bad management theories persist? A comment on Ghoshal. Academy of Management Learning and Education, 4(1): 96 -100.
  47. Pfeffer, J., & Fong, C. 2002. The end of business schools? Less success than meets the eye. Academy of Management Learning and Education, 1(1): 78 -95.
  48. Rynes, S. L. 2007a. Academy of Management Journal editor's forum on citations: Editor's forward. Academy of Manage- ment Journal, 50(3): 489 -490.
  49. Rynes, S. L. 2007b. Academy of Management Journal editor's forum on research with relevance to practice. Editor's fore- word: Carrying Sumantra Ghoshal's Torch: Creating More Positive, Relevant, and Ecologically Valid Research. Acad- emy of Management Journal, 50(4): 745-747.
  50. Scott, W. R. 1987. The adolescence of institutional theory. Ad- ministrative Science Quarterly, 32: 493-511.
  51. Sauder, M. 2008. Interlopers and field change: The entry of U.S. news into the field of legal education. Administrative Sci- ence Quarterly, 53: 209 -234.
  52. Sauder, M., & W. Espeland, W. 2006. Strength in numbers? A comparison of law and business school rankings. Indiana Law Journal, 81: 205-227.
  53. Segalla, M. 2008a. Editorial: Publishing in the right place or publishing the right thing. European Journal of Interna- tional Management, 2: 122-127.
  54. Segalla, M. 2008b. Publishing in the right place or publishing the right thing: Journal targeting and citations strategies for promotion and tenure committees. Unpublished working paper (expanded version of Segalla 2008a). HEC School of Management, Paris, January 20.
  55. Singh, G., Haddad, K. M., & Chow, C. W. 2007. Are articles in "top" management journals necessarily of higher quality? Journal of Management Inquiry, 16(4): 319 -331.
  56. Spreitzer, G., & Sonenshein, S. 2003. Positive Deviance and Extraordinary Organizing in Cameron, K. S., Dutton, J. E., & Quinn, R. E. (Eds.), Positive organizational scholarship: 207- 224. San Francisco: Berrett-Kohler.
  57. Starbuck, W. H. 2005. How much better are the most-prestigious journals? The statistics of academic publication. Organiza- tion Science, 16(2): 180 -200.
  58. Starbuck, W. H. 2007. Living in mythical spaces. Organizational Studies, 28(1): 21-25
  59. Swanson, E. 2004. Publishing in the majors: A comparison of accounting, finance, management, and marketing. Contem- porary Accounting Research, 21(1): 223-255.
  60. Testa, J. nd. The Thomson Scientific journal selection process. http://www.thomsonreuters.com/business_units/scientific/ free/essays/journalselection/, accessed 15 October 2008.
  61. Tsui, A. S. 2007. From homogenization to pluralism: Interna- tional research in the academy and beyond. Academy of Management Journal, 50 (6): 1353-1364.
  62. Tung, R. L., & Witteloostuijn, A. van 2008. What makes a study sufficiently international? Journal of International Business Studies, 39(2): 180 -183.
  63. Tushman, M., & O'Reilly, C. III 2007. Research and relevance: Implications of Pasteur's quadrant for doctoral programs and faculty development. Academy of Management Jour- nal, 50(4): 769 -774.
  64. Van Fleet, D., McWilliams, A., & Siegel, D. 2000. A theoretical and empirical analysis of journal rankings: The case of formal lists. Journal of Management, 26(5): 839 -861.
  65. Vermeulen, F. 2005. On rigor and relevance: Fostering dialectic progress in management research. Academy of Manage- ment Journal, 48(6): 978 -982.
  66. Vermeulen, F. 2007. I shall not remain insignificant: Adding a second loop to matter more. Academy of Management Jour- nal, 50(4): 754 -761.
  67. Werner, S., & Brouthers, L. E. 2002. How international is man- agement? Journal of International Business Studies, 33(3): 583-591.
  68. Westney, E. D. 2005. Institutional theory and the multinational corporation. In Sumantra Ghoshal and D. Eleanor Westney (Eds.), Organization theory and the multinational corpora- tion: 47-67. NY: Palgrave MacMillan.
  69. Xu, S., Yalcinkaya, G., & Seggie, S. H. 2008. Prolific authors and institutions in leading international business journals. Asia Pacific Journal of Management, 25, published on-line No- vember 2007.
  70. Zemsky, R. 2008. The rain man cometh-Again. Academy of Management Perspectives, 22(1): 5-14.
  71. Zucker, L. G. 1987. Institutional theories of organization. Annual Review of Sociology, 13: 443-464.

Cited by

Sorted by citations

Authors
Adler, N. J., & Harzing, A.-W.
Affiliations
McGill University (CA)
Venue
Academy of Management Learning & Education
Published
2009-03
Volume
8
Issue
1
Pages
72-95
ISSN
  • 1537-260X (ISSN-L)
  • 1944-9585
OpenAlex
W2135547856
Cited by
853
References
3
Citations by year
2026: 20 · 2025: 35 · 2024: 30 · 2023: 30 · 2022: 37 · 2021: 48 · 2020: 54 · 2019: 65 · 2018: 46 · 2017: 57 · 2016: 64 · 2015: 56 · 2014: 53 · 2013: 62 · 2012: 68
FWCI
80.41
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • Academy of Management Learning and Education (publishedVersion)
  • eScholarship@McGill (McGill) (submittedVersion, other-oa)
  • eScholarship@McGill (McGill) (submittedVersion)
  • Minerva Access (University of Melbourne) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Political Science and International Relations › Higher Education Governance and Development
  • Social Sciences › Decision Sciences › Information Systems and Management › Research, Science, and Academia
Keywords
academic rankings, university rankings, management research, research evaluation, international business research, university mission

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review