Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 178

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Can Harman’s single-factor test reliably distinguish between research designs? Not in published management studies

Howard, Matt C., Boudreaux, Melanie, Oglesby, Matthew · 2024 · European Journal of Work and Organizational Psychology

Abstract

The goal of the current article is to conduct a widescale empirical investigation on the (in)efficacy of Harman’s single-factor test by showing that the approach is insensitive to aspects of research design known to influence common method bias (CMB). Our systematic literature review of 1,619 sources demonstrates that the amount of variance explained by the first factor of Harman’s single-factor test does not differ between cross-sectional and multi-wave, single-source and multi-source, or mono-method and multi-method studies. We instead find that extraneous aspects of studies influence the amount of variance explained, including the number of studied indicators and retained factors. These results therefore suggest that Harman’s single-factor test is not a reliable assessment of CMB, and we hope these results prevent future researchers from applying the analysis. Our discussion concludes with alternative suggestions for identifying and addressing CMB, such as the application of sophisticated research designs and marker variable techniques.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/1359432x.2024.2393462

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 15 references

Research works printed but not linked: 54

  1. 10.1145/3330472.3330477 · DOI · Paper page
  2. 10.5296/ije.v4i2.1962 · DOI
  3. 10.1177/014920639101700303 · DOI · Paper page
  4. 10.1016/j.jwb.2016.12.002 · DOI
  5. 10.1177/1069031X21995871 · DOI
  6. 10.1007/s11747-020-00766-8 · DOI · Paper page
  7. Becker, R. (2021). Have you ever seen the rain? The causal impact of the weather situation and the season on survey participation in a multi-wave panel study. Survey Research Methods, 15(1), 27–41.
  8. 10.1111/1748-8583.12398 · DOI · Paper page
  9. 10.1177/1094428109360993 · DOI · Paper page
  10. CABS. (2021). Academic journal guide 2018. https://charteredabs.org/academic-journal-guide-2021/
  11. 10.1037/h0046016 · DOI · Paper page
  12. 10.1207/s15327906mbr0204_1 · DOI
  13. Campbell, D. T., & O’Connell, E. J. (1982). Methods as diluting trait relationships rather than adding irrelevant systematic variance. New Directions for Methodology of Social & Behavioral Science.
  14. 10.1007/978-3-030-22113-3_20 · DOI · Paper page
  15. 10.2307/41703491 · DOI · Paper page
  16. 10.1037/0033-2909.112.1.155 · doi:10.1037/0033-2909.112.1.155 · Paper page
  17. 10.1207/s15327043hup1101_2 · DOI
  18. 10.1007/s10869-010-9181-6 · DOI · Paper page
  19. 10.1016/j.jvb.2020.103472 · DOI · Paper page
  20. 10.1177/002224378702400308 · DOI · Paper page
  21. 10.1002/smj.701 · DOI · Paper page
  22. 10.1002/job.2673 · DOI
  23. 10.1037/h0035197 · DOI
  24. Eid, M., Koch, T., & Geiser, C. (2023). Multitrait–multimethod models. In R. H. Hoyle (Ed.), Handbook of Structural Equation Modeling (2nd ed, pp. 349–266). Guilford Press.
  25. 10.1108/S0742-730120220000040001 · DOI
  26. 10.1016/0749-5978(85)90002-0 · DOI · Paper page
  27. 10.1111/joms.12415 · DOI · Paper page
  28. 10.1016/j.ijnurstu.2013.10.005 · DOI · Paper page
  29. 10.1111/jopy.12625 · DOI
  30. 10.1002/wics.27 · DOI · Paper page
  31. Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis. Hampshire. Cengage Learning EMEA.
  32. Helm, J. L. (Ed.). (2022). Advanced multitrait-multimethod analyses for behavioral and social sciences. Routledge.
  33. 10.1177/0149206320960533 · DOI · Paper page
  34. 10.1016/j.jbusres.2023.113969 · DOI
  35. 10.1016/j.jbusres.2022.113328 · DOI · Paper page
  36. 10.1037/a0021504 · DOI · Paper page
  37. 10.1007/s12144-020-00645-z · DOI
  38. 10.1057/palgrave.jibs.8400373 · DOI · Paper page
  39. 10.1177/10944281221075361 · DOI · Paper page
  40. 10.1016/j.ijhm.2016.04.010 · DOI · Paper page
  41. 10.22237/jmasm/1130804820 · doi:10.22237/jmasm/1130804820
  42. 10.1146/annurev-psych-120710-100452 · DOI · Paper page
  43. 10.1146/annurev-orgpsych-110721-040030 · DOI · Paper page
  44. 10.1002/mpr.352 · DOI · Paper page
  45. 10.1080/10705511.2018.1558060 · DOI
  46. 10.1145/3051473.3051479 · DOI · Paper page
  47. 10.1177/1094428109351241 · DOI · Paper page
  48. 10.1108/IJCHM-03-2017-0136 · DOI
  49. 10.1177/0149206316687295 · DOI · Paper page
  50. 10.1007/s11747-020-00745-z · DOI
  51. 10.1016/j.jretai.2012.10.002 · DOI
  52. 10.1007/s10869-015-9422-9 · DOI · Paper page
  53. 10.1177/0165025413506143 · DOI
  54. 10.1037/met0000198 · doi:10.1037/met0000198

Cited by

See all 178

Metadata

Authors
Howard, M. C., Boudreaux, M., & Oglesby, M.
Venue
European Journal of Work and Organizational Psychology
Published
2024-08-27
Volume
33
Issue
6
Pages
790-804
ISSN
  • 1359-432X (ISSN-L)
  • 1464-0643
OpenAlex
W4401966679
Cited by
178
References
15
FWCI
55.33
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Can Harman’s single-factor test reliably distinguish between research designs? Not in published management studies. https://app.cochat.ai/papers/10.1080/1359432x.2024.2393462
The paper
Howard, M. C., Boudreaux, M., & Oglesby, M. (2024). Can Harman’s single-factor test reliably distinguish between research designs? Not in published management studies. European Journal of Work and Organizational Psychology. https://doi.org/10.1080/1359432x.2024.2393462

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.