CoChat
Sign Up
Cited by 565Business Strategies and Management ResearchMeta-analysis and systematic reviewsAdvanced Causal Inference Techniques

In Pursuit of Moderation: Nine Common Errors and Their Solutions1

Open access
Carte, Traci A., Russell, Craig J.·2003·MIS Quarterly
DOI 10.2307/30036541
Open this paper in CoChat — free accountOpen PDF

Abstract

One result of the increasing sophistication and complexity of MIS theory and research is the number of studies hypothesizing and testing for moderation effects. A review of the MIS and broader management literatures suggests researchers investigating moderated relationships often commit one or more errors falling into three broad categories: inappropriate use or interpretation of statistics, misalignment of research design with phenomena of interest, and measurement or scaling issues. Examples of nine common errors are presented. Commission of these errors is expected to yield literatures characterized by mixed results at best, and thoroughly erroneous results at worse. Procedures representing examples of best practice and reporting guidelines are provided to help MIS investigators avoid or minimize these errors.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2307/30036541

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 70

  1. Aguinis, H. "Statistical Power Problems in Multiple Moderated Regression in Management Research," Journal of Management (21:6), 1995, pp. 1141-1158.
  2. Ahituv, N., Igbaria, M., and Sella, A. "The Effects of Time Pressure and Completes of Information on Decision Making," Journal of Management Information Systems (15:2), 1998, pp. 153-172.
  3. Aiken, L. S., and West, S. G. Multiple Regres- sion: Testing and Interpreting Interactions, Sage Publications, Newberry Park, CA, 1996.
  4. Armstrong, C. P., and Sambamurthy, V. "Informa- tion Technology Assimilation in Firms: The Influence of Senior Leadership and IT Infrastructures," Information Systems Research (10:4), 1999, pp. 304-327.
  5. Arnold, H. J. "Moderator Variables: A Clarifica- tion of Conceptual, Analytic, and Psychometric Issues," Organizational Behavior and Human Performance (29:2), 1982, pp. 143-174.
  6. Arnold, H. J. "Test Moderator Variable Hypothe- sis: A Reply to Stone and Hollenbeck," Organi- zational Behavior and Human Performance (34), 1984, pp. 214-224.
  7. Arnold, H. J. "A Test of the Multiplicative Hypo- thesis of Expectancy Valence Theories of Work Motivation," Academy of Management Journal, (24) 1981, pp. 128-141.
  8. Banerjee, D., Cronan, T. P., Jones, T. W. "Modeling IT Ethics: A Study of Situational Ethics," MIS Quarterly (22:1), 1998, pp. 31-60.
  9. Banker, R. D., and Slaughter, S. A. "The Moder- ating Effects of Structure on Volatility and Com- plexity in Software Enhancement," Information Systems Research (11:3), 2000, pp. 219-240.
  10. Baron, R. M., and Kenny, D. A. "The Moderator- Mediator Variable Distinction in Social Psycho- logy Research: Conceptual, Strategic, and Sta- tistical Considerations," Journal of Personality and Social Psychology (51:6), 1986, pp. 1173- 1182.
  11. Baroudi, J. J., and Orlikowski, W. J. "The Prob- lem of Statistical Power in MIS Research," MIS Quarterly (13:1), 1989, pp. 87-106.
  12. Bartlett, M. S. "The Use of Transformations," Biometrics (3), 1947, pp. 22-38.
  13. Birnbaum, M. H. "The Devil Rides Again: Cor- relation as an Index of Fit," Psychological Bulletin (79), 1973, pp. 239-242.
  14. Birnbaum, M. H. Measurement, Judgement, and Decision Making, Academic Press, San Diego, CA, 1998.
  15. Birnbaum, M. H. "Relationships Among Models of Salary Bias," American Psychologist (40), 1985, pp. 862-866.
  16. Birnbaum, M. H. "Reply to the Devil's Advocates: Don't Confound Model Testing with Measure- ment," Psychological Bulletin (81), 1974, pp. 854-859.
  17. Birnbaum, M. H. "To Resolve Fechner vs. Stevens: Settle the Dispute Concerning 'Ratios' and 'Differences,'" Behavioral and Brain Sciences (12), 1989, pp. 270-271.
  18. Bobko, P. Correlation and Regression: Principles and Applications of Industrial/Qrganizational Psychology and Management. McGraw-Hill Inc., New York, 1995.
  19. Bobko, P. "A Solution to Some Dilemmas When Testing Hypotheses About Ordinal Inter- actions," Journal of Applied Psychology (71), 1986, pp. 323-326.
  20. Boudreau, M. C, Gefen, D., and Straub, D. W. "Validation in Information Systems Research: A State-of-the-Art Assessment," MS Quarterly (25:1), 2001, pp. 1-16.
  21. Busemeyer, J. R., and Jones, J. E. "Analysis of Multiplicative Combination Rules When the Causal Variables Are Measured with Error," Psychological Bulletin (93), 1983, pp. 549-562.
  22. Chin, W. W., Marcolin, B. L., and Newsted, P. R. "A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and Voice Mail Emotion/Adoption Study," in Proceedings of the Seventeenth Intemational Conference on Information Systenris, J. I.
  23. Choe, J. M. "The Relationships Among Perfor- mance of Accounting Information Systems, Influence Factors, and Evolution Level of Infor- mation Systems" Journal of Management Information Systems (12:4), 1996, pp.215-239.
  24. Chow, G. "Test of Equality between Sets of Coef- ficients in Two Linear Regressions," Econo- metrica (28:3), 1960, pp. 591-604.
  25. Cicchetti, D. V., Showalter, D., and Tyrer, P. J. "The Effect of Number of Rating Scale Cate- gories on Levels of Interrater Reliability: A Monte Carlo Investigation," Applied Psycho- logical Measurement {9), 1985, pp. 31-36.
  26. Cohen, J. "Partial Products Are Interactions; Partial Powers Are Curve Components," Psych- ological Bulletin (85), 1978, pp. 858-866.
  27. Cohen, J., and Cohen, P. Applied Multiple Regression/Correiation Anaiysis for the Beha- vioral Sciences (2"" ed.), Lawrence Erlbaum, Hillsdale, NJ, 1983.
  28. Cortina, J. M. "Interaction, Nonlinearity, and Multi- colinearity: Implications for Multiple Regres- sion," Journal of Management (19), 1993, pp. 915-922.
  29. Devaraj, S., and Kohli, R. "Information Techno- logy Payoff in the Health-Care Industry: A Longitudinal Study," Journal of Management Information Systems (16:4), 2000, pp. 41-67.
  30. Fritz, M. B. W., Yarasimhan, S., and Rhee, H.-S. "Communications and Coordinations on the Virtual Office" Journal of Management Infor- mation Systems (14:4), 1998, pp. 7-28.
  31. Gefen, D., Straub, D. W., and Boudreau, M. C. "Structural Equation Modeling and Regression: Guidelines for Research Practice," Communi- cations of the AIS (1:7), 2000, pp. 1-78.
  32. Grover, V., Cheon, M. J., and Teng, J. T. C. "The Effect of Service Quality and Partnership on the Outsourcing of Information Systems Functions," Journal of Management Information Systems (12:4), 1996, pp. 89-116.
  33. Hardgrave, B. C, Wilson, R. L., and Eastman, K. "Toward a Contingency Model for Selecting an Information System Prototyping Strategy," Jour- nal of Management Information Systems (16:2), 1999, pp. 113-136.
  34. Harman, H. H. Modern Factor Analysis (3'" ed.). University of Chicago Press, Chicago, 1976.
  35. Harris, R. J. "Reforming Significance Testing Via Three-Valued Logic," In What If There Were No Significance Testing?, L. L. Harlow, S. A. Mulaik, and J. H. Steiger (eds.), Lawrence Erlbaum, Mahwah, NJ, 1997.
  36. Harrington, S. "The Effect of Codes of Ethics and Personal Denial of Responsibility on Computer Abuse and Intention Judgments," MIS Quarterly (20:3), 1996, pp. 257-278.
  37. Harrison, D. A., Mykytyn Jr., P. P., and Riemen- schneider, C. K. "Executive Decisions About Adoption of Information Technology in Small Businesses: Theory and Empirical Tests," Inforation Systems Research (8:2), 1997, pp. 171-195.
  38. Igbaria, M., and Guimaraes, T. "Antecedents and Consequence of Job Satisfaction Among Infor- mation Center Employees" Journai of Manage- ment Information Systems (9:4), 1993, pp. 145- 174.
  39. Igbaria, M., and Guimaraes, T. "Exploring Dif- ferences in Employee Turnover Intentions and Its Determinants Among Telecommuters and Non-Telecommuters," Journal of Management Information Systems (16:1), 1999, pp. 147-164.
  40. Igbaria, M., Parasuraman, S., and Badaway, M. K. "Work Experiences, Job Involvement, and Quality of Work Life Among Information Sys- tems Personnel," MIS Quarterly (18:2), 1994, pp. 175-201.
  41. Jarvenpaa, S. L., Dickson, G. W., and DeSanctis, G. "Methodological Issues in Experimental IS Research: Experiences and Recommenda- tions," MIS Quarterly (9:2), 1985, pp. 141-156.
  42. James, L., and Brett, J. "Mediators, Moderators, and Tests for Mediations," Journal of Applied Psychology (69:2), 1984, pp. 307-321.
  43. Keil, M., Tan, B. C. Y., Wei, K. K., Saarinen, T., Tuunainen, V., and Wassenaar, A. "A Cross- Cultural Study on Escalation of Commitment Behavior in Software Projects," MIS Quarterly, 2000 (24:2), pp. 299-325.
  44. Landis, R. S., and Dunlap, W. P. "Moderated Multiple Regression Tests Are Criterion Spe- cific," Qrganizational Research Methods (3), 2000, pp. 254-266.
  45. Lee, A. "Executive Overview. Validation in Infor- mation Systems Research: A State of the Art Assessment," MIS Quarterly (25:1), 2001, p. xi.
  46. Lee, B., Barua, A., and Whinston, A. B. "Dis- covery and Representation of Causal Relation- ships in MIS Research: A Methodological Framework," MIS Quarterly (21:1), 1997, pp. 109-136.
  47. Lubinski, D., and Humphreys, L. G. "Assessing Spurious 'Moderator Effects': Illustrated Sub- stantively with the Hypothesized ('Synergistic') Relation between Spatial and Mathematical ability," Psychological Bulletin (107), 1990, pp. 385-393.
  48. McClelland, G. H., and Judd, C. M. "Statistical Difficulties of Detecting Interactions and Moder- ator Effects," Psychological Bulletin (114), 1993, pp. 376-390.
  49. McKeen, J. D., and Guimaraes, T. "Successful Strategies for User Participation in Systems Development," Journal of Management Infor- mation Systems (14:2), 1997, pp. 133-150.
  50. McKeen, J. D., Guimaraes, T., and Wetherbe, J. C. "The Relationship between User Participa- tion and User Satisfaction: An Investigation of Four Contingency Factors," MIS Quarterly (18:4), 1994, pp. 427-451.
  51. Nunnally, J. Psychometric Theory, McGraw-Hill, New York, 1967.
  52. Podsakoff, P. M., MacKenzie, S. B., Ahearne, M., and Bommer, W. H. "Searching fora Needle in a Haystack: Trying to Identify the Illusive Moderators of Leadership Behaviors," Journal of Management (21), 1995, pp. 422-470.
  53. Raelin, J. A. "Work Patterns in the Professional Lifecycle," Journal of Qccupational Psychology (58:3), 1985, pp. 177-187.
  54. Rice, R. E., and Contractor, N. S. "Conceptua- lizing Effects of Office Information Systems: A Methodology and Application for the Study of Alpha, Beta, and Gamma Changes," Decision Science (21), 1990, pp. 301-317.
  55. Russell, C. J., and Bobko, P. "Moderated Regression Analysis and Likert Scales: Too Coarse for Comfort," Journal of Applied Psychology (77), 1992, pp. 336-342.
  56. Russell, C. J., and Dean, M. A. "To Log or Not to Log: Bootstrap as an Alternative to Parametric Estimation of Moderation Effects in the Pre- sence of Skewed Dependent Variables," Organizational Research Methods (3), 2000, pp. 167-185.
  57. Saleem, Y, "An Empirical Test of the Contingency Approach to User Participation in Information Systems Development," Journal of Manage- ment Information Systems (13:1), 1996, pp. 145-166.
  58. Saunders, D. R. "Moderator Variables in Predic- tion," Educational and Psychological Measure- ment (16), 1956, pp. 209-222.
  59. Schepanski, A. "The Predictive Ability Criterion in Experimental Judgment Research in Accounting," Dec/s/onSc/ences (14), 1983, pp. 503-512.
  60. Schmidt, F. E. "Implications of a Measurement Problem for Expectancy Theory Research," Organizational Behavior and Human Perfor- mance (10), 1973, pp. 243-251.
  61. Schmitt, N. W. "The Use of Analysis of Covari- ance Structures to Assess Beta and Gamma Change," Multivariate Behavioral Research (17), 1982, pp. 343-358.
  62. Schmitt, N. W., Pulakos, E. D., and Liebiein, A. "Comparison of Three Techniques to Assess Group-Levei Beta and Gamma Change," Applied Psychological Measurement (8), 1984, pp. 249-260.
  63. Sharma, S., Durand, R. IVI., and Gur-Aire, O. "Identification and Analysis of Moderator Vari- ables," Journal of Marketing Research (18), 1981,August, pp. 291-300.
  64. Stevens, S. S. "Problems and Methods of Psychophysics," Psychological Bulletin (55), 1958, pp. 177-196.
  65. Thompson, R. L., Higgins, C. A., and Howell, J. M. "Influence of Experience on Personal Computer Utilization: Testing a Conceptual Model," Journal of Management Information Systems (11:1), 1994, pp. 167-187.
  66. Venkatesh, V., and Morris, M. G. "Why Don't Men Ever Stop to Ask for Directions? Gender, Social Influence, and Their Role in Technology Acceptance and Usage Behavior," MiS Quarterly (24:1), 2000 pp. 115-139.
  67. Vroom, V. H. Work and Motivation, Wiley, New York, 1964.
  68. Wagner, J. A. "The Organizational Tenure-Job Involvement Relationship: A Job-Career Experience Explanation," Journai of Organiza- tional Behavior (8A), 1987, pp. 63-70.
  69. Weill, P. "The Relationship between Investment in Information Technology and Firm Perfor- mance: A Study of the Valve Manufacturing Sector," information Systems Research (3:4), 1992, pp. 307-333.
  70. Winer, B. J. Statistical Principies in Experimentai Design, McGraw Hill, New York, 1974.

Cited by

Sorted by citations

Authors
Carte, T. A., & Russell, C. J.
Affiliations
  • University of Oklahoma (US)
  • Division of Management, Price College of Business, University of Oklahoma, 307 W. Brooks Drive, Norman, OK 73019-0450, U.S.A.
ORCID
Carte, Traci A.
Venue
MIS Quarterly
Publisher
MIS Quarterly
Published
2003-09-01
Volume
27
Issue
3
Pages
479-501
ISSN
  • 0276-7783 (ISSN-L)
  • 2162-9730
DOI
10.2307/30036541
OpenAlex
W2127450621
Cited by
565
References
6
Citations by year
2026: 10 · 2025: 17 · 2024: 6 · 2023: 10 · 2022: 24 · 2021: 25 · 2020: 25 · 2019: 29 · 2018: 14 · 2017: 28 · 2016: 23 · 2015: 37 · 2014: 34 · 2013: 42 · 2012: 37
FWCI
17.11
Type
journal article
Language
en
Open access
Locations
  • MIS Quarterly (publishedVersion)
  • SHAREOK (University of Oklahoma; Oklahoma State University; Central Oklahoma University) (submittedVersion)
  • Journal of the Association for Information Systems (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Business Strategies and Management Research
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
moderating effect, statistical interpretation, research design, measurement challenges, scaling, reporting guidelines, best practices, management information systems research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review