Eight Obstacles to Overcome in the Theory Testing Genre
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
Theory testing work is popular in information systems (IS), with many studies using questionnaires, experiments, or other methods to gather quantitative data and test hypotheses with statistical techniques. This editorial note highlights some of the obstacles that theory testing researchers face, and assists authors so that their papers will not be rejected outright on submission, nor slowed unnecessarily in the review process. In particular, we identify three obstacles relating to theorizing and five related to methods. We provide guidance on how authors can deal with each obstacle, and include examples of studies that have successfully addressed the obstacle. We hope that our editorial will encourage authors to take better note of these obstacles and to give further consideration to avoiding them when they plan and conduct their research studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194
MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011) · MIS Quarterly · doi:10.2307/23044045 · Cited by 3,165
Ringle, C. M., Sarstedt, M., & Straub, D. W. (2012) · MIS Quarterly · doi:10.2307/41410402 · Cited by 2,184
In Pursuit of Moderation: Nine Common Errors and Their Solutions1
Carte, T. A., & Russell, C. J. (2003) · MIS Quarterly · doi:10.2307/30036541 · Cited by 565 · cited 1×
“Make certain that common problems are avoided, such as the incorrect testing of moderation effects (Carte & Russell, 2003) and the application of difference scores (Klein, Jiang, & Cheney, 2009).”
BackgroundAssessing Common Method Bias: Problems with the ULMC Technique1
Chin, W. W., Thatcher, J. B., & Wright, R. T. (2012) · MIS Quarterly · doi:10.2307/41703491 · Cited by 505
Sharma, R., Yetton, P., & Crawford, J. (2009) · MIS Quarterly · doi:10.2307/20650305 · Cited by 496
The Fallacy of Formative Measurement
Edwards, J. R. (2011) · Organizational Research Methods · doi:10.1177/1094428110378369 · Cited by 495
Evaluating and Developing Theories in the Information Systems Discipline
Monash University, & Weber, R. (2012) · Journal of the Association for Information Systems · doi:10.17705/1jais.00284 · Cited by 344
Cited by
Big Data Research in Information Systems: Toward an Inclusive Research Agenda
University of Virginia, Abbasi, A., Sarker, S., University of Virginia, Aalto University, Chiang, R., & University of Cincinnati (2016) · Journal of the Association for Information Systems · doi:10.17705/1jais.00423 · Cited by 688
“…traditional scientific model of research (which often involves constructing hypotheses based on guesses and then deductively testing the hypotheses using carefully sampled data (e.g., Gregor & Klein, 2014)) in a data-abundant environment associated with big data (Anderson, 2008; Kitchin, 2014b).”
BackgroundZeng, N., Liu, Y., Gong, P., Hertogh, M., & König, M. (2021) · Frontiers of Engineering Management · doi:10.1007/s42524-021-0153-5 · Cited by 300
Menard, P., Bott, G. J., & Crossler, R. E. (2017) · Journal of Management Information Systems · doi:10.1080/07421222.2017.1394083 · Cited by 279
Context is king! Considering particularism in research design and reporting
Davison, R. M., & Martinsons, M. G. (2016) · Journal of Information Technology · doi:10.1057/jit.2015.19 · Cited by 259
“As Gregor and Klein (2014) suggest, in building theoretical models, researchers must not only justify the cause and effect linkages between constructs but also seek to ensure that the prior empirical studies they cite are appropriate.”
The Usability Construct: A Dead End?
Tractinsky, N. (2018) · Human–Computer Interaction · doi:10.1080/07370024.2017.1298038 · Cited by 87
Metadata
- Authors
- Australian National University, Gregor, S., Klein, G., & University of Colorado, C. S.
- Venue
- Journal of the Association for Information Systems
- Publisher
- Association for Information Systems
- Published
- 2014-11
- Volume
- 15
- Issue
- 11
- Pages
- I-XIX
- ISSN
- 1536-9323 (ISSN-L)
- OpenAlex
- W190672341
- Cited by
- 38
- References
- 8
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Eight Obstacles to Overcome in the Theory Testing Genre. https://app.cochat.ai/papers/10.17705/1jais.00382
- The paper
- Australian National University, Gregor, S., Klein, G., & University of Colorado, C. S. (2014). Eight Obstacles to Overcome in the Theory Testing Genre. Journal of the Association for Information Systems. https://doi.org/10.17705/1jais.00382
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.