The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Research Commentary—Generalizability of Information Systems Research Using Student Subjects—A Reflection on Our Practices and Recommendations for Future Research
Abstract
Information systems researchers, like those in many other disciplines in the social sciences, have debated the value and appropriateness of using students as research subjects. This debate appears in several articles that have been published on the subject as well as in the review process. In this latter arena, however, the debate has become increasingly like a script—the actors (authors and reviewers) simply read their parts of the script; some avoid the underlying issues whereas others cursorily address generalizability without real consideration of those issues. As a result, despite the extent of debate, we seem no closer to a resolution. Authors who use student subjects rely on their scripted arguments to justify the use of student subjects and do not always consider whether those arguments are valid. But reviewers who oppose the use of student subjects are equally culpable. They, too, rely on scripted arguments to criticize work using student subjects, and do not always consider whether those arguments are salient to the particular study. By presenting and reviewing one version of this script in the context of theoretical discussions of generalizability, we hope to demonstrate its limitations so that we can move beyond these scripted arguments into a more meaningful discussion. To do this, we review empirical studies from the period 1990–2010 to examine the extent to which student subjects are being used in the field and to critically assess the discussions within the field about the use of student samples. We conclude by presenting recommendations for authors and reviewers, for determining whether the use of students is appropriate in a particular context, and for presenting and discussing work that uses student subjects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587
Lincoln, Y. S., Guba, E. G., & Pilotta, J. J. (1985) · International Journal of Intercultural Relations · doi:10.1016/0147-1767(85)90062-8 · Cited by 45,787
Case Study Research: Design and Methods
Chaiklin, H. (1991) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199102000-00025 · Cited by 39,535
User Acceptance of Computer Technology: A Comparison of Two Theoretical Models
Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989) · Management Science · doi:10.1287/mnsc.35.8.982 · Cited by 25,709
A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies
Venkatesh, V., & Davis, F. D. (2000) · Management Science · doi:10.1287/mnsc.46.2.186.11926 · Cited by 23,066
(1979) · Medical Entomology and Zoology · Cited by 11,252
Understanding information technology usage: a test of competing models
Taylor, S., & Todd, P. A. (1995) · Information Systems Research · doi:10.1287/isre.6.2.144 · Cited by 8,815 · cited 1×
“The paper in our sample script relies on the decomposed theory of planned behavior (Taylor and Todd 1995), an adaptation of Ajzen’s (1991) theory of planned behavior (TPB), a broad theory of how human behavior unfolds.”
MethodTrust and TAM in Online Shopping: An Integrated Model1
Gefen, D., Karahanna, E., & Straub, D. W. (2003) · MIS Quarterly · doi:10.2307/30036519 · Cited by 8,635
Venkatesh, V. (2000) · Information Systems Research · doi:10.1287/isre.11.4.342.11872 · Cited by 6,664
Quasi-experimentation: Design and analysis issues for field settings
Shrout, P. E. (1980) · Evaluation and Program Planning · doi:10.1016/0149-7189(80)90063-4 · Cited by 6,010
Agarwal, R., & Karahanna, E. (2000) · MIS Quarterly · doi:10.2307/3250951 · Cited by 4,726
Validating Instruments in MIS Research1
Straub, D. W. (1989) · MIS Quarterly · doi:10.2307/248922 · Cited by 2,526
Sears, D. O. (1986) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.51.3.515 · Cited by 2,119
Piccoli, G., Ahmad, R., & Ives, B. (2001) · MIS Quarterly · doi:10.2307/3250989 · Cited by 1,587
Generalizing Generalizability in Information Systems Research
Lee, A. S., & Baskerville, R. L. (2003) · Information Systems Research · doi:10.1287/isre.14.3.221.16560 · Cited by 1,551
Venkatesh, V., & Brown, S. A. (2001) · MIS Quarterly · doi:10.2307/3250959 · Cited by 1,397
University of British Columbia, C., Benbasat, I., Barki, H., & Montréal, H. (2007) · Journal of the Association for Information Systems · doi:10.17705/1jais.00126 · Cited by 1,363
In defense of external invalidity.
Mook, D. G. (1983) · American Psychologist · doi:10.1037/0003-066x.38.4.379 · Cited by 1,212
Factors relevant to the validity of experiments in social settings.
Campbell, D. T. (1957) · Psychological Bulletin · doi:10.1037/h0040950 · Cited by 1,140
Designing Evaluations of Educational and Social Programs.
Hendrickson, L., & Cronbach, L. J. (1983) · Contemporary Sociology · doi:10.2307/2068975 · Cited by 1,042
Students and Housewives: Differences in Susceptibility to Reference Group Influence
Park, C. W., & Lessig, V. P. (1977) · Journal of Consumer Research · doi:10.1086/208685 · Cited by 746
Proceedings of the international conference on Information systems
Hirschheim, R., Newman, M., & DeGross, J. I. (1998) · Cited by 683
Sienkiewicz, H. (1895) · Cited by 669
The "Science of the Sophomore" Revisited: From Conjecture to Empiricism
Gordon, M. E., Slade, L. A., & Schmitt, N. (1986) · The Academy of Management Review · doi:10.2307/258340 · Cited by 574
Understanding it adoption decisions in small business: integrating current theories
Riemenschneider, C. K., Harrison, D. A., & Mykytyn, P. P. (2003) · Information & Management · doi:10.1016/s0378-7206(02)00010-1 · Cited by 570
Research Note: The Influence of Recommendations and Consumer Reviews on Evaluations of Websites
Kumar, N., & Benbasat, I. (2006) · Information Systems Research · doi:10.1287/isre.1060.0107 · Cited by 504
Research in Information Systems: An Empirical Study of Diversity in the Discipline and Its Journals
Vessey, I., Ramesh, V., & Glass, R. L. (2002) · Journal of Management Information Systems · doi:10.1080/07421222.2002.11045721 · Cited by 391
The College Sophomore as Guinea Pig: Setting the Record Straight
Greenberg, J. (1987) · The Academy of Management Review · doi:10.2307/258001 · Cited by 350
Complementarities Between Organizational IT Architecture and Governance Structure
Tiwana, A., & Konsynski, B. (2010) · Information Systems Research · doi:10.1287/isre.1080.0206 · Cited by 310
A Task-Based Model of Perceived Website Complexity1
Nadkarni, S., & Gupta, R. (2007) · MIS Quarterly · doi:10.2307/25148805 · Cited by 257
Relabeling internal and external validity for applied social scientists
Campbell, D. T. (1986) · New Directions for Program Evaluation · doi:10.1002/ev.1434 · Cited by 234
Berthon, P., Pitt, L., Ewing, M., & Carr, C. L. (2002) · Information Systems Research · doi:10.1287/isre.13.4.416.71 · Cited by 233
Fuller, R. M., & Dennis, A. R. (2009) · Information Systems Research · doi:10.1287/isre.1070.0167 · Cited by 231
The human subject in psychological research
Schultz, D. P. (1968) · PsycEXTRA Dataset · doi:10.1037/e515702009-001 · Cited by 209
The human subject in psychological research.
Schultz, D. P. (1969) · Psychological Bulletin · doi:10.1037/h0027880 · Cited by 194
Wheeler, B. C., & Valacich, J. S. (1996) · Information Systems Research · doi:10.1287/isre.7.4.429 · Cited by 181
An Assessment of Laboratory Experiments in Accounting
Swieringa, R. J., & Weick, K. E. (1982) · Journal of Accounting Research · doi:10.2307/2674675 · Cited by 176
Brass, D. J., & Locke, E. A. (1987) · Administrative Science Quarterly · doi:10.2307/2392922 · Cited by 152
Student Guinea Pigs: Porcine Predictors and Particularistic Phenomena
Gordon, M. E., Slade, L. A., & Schmitt, N. (1987) · Academy of Management Review · doi:10.5465/amr.1987.4306524 · Cited by 141
An appraisal of the use of student subjects in marketing research
Burnett, J. J., & Dune, P. M. (1986) · Journal of Business Research · doi:10.1016/0148-2963(86)90024-x · Cited by 138
Griffith, T. L., & Northcraft, G. B. (1996) · MIS Quarterly · doi:10.2307/249544 · Cited by 129
Subject selection bias in psychological research.
Smart, R. G. (1966) · The Canadian Psychologist · doi:10.1037/h0083096 · Cited by 126
External Validity in IS Survey Research
King, W. R., & He, J. (2005) · Communications of the Association for Information Systems · doi:10.17705/1cais.01645 · Cited by 120
Structuring Time and Task in Electronic Brainstorming1
Dennis, A. R., Aronson, J. E., Heninger, W. G., & Walker, E. D. (1999) · MIS Quarterly · doi:10.2307/249411 · Cited by 117
Students as Surrogates for Managers in a Decision-making Environment: An Experimental Study
Hughes, C. T., & Gibson, M. L. (1991) · Journal of Management Information Systems · doi:10.1080/07421222.1991.11517925 · Cited by 92
Meta-analysis : new developments and applications in medical and social sciences
Schulze, R., Holling, H., & Böhning, D. (2003) · Hogrefe & Huber eBooks · Cited by 77
OTHER-SETTINGS GENERALIZATION IN IS RESEARCH
Seddon, P. B., & Scheepers, R. (2006) · Deakin Research Online (Deakin University) · Cited by 24
Cited by
Synthesizing information systems knowledge: A typology of literature reviews
Paré, G., Trudel, M.-C., Jaana, M., & Kitsiou, S. (2015) · Information & Management · doi:10.1016/j.im.2014.08.008 · Cited by 1,987
Maier, C., Laumer, S., Weinert, C., & Weitzel, T. (2015) · Information Systems Journal · doi:10.1111/isj.12068 · Cited by 609
Mobile Application Usability: Conceptualization and Instrument Development1
Hoehle, H., & Venkatesh, V. (2015) · MIS Quarterly · doi:10.25300/misq/2015/39.2.08 · Cited by 435
Generalizing from Research Findings: The Merits of Case Studies
Tsang, E. W. K. (2014) · International Journal of Management Reviews · doi:10.1111/ijmr.12024 · Cited by 420
What catalyses mobile apps usage intention: An empirical analysis
Hew, J.-J., Lee, V.-H., Ooi, K.-B., & Wei, J. (2015) · Industrial Management & Data Systems · doi:10.1108/imds-01-2015-0028 · Cited by 383
Metadata
- Authors
- Compeau, D., Marcolin, B., Kelley, H., & Higgins, C.
- Venue
- Information Systems Research
- Published
- 2012-12
- Volume
- 23
- Issue
- 4
- Pages
- 1093-1109
- ISSN
- 1047-7047 (ISSN-L)
- 1526-5536
- OpenAlex
- W2059528567
- Cited by
- 256
- References
- 47
- FWCI
- 81.42
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.