The DOI matches its Crossref record; 68 of the 72 references it cites were found in a registry or an index.
Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires.
Abstract
The rapid growth of the Internet provides a wealth of new research opportunities for psychologists. Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N = 361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods. It is concluded that Internet methods can contribute to many areas of psychology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The proportion of females (62%) to males in this study was more representative than in traditional survey methods, where women often comprise 71% to 77% of the participants [39]”
From Effects of Mock Facebook Workday Comments on Public Perception of Professional Credibility: A Field Study in Canada (2019) · cites this paper as [39]
In the citing paper
“Self-presentation is an unlikely confound in Study 1 because the motivation to self-present is absent when participants feel anonymous (Paulhus, 1984(Paulhus, , 1991) ) as they do in anonymous online studies such as the present one, where they don't provide identifying information (Gosling et al., 2004).”
From Cultural norm fulfillment, interpersonal belonging, or getting ahead? A large-scale cross-cultural test of three perspectives on the function of self-esteem. (2015) · cites this paper as Gosling et al., 2004
In the citing paper
“As described in the Appendix, the reliability and validity of internet data collection has become increasingly clear in recent years.”
From Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention (2008) · cites this paper as Gosling et al., 2004
In the citing paper
“The validity of internet testing has been well supported in recent reviews (Gosling et al., 2004), with excellent correspondence between results obtained with internet samples are results obtained in the laboratory (e.g., Birnbaum, 1999;Krantz & Dalal, 2000;McGraw, Tew, & Williams, 2000;Reips, 2002).”
From Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention (2008) · cites this paper as Gosling et al., 2004
In the citing paper
“Furthermore, previous research has suggested populations for online studies and Facebook are quite representative [23], [24].”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [23]
In the citing paper
“Still, the sample sizes and availability of demographic information afforded by social media bring us closer to a more ideal representative sample [99].”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [99]
Citations
Cites
Mail and internet surveys : the tailored design method
Dillman, D. A. (2007) · Cited by 9,587
(1999) · Cited by 7,995
Self-citationInternet paradox: A social technology that reduces social involvement and psychological well-being?
Kraut, R., Patterson, M., Lundmark, V., Kiesler, S., Mukophadhyay, T., & Scherlis, W. (1998) · American Psychologist · doi:10.1037/0003-066x.53.9.1017 · Cited by 4,793
Handbook of Personality : Theory and Research
Pervin, L. A. (1992) · Cited by 4,543
Orne, M. T. (1962) · American Psychologist · doi:10.1037/h0043424 · Cited by 4,041
Mail and Internet Surveys: The Tailored Design Method
Chisnall, P. M. (2007) · Journal of Advertising Research · doi:10.2501/s0021849907070237 · Cited by 3,463
Gender differences in personality traits across cultures: Robust and surprising findings.
Costa, P. T., Terracciano, A., & McCrae, R. R. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.81.2.322 · Cited by 2,986
Kraut, R. E., Kiesler, S. B., Boneva, B. S., Cummings, J. N., Helgeson, V. S., & Crawford, A. M. (2002) · Journal of Social Issues · doi:10.1111/1540-4560.00248 · Cited by 2,367
Gender differences in personality: A meta-analysis.
Feingold, A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.3.429 · Cited by 2,240
Handbook of Personality Theory and Research.
Schwartz, M. L., Borgatta, E. F., & Lambert, W. W. (1970) · American Sociological Review · doi:10.2307/2093310 · Cited by 2,182
Research works printed but not linked: 7
- Azar, 2000, A web of research: They're fun, they're fast, and they save money, but do Web experiments yield quality results?, Monitor on Psychology, 31, 42
- 10.1037/0022-3514.75.3.729 · DOI · Paper page
- Dillman. 1998 · DOI
- Locke, 1995, Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats., Journal of Social Behavior and Personality, 10, 255
- 2000, Digest of education statistics 2000.
- Skitka, (in press). Social psychological research and me Internet: The promise and the perils of a new methodological frontier.
- 2000, Population, 1
Cited by
Sorted by citations
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
“To examine how MTurk samples compare with the diversity of standard Internet samples, we compared the demographics of 3,006 MTurk participants with those in a large Internet sample (Gosling et al., 2004).”
Self-citationPositive Psychology Progress: Empirical Validation of Interventions.
Seligman, M. E. P., Steen, T. A., Park, N., & Peterson, C. (2005) · American Psychologist · doi:10.1037/0003-066x.60.5.410 · Cited by 5,947
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
“Based on a comprehensive analysis, for example, Gosling et al. (2004) conclude that Internet samples tend to be diverse, are not adversely affected by nonserious or habitual responders, and produce findings consistent with traditional methods.”
ResultRunning experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
“More importantly, as we demonstrate in Section 4, non-response error seems to be less of a concern in Mechanical Turk samples than in Internet convenience samples recruited through other means.”
BackgroundResultInfluentialDevelopment and validation of the short grit scale (grit-s)
Duckworth, A. L., & Quinn, P. D. (2009) · Journal of Personality Assessment · doi:10.1080/00223890802634290 · Cited by 3,376
“Gosling, Vazire, Srivastava, and John (2004) reviewed evidence that questionnaires completed on the Internet demonstrate psychometric properties that are very similar to those of paper-and-pencil measures.”
Metadata
- Authors
- Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P.
- Affiliations
- The University of Texas at Austin (US)
- Stanford University (US)
- University of California, Berkeley (US)
- Venue
- American Psychologist
- Published
- 2004
- Volume
- 59
- Issue
- 2
- Pages
- 93-104
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W1992490637
- Cited by
- 2,932
- References
- 72
- Citations by year
- 2026: 40 · 2025: 64 · 2024: 72 · 2023: 85 · 2022: 123 · 2021: 135 · 2020: 157 · 2019: 163 · 2018: 148 · 2017: 170 · 2016: 181 · 2015: 192 · 2014: 218 · 2013: 184 · 2012: 189
- FWCI
- 261.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Psychologist (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- online data collection, paper-and-pencil questionnaires, sample representativeness, data quality, response format effects, generalizability, self-report questionnaire
- MeSH terms
- Adult, Aged, Attitude, Female, Humans, Male, Middle Aged, Psychology, Surveys and Questionnaires, Evidence-Based Medicine, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
