The DOI matches its Crossref record; 68 of the 72 references it cites were found in a registry or an index.
Should We Trust Web-Based Studies? A Comparative Analysis of Six Preconceptions About Internet Questionnaires.
Abstract
The rapid growth of the Internet provides a wealth of new research opportunities for psychologists. Internet data collection methods, with a focus on self-report questionnaires from self-selected samples, are evaluated and compared with traditional paper-and-pencil methods. Six preconceptions about Internet samples and data quality are evaluated by comparing a new large Internet sample (N = 361,703) with a set of 510 published traditional samples. Internet samples are shown to be relatively diverse with respect to gender, socioeconomic status, geographic region, and age. Moreover, Internet findings generalize across presentation formats, are not adversely affected by nonserious or repeat responders, and are consistent with findings from traditional methods. It is concluded that Internet methods can contribute to many areas of psychology.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The proportion of females (62%) to males in this study was more representative than in traditional survey methods, where women often comprise 71% to 77% of the participants [39]”
From Effects of Mock Facebook Workday Comments on Public Perception of Professional Credibility: A Field Study in Canada (2019) · cites this paper as [39]
In the citing paper
“Self-presentation is an unlikely confound in Study 1 because the motivation to self-present is absent when participants feel anonymous (Paulhus, 1984(Paulhus, , 1991) ) as they do in anonymous online studies such as the present one, where they don't provide identifying information (Gosling et al., 2004).”
From Cultural norm fulfillment, interpersonal belonging, or getting ahead? A large-scale cross-cultural test of three perspectives on the function of self-esteem. (2015) · cites this paper as Gosling et al., 2004
In the citing paper
“As described in the Appendix, the reliability and validity of internet data collection has become increasingly clear in recent years.”
From Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention (2008) · cites this paper as Gosling et al., 2004
In the citing paper
“The validity of internet testing has been well supported in recent reviews (Gosling et al., 2004), with excellent correspondence between results obtained with internet samples are results obtained in the laboratory (e.g., Birnbaum, 1999;Krantz & Dalal, 2000;McGraw, Tew, & Williams, 2000;Reips, 2002).”
From Spacing Effects in Learning: A Temporal Ridgeline of Optimal Retention (2008) · cites this paper as Gosling et al., 2004
In the citing paper
“Furthermore, previous research has suggested populations for online studies and Facebook are quite representative [23], [24].”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [23]
In the citing paper
“Still, the sample sizes and availability of demographic information afforded by social media bring us closer to a more ideal representative sample [99].”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [99]
Citations
Cites
Mail and internet surveys : the tailored design method
Dillman, D. A. (2007) · Cited by 9,587
(1999) · Cited by 7,995
Self-citationInternet paradox: A social technology that reduces social involvement and psychological well-being?
Kraut, R., Patterson, M., Lundmark, V., Kiesler, S., Mukophadhyay, T., & Scherlis, W. (1998) · American Psychologist · doi:10.1037/0003-066x.53.9.1017 · Cited by 4,793
Handbook of Personality : Theory and Research
Pervin, L. A. (1992) · Cited by 4,543
Orne, M. T. (1962) · American Psychologist · doi:10.1037/h0043424 · Cited by 4,041
Mail and Internet Surveys: The Tailored Design Method
Chisnall, P. M. (2007) · Journal of Advertising Research · doi:10.2501/s0021849907070237 · Cited by 3,463
Gender differences in personality traits across cultures: Robust and surprising findings.
Costa, P. T., Terracciano, A., & McCrae, R. R. (2001) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.81.2.322 · Cited by 2,986
Kraut, R. E., Kiesler, S. B., Boneva, B. S., Cummings, J. N., Helgeson, V. S., & Crawford, A. M. (2002) · Journal of Social Issues · doi:10.1111/1540-4560.00248 · Cited by 2,367
Gender differences in personality: A meta-analysis.
Feingold, A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.3.429 · Cited by 2,240
Handbook of Personality Theory and Research.
Schwartz, M. L., Borgatta, E. F., & Lambert, W. W. (1970) · American Sociological Review · doi:10.2307/2093310 · Cited by 2,182
Research works printed but not linked: 7
- Azar, 2000, A web of research: They're fun, they're fast, and they save money, but do Web experiments yield quality results?, Monitor on Psychology, 31, 42
- 10.1037/0022-3514.75.3.729 · DOI · Paper page
- Dillman. 1998 · DOI
- Locke, 1995, Method of psychological assessment, self-disclosure, and experiential differences: A study of computer, questionnaire, and interview assessment formats., Journal of Social Behavior and Personality, 10, 255
- 2000, Digest of education statistics 2000.
- Skitka, (in press). Social psychological research and me Internet: The promise and the perils of a new methodological frontier.
- 2000, Population, 1
Cited by
Sorted by citations
Items 1,001–1,012 of 1,012
Purchase Intentions in a Chatbot Environment: An Examination of the Effects of Customer Experience
Martínez Puertas, S., Illescas Manzano, M. D., Segovia López, C., & Ribeiro-Cardoso, P. (2024) · Oeconomia Copernicana · doi:10.24136/oc.2914 · Cited by 34
Implicit and explicit emotional behavior and mindfulness
Sauer, S., Walach, H., Schmidt, S., Hinterberger, T., Horan, M., & Kohls, N. (2011) · Consciousness and Cognition · doi:10.1016/j.concog.2011.08.002 · Cited by 32
Hui, A., Wong, P. W.-C., & Fu, K.-W. (2015) · JMIR Mental Health · doi:10.2196/mental.3649 · Cited by 33
“Online campaigns, through the use of websites, online forums, and social media in particular, enable reaching diverse populations and providing motivation through reminders and feedback to the participants [14].”
Moritz, S., Stepulovs, O., Schröder, J., Hottenrott, B., Meyer, B., & Hauschildt, M. (2016) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2016.04.001 · Cited by 32
A measure of survey mode differences
Homola, J., Jackson, N., & Gill, J. (2016) · Electoral Studies · doi:10.1016/j.electstud.2016.06.010 · Cited by 21
“Despite some optimistic views (e.g. Gosling et al., 2004), there is strong evidence of measurement error between face-to-face and Internet surveys; it just turns out to be more subtle than expected.”
BackgroundComparing In-Person, Sona, and Mechanical Turk Measurements of Three Prejudice-Relevant Constructs
Gamblin, B. W., Winslow, M. P., Lindsay, B., Newsom, A. W., & Kehn, A. (2017) · Current Psychology · doi:10.1007/s12144-015-9403-1 · Cited by 19
“…et al. 1978; Ekehammar et al. 2004) and there is some evidence for non-equivalency in general (e.g., Chandler et al. 2014; Goodman et al. 2013), the majority of the literature points to equivalency across modality for psychological constructs (e. g., Buhrmester et al. 2011; Gosling et al. 2004).”
BackgroundResultCollins, C. L., Pina, A., Carrillo, A., Ghil, E., Smith-Peirce, R. N., Gomez, M., Okolo, P., Chen, Y., Pahor, A., Jaeggi, S. M., et al. (2022) · Journal of Cognitive Enhancement · doi:10.1007/s41465-022-00240-z · Cited by 15
“It has been found that data collected online are comparable to inlab studies (Brock et al., 2012; Gosling et al., 2004; Weigold et al., 2013, Germine et al., 2012), with some exceptions (Buchanan et al.”
MethodWeijs, C., Coe, J., Desmarais, S., Majowicz, S., & Jones-Bitton, A. (2019) · Journal of Medical Internet Research · doi:10.2196/12024 · Cited by 17
“%) to males in this study was more representative than in traditional survey methods, where women often comprise 71% to 77% of the participants [39], and the average age of participants in this study (40 years) is similar to the average age of a social media user (38 years) [29].”
Gebauer, J. E., Haddock, G., Broemer, P., & von Hecker, U. (2013) · Journal of Personality and Social Psychology · doi:10.1037/a0033482 · Cited by 14
Poulton, A., Chen, L. P. E., Dali, G., Fox, M., & Hester, R. (2022) · Journal of Medical Internet Research · doi:10.2196/32922 · Cited by 11
“Web-based survey–based psychological assessment has been recognized as a cost-effective, efficient, and psychometrically sound means of recruiting large, diverse samples [26,28,44-48].”
Background“It's not just about being judged”: interpersonal distrust uniquely contributes to social anxiety
Fetzner, M. G., Teale Sapach, M. J. N., Mulvogue, M., & Carleton, R. N. (2016) · Journal of Experimental Psychopathology · doi:10.5127/jep.042414 · Cited by 11
Buz, J., Á. Gómez-Martínez, M., Crego, A., Yela, J. R., & Sánchez-Zaballos, E. (2022) · Assessment · doi:10.1177/10731911211018855 · Cited by 3
“In general, findings from online surveys are consistent with those obtained using traditional methods (Gosling et al., 2004).”
Result
Metadata
- Authors
- Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P.
- Affiliations
- The University of Texas at Austin (US)
- Stanford University (US)
- University of California, Berkeley (US)
- Venue
- American Psychologist
- Published
- 2004
- Volume
- 59
- Issue
- 2
- Pages
- 93-104
- ISSN
- 0003-066X (ISSN-L)
- 1935-990X
- OpenAlex
- W1992490637
- Cited by
- 2,932
- References
- 72
- Citations by year
- 2026: 40 · 2025: 64 · 2024: 72 · 2023: 85 · 2022: 123 · 2021: 135 · 2020: 157 · 2019: 163 · 2018: 148 · 2017: 170 · 2016: 181 · 2015: 192 · 2014: 218 · 2013: 184 · 2012: 189
- FWCI
- 261.73
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Psychologist (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Focus Groups and Qualitative Methods
- Keywords
- online data collection, paper-and-pencil questionnaires, sample representativeness, data quality, response format effects, generalizability, self-report questionnaire
- MeSH terms
- Adult, Aged, Attitude, Female, Humans, Male, Middle Aged, Psychology, Surveys and Questionnaires, Evidence-Based Medicine, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
