Measurement Equivalence of Paper‐and‐Pencil and Internet Organisational Surveys: A Large Scale Examination in 16 Countries
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
In multinational surveys, mixed‐mode administration modes (e.g. combining Internet and paper‐and‐pencil administration) are increasingly used. To date, no studies have investigated whether measurement equivalence exists between Internet data collection and data collection using the conventional paper‐and‐pencil method in organisational surveys which include a large number of countries. This paper examined the measurement equivalence of a truly global organisational survey across Internet and paper‐and‐pencil survey administrations. Data from an organisational survey in 16 countries ( N = 52,461) across the globe were used to assess the measurement equivalence of an organisational climate measure within each country in which the survey was administered. The empirical data provided strong indications which support the measurement equivalence of the multi‐item survey instrument across Internet and paper‐and‐pencil surveys in virtually all countries in which the survey was conducted. These findings suggest that merging data obtained through Internet and paper‐and‐pencil data administration in a particular country is legitimate as no evidence was found for differential effects across both modes of data collection. Dans les enquêtes internationales, les modalités d’administration mixtes, c’est‐à‐dire combinant les solution Internet et papier‐crayon, sont de plus en plus utilisées. Jusqu’à présent, on ignore si le recueil de données par Internet est méthodologiquement équivalent au recueil traditionnel sur papier dans les enquêtes organisationnelles qui couvrent un grand nombre de pays. Dans cet article, on analyse l’analogie des mesures d’une enquête organisationnelle véritablement universelle utilisant à la fois une administration Internet et papier‐crayon. Des données provenant de seize pays ( N = 52,461) répartis sur l’ensemble de la planète ont été exploitées pour évaluer l’équivalence méthodologique d’une mesure du climat organisationnel dans chacun de ces pays. Les résultats empiriques sont en faveur de l’équivalence méthodologique de l’ensemble des items pour les recueils Internet et papier‐crayon dans pratiquement tous les pays retenus pour l’enquête. Ces résultats montrent qu’il est légitime de traiter ensemble les données obtenues par Internet et par papier‐crayon dans un même pays puisque rien ne nous permet d’affirmer l’existence d’un effet différentiel dû aux deux modes d’administration.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance
Cheung, G. W., & Rensvold, R. B. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0902_5 · Cited by 16,280
Vandenberg, R. J., & Lance, C. E. (2000) · Organizational Research Methods · doi:10.1177/109442810031002 · Cited by 7,997
Structural model evaluation and modification: An interval estimation approach
Steiger, J. H. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2502_4 · Cited by 7,408 · cited 2×
“In particular, the following goodnessof-fit measures were used: (1) the Comparative Fit Index (Bentler, 1990), (2) the Tucker-Lewis Index (TLI), which is also referred to as the Bentler-Bonett Non-Normed Fit Index (NNFI; Bentler & Bonett, 1980), (3) the Root Mean Square Error of Approximation (RMSEA; Steiger, 1990 ),…”
BackgroundMethodMeasurement Invariance, Factor Analysis and Factorial Invariance
Meredith, W. (1993) · Psychometrika · doi:10.1007/bf02294825 · Cited by 4,536 · cited 2×
“The equality of factor loadings and item intercepts (also known as scalar equivalence ) is required whenever (estimated) construct means are to be compared across groups or countries (see Chan, 2000; Little, 1997; Meredith, 1993).”
BackgroundStructural equation modelling in perspective
Kelloway, E. K. (1995) · Journal of Organizational Behavior · doi:10.1002/job.4030160304 · Cited by 495 · cited 1×
“348 DE BEUCKELAER AND LIEVENS sensitive to sample size (Brannick, 1995; Kelloway, 1995), we used the difference in Comparative Fit Index (CFI) between nested equivalence models because a recent simulation study (Cheung & Rensvold, 2002) found that the difference in Comparative Fit Index was a…”
MethodThe effects of climate and personal influences on individual behavior and attitudes in organizations
Ostroff, C. (1993) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1993.1045 · Cited by 306
From paper to pixels: Moving personnel surveys to the Web
THOMPSON, L. F., SURFACE, E. A., MARTIN, D. L., & SANDERS, M. G. (2003) · Personnel Psychology · doi:10.1111/j.1744-6570.2003.tb00149.x · Cited by 193
Cited by
The viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,018
Risk, uncertainty and the theory of planned behavior: A tourism example
Quintal, V. A., Lee, J. A., & Soutar, G. N. (2010) · Tourism Management · doi:10.1016/j.tourman.2009.08.006 · Cited by 855
Chew, E. Y. T., & Jahari, S. A. (2014) · Tourism Management · doi:10.1016/j.tourman.2013.07.008 · Cited by 788
Social desirability is the same in offline, online, and paper surveys: A meta-analysis
Dodou, D., & de Winter, J. C. F. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.04.005 · Cited by 325
“For further discussion on cross-cultural differences in social desirability, see De Beuckelaer and Lievens, (2009), Fang, Wen, and Prybutok (2013), and Furner (2011).”
BackgroundHoerger, M. (2010) · Cyberpsychology, Behavior, and Social Networking · doi:10.1089/cyber.2009.0445 · Cited by 292
Metadata
- Authors
- De Beuckelaer, A., & Lievens, F.
- Venue
- Applied Psychology
- Publisher
- Wiley
- Published
- 2009-03-05
- Volume
- 58
- Issue
- 2
- Pages
- 336-361
- ISSN
- 0269-994X (ISSN-L)
- 1464-0597
- OpenAlex
- W2161962548
- Cited by
- 149
- References
- 8
- FWCI
- 26.72
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Measurement Equivalence of Paper‐and‐Pencil and Internet Organisational Surveys: A Large Scale Examination in 16 Countries. https://app.cochat.ai/papers/10.1111/j.1464-0597.2008.00350.x
- The paper
- De Beuckelaer, A., & Lievens, F. (2009). Measurement Equivalence of Paper‐and‐Pencil and Internet Organisational Surveys: A Large Scale Examination in 16 Countries. Applied Psychology. https://doi.org/10.1111/j.1464-0597.2008.00350.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.