Convergent and discriminant validation by the multitrait-multimethod matrix.
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
This data set was collected to examine the eHealth Literacy Scale as a means of assessing eHealth literacy among German athletes. In this context, we pursued two objectives with this study: first, to test the factorial structure of the GR-eHEALS and assess its construct validity by examining both convergent and discriminant validity; and second, to examine the associations between eHealth literacy and health-related outcomes (i.e., substance use and injuries). This digital survey was conducted as a cross-sectional study, adhering to the approval guidelines of the Ethics Committee of the Faculty of Medicine at the University of Duisburg-Essen (19-8947-BO). Prior to the survey, each participant provided electronic informed consent. Participation was both anonymous and voluntary, without any form of reimbursement. We utilized the Unipark software (Tivian XI GmbH) that was distributed through social media, sports clubs (involving athletes competing in regional and nationwide tournaments), and sports associations (both regional and nationwide) from December 2021 to December 2022. Methods This study collected sociodemographic data from participants through self-report measures, including information on their sex, age, marital status, education level, occupation, and financial situation. Furthermore, sports-related data (i.e., type of sports and whether they do individual or team sports) was assessed. eHealth literacy of participants was assessed using the GR-eHEALS (1), which is based on the eHEALS by Norman and Skinner (2). The GR-eHEALS consists of eight items that are rated on a 5-point Likert scale (1 = do not agree at all; 5 = fully agree). To test the convergent validity of the GR-eHEALS, established scales measuring confidence in using digital media (3, 4) were administered. Furthermore, the length of daily internet use for personal and professional purposes was evaluated using a single self-developed item rated on a 5-point Likert scale (1 = not at all; 5 = more than 5 hours). In addition, three items each were inquired after internet anxiety and digital overload (3, 5), and they were rated on a 5-point Likert scale (1 = totally disagree; 5 = totally agree). These measures were expected to correlate significantly with the GR-eHEALS, as per Campbell and Fiske (6) guidelines. To evaluate the discriminant validity of the GR-eHEALS, we used the 8-item Impulsive Behavior–8 Scale (7) to measure impulsivity as a personal trait that was expected to be independent of eHealth literacy. These items were also rated on a 5-point Likert scale. Additionally, participants provided medical data through self-report measures. It was assessed how often the following substances were consumed on a 5-point Likert scale (1 = never, 5 = daily): Cannabis, nicotine, sedatives prescribed by physicians (e.g. benzodiazepines), painkillers prescribed by physicians (e.g. tramadol), sedatives not prescribed by physicians / over-the-counter sedatives, painkillers not prescribed by physicians / over-the-counter painkillers (e.g. ibuprofen, diclofenac). Moreover, number and severity of injuries was assessed. For this purpose, athletes indicated on a 5-point Likert scale (1 = not at all, 5 = more than 20 times a year) how often they had suffered minor, moderate and severe injuries within the last year and how often surgery had been necessary. Statistical analyses Statistical analyses were conducted with R version 4.2.2.2 and R Studio 2023.06.1+524. A confirmatory factor analysis (CFA) was performed in order to affirm the factor structure of the GR-eHEALS scale in the present sample. Results were interpreted according to Hu and Bentler (8) assuming the comparative fit index (CFI) and Tucker Lewis index (TLI) of at least 0.95 and root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) of below 0.06 and 0.08, respectively (8). As the GR-eHEALS consists of items on ordinal scale, a robust likelihood estimator (WLSMV; 9) was chosen to avoid biases in the model. Internal consistencies (reliability) of the convergent and discriminant validity scales, the GR-eHEALS and its two subscales were examined. Subsequently, two-tailed Pearson correlations were conducted between the validity scales, the outcome measurements, and sociodemographic variables with the GR-eHEALS. Sex differences on GR-eHEALS were assessed by a two-tailed independent t-Test. Results were considered as significant with p = .05. Incomplete data was deleted list wise.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This tripartite alignment confirms cross-modal consistency between subjective scales (NASA-TLX) and objective physiological metrics (eye-tracking data), satisfying the multitrait-multimethod matrix (MTMM) validation framework proposed by Campbell and Fiske (1959).”
From The impact of usage experience and input modality on trust experience and cognitive load in older adults (2025) · cites this paper as Campbell and Fiske (1959)
In the citing paper
“Such convergent findings at different levels of empirical evidence (experimental, correlational, and neural) is a successful example of triangulation that does not leave much room for further debate (Campbell & Fiske, 1959;Munafò & Smith, 2018).”
From Working memory training in typically developing children: A multilevel meta-analysis (2020) · cites this paper as Campbell & Fiske, 1959
In the citing paper
“Adopting a convergent operations approach (Campbell & Fiske, 1959), we implemented an alternative, more immersive manipulation of nostalgia for generalizability purposes.”
From Nostalgia-Evoked Inspiration: Mediating Mechanisms and Motivational Implications (2015) · cites this paper as Campbell & Fiske, 1959
In the citing paper
“Second, we obtained stronger support for the theoretically grounded sequence of influence than for assorted alternative sequences, casting doubt on the idea that the present findings merely reflect a halo of shared response orientations (i.e., common method variance; Campbell & Fiske, 1959).”
From Nostalgia-Evoked Inspiration: Mediating Mechanisms and Motivational Implications (2015) · cites this paper as Campbell & Fiske, 1959
In the citing paper
“Across the five studies, we used a convergent operations approach (Campbell & Fiske, 1959) to test the regulatory model (Figure 1).”
From The mnemonic mover: Nostalgia regulates avoidance and approach motivation (2014) · cites this paper as Campbell & Fiske, 1959
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hu, L.-T., & Bentler, P. M. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.424 · Cited by 12,298
Construct validity in psychological tests.
Cronbach, L. J., & Meehl, P. E. (1955) · Psychological Bulletin · doi:10.1037/h0040957 · Cited by 11,195
Essentials of Psychological Testing
Rosner, B., & Cronbach, L. J. (1960) · The American Journal of Psychology · doi:10.2307/1419921 · Cited by 4,513
Li, C.-H. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0619-7 · Cited by 3,031
eHEALS: The eHealth Literacy Scale
Norman, C. D., & Skinner, H. A. (2006) · Journal of Medical Internet Research · doi:10.2196/jmir.8.4.e27 · Cited by 2,680
A constant error in psychological ratings.
Thorndike, E. L. (1920) · Journal of Applied Psychology · doi:10.1037/h0071663 · Cited by 2,163
The Social Desirability Variable in Personality Assessment and Research.
Sprott, D. A., & Edwards, A. L. (1959) · Journal of the American Statistical Association · doi:10.2307/2282171 · Cited by 1,491
Bridgman, P. W. (1927) · CERN Document Server (European Organization for Nuclear Research) · Cited by 1,075
Consistency of the factorial structures of personality ratings from different sources.
Fiske, D. W. (1949) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0057198 · Cited by 980
Self-citationResponse Sets and Test Validity
Cronbach, L. J. (1946) · Educational and Psychological Measurement · doi:10.1177/001316444600600405 · Cited by 898
Cited by
Evaluating Structural Equation Models with Unobservable Variables and Measurement Error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.2307/3151312 · Cited by 109,314
Evaluating structural equation models with unobservable variables and measurement error
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800104 · Cited by 109,190
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.879 · Cited by 82,194
Baron, R. M., & Kenny, D. A. (1986) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.51.6.1173 · Cited by 91,532
“Because of the conceptual status of this assessment in the mediator case, one's main concern is the demonstration of construct validity, a situation that ideally requires multiple independent and converging measurements (Campbell & Fiske, 1959).”
BackgroundPerceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Metadata
- Authors
- Campbell, D. T., & Fiske, D. W.
- Venue
- Psychological Bulletin
- Published
- 1959
- Volume
- 56
- Issue
- 2
- Pages
- 81-105
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2024509488
- Cited by
- 17,116
- References
- 35
- Citations by year
- 2026: 481 · 2025: 501 · 2024: 525 · 2023: 548 · 2022: 544 · 2021: 688 · 2020: 734 · 2019: 584 · 2018: 546 · 2017: 540 · 2016: 557 · 2015: 678 · 2014: 626 · 2013: 599 · 2012: 562
- FWCI
- 366.11
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Bulletin (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- digital health literacy, eHealth Literacy Scale, athletes, construct validity, convergent validity, discriminant validity, factor structure, substance use, sports injuries, impulsivity, internet anxiety, digital overload, German athletes, sports clubs, sports organizations, social media survey, multitrait-multimethod matrix
- MeSH terms
- Humans, Psychological Tests
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.