Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic
Open accessAbstract
Objective: To compare different statistical methods for assessing the relative validity of a self-administered, 150-item, semi-quantitative food-frequency questionnaire (FFQ) with 4-day weighed diet records (WR). Design: Subjects completed the Scottish Collaborative Group FFQ and carried out a 4-day WR. Relative agreement between the FFQ and WR for energy-adjusted nutrient intakes was assessed by Pearson and Spearman rank correlation coefficients, the percentages of subjects classified into the same and opposite thirds of intake, and Cohen's weighted kappa. Subjects: Forty-one men, mean age 36 (range 21-56) years, and 40 women, mean age 33 (range 19-58) years, recruited from different locations in Aberdeen, Scotland. Results: Spearman correlation coefficients tended to be lower than Pearson correlation coefficients, and were above 0.5 for 10 of the 27 nutrients in men and 17 of the 27 nutrients in women. For nutrients with Spearman correlation coefficients above 0.5, the percentage of subjects correctly classified into thirds ranged from 39 to 78%, and weighted kappa values ranged from 0.23 to 0.66. Conclusions: Both Spearman correlation coefficients and weighted kappa values are useful in assessing the relative validity of estimates of nutrient intake by FFQs. Spearman correlation coefficients above 0.5, more than 50% of subjects correctly classified and less than 10% of subjects grossly misclassified into thirds, and weighted kappa values above 0.4 are recommended for nutrients of interest in epidemiological studies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the present study, 22 of 30 nutrients assessed had a correlation coefficient greater than the .4 threshold proposed by Cade et al., and 13 of 30 achieved a correlation greater than or equal to the .5 proposed by Masson et al.
“In the present study, 22 of 30 nutrients assessed had a correlation coefficient greater than the .4 threshold that was proposed by Cade at al. [23], and 13 of 30 achieved a correlation greater than or equal to the “desirable” .5 proposed by Masson et al [42].”
From Online dietary intake estimation: reproducibility and validity of the Food4Me Food frequency questionnaire against a 4-day weighed food record (2014) · cites this paper as [42]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
Beckett, A. H., Rowland, M., & TURNER, P. (1965) · The Lancet · doi:10.1016/s0140-6736(65)91037-8 · Cited by 16,917 · see citing works
Practical Statistics for Medical Research.
Walter, S., & Altman, D. G. (1992) · Biometrics · doi:10.2307/2532320 · Cited by 11,665 · see citing works
Practical Statistics for Medical Research
Altman, D. G. (1990) · doi:10.1201/9780429258589 · Cited by 10,668 · see citing works
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · see citing works
Willett, W. C. (1998) · doi:10.1093/acprof:oso/9780195122978.001.0001 · Cited by 4,541 · see citing works
Total energy intake: implications for epidemiologic analyses.
WILLETT, W., & STAMPFER, M. J. (1986) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a114366 · Cited by 2,751 · see citing works
RE: “TOTAL ENERGY INTAKE: IMPLICATIONS FOR EPIDEMIOLOGIC ANALYSES”
Palmgren, J., & Kushi, L. H. (1991) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a115841 · Cited by 2,712 · see citing works
Goldberg, G. R., Black, A., Jebb, S. A., Cole, T. J., Pr, M., Coward, W. A., & Prentice, A. M. (1991) · PubMed · Cited by 1,927 · see citing works
McCance and widdowson's the composition of foods
Pennington, J. A. T. (1992) · Journal of Food Composition and Analysis · doi:10.1016/0889-1575(92)90046-m · Cited by 765 · see citing works
Bingham, S. (1997) · International Journal of Epidemiology · doi:10.1093/ije/26.suppl_1.s137 · Cited by 634 · see citing works
Nelson, M., Black, A. E., Morris, J., & Cole, T. J. (1989) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/50.1.155 · Cited by 530 · see citing works
Brunner, E. J., Stallone, D. D., Juneja, M., Bingham, S. A., & Marmot, M. G. (2001) · British Journal Of Nutrition · doi:10.1079/bjn2001414 · Cited by 312 · see citing works
Future directions in the development of food-frequency questionnaires
Willett, W. C. (1994) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/59.1.171s · Cited by 194 · see citing works
Yarnell, J. W. G., Am, F., Je, M., Pm, S., & Walker, C. L. (1983) · PubMed · Cited by 193 · see citing works
(1991) · Royal Society of Chemistry eBooks · doi:10.1039/9781849732499 · Cited by 166 · see citing works
Milk Products and Eggs: Fourth Supplement to McCance and Widdowson's the Composition of Foods
Holland, B. R., Unwin, I. D., Buss, D. H., McCance, R. A., & Widdowson, E. M. (1989) · Medical Entomology and Zoology · Cited by 154 · see citing works
Vegetables, herbs and spices: fifth supplement to McCance and Widdowson's The Composition of Foods.
Holland, B. R., Unwin, I. D., & Buss, D. H. (1991) · Medical Entomology and Zoology · Cited by 149 · see citing works
(1993) · Royal Society of Chemistry eBooks · doi:10.1039/9781849732611 · Cited by 143 · see citing works
Holland, B. R., BROWN, J. H., & Buss, D. H. (1993) · Cited by 140 · see citing works
(1992) · Royal Society of Chemistry eBooks · doi:10.1039/9781849732482 · Cited by 135 · see citing works
Invited Commentary: Some Limitations of Semiquantitative Food Frequency Questionnaires
Sempos, C. T. (1992) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a116212 · Cited by 134 · see citing works
(1995) · Royal Society of Chemistry eBooks · doi:10.1039/9781849732475 · Cited by 118 · see citing works
Cereals and cereal products: third supplement to McCance and Widdowsons the Composition of foods
Holland, B. R., McCance, R. A., Widdowson, E. M., Unwin, I. D., & Buss, D. H. (1988) · Royal Society of Chemistry eBooks · Cited by 108 · see citing works
(1994) · Royal Society of Chemistry eBooks · doi:10.1039/9781849732604 · Cited by 106 · see citing works
The Scottish Heart Health Study: objectives and development of methods.
Smith, W. C., Crombie, I. K., Tavendale, R., et al. (1987) · PubMed · Cited by 66 · see citing works
Implications of Measurement Error in Exposure for the Sample Sizes of Case-Control Studies
McKeown‐Eyssen, G., & Tibshirani, R. J. (1994) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a117014 · Cited by 41 · see citing works
Hébert, J. R., & Miller, D. R. (1991) · European Journal of Epidemiology · doi:10.1007/bf00144997 · Cited by 39 · see citing works
Validation of methods for estimating habitual diet: proposed guidelines.
Js, G. (1995) · PubMed · Cited by 38 · see citing works
Guidelines for reports on validation studies.
Burema, J., van Staveren, W. A., & Feunekes, G. I. J. (1995) · PubMed · Cited by 14 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Holland B, Welch AA, Unwin ID, Buss DH, Paul AA, Southgate DAT. McCance & Widdowson's The Composition of Foods, 5th ed. Cambridge: Royal Society of Chemistry, 1991.
- Holland B, Welch AA, Buss DH. Vegetable Dishes. Second Supplement to McCance & Widdowson's The Composition of Foods, 5th ed. Cambridge: Royal Society of Chemistry, 1992.
- Chan W, Brown J, Church SM, Buss DH. Meat Products and Dishes. Sixth Supplement to McCance & Widdowson's The Composition of Foods, 5th ed. Cambridge: Royal Society of Chemistry, 1996.
Cited by
Sorted by citations
Susceptibility of Human Metabolic Phenotypes to Dietary Modulation
Stella, C., Beckwith-Hall, B., Cloarec, O., Holmes, E., Lindon, J. C., Powell, J., van der Ouderaa, F., Bingham, S., Cross, A. J., & Nicholson, J. K. (2006) · Journal of Proteome Research · doi:10.1021/pr060265y · Cited by 376 · see citing works
Fallaize, R., Forster, H., Macready, A. L., Walsh, M. C., Mathers, J. C., Brennan, L., Gibney, E. R., Gibney, M. J., & Lovegrove, J. A. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3355 · Cited by 210 · see citing works
Validity and reproducibility of a web-based, self-administered food frequency questionnaire
Labonté, M.-È., Cyr, A., Baril-Gravel, L., Royer, M.-M., & Lamarche, B. (2012) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2011.163 · Cited by 189 · see citing works
Online dietary intake estimation: the Food4Me food frequency questionnaire
Forster, H., Fallaize, R., Gallagher, C., O’Donovan, C. B., Woolhead, C., Walsh, M. C., Macready, A. L., Lovegrove, J. A., Mathers, J. C., Gibney, M. J., et al. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.3105 · Cited by 151 · see citing works
The associations between personality, diet and body mass index in older people.
Mõttus, R., McNeill, G., Jia, X., Craig, L. C. A., Starr, J. M., & Deary, I. J. (2013) · Health Psychology · doi:10.1037/a0025537 · Cited by 171 · see citing works
Self-citationMediterranean-Type diet and brain structural change from 73 to 76 years in a Scottish cohort
Luciano, M., Corley, J., Cox, S. R., Valdés Hernández, M. C., Craig, L. C. A., Dickie, D. A., Karama, S., McNeill, G. M., Bastin, M. E., Wardlaw, J. M., et al. (2017) · Neurology · doi:10.1212/wnl.0000000000003559 · Cited by 140 · see citing works
Caffeine Consumption and Cognitive Function at Age 70: The Lothian Birth Cohort 1936 Study
Corley, J., Jia, X., Kyle, J. A. M., Gow, A. J., Brett, C. E., Starr, J. M., McNeill, G., & Deary, I. J. (2010) · Psychosomatic Medicine · doi:10.1097/psy.0b013e3181c92a9c · Cited by 71 · see citing works
Self-citationTurunen, A. W., Männistö, S., Kiviranta, H., Marniemi, J., Jula, A., Tiittanen, P., Suominen-Taipale, L., Vartiainen, T., & Verkasalo, P. K. (2010) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2009.147 · Cited by 43 · see citing works
- Authors
- Masson, L., MCNeill, G., Tomany, J., Simpson, J., Peace, H., Wei, L., Grubb, D., & Bolton-Smith, C.
- Affiliations
- University of Aberdeen (GB)
- MRC Elsie Widdowson Laboratory (GB)
- Corresponding author
- MCNeill, G
- ORCID
- Venue
- Public Health Nutrition
- Publisher
- Cambridge University Press (CUP)
- Published
- 2003-06
- Volume
- 6
- Issue
- 3
- Pages
- 313-321
- ISSN
- 1368-9800 (ISSN-L)
- 1475-2727
- OpenAlex
- W2001170097
- Cited by
- 423
- References
- 30
- Citations by year
- 2026: 15 · 2025: 22 · 2024: 16 · 2023: 21 · 2022: 22 · 2021: 19 · 2020: 24 · 2019: 19 · 2018: 26 · 2017: 19 · 2016: 17 · 2015: 18 · 2014: 29 · 2013: 20 · 2012: 23
- FWCI
- 10.50
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Public Health Nutrition (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Nutritional Studies and Diet
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Keywords
- food frequency questionnaire, questionnaire validity, nutrient intake assessment, Spearman correlation, Pearson correlation, weighted kappa, dietary assessment agreement
- MeSH terms
- Adult, Diet Surveys, Feeding Behavior, Female, Humans, Male, Middle Aged, Surveys and Questionnaires, Scotland, Sex Factors, Reproducibility of Results, Nutrition Assessment, Diet Records, Statistics, Nonparametric
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
