The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Sample size requirements for the design of reliability study: review and new results
Abstract
The reliability of continuous or binary outcome measures is usually assessed by estimation of the intraclass correlation coefficient (ICC). A crucial step for this purpose is the determination of the required sample size. In this review, we discuss the contributions made in this regard and derive the optimal allocation for the number of subjects k and the number of repeated measurements n that minimize the variance of the estimated ICC. Cost constraints are discussed for both normally and non-normally distributed responses, with emphasis on the case of dichotomous assessments. Tables showing optimal choices of k and n are given along with the guidelines for the efficient design of reliability studies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our sample size is adequate for this type of reliability research as demonstrated by similar studies (Shoukri etal., 2004; Truong et al., 2020).”
From Investigating state and trait aspects of resilience using Generalizability theory (2024) · cites this paper as Shoukri et al., 2004
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Statistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
The Advanced Theory of Statistics.
Clarke, G. M., Kendall, M., & Stuart, A. (1978) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344782 · Cited by 6,658
SCHEFFE, H. (1960) · Soil Science · doi:10.1097/00010694-196006000-00016 · Cited by 5,516
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354
(2008) · The Concise Encyclopedia of Statistics · doi:10.1007/978-0-387-32833-1_5 · Cited by 2,670
Optimization—Theory and Applications
Cesari, L. (1983) · Springer eBooks · doi:10.1007/978-1-4613-8165-5 · Cited by 2,559
Statistical Methods for Research Workers.
I., L., & Fisher, R. A. (1926) · Journal of the Royal Statistical Society · doi:10.2307/2341488 · Cited by 2,128
Sample size requirements for estimating intraclass correlations with desired precision
Bonett, D. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1108 · Cited by 987
Research works printed but not linked: 2
- Hemmersley IM., 1949, Metron, 15, 189
- Mak TK., 1988, Applied Statistics, 20, 37
Cited by
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,174
Self-citationGuidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audige, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.016 · Cited by 997
Self-citationMethod agreement analysis: A review of correct methodology
Watson, P. F., & Petrie, A. (2010) · Theriogenology · doi:10.1016/j.theriogenology.2010.01.003 · Cited by 881
Seel, R. T., Sherer, M., Whyte, J., Katz, D. I., Giacino, J. T., Rosenbaum, A. M., Hammond, F. M., Kalmar, K., Pape, T. L.-B., Zafonte, R., et al. (2010) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2010.07.218 · Cited by 625
Magerl, W., Krumova, E. K., Baron, R., Tölle, T., Treede, R.-D., & Maier, C. (2010) · Pain · doi:10.1016/j.pain.2010.07.026 · Cited by 604
Metadata
- Authors
- Shoukri, M. M., Asyali, M. H., & Donner, A.
- Affiliations
- Western University (CA)
- Department of Biostatistics, Epidemiology and Scientific Computing, King Faisal Specialist Hospital and Research Centre, Riyadh, Kingdom of Saudi Arabia
- Corresponding author
- Shoukri, M M
- ORCID
- Shoukri, M M
- Venue
- Statistical Methods in Medical Research
- Publisher
- SAGE Publications
- Published
- 2004-08
- Volume
- 13
- Issue
- 4
- Pages
- 251-271
- ISSN
- 0962-2802 (ISSN-L)
- 1477-0334
- OpenAlex
- W1829168467
- Cited by
- 459
- References
- 38
- Citations by year
- 2026: 17 · 2025: 30 · 2024: 38 · 2023: 36 · 2022: 39 · 2021: 34 · 2020: 41 · 2019: 37 · 2018: 28 · 2017: 34 · 2016: 21 · 2015: 19 · 2014: 14 · 2013: 23 · 2012: 10
- FWCI
- 1.70
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Statistical Methods in Medical Research (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- sample size determination, intraclass correlation coefficient, optimal allocation, repeated measures, cost constraints, continuous outcomes, variance minimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sample size requirements for the design of reliability study: review and new results. https://app.cochat.ai/papers/10.1191/0962280204sm365ra
- The paper
- Shoukri, M. M., Asyali, M. H., & Donner, A. (2004). Sample size requirements for the design of reliability study: review and new results. Statistical Methods in Medical Research. https://doi.org/10.1191/0962280204sm365ra
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.