Clinical Scores in Veterinary Medicine: What Are the Pitfalls of Score Construction, Reliability, and Validation? A General Methodological Approach Applied in Cattle
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Clinical scores are commonly used for cattle. They generally contain a mix of categorical and numerical variables that need to be assessed by scorers, such as farmers, animal caretakers, scientists, and veterinarians. This article examines the key concepts that need to be accounted for when developing the test for optimal outcomes. First, the target condition or construct that the scale is supposed to measure should be defined, and if possible, an adequate proxy used for classification should be determined. Then, items (e.g., clinical signs) of interest that are either caused by the target condition (reflective items) or that caused the target condition (formative items) are listed, and reliable items (inter and intra-rater reliability) are kept for the next step. A model is then developed to determine the relative weight of the items associated with the target condition. A scale is then built after validating the model and determining the optimal threshold in terms of sensitivity (ability to detect the target condition) and specificity (ability to detect the absence of the target condition). Its robustness to various scenarios of the target condition prevalence and the impact of the relative cost of false negatives to false positives can also be assessed to tailor the scale used based on specific application conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Treemap of the relative weights of different clinical signs included in various scoring systems used to detect bovine respiratory disease complex in calves. The size of the square is proportional to the specific clinical sign weight in the Wisconsin calf respiratory score (A) [7], California score (B) [8], Québec modified California score (C) [9], and Québec veal calf respiratory score (D) [10]. The map clearly shows the variable signs included and the relative impact on the total score of the same clinical sign.](https://figures.cochat.ai/paper-figures/95fb038eec00443f81f6cfacf02f6bb0/clinical-scores-in-veterinary-medicine-what-are-the-pitfalls-of-score-figure-1.jpg)


![Figure 4 Determination of the optimal threshold based on a specific score. This figure represents the California respiratory score threshold for detecting ultrasonographic lung consolidation (target condition defined as a maximal consolidation depth ≥ 1 cm) in 608 pre-weaned calves from 39 different dairy herds [9]. The accuracy of the score is presented in the y-axis with the sum of sensitivity (Se) and specificity (Sp) of the score depending on the threshold (≥ threshold to define a positive score, < for a negative score). The dots represent the observed value of the score, and the 95% confidence interval band was obtained after 1000 bootstrap estimates. Even though the threshold of ≥ 4 is numerically associated with the highest accuracy (dashed blue line), there is no evidence that this threshold is better than any threshold between 2 and 7 (dotted red lines) due to the confidence interval width.](https://figures.cochat.ai/paper-figures/b26bd54f2e754e359618c4dedb304706/clinical-scores-in-veterinary-medicine-what-are-the-pitfalls-of-score-figure-4.jpg)

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Current diagnostic practices are largely dependent on observing clinical signs by skilled personnel [20,21].”
From Strategies for Bovine Respiratory Disease (BRD) Diagnosis and Prognosis: A Comprehensive Overview (2024) · cites this paper as [21]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This complicates the establishment of an accurate standard reference diagnostic test that could prevent misclassification of the disease [21].”
From Strategies for Bovine Respiratory Disease (BRD) Diagnosis and Prognosis: A Comprehensive Overview (2024) · cites this paper as [21]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 2×
“Cohen’s kappa [24] is by far the most commonly reported agreement parameter in veterinary medicine and many other medical fields.”
BackgroundMethodA guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709 · cited 3×
“This family of reliability parameters is also called intra-class correlation (ICC) measures [33].”
BackgroundMethodInfluentialBoateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018) · Frontiers in Public Health · doi:10.3389/fpubh.2018.00149 · Cited by 5,925
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Measurement in Medicine: A Practical Guide
de Vet, H. C. W., Terwee, C. B., Mokkink, L. B., & Knol, D. L. (2011) · doi:10.1017/cbo9780511996214 · Cited by 2,321
The Statistical Evaluation of Medical Tests for Classification and Prediction
Pepe, M. S. (2003) · doi:10.1093/oso/9780198509844.001.0001 · Cited by 2,198
Roche, J. R., Friggens, N. C., Kay, J. K., Fisher, M. W., Stafford, K. J., & Berry, D. P. (2009) · Journal of Dairy Science · doi:10.3168/jds.2009-2431 · Cited by 1,254 · cited 1×
“Other scoring systems exist, for example, to determine calf diarrhea [7], lameness [11], body condition [12], and pain [13].”
BackgroundGuidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audige, L., Brorson, S., Donner, A., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. L. (2011) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2011.01.016 · Cited by 997
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Surgery · doi:10.1002/bjs.9736 · Cited by 889
Disease Management of Dairy Calves and Heifers
McGuirk, S. M. (2008) · Veterinary Clinics of North America: Food Animal Practice · doi:10.1016/j.cvfa.2007.10.003 · Cited by 629 · cited 2×
“The size of the square is proportional to the specific clinical sign weight in the Wisconsin calf respiratory score (A) [7], California score (B) [8], Québec modified California score (C) [9], and Québec veal calf respiratory score (D) [10].”
BackgroundMethodEstimation of diagnostic-test sensitivity and specificity through Bayesian modeling
Branscum, A. J., Gardner, I. A., & Johnson, W. O. (2005) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2004.12.005 · Cited by 533 · cited 1×
“, for BRD, using an association between thoracic ultrasound, serum haptoglobin, and bronchoalveolar lavage) although this has its own risk of bias [36]; and finally (iii) accounting for the imperfect accuracy of a test that can be done in a practical setting to define the latent status of the animal [37].”
MethodZapf, A., Castell, S., Morawietz, L., & Karch, A. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0200-9 · Cited by 377 · cited 2×
“See [23,32] for information on how to calculate this reliability index and associated confidence intervals.”
BackgroundClinicians are right not to like Cohen's
de Vet, H. C. W., Mokkink, L. B., Terwee, C. B., Hoekstra, O. S., & Knol, D. L. (2013) · BMJ · doi:10.1136/bmj.f2125 · Cited by 250
Should scoring rules be based on odds ratios or regression coefficients?
Moons, K. G. M., Harrell, F. E., & Steyerberg, E. W. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00453-5 · Cited by 241 · cited 1×
“) link and not with the odds ratio derived from the coefficients in a logistic regression approach [41].”
MethodLove, W. J., Lehenbauer, T. W., Kass, P. H., Van Eenennaam, A. L., & Aly, S. S. (2014) · PeerJ · doi:10.7717/peerj.238 · Cited by 228 · cited 3×
“Therefore, the imperfect accuracy of the test means that unbiased model characteristics can be determined when the outcome is measured with error [8,38].”
BackgroundMethodNuffel, A. V., Zwertvaegher, I., Pluym, L. M., Weyenberg, S. V., Thorup, V. M., Pastell, M., Sonck, B., & Saeys, W. (2015) · Animals · doi:10.3390/ani5030387 · Cited by 154 · cited 1×
“For example, the lameness score based on visual gait analysis could be initially compared with the weight repartition forces on the ground and specific gait analysis measurement [14].”
MethodPerryman, L. E., Kapil, S. J., Jones, M. L., & Hunt, E. L. (1999) · Vaccine · doi:10.1016/s0264-410x(98)00477-0 · Cited by 153 · cited 1×
“Again, the neonatal calf diarrhea scoring scale can be objectively determined in calves using metabolic cages that assess fecal output and dry fecal matter [15].”
BackgroundGreiner, M. (1996) · Journal of Immunological Methods · doi:10.1016/0022-1759(96)00013-0 · Cited by 135 · cited 1×
“Various test applications in terms of the possible prevalence of the target condition (p) and the relative cost of false-negative versus false-positive (FN:FP) cases (r) can be determined by calculating the misclassification cost term (MCT), which should be minimized [43].”
BackgroundModelling risk when binary outcomes are subject to error
McInturff, P., Johnson, W. O., Cowling, D., & Gardner, I. A. (2004) · Statistics in Medicine · doi:10.1002/sim.1656 · Cited by 107 · cited 1×
“Therefore, the imperfect accuracy of the test means that unbiased model characteristics can be determined when the outcome is measured with error [8,38].”
BackgroundBovine Respiratory Disease Diagnosis
Buczinski, S., & Pardon, B. (2020) · Veterinary Clinics of North America: Food Animal Practice · doi:10.1016/j.cvfa.2020.03.004 · Cited by 89 · cited 1×
“The bovine respiratory disease complex (BRD), which is a complex mixture of infectious agents affecting the upper and lower respiratory tract, is a common example in cattle medicine [17].”
BackgroundSelf-citationCarslake, C., Vázquez-Diosdado, J. A., & Kaler, J. (2020) · Sensors · doi:10.3390/s21010088 · Cited by 81
Knauer, W. A., Godden, S. M., Dietrich, A., & James, R. E. (2017) · Journal of Dairy Science · doi:10.3168/jds.2016-12372 · Cited by 71
Bias due to composite reference standards in diagnostic accuracy studies
Schiller, I., van Smeden, M., Hadgu, A., Libman, M., Reitsma, J. B., & Dendukuri, N. (2016) · Statistics in Medicine · doi:10.1002/sim.6803 · Cited by 68 · cited 1×
“, for BRD, using an association between thoracic ultrasound, serum haptoglobin, and bronchoalveolar lavage) although this has its own risk of bias [36]; and finally (iii) accounting for the imperfect accuracy of a test that can be done in a practical setting to define the latent status of the animal [37].”
MethodFeeding behaviour and activity as early indicators of disease in pre-weaned dairy calves
Duthie, C.-A., Bowen, J. M., Bell, D. J., Miller, G. A., Mason, C., & Haskell, M. J. (2021) · Animal · doi:10.1016/j.animal.2020.100150 · Cited by 63
Schlageter-Tello, A., Bokkers, E. A. M., Groot Koerkamp, P. W. G., Van Hertem, T., Viazzi, S., Romanini, C. E. B., Halachmi, I., Bahr, C., Berckmans, D., & Lokhorst, K. (2014) · Journal of Dairy Science · doi:10.3168/jds.2014-8129 · Cited by 55 · cited 2×
“The natural drawback of using multiple categories is that it is associated with an inflated risk of altered reliability due to more choices in the item scale [11].”
BackgroundDevelopment of a Clinical Scoring System for Bovine Respiratory Disease in Weaned Dairy Calves
Maier, G. U., Rowe, J. D., Lehenbauer, T. W., Karle, B. M., Williams, D. R., Champagne, J. D., & Aly, S. S. (2019) · Journal of Dairy Science · doi:10.3168/jds.2018-15474 · Cited by 51 · cited 1×
“[21] included reflective items (sunken eyes, cough, breathing pattern, and rectal temperature) and formative items (e.”
BackgroundKnauer, W. A., Godden, S. M., Dietrich, A., Hawkins, D. M., & James, R. E. (2018) · Journal of Dairy Science · doi:10.3168/jds.2017-13947 · Cited by 48
Buczinski, S., Buathier, C., Bélanger, A. M., Michaux, H., Tison, N., & Timsit, E. (2018) · Journal of Veterinary Internal Medicine · doi:10.1111/jvim.15257 · Cited by 45 · cited 1×
“Imaging methods are also affected by operator technique and image interpretation, such as quantifying lung lesions using thoracic ultrasonography in cattle [20].”
BackgroundSelf-citationFactors affecting rectal temperature measurement using commonly available digital thermometers
Naylor, J. M., Streeter, R. M., & Torgerson, P. (2012) · Research in Veterinary Science · doi:10.1016/j.rvsc.2010.10.027 · Cited by 44 · cited 1×
“For example, the rectal temperature measurement may not only be based on the type of thermometer used, but also on how the probe is inserted [19], which raises the same type of concern as tape application for heart girth measurement.”
BackgroundBuczinski, S., Fecteau, G., Dubuc, J., & Francoz, D. (2018) · Preventive Veterinary Medicine · doi:10.1016/j.prevetmed.2018.05.004 · Cited by 43 · cited 2×
“This figure represents the California respiratory score threshold for detecting ultrasonographic lung consolidation (target condition defined as a maximal consolidation depth ≥ 1 cm) in 608 pre-weaned calves from 39 different dairy herds [9].”
BackgroundMethodSelf-citationWalsh, P. F., Thornton, J. M., Walker, N., McCoy, J. G., Baal, J., Baal, J., Mendoza, N., Banimahd, F., & Asato, J. (2014) · PeerJ · doi:10.7717/peerj.651 · Cited by 37
Short communication: Verifying Holstein heifer heart girth to body weight prediction equations
Heinrichs, A. J., Heinrichs, B. S., Jones, C., Erickson, P. S., Kalscheur, K. F., Nennich, T. D., Heins, B., & Cardoso, F. C. (2017) · Journal of Dairy Science · doi:10.3168/jds.2016-12496 · Cited by 35 · cited 1×
“Nevertheless, the same variable (weight) can be associated with other sources of measurement error if, for example, weight is estimated using heart girth measurement [18].”
BackgroundPardon, B., Buczinski, S., & Deprez, P. R. (2019) · Veterinary Record · doi:10.1136/vr.105238 · Cited by 35 · cited 1×
“, the different scoring sessions where different raters assess different animals [31].”
BackgroundSelf-citationCommentary: On NRI, IDI, and “Good-Looking” Statistics with Nothing Underneath
Hilden, J. (2014) · Epidemiology · doi:10.1097/ede.0000000000000063 · Cited by 33 · cited 1×
“Determining K confidence intervals has been criticized from a frequentist perspective [25].”
BackgroundDo we automatically detect health- or general welfare-related issues? A framework
Stachowicz, J., & Umstätter, C. (2021) · Proceedings of the Royal Society B Biological Sciences · doi:10.1098/rspb.2021.0190 · Cited by 30 · cited 1×
“Stress conditions and the welfare status of animals are also construct categories that are complex to determine and define precisely [3].”
BackgroundBerman, J., Francoz, D., Abdallah, A., Dufour, S., & Buczinski, S. (2021) · Journal of Dairy Science · doi:10.3168/jds.2021-20503 · Cited by 30 · cited 2×
“The AC parameters complement traditional K statistics [26,30].”
BackgroundSelf-citationTomacheuski, R. M., Monteiro, B. P., Evangelista, M. C., Luna, S. P. L., & Steagall, P. V. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0251435 · Cited by 11 · cited 1×
“Other scoring systems exist, for example, to determine calf diarrhea [7], lameness [11], body condition [12], and pain [13].”
Backgroundvan Oest, R., & Girard, J. M. (2021) · Psychological Methods · doi:10.1037/met0000412 · Cited by 4
van Oest, R. D., & Girard, J. M. (2021) · doi:10.31234/osf.io/s5n4e · Cited by 1 · cited 1×
“The reliability measures generally rely on more extensive coding ability and, although promising, are only used by few researchers [27,28].”
Background
Research works printed but not linked: 1
- Harrell, F.E. (2019). Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis, Springer. [2nd ed.].
Cited by
Strategies for Bovine Respiratory Disease (BRD) Diagnosis and Prognosis: A Comprehensive Overview
Kamel, M. S., Davidson, J. L., & Verma, M. S. (2024) · Animals · doi:10.3390/ani14040627 · Cited by 76
“This complicates the establishment of an accurate standard reference diagnostic test that could prevent misclassification of the disease [21].”
BackgroundO’Donoghue, S., Waters, S. M., Morris, D. W., & Earley, B. (2025) · Veterinary Sciences · doi:10.3390/vetsci12080778 · Cited by 58
“Clinical scoring is used in the monitoring and diagnostic of a range of animal health applications across a range of different cohorts [126].”
Clinical Scoring Systems in the Newborn Calf: An Overview
Probo, M., & Veronesi, M. C. (2022) · Animals · doi:10.3390/ani12213013 · Cited by 42
“A valuable tool in this purpose is represented by scoring systems, which enable the scorers (farmers, technicians, and veterinarians) to adopt a systematic approach in this regard [9].”
BackgroundA Novel Scoring System for Humane Endpoints in Mice with Cecal Ligation and Puncture-Induced Sepsis
Ferguson, L. T., RASHIED, A. A., Liang, Z., Yumoto, T., Anyalebechi, J. C., Swift, D. A., Hernandes, M. S., Krafty, R. T., Coopersmith, C. M., & Lee, V. K. (2023) · Comparative Medicine · doi:10.30802/aalas-cm-22-000124 · Cited by 14
Buczinski, S., Ollivett, T. L., & Pardon, B. (2025) · Journal of Dairy Science · doi:10.3168/jds.2025-26764 · Cited by 9
Self-citation
Metadata
- Authors
- Buczinski, S., Boccardo, A., & Pravettoni, D.
- Affiliations
- Université de Montréal (CA)
- University of Milan (IT)
- Corresponding author
- Buczinski, Sébastien
- Venue
- Animals
- Publisher
- MDPI AG
- Published
- 2021-11-13
- Volume
- 11
- Issue
- 11
- Pages
- 3244
- ISSN
- 2076-2615 (ISSN-L)
- PMC
- PMC8614512
- OpenAlex
- W3212240198
- Cited by
- 15
- References
- 38
- Citations by year
- 2026: 3 · 2025: 5 · 2024: 2 · 2023: 4 · 2022: 1
- FWCI
- 0.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Animals (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Archivio Istituzionale della Ricerca (Universita Degli Studi Di Milano) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Life Sciences › Agricultural and Biological Sciences › Animal Science and Zoology › Animal health and immunology
- Life Sciences › Agricultural and Biological Sciences › Animal Science and Zoology › Animal Behavior and Welfare Studies
- Keywords
- clinical scoring systems, cattle, scale development, reliability statistics, validation, inter-rater reliability, intra-rater reliability, item weighting, sensitivity and specificity, threshold optimization, prevalence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.