Combined result and associated uncertainty from interlaboratory evaluations based on the ISO Guide
Open accessAbstract
We address the problem of determining the combined result and its associated uncertainty in the measurement of a common measurand by a group of competent laboratories. Most data analyses of interlaboratory evaluations are based on the assumption that the expected values of the individual laboratory results are all equal to the value of the common measurand. This means that the laboratory results are subject to random effects only with respect to the value of the measurand. This assumption is frequently unjustified. We use the more realistic assumption that the laboratory results are subject to both random and systematic effects with respect to the value of the measurand. In this case, the value of the measurand may fall anywhere within the range of results. Therefore, a combined result and its associated standard uncertainty that place a non-negligible fraction of the results outside the 2-standard-uncertainty interval are unsatisfactory representations of the value of the common measurand provided by the set of laboratory results. The more realistic assumption requires us to deal with the uncertainty arising from possible systematic effects in the laboratory results. Following the approach of the ISO Guide to deal with systematic effects, we propose a three-step method to determine a combined result and its associated standard uncertainty such that the 2-standard-uncertainty interval would include a sufficiently large fraction of the results. When the interlaboratory evaluation is an International Committee for Weights and Measures (CIPM) key comparison, we suggest that the combined result and its associated standard uncertainty determined by the three-step method be identified with the key comparison reference value and its associated standard uncertainty. These quantities can then be used to specify the degree of equivalence of the individual laboratory results. We illustrate the three-step method by applying it to the results of an international comparison of cryogenic radiometers recently organized by the Consultative Committee for Photometry and Radiometry (CCPR).
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
With the lock scheme to the transitions from the mF=±9/2 states, online information on the magnetic field strength at the position of the atoms is obtained provided that the lattice vector shift is sufficiently small.
“With the lock scheme to the transitions from the m F = ± 9 / 2 states, we obtain online information on the magnetic field strength at the position of the atoms provided that the lattice vector shift [48] is sufficiently small.”
From Long-distance chronometric leveling with a portable optical clock (2024) · cites this paper as [48]
The citation is attached to the condition about a sufficiently small lattice vector shift. Our copy’s section 5 discusses comparisons of cryogenic radiometers, not lattice vector shifts or atomic transitions. Searches for those terms and related optical-lattice terminology found no passage addressing the cited condition. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Linear Statistical Inference and its Applications
Rao, C. R. (1973) · Wiley series in probability and statistics · doi:10.1002/9780470316436 · Cited by 10,474 · see citing works
Continuous Univariate Distributions.
Omori, Y., Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995) · Journal of the American Statistical Association · doi:10.2307/2291547 · Cited by 9,085 · see citing works
Guidelines for evaluating and expressing the uncertainty of NIST measurement results
Taylor, B. N., & Kuyatt, C. E. (2018) · doi:10.6028/nist.tn.1297-1993 · Cited by 2,582 · see citing works
IV.—On Least Squares and Linear Combination of Observations
Aitken, A. C. (1936) · Proceedings of the Royal Society of Edinburgh · doi:10.1017/s0370164600014346 · Cited by 950 · see citing works
Introduction to Mathematical Statistics.
Hochberg, M., & Hoel, P. G. (1974) · American Mathematical Monthly · doi:10.2307/2319241 · Cited by 766 · see citing works
Taylor, B. N., Parker, W. H., & Langenberg, D. N. (1969) · Reviews of Modern Physics · doi:10.1103/revmodphys.41.375 · Cited by 611 · see citing works
Consensus Values and Weighting Factors
Paule, R. C., & Mandel, J. (1982) · Journal of Research of the National Bureau of Standards · doi:10.6028/jres.087.022 · Cited by 580 · see citing works
The Calculation of Errors by the Method of Least Squares
Birge, R. T. (1932) · Physical Review · doi:10.1103/physrev.40.207 · Cited by 525 · see citing works
Problems Arising in the Analysis of a Series of Similar Experiments
Cochran, W. G. (1937) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.2307/2984123 · Cited by 444 · see citing works
Introduction to Mathematical Statistics (4th ed.).
Reinsel, G. C., Hogg, R. V., & Craig, A. T. (1980) · Journal of the American Statistical Association · doi:10.2307/2287488 · Cited by 337 · see citing works
CODATA Recommended Values of the Fundamental Physical Constants: 1998
Mohr, P. J., & Taylor, B. N. (1999) · Journal of Physical and Chemical Reference Data · doi:10.1063/1.556049 · Cited by 312 · see citing works
An approach to combining results from multiple methods motivated by the ISO GUM
Levenson, M. S., Banks, D. L., Eberhardt, K. R., Gill, L. M., Guthrie, W. F., Liu, H. K., Vangel, M. G., Yen, J. H., & Zhang, N. F. (2000) · Journal of Research of the National Institute of Standards and Technology · doi:10.6028/jres.105.047 · Cited by 135 · see citing works
Estimation of a Common Mean and Weighted Means Statistics
Rukhin, A. L., & Vangel, M. G. (1998) · Journal of the American Statistical Association · doi:10.1080/01621459.1998.10474111 · Cited by 115 · see citing works
Robust estimation and identifying outliers
Rousseeuw, P. J. (1990) · Cited by 58 · see citing works
ILAC mutual recognition arrangement
Squirrell, A. (2001) · Accreditation and Quality Assurance · doi:10.1007/s007690000258 · Cited by 23 · see citing works
Comparison of infrared reflectance spectra of fluoride perovskites
Hofmeister, A. M., & Billups, K. (1991) · Spectrochimica Acta Part A Molecular Spectroscopy · doi:10.1016/0584-8539(91)80256-i · Cited by 11 · see citing works
Flagged first, then by citations
Research works printed but not linked: 10
- Birge R. T., Phys. Rev., 1932, 40, 207-227.
- Levenson M. S., Banks D. L., Eberhardt K. R., Gill L. M., Guthrie W. F., Liu H. K., Vangel M. G., Yen J. H., Zhang N. F., J. Res. Natl. Inst. Stand. Technol., 2000, 105, 571-579.
- Paule R. C., Mandel J., J. Res. Natl. Bur. Stand., 1982, 87, 377-385.
- Rukhin A. L., Vangel M. G., J. Am. Stat. Assoc., 1998, 93, 303-308.
- Taylor B. N., Parker W. H., Langenberg D. N., Rev. Mod. Phys., 1969, 41, 375-496 (Section III.A.1.).
- Mohr P. J., Taylor B. N., J. Phys. Chem. Ref. Data, 1999, 28, 1713-1852, Section 4.1 and Appendix E.
- Box G. E. P., Tiao G. C., Bayesian Inference in Statistical Analysis, Theorem 2.4.1, Reading, Mass., Addison-Wesley, 1973.
- Evans M., Hastings N., Peacock B., Statistical Distribu- tions, 3rd ed., New York, John Wiley & Sons, 2000.
- Aitken A. C., Proc. R. Soc. Edinburgh, A, 1935, 55, 42-48.
- Pearson E. S., Hartley H. O., Biometrika Tables for Statistician, 3rd ed., Table 31a, Cambridge, Cambridge University Press, 1966.
Cited by
Sorted by citations
Traceable size determination of nanoparticles, a comparison among European metrology institutes
Meli, F., Klein, T., Buhr, E., Frase, C. G., Gleber, G., Krumrey, M. K., Duta, A., Duta, S., Korpelainen, V., Bellotti, R., et al. (2012) · Measurement Science and Technology · doi:10.1088/0957-0233/23/12/125005 · Cited by 112 · see citing works
Long-term comparison of caesium fountain primary frequency standards
Parker, T. E. (2009) · Metrologia · doi:10.1088/0026-1394/47/1/001 · Cited by 77 · see citing works
Determination of a reference value and its uncertainty through a power-moderated mean
Pommé, S. G., & Keightley, J. D. (2015) · Metrologia · doi:10.1088/0026-1394/52/3/s200 · Cited by 76 · see citing works
Trapezoidal and triangular distributions for Type B evaluation of standard uncertainty
Kacker, R. N., & Lawrence, J. F. (2007) · Metrologia · doi:10.1088/0026-1394/44/2/003 · Cited by 50 · see citing works
Self-citationState-of-the-art mass determination of28Si spheres for the Avogadro project
Picard, A., Barat, P., Borys, M., Firlus, M., & Mizushima, S. (2011) · Metrologia · doi:10.1088/0026-1394/48/2/s14 · Cited by 46 · see citing works
Statistical analysis of CIPM key comparisons based on the ISOGuide
Kacker, R. N., Datla, R. U., & Parr, A. C. (2004) · Metrologia · doi:10.1088/0026-1394/41/4/017 · Cited by 43 · see citing works
Self-citationGómez-Ros, J. M., Pröhl, G., Ulanovsky, A. V., & Lis, M. (2008) · Journal of Environmental Radioactivity · doi:10.1016/j.jenvrad.2008.01.005 · Cited by 36 · see citing works
Mana, G., Massa, E., & Predescu, M. M. (2012) · Metrologia · doi:10.1088/0026-1394/49/4/492 · Cited by 35 · see citing works
Anderson, P. M., Guo, H., & Sunderland, P. B. (2017) · Journal of Aerosol Science · doi:10.1016/j.jaerosci.2017.10.002 · Cited by 34 · see citing works
Long-distance chronometric leveling with a portable optical clock
Grotti, J., Nosske, I., Koller, S. B., Herbers, S., Denker, H., Timmen, L., Vishnyakova, G., Grosche, G., Waterholter, T., Kuhl, A., et al. (2024) · Physical Review Applied · doi:10.1103/physrevapplied.21.l061001 · Cited by 33 · see citing works
“813 m s − 2 , and the height difference between the two clocks ∆ h day = h Sr2 , day − h Sr1 . h Sr2 , day is constant within each local campaign at PTB, but differs slightly between both campaigns.”
Elster, C., & Toman, B. (2013) · Metrologia · doi:10.1088/0026-1394/50/5/549 · Cited by 32 · see citing works
Consistency tests for key comparison data
Iyer, H. K., Wang, C. M., & Vecchia, D. F. (2004) · Metrologia · doi:10.1088/0026-1394/41/4/001 · Cited by 30 · see citing works
Forming a comparison reference value from different distributions of belief
Willink, R. (2005) · Metrologia · doi:10.1088/0026-1394/43/1/002 · Cited by 30 · see citing works
Analysis of key comparison data and laboratory biases
Гурьевна, Ч. А., Elster, C., Lira, I. H., & Wöger, W. (2008) · Metrologia · doi:10.1088/0026-1394/45/2/010 · Cited by 29 · see citing works
Statistical interpretation of key comparison reference value and degrees of equivalence
Kacker, R. N., Datla, R. U., & Parr, A. C. (2003) · Journal of Research of the National Institute of Standards and Technology · doi:10.6028/jres.108.038 · Cited by 28 · see citing works
Self-citationA generalized confidence interval for a measurand in the presence of type-A and type-B uncertainties
Wang, C. M., & Iyer, H. K. (2006) · Measurement · doi:10.1016/j.measurement.2006.04.011 · Cited by 28 · see citing works
Jahan, F., & Ballico, M. J. (2007) · International Journal of Thermophysics · doi:10.1007/s10765-007-0304-x · Cited by 23 · see citing works
Nicolet, A., Meli, F., van der Pol, E., Yuana, Y., Gollwitzer, C., Krumrey, M. K., Čižmár, P., Buhr, E., Pétry, J., Sebaïhi, N., et al. (2016) · Measurement Science and Technology · doi:10.1088/0957-0233/27/3/035701 · Cited by 22 · see citing works
Traceable pico-meter level step height metrology
Orji, N. G., Dixson, R. G., Fu, J. Y., & Vorburger, T. V. (2004) · Wear · doi:10.1016/j.wear.2004.04.009 · Cited by 22 · see citing works
Principles of probability and statistics for metrology
Willink, R. (2006) · Metrologia · doi:10.1088/0026-1394/43/4/s07 · Cited by 21 · see citing works
Pommé, S. G., & Spasova, Y. (2007) · Accreditation and Quality Assurance · doi:10.1007/s00769-007-0330-1 · Cited by 17 · see citing works
Comparison of Frequency Standards Used for TAI
Petit, G., & Panfilo, G. (2012) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2012.2228749 · Cited by 16 · see citing works
Decker, J. E., Brown, N., Cox, M. G., Steele, A. G., & Douglas, R. J. W. (2006) · Metrologia · doi:10.1088/0026-1394/43/6/n06 · Cited by 16 · see citing works
Description and Results: Antenna Measurement Facility Comparisons [Measurements Corner]
Saporetti, M. A., Foged, L. J., Castañer, M. S., Pivnenko, S., Cornelius, R., & Heberling, D. (2017) · IEEE Antennas and Propagation Magazine · doi:10.1109/map.2017.2686097 · Cited by 15 · see citing works
Schuster, M., Nevas, S., & Sperling, A. (2014) · Applied Optics · doi:10.1364/ao.53.002815 · Cited by 12 · see citing works
New Evaluation of $$T-T_{2000}$$ T - T 2000 from 0.02 K to 1 K by Independent Thermodynamic Methods
Engert, J., Kirste, A., Shibahara, A., Casey, A. J., Levitin, L. V., Saunders, J., Hahtela, O., Kemppinen, A., Mykkänen, E., Prunnila, M., et al. (2016) · International Journal of Thermophysics · doi:10.1007/s10765-016-2123-4 · Cited by 12 · see citing works
Temperature determination of the Si–SiC eutectic fixed point using thermocouples
Suherlan, Kim, Y.-G., Joung, W., & Yang, I. (2015) · Metrologia · doi:10.1088/0026-1394/52/2/330 · Cited by 9 · see citing works
Kanisch, G. (2017) · Nuclear Instruments and Methods in Physics Research Section A Accelerators Spectrometers Detectors and Associated Equipment · doi:10.1016/j.nima.2017.02.092 · Cited by 9 · see citing works
Kim, Y. G., Wei, Z., Ogura, H., Jahan, F., & Singh, Y. (2016) · Metrologia · doi:10.1088/0026-1394/53/1a/03003 · Cited by 9 · see citing works
An Introduction to Data Modeling Principles in Metrology and Testing
Pavese, F. (2008) · Modeling and simulation in science, engineering & technology · doi:10.1007/978-0-8176-4804-6_1 · Cited by 8 · see citing works
The Critical Role of International Comparisons in Global Metrology System: An Overview
Salzenstein, P., Wu, T. Y., & Pavlyuchenko, E. (2025) · Metrology · doi:10.3390/metrology5040074 · Cited by 7 · see citing works
Angle comparison using an autocollimator
Geckeler, R. D., Just, A., Vasilev, V., Prieto, E., Dvořáček, F., Zelenika, S., Przybylska, J., Duta, A., Victorov, I., Pisani, M., et al. (2018) · Metrologia · doi:10.1088/0026-1394/55/1a/04001 · Cited by 7 · see citing works
Cundeva–Blajer, M. (2017) · Journal of Electrical Engineering and Information Technologies · doi:10.51466/jeeit171015ch · Cited by 2 · see citing works
Fotowicz, P. (2009) · PRZEGLĄD ELEKTROTECHNICZNY · Cited by 1 · see citing works
Spherical polystyrene particle deformation measured with the AFM
Nicolet, A., & Meli, F. (2017) · Measurement Science and Technology · doi:10.1088/1361-6501/28/3/034003 · Cited by 1 · see citing works
Approaches to Data Assessment and Uncertainty Estimation in Testing
Bremser, W., & Hässelbarth, W. (2008) · Modeling and simulation in science, engineering & technology · doi:10.1007/978-0-8176-4804-6_10 · Cited by 1 · see citing works
- Authors
- Kacker, R., Datla, R., & Parr, A.
- Affiliations
- National Institute of Standards and Technology (US)
- Venue
- Metrologia
- Publisher
- IOP Publishing
- Published
- 2002-06
- Volume
- 39
- Issue
- 3
- Pages
- 279-293
- ISSN
- 0026-1394 (ISSN-L)
- OpenAlex
- W2031377355
- Cited by
- 36
- References
- 16
- Citations by year
- 2025: 1 · 2024: 1 · 2018: 1 · 2017: 5 · 2016: 3 · 2015: 2 · 2014: 1 · 2013: 1 · 2012: 3
- FWCI
- 2.63
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Metrologia (publishedVersion)
- Funders
- Advanced Low Carbon Technology Research and Development Program
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Scientific Measurement and Uncertainty Evaluation
- Physical Sciences › Engineering › Mechanical Engineering › Advanced Measurement and Metrology Techniques
- Physical Sciences › Engineering › Aerospace Engineering › Calibration and Measurement Techniques
- Keywords
- interlaboratory comparison, measurement uncertainty, systematic effects, random effects, key comparison, cryogenic radiometer, reference values
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review