Cited by 13Optimal Experimental Design MethodsAdvanced Statistical Methods and ModelsSimulation Techniques and Applications

Optimal Blocked and Split-Plot Designs Ensuring Precise Pure-Error Estimation of the Variance Components

Open access
Mylona, Kalliopi, Gilmour, Steven G., Goos, Peter·2020·Technometrics

Abstract

Textbooks on response surface methodology generally stress the importance of lack-of-fit tests and estimation of pure error. For lack-of-fit tests to be possible and other inference to be unbiased, experiments should allow for pure-error estimation. Therefore, they should involve replicated treatments. While most textbooks focus on lack-of-fit testing in the context of completely randomized designs, many response surface experiments are not completely randomized and require block or split-plot structures. The analysis of data from blocked or split-plot experiments is generally based on a mixed regression model with two variance components instead of one. In this article, we present a novel approach to designing blocked and split-plot experiments which ensures that the two variance components can be efficiently estimated from pure error and guarantees a precise estimation of the response surface model. Our novel approach involves a new Bayesian compound D-optimal design criterion which pays attention to both the variance components and the fixed treatment effects. One part of the compound criterion (the part concerned with the treatment effects) is based on the response surface model of interest, while the other part (which is concerned with pure-error estimates of the variance components) is based on the full treatment model. We demonstrate that our new criterion yields split-plot designs that outperform existing designs from the literature both in terms of the precision of the pure-error estimates and the precision of the estimates of the factor effects.

Integrity signals

Registry

Crossref · 10.1080/00401706.2019.1595153

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

4 of 19 cited sources checked · 3 supported · 0 flagged · accuracy not yet measured

  • Interpretationreasoning

    The new-criterion designs differ fundamentally from the split-plot designs of Vining et al. (2005), Parker et al. (2006), Kowalski et al. (2002), and Kowalski et al. (2006).

    “Consequently, the designs produced by the new criterion are fundamentally dierent from the split-plot designs proposed by Vining et al. (2005), Parker et al. (2006), Kowalski et al. (2002) and Kowalski et al. (2006).”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Vining et al. (2005) source
    • cites Parker et al. (2006) source
    • cites Kowalski et al. (2002) source
  • Premisecitation

    Those split-plot designs replicate a few treatments many times, mainly center and/or axial points, with much replication inside whole plots.

    “These designs involve many replicates of a limited number of treatments (namely, the treatments corresponding to center points and/or axial points).Much of the replication in these designs is within the whole plots.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Vining et al. (2005) source
    • cites Parker et al. (2006) source

      Section 5.1 describes adding subplot-center replicates to balance a Box–Behnken design, and Section 5.2 says subplot centers can be added within each whole plot. But Section 5.3 says its base crossed design has no replicates, while Section 6 describes a design without replication. The paper therefore supports within-whole-plot center replication as an option, not the general characterization that these designs contain many replicates concentrated at center and/or axial points. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites Kowalski et al. (2002) source
  • Findingown data

    For estimating fixed effects, the benchmark design's D-efficiency relative to the optimal design is smaller than 60%.

    “For estimating the xed eects, the benchmark design's D-eciency relative to the optimal design is therefore smaller than 60%.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • Findingown data

    The value of the new Bayesian composite criterion is 2.2965 for the optimal design versus 1.7961 for the benchmark design.

    “The value for the new Bayesian composite criterion equals 2.2965 for the optimal design versus 1.7961 for the benchmark design.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • Findingown data

    Of the three screening designs, the OriCC/TradD design has the largest log|M| value, 45.2136.

    “Of the three design under consideration, the `OriCC/TradD' design has the largest log |M| value, 45.2136.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims. 32
  • The quote only partly states this claim. 2
  • Classified as probably not a substantial claim. 3
  • May combine two claims.

    Interpretationreasoning

    The resulting designs enable unbiased and efficient inferences and provide efficient pure-error estimates of the variance components required by the lack-of-fit test proposed by Goos and Gilmour (2017).

    “The resulting designs enable unbiased and ecient inferences and provide ecient pure-error estimates of the variance components required by the lackof-t test proposed by Goos and Gilmour (2017) for data from blocked and split-plot experiments.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Goos and Gilmour (2017) source
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Interpretationreasoning

    The full treatment model does not exhibit any lack of fit, and the variance component estimates resulting from it are pure-error estimates.

    “Therefore, the model does not exhibit any lack of t, and the variance component estimates resulting from it are pure-error estimates.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Gilmour et al. (2019)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Goos and Gilmour (2017) source
    • The authors state this as an assumption: The effects of runs and treatment factors are additive, so there is no block-by-treatment interaction.
    • The authors state this as an assumption: The randomization of the experiment is correct.
    • The authors state this as an assumption: The random block or whole-plot effects γ are normally distributed with mean zero and covariance σ²γ I_b.
    • The authors state this as an assumption: The random errors ε are normally distributed with mean zero and covariance σ² I_n.
    • The authors state this as an assumption: The random block or whole-plot effects and the random errors are independent.
  • May combine two claims.

    Findingown data

    In the classical design, four pairs of whole plots have two treatments in common; in the design produced by the new criterion, every pair of whole plots has at most one treatment in common, and every whole plot contains exactly two of the duplicated treatments.

    “In the classical design, four pairs of whole plots have two treatments in common.For instance, treatments 1 and 21 both appear in whole plots 1 and 4.This is shown in Table 5.In the design produced by the new criterion, every pair of whole plots has at most one treatment in common, while every whole plot contains exactly two of the duplicated treatments.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Findingown data

    Both the design produced by the new composite criterion and the classical design involve 16 treatments and eight replicates, with eight of the 16 treatments duplicated.

    “Both the design produced by the new composite criterion and the classical design involve 16 treatments and thus eight replicates.More specically, eight of the 16 treatments are duplicated.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Findingown data

    For the ceramic pipe experiment, the log determinant of the information matrix for fixed effects M is 36.7513 for the optimal design and 29.0329 for the benchmark design when η=1 and σ2=1.

    “The log determinant of the information matrix for the xed eects in the response surface model, M, is 36.7513for the optimal design when η = 1 and σ 2 = 1, and 29.0329 for the benchmark design.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random block or whole-plot effects γ are normally distributed with mean zero and covariance σ²γ I_b.
    • The authors state this as an assumption: The random errors ε are normally distributed with mean zero and covariance σ² I_n.
    • The authors state this as an assumption: The random block or whole-plot effects and the random errors are independent.
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Findingown data

    The assignment of duplicates to whole plots in the classical design is better for pure-error variance component estimation than the assignment in the design produced by the new criterion, since log|N_t| is larger for the former (0.5754) than for the latter (0).

    “The assignment of the duplicates to the whole plots in the classical design is better for the pure-error variance component estimation than the assignment in the design produced by the new criterion, since the log |N t | value is larger for the former design (0.5754) than for the latter (0).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random block or whole-plot effects γ are normally distributed with mean zero and covariance σ²γ I_b.
    • The authors state this as an assumption: The random errors ε are normally distributed with mean zero and covariance σ² I_n.
    • The authors state this as an assumption: The random block or whole-plot effects and the random errors are independent.
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Findingown data

    The existing criteria produce a design that is best for fixed effects estimation, but it has no replicates and its information matrix for pure-error variance component estimation is singular.

    “The existing criteria produce a design that is best for xed eects estimation, but it has no replicates and its information matrix for pure-error variance component estimation is singular (so, its log |N t | value goes to -∞).”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The random block or whole-plot effects γ are normally distributed with mean zero and covariance σ²γ I_b.
    • The authors state this as an assumption: The random errors ε are normally distributed with mean zero and covariance σ² I_n.
    • The authors state this as an assumption: The random block or whole-plot effects and the random errors are independent.
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims. The quote only partly states this claim.

    Findingreasoning

    The new criterion yields split-plot designs that outperform existing designs from the literature in terms of precision of pure-error estimates and precision of estimates of factor effects.

    “We demonstrate that our new criterion yields split-plot designs that outperform existing designs from the literature both in terms of the precision of the pure-error estimates”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Findingnone given

    In the new-criterion designs, replication is entirely between blocks for blocked designs or between whole plots for split-plot designs.

    “One striking feature of the designs produced by the new criterion and reported in this paper is that all replication occurs between the blocks (for blocked designs) or between the whole plots (for split-plot designs).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The 48-run blocked design optimizing the new criterion has no within-block replication, but four pairs of whole plots share two treatments.

    “It does not involve within-block replication, but, as in the classical design in the previous illustration, four pairs of whole plots have two treatments in common.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Because of the nine duplicated points, the full-treatment-model REML information matrix is non-singular; its log determinant is 0.8344 for η=1 and σ2=1, much lower than 2.2794 for the new-criterion design.

    “Due to the nine duplicated points, the REML information matrix for the full treatment model is non-singular.Its log determinant is 0.8344 when η = 1 and σ 2 = 1, which is substantially smaller than the value 2.2794 for the design produced by the new composite criterion.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The new-criterion 48-run split-plot design has no within-whole-plot replication; two whole plots share three treatments, and five other whole-plot pairs share two.

    “The design does not have any replication within whole plots.Two of the eight whole plots have three treatments in common, while ve other pairs of whole plots have two treatments in common.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    That benchmark design has 39 distinct treatments, with nine duplicated; all duplication is between whole plots.

    “The design has 39 distinct treatments, nine of which are duplicated.All duplication is between the whole plots, rather than within the whole plots.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning

    The benchmark 48-run blocked design does not involve any replicates and therefore does not allow pure-error variance component estimation.

    “Despite the fact it involves 48 runs, which is substantially more than the number of xed eects, the benchmark design does not involve any replicates.The benchmark design therefore does not allow pure-error variance component estimation.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingreasoning

    For the 48-run blocked design based on the new composite criterion, the REML information matrix under the full treatment model Nt is non-singular; for the benchmark design it is not, due to lack of replicated treatments.

    “For the design based on the new composite criterion, the REML information matrix under the full treatment model, N t , is non-singular.Due to lack of replicated treatments, this is not the case for the benchmark design.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In the 48-run blocked example, the new-criterion design contains 31 treatments, with 17 appearing twice.

    “The design optimizing the new criterion involves 31 treatments, 17 of which are duplicated.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In the simulation, the mean σ2 estimate under REML with the response surface model is much larger than the true value 10, whereas under REML with the full treatment model it is near 10.

    “The mean estimate for σ 2 by far exceeds the true value of 10 when REML is applied to the response surface model, while it is close to 10 when REML is applied to the full treatment model.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In the 48-run split-plot example, the new-criterion design contains 30 treatments, with one appearing three times and 16 appearing twice.

    “The split-plot design optimizing the new criterion involves 30 treatments, one of which is triplicated and 16 of which are duplicated.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In that design, nearly all replicated points are factorial points, and no center run is replicated.

    “Almost all replicated points are factorial points.None of them is the center run.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisereasoning

    The new designs aim to be optimal for all parameters, whereas Cao et al. (2017) and Trinca and Gilmour (2017) aim to be optimal only for fixed effects and only stratum by stratum.

    “Our designs aim to be optimal for all parameters, whereas theirs aim to be optimal only for xed eects and only stratum by stratum.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Cao et al. (2017) source
    • cites Trinca and Gilmour (2017) source
  • May combine two claims.

    Interpretationnone given

    The new method is generic: it works for blocked and split-plot experiments, handles main-effects, main-effects-plus-interactions and full quadratic response surface models, and works for any number of blocks/whole plots and block/whole-plot sizes dictated by logistics.

    “Our method is generic in that it works for blocked and split-plot experiments, it is able to handle main-eects, maineects-plus-interactions and full quadratic response surface models, and it works for any number of blocks/whole plots and block/whole-plot sizes dictated by the logistics of the experiment.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    The benchmark design is inferior to the optimal design both in precision of pure-error estimates and in precision of fixed effects estimates.

    “The benchmark design is thus inferior both in terms of the precision of the pure-error estimates and in terms of the precision of the xed eects estimates.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • Classified as probably not a substantial claim.

    Interpretationnone given

    The composite D-optimality criterion focuses on precise estimation of fixed treatment effects and precise pure-error estimation of variance components.

    “we suggest a composite D-optimality criterion that focuses on precise estimation of the xed treatment eects, as well as on precise pure-error estimation of the variance components.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Premisereasoning

    The information matrix N in the existing composite criterion of Mylona et al. (2014) depends on the assumed response surface model through P and X, so using its determinant to quantify information on variance components is not model-robust.

    “The main weakness of the composite D-optimality criterion of Mylona et al. (2014) is that the information matrix for the variance components, N, depends on the assumed response surface model through the matrices P and X.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Mylona et al. (2014) source
  • May combine two claims.

    Interpretationreasoning

    Designs obtained using w=1/2 do not allow any additional model terms to be studied or lack of fit to be tested.

    “While this is ideal when it comes to pure-error estimation of the variance components, these designs do not allow any additional model terms to be studied or lack of t to be tested.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim.

    Interpretationcitation

    To obtain high-quality pure-error estimates of both variance components in a split-plot model, it is advisable to replicate primarily between whole plots rather than within whole plots.

    “This implies that, to obtain high-quality pure-error estimates of both variance components in a splitplot model, it is advisable to replicate primarily between whole plots rather than within whole plots.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Trinca and Gilmour (2017) source
    • The authors state this as an assumption: The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.
  • May combine two claims.

    Interpretationreasoning

    When the response surface model is not correctly specified, REML variance component estimates from it are generally biased and can damage the whole analysis.

    “As soon as that is no longer true, the variance component estimates obtained by applying REML to the response surface model will generally be biased, which may have detrimental consequences for the entire data analysis.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationown data

    This is the third example where assigning replicates in this way helps obtain precise pure-error variance component estimates.

    “This is the third illustration in which this type of assignment of the replicates turns out to be useful to obtain precise pure-error variance component estimates.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The largest σ2γ precision gap occurs for the traditional/original design, explained by its large log|N| versus log|Nt| difference (2.9882 vs 0.8344).

    “That the dierence in precision is largest for the design produced by the traditional D-optimality criterion and the original composite criterion can be explained by the large dierence in that design's log |N| and log |N t | values (which equal 2.9882 and 0.8344, respectively; see Table 11).”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    The new criterion combines two information matrices: one for fixed factor effects from a traditional response surface model, and one for pure-error variance component estimates from REML applied to the full treatment model.

    “The new criterion is a composite criterion.One component of the criterion is the information matrix for the xed factor eects, based on a traditional response surface model.”
    • Also stated in: § 6 Discussion · plain
      “The second component is the information matrix for the pure-error variance component estimates.That information matrix is based on the application of restricted or residual maximum likelihood (REML) to the full treatment model.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationown data

    The two 48-run split-plot designs differ little in log|M|, so the new-criterion design is highly D-efficient for fixed effects, with 97.9% D-efficiency relative to the benchmark.

    “The dierence in log |M| value between the two designs is small, which means that the design produced by the new criterion is highly D-ecient for estimating the xed eects too.Its D-eciency relative to that of the benchmark design is 97.9%.”
    Quote in the paperSubstantial claim
  • May combine two claims. The quote only partly states this claim.

    Interpretationown data

    For the 48-run split-plot designs, log|N| exceeds log|Nt|, which implies that variance components estimated by REML under the response surface model are more precise than pure-error estimates from the full treatment model.

    “One nal observation concerningTable 11 is that the log |N| values are larger than the log |N t | values.This implies that the variance components obtained by applying REML to”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The response surface model is correctly specified for the reported log|N| values.
  • May combine two claims.

    Interpretationreasoning

    For designs saturated for the response surface model, N and Nt are equal, so log|N| and log|Nt| have the same value.

    “For such designs, N and N t are equal, so that log |N| and log |N t | have the same value.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationreasoning

    The new designs resemble balanced incomplete block designs, which are optimal for unstructured treatments and use a small number of replicates for every treatment.

    “The newly generated designs are akin to balanced incomplete block designs, which are known to be optimal designs for a set of unstructured treatments and involve a limited number of replicates for all treatments.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The arrangement of four pairs of whole plots sharing two treatments was found to produce precise pure-error estimates in the previous split-plot example and also arises in the new blocked example.

    “This kind of arrangement was found to produce precise pure-error estimates in the previous example (involving a split-plot experiment).It is interesting that it arises in the new example (involving a blocked experiment) as well.”
    Quote in the paperNot a substantial claim
35 of 35
  • The authors state this as an assumption.

    The effects of runs and treatment factors are additive, so there is no block-by-treatment interaction.

    “assuming additivity between the runs and the treatment factors' eects when calculating pure-error estimates”
    “A consequence of this assumption is that there is no block-by-treatment interaction.”
  • The authors state this as an assumption.

    The randomization of the experiment is correct.

    “follows immediately from the assumption of treatment-run additivity and a correct randomization”
  • The authors state this as an assumption.

    The random block or whole-plot effects γ are normally distributed with mean zero and covariance σ²γ I_b.

    “We further assume that γ ∼ N 0 b , σ 2 γ I b”
  • The authors state this as an assumption.

    The random errors ε are normally distributed with mean zero and covariance σ² I_n.

    “∼ N (0 n , σ 2 I n )”
  • The authors state this as an assumption.

    The random block or whole-plot effects and the random errors are independent.

    “and that γ and are independent.”
  • The authors state this as an assumption.

    The variance ratio η follows a lognormal prior distribution with parameters µ=0 and σ=0.75.

    “we use a lognormal prior distribution for the variance ratio”
    “The designs we present in this article were all constructed using the parameter values µ = 0 and σ = 0.75 for the lognormal prior distribution.”
  • The authors state this as an assumption.

    The response surface model is the model any researcher will be primarily interested in.

    “because we believe that this is the model any researcher will be primarily interested in.”
  • The authors state this as an assumption.

    The response surface model is correctly specified for the reported log|N| values.

    “Of course, the log |N| values reported here assume that the response surface model is correctly specied.”
8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Duplicated whole-plot treatment is misidentified

    Minor

    The text names treatment 29 as part of a duplicated whole plot with treatments 19, 21 and 25. Table 2 instead places treatment 27 with those three treatments in whole plots 5 and 6; treatment 29 appears only in whole plot 7.

  • Discussion denies within-whole-plot replication shown earlier

    Minor

    The discussion says every replicate in the reported designs occurs between blocks or whole plots. The paper earlier explicitly states that four treatments in its Table 2 design are duplicated within a whole plot.

2 of 2

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Mylona, K., Gilmour, S. G., & Goos, P.
Affiliations
  • King's College London (GB)
  • University of Antwerp (BE)
Corresponding author
Goos, Peter
Venue
Technometrics
Published
2019-06-21
Volume
62
Issue
1
Pages
57-70
ISSN
  • 0040-1706 (ISSN-L)
  • 1537-2723
OpenAlex
W2946960690
Cited by
13
References
29
Citations by year
2026: 1 · 2025: 3 · 2024: 3 · 2022: 3 · 2021: 1 · 2020: 1 · 2019: 1
FWCI
1.07
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • Technometrics (publishedVersion)
  • Lirias (submittedVersion, other-oa)
  • Research Portal (King's College London) (submittedVersion, other-oa)
Funders
  • Banco Santander (600371)
  • University of Southampton
  • European Commission (600371)
  • Ministerio de Educación, Cultura y Deporte (CEI‐15‐17)
  • Ministerio de Economía y Competitividad (COFUND2013‐40258)
  • Universidad Carlos III de Madrid (COFUND2013-40258, 600371)
  • H2020 European Research Council
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Statistical Methods and Models
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Simulation Techniques and Applications
Keywords
blocked designs, split-plot design, variance components, response surface methodology, lack-of-fit test, mixed model analysis, experimental design optimization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-02.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-02.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review