Cited by 7Advanced Measurement and Detection MethodsMilitary Defense Systems AnalysisReliability and Maintenance Optimization

Support vector regression-based operational effectiveness evaluation approach to reconnaissance satellite system

Open access
Han, Chi, Xiong, Wei, Xiong, Minghui, Liu, Zhen·2023·Journal of Systems Engineering and Electronics

Abstract

As one of the most important part of weapon system of systems (WSoS), quantitative evaluation of reconnaissance satellite system (RSS) is indispensable during its construction and application. Aiming at the problem of nonlinear effectiveness evaluation under small sample conditions, we propose an evaluation method based on support vector regression (SVR) to effectively address the defects of traditional methods. Considering the performance of SVR is influenced by the penalty factor, kernel type, and other parameters deeply, the improved grey wolf optimizer (IGWO) is employed for parameter optimization. In the proposed IGWO algorithm, the opposition-based learning strategy is adopted to increase the probability of avoiding the local optima, the mutation operator is used to escape from premature convergence and differential convergence factors are applied to increase the rate of convergence. Numerical experiments of 14 test functions validate the applicability of IGWO algorithm dealing with global optimization. The index system and evaluation method are constructed based on the characteristics of RSS. To validate the proposed IGWO-SVR evaluation method, eight benchmark data sets and combat simulation are employed to estimate the evaluation accuracy, convergence performance and computational complexity. According to the experimental results, the proposed method outperforms several prediction based evaluation methods, verifies the superiority and effectiveness in RSS operational effectiveness evaluation.

Integrity signals

Registry

Crossref · 10.23919/jsee.2023.000020

Cites

43 of 43 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

2 of 7 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    GWO has advantages over PSO, ALO, MFO, SCA, WOA and MVO in global search ability, solution accuracy and convergence speed.

    “Compared with some proposed intelligent optimization algorithms, such as PSO, ALO, moth-flame optimization (MFO), SCA, WOA and multi-verse optimizer (MVO), GWO has advantages in global search ability, solution accuracy and convergence speed [21].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [21] source

      The Introduction describes GWO as having strong global search ability and says it outperforms GA and PSO in convergence speed and solution accuracy. It does not establish those advantages over ALO, MFO, SCA, WOA, or MVO; the paper’s reported comparisons instead involve ABC and SA. (LLM, conf=0.90)

      Experimental — accuracy not yet measured
  • Findingown data

    A comprehensive method optimizing SVR parameters for OEE with the IGWO algorithm is proposed in this paper.

    “A comprehensive method optimizing SVR parameters for OEE with the IGWO algorithm is proposed in this paper.”
    Quote in the paperSubstantial claim
  • Findingown data

    Two datasets meet the reject condition of the null hypothesis when IGWO-SVR is compared with BPNN.

    “In particular, two datasets meet the reject condition of null hypothesis when compared with BPNN.”
    Quote in the paperSubstantial claim
  • Findingown data

    The running time of IGWO is less than 0.5 s.

    “Furthermore, the running time of the proposed IGWO is less than 0.5 s, demonstrating its high execution efficiency.”
    Quote in the paperNot a substantial claim
  • Findingown data

    IGWO has the best standard deviation among compared methods for fixed-dimensional multimodal test functions f12-f14.

    “In the fixed-dimensional multimodal test functions ( -), IGWO has the best standard deviation among these methods.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 17
  • The quote only partly states this claim. 9
  • May combine two claims. 20
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The test functions are widely used to assess the performance of state-of-the-art algorithms.

    “The test functions are widely used to assess the performance of state-of-the-art algorithms [28].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [28] source

      The Case Study section describes using ten UCI data sets to compare SVR algorithms, but it does not discuss test functions or say they are widely used. A search for benchmark test functions found no passage addressing that claim; page numbers were unavailable in our copy. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim.

    Findingreasoning

    A total of six indicators-coverage rate, revisit time, communication duration, response time, reconnaissance frequency, and transmission delay-are determined as data characteristics for the RSS effectiveness evaluation.

    “Based on the specific application of the RSS and availability of data, a total of six indicators including coverage rate, revisit time, communication duration, response time, reconnaissance frequency, and transmission delay are determined as data characteristics.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    IGWO obtains global optimal objective values for functions f5, f9, and f11 while other methods fail.

    “For example, the global optimal objective values of several functions ( , , and ) are obtained by the proposed IGWO algorithm triumphantly while the other methods fail to make it”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim.

    Findingown data

    On the remaining five data sets, IGWO-SVR shows a significant difference compared with GWO-SVR.

    “On the remaining five data sets, IGWO-SVR is with significant difference.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    Although IGWO, MVO, SCA, and WOA converge at a similar speed in f7, the accuracy of IGWO is the best.

    “Although IGWO, MVO, SCA, and WOA converge at a similar speed, the accuracy of IGWO is the best.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    Regarding MAE, the values for SVR, IGWO-SVR, BPNN, and GWO-SVR are 0.02, 0.006, 0.04, and 0.02, respectively.

    “Regarding MAE, the values are 0.02, 0.006, 0.04, and 0.02, respectively.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    When IGWO-SVR is compared with GWO-SVR, there is no significant difference on datasets c, d, and f.

    “When it comes to GWO-SVR, there is no significant difference on datasets c, d, and f .”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Findingreasoning

    The effectiveness evaluation index system of RSS has been established through the iterative process shown in Fig. 7.

    “Through the process shown in Fig. 7, the effectiveness evaluation index system of RSS has been established.”
    • Also stated in: § Determination of the evaluation index hierarchy · plain
      “Finally, a layered decomposition indicator system for evaluating the effectiveness of RSS is obtained, as shown in Fig. 8.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Each of the four operational effectiveness indicators can be decomposed into multiple quantifiable individual performance indicators.

    “Each of the above indicators can be decomposed into multiple quantifiable individual performance indicators.”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim.

    Findingown data

    The R2 between the predicted value and the actual value for IGWO-SVR reaches 99.7%.

    “The between the predicted value and the actual value reaches 99.7%.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    The distribution of MSE for IGWO-SVR is more concentrated than for the other methods on datasets b, e, f, and h.

    “Meanwhile, the distribution of MSE is more concentrated, verifying the stability of the proposed approach.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    The proposed method achieves optimum evaluation accuracy and corresponding SVR parameters.

    “Finally, the method achieves optimum evaluation accuracy and corresponding SVR parameters.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    For GWO-SVR, the optimal parameters are penalty factor C = 5.34, Gauss kernel parameter σ = 1.0921, and insensitive loss function ε = 0.517.

    “For GWO-SVR, the penalty factor C = 5.34, the parameter of Gauss kernel function σ = 1.092 1 and the insensitive loss function ε = 0.517.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Findingown data

    In multimodal functions f9, f10, and f11, IGWO converges to the global optimum at about 200 iterations, faster than compared algorithms.

    “In multimodal functions f 9 , f 10 , and f 11 , IGWO converges to the global optimum solution as the iteration is about 200, which is faster than the algorithms in comparison.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The running time of PSO and MVO is shorter than IGWO in f1, f2, and f4 with a slight difference.

    “It should be noted that the running time of PSO and MVO is shorter than the proposed IGWO in f 1 , f 2 , and f 4 with a slight dif-”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Findingnone given

    Four operational effectiveness indicators-signal reconnaissance capability, imaging reconnaissance capability, information transmission capability, and reconnaissance information processing capability-are established for RSS.

    “Four operational effectiveness indicators: signal reconnaissance capability, imaging reconnaissance capability, information transmission capability, and reconnaissance information processing capability are established, respectively.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Findingown data

    In f3, WOA, GWO, and IGWO have similar performance, while IGWO is slightly superior in accuracy.

    “In f 3 , WOA, GWO, and IGWO have similar performance, while IGWO is slightly superior in accuracy.”
    Quote in the paperNot a substantial claim
  • May combine two claims. The quote only partly states this claim.

    Findingown data

    IGWO-SVR obtains the best score in RMSE, MAE, and R2.

    “IGWO-SVR obtains the best score in RMSE, MAE, and .”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    For the remaining datasets, the computational cost of IGWO-SVR is competitive.

    “For rest of datasets, the consequence of IGWO-SVR is also competitive.”
    Quote in the paperNot a substantial claim
  • The quote only partly states this claim.

    Findingown data

    According to Fig. 12, the proposed IGWO-SVR method wins the first place in absolute error.

    “According to Fig. 12, the proposed method wins the first place.”
    Quote in the paperSubstantial claim
  • The quote only partly states this claim.

    Findingown data

    IGWO outperforms standard GWO and other compared algorithms in most test functions.

    “According to Table 3, in most test functions, the proposed method has significantly better performance than the standard GWO algorithm.”
    • Also stated in: § Function · plain
      “In summary, IGWO has advantages in most test functions.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Interpretationreasoning

    IGWO with opposition-based learning, mutation operator, and differential convergence factor is proposed to escape local optima and relieve premature convergence of GWO.

    “The IGWO algorithm with opposition-based learning strategy, mutation operator, and differential convergence factor is proposed to escape from local optima and relieve the premature convergence of the GWO algorithm.”
    • Also stated in: § Abstract · plain
      “In the proposed IGWO algorithm, the opposition-based learning strategy is adopted to increase the probability of avoiding the local optima, the mutation operator is used to escape from premature convergence and differential convergence factors are applied to increase the rate of convergence.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    In standard GWO, the balance between exploration and development is controlled by parameter a, which decreases linearly over [0,2].

    “In standard GWO, the balance between exploration and development capabilities is controlled by parameter a, which decreases linearly over [0,2].”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The experimental results confirm that IGWO-SVR is superior to other methods in terms of prediction accuracy and convergence stability.

    “The experimental results above confirm that IGWO-SVR is superior to other methods in terms of prediction accuracy and convergence stability.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    To ensure reliability of results, all comparison algorithms uniformly set the maximum population size to 30 and the maximum number of iterations to 500.

    “To ensure the reliability of the results, all comparison algorithms uniformly set the maximum population size to 30 and the maximum number of iterations to 500.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim.

    Premisereasoning

    The choice of an appropriate radius parameter is a key issue in parameter selection of the RBF core SVR model.

    “The choice of the appropriate radius parameter is a key issue in parameter selection of radial basis function (RBF) core SVR model.”
    Quote in the paperSubstantial claim
  • May combine two claims.

    Premisereasoning

    Standard GWO has poor population diversity due to its simple search mechanism, resulting in a high probability of premature convergence on multimodal problems.

    “The diversity of population is usually poor due to the simple search mechanism, resulting in a high probability in premature convergence when dealing with multimodal problems.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisereasoning

    The opposition-based learning strategy increases the probability of finding better solutions and enhances global exploitation and local exploration.

    “Therefore, if the current solution is local optimal, then the algorithm successfully jumps out of it, which will increase the probability of finding better solutions.As a result, the global exploitation and local exploration of the population can be effectively enhanced.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [39] source
  • May combine two claims.

    Premisenone given

    Standard GWO's control parameter decreases linearly, leading to a restricted population distribution range.

    “Meanwhile, the control parameter decreases linearly, leading to the restricted population distribution range.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationnone given

    In order to pursue accuracy in prediction, the integrated method sacrifices a fraction of stability of SVR; IGWO-SVR realizes a great balance in this term.

    “Actually, in order to pursue accuracy in prediction, the integrated method sacrifices a fraction of stability of SVR.Regarding this term, IGWO-SVR realizes a great balance.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    The radius parameter of the RBF kernel has a great influence on the learning performance of SVR.

    “σ where the radius parameter of the kernel function has a great influence on the learning performance of SVR.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Optimization of SVR is a typical multi-parameter optimization problem in which obtaining a balance between exploration and exploitation is vital.

    “The optimization of SVR is a typical multi-parameter optimization problem, in which it is of vital importance of obtaining balance between exploration and exploitation.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    Traditional evaluation methods have low evaluation efficiency.

    “Second, the efficiency of evalua-tion is low.”
    Quote in the paperNot a substantial claim
  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given

    The kernel function adopts a radial basis function with strong generalization ability.

    “The kernel function adopts a radial basis function with strong generalization ability.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Premisecitation

    GWO is a biological heuristic optimization method derived from simulating the social hierarchy and hunting mechanism of gray wolf, proven to have a more reasonable global optimal search mechanism and suitable for parameter optimization.

    “GWO is a biological heuristic optimization method derived from simulating the social hierarchy and hunting mechanism of gray wolf in nature, which has been proven to have a more reasonable global optimal search mechanism and suitable for processing parameter optimization problems [36].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [36] source
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    The proposed IGWO-SVR method can realize nonlinear fitting of complex systems, obtain deep relationships implicit in data, and achieve prediction and evaluation of effectiveness.

    “It can realize the nonlinear fitting of complex systems and obtain the deep relationship implicit in data, and achieve the prediction and evaluation of effectiveness.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: For the RSS studied in this work, the objective function space of effectiveness evaluation is so complicated that it cannot be described by an analytical formula.
    • Rests on an assumption: The essence of the effectiveness evaluation process is to solve a multi-objective input nonlinear equation.
  • May combine two claims.

    Premisecitation

    GWO has very few parameters and requires no derivation information in the initial search.

    “Also it has very few parameters, and no derivation information is required in the initial search, thus GWO is used to optimize SVR.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [36] source
  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    SVR follows the principle of structural risk minimization, considering confidence range and empirical risk to achieve global optimization at the same time.

    “SVR follows the principle of structural risk minimization, considering the range of confidence and empirical risk to achieve global optimization at the same time [33].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [33] source
  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    For each test function, thirty runs are conducted independently to reduce the negative effect of randomness.

    “For each test function, thirty runs are conducted independently to reduce the negative effect of randomness.”
    Quote in the paperSubstantial claim
  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    Traditional evaluation methods have the limitation that subjective influence cannot be avoided, leading to an impact on assessment accuracy.

    “First, subjective influence cannot be avoided, leading to an impact on the accuracy of the assessment.”
    Quote in the paperNot a substantial claim
  • May combine two claims.

    Interpretationown data

    The SVR-based evaluation method avoids the influence of subjectivity and obtains higher evaluation accuracy.

    “The proposed method not only avoids the influence of subjectivity, but also obtains a higher evaluation accuracy.”
    Quote in the paperSubstantial claim
41 of 41
  • The paper relies on this without support.

    For the RSS studied in this paper, the number of evaluation objects is so limited that it is difficult to obtain a large amount of usable historical data.

    “For RSS studied in this paper, the number of evaluation object is so limited that it is difficult to obtain a large amount of usable historical data.”
  • The paper relies on this without support.

    SVR has fewer parameters and faster running speed on small samples.

    “In comparison, SVR has fewer parameters and faster running speed on small samples.”
  • The paper relies on this without support.

    The setting of SVR parameters has a large impact on accuracy and generalization performance, especially when dealing with high-dimensional data.

    “The setting of parameters has large impact on accuracy and generalization performance of SVR, especially when dealing with high-dimensional data.”
  • The paper relies on this without support.

    The performance of SVR is deeply influenced by the penalty factor, kernel type, and other parameters.

    “Considering the performance of SVR is influenced by the penalty factor, kernel type, and other parameters deeply”
  • The paper relies on this without support.

    GWO has a special ability to strike the right balance between exploration and exploitation.

    “The special capability to strike the right balance between exploration and exploitation is precisely the advantage of GWO”
  • The paper relies on this without support.

    The effectiveness and connotation measurement framework of RSS are still unclear.

    “However, the effectiveness and connotation measurement framework of RSS are still unclear.”
  • The paper relies on this without support.

    For the RSS studied in this work, the objective function space of effectiveness evaluation is so complicated that it cannot be described by an analytical formula.

    “For the RSS studied in this work, the objective function space of effectiveness evaluation is so complicated that cannot be described by analytical formula.”
  • The paper relies on this without support.

    Traditional evaluation methods have defects dealing with nonlinear evaluation under small sample conditions, and are incompetent for effectiveness evaluation of RSS.

    “Traditional evaluation methods have defects dealing with nonlinear evaluation under small sample conditions, thus these methods are incompetent for effectiveness evaluation of RSS.”
  • The paper relies on this without support.

    Due to spatiotemporal dynamic characteristics of RSS, satellite applications are constrained by satellite system orbits and combat units, resulting in vagueness and uncertainty of RSS's operational effectiveness.

    “Meanwhile, due to the spatiotemporal dynamic characteristics of RSS, the satellite applications are constrained by satellite system orbits and combat units, resulting in vagueness and uncertainty of RSS's operational effectiveness.”
  • The paper relies on this without support.

    The main ideas of effectiveness evaluation and machine learning are similar.

    “The main ideas of effectiveness evaluation and machine learning are similar.”
  • The paper relies on this without support.

    The essence of the effectiveness evaluation process is to solve a multi-objective input nonlinear equation.

    “Based on this, the essence of the effectiveness evaluation process is to solve a multi-objective input nonlinear equation, which can be described by the following model:”
  • The paper relies on this without support.

    The simulation data under specific operational scenarios adequately represent RSS operational effectiveness for the purpose of verifying the IGWO-SVR model.

    “Data in experiment are derived from simulation under specific operational scenarios, as shown in Fig. 9.”
  • The paper relies on this without support.

    The TOPSIS-derived effectiveness values are a valid ground truth for training and testing the IGWO-SVR model.

    “TOPSIS method is applied to obtain the effectiveness of RSS according to procedures in Section 4.”
13 of 13

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Reported RMSE is smaller than MAE

    Major

    For the same prediction errors, root mean squared error cannot be smaller than mean absolute error. The IGWO-SVR row reports 0.000935 versus 0.006358, and the BPNN row also reverses this required ordering.

  • Reported effectiveness values exceed the TOPSIS range

    Major

    The paper uses TOPSIS closeness as the effectiveness value. Its stated formula divides a nonnegative distance by the sum of two nonnegative distances, so the value cannot exceed 1, yet the sample data include 2.0857.

  • The f8 optimum uses five terms for a 30-dimensional sum

    Minor

    The stated f8 function sums one identically bounded contribution per dimension and has dimension 30. Its listed optimum instead multiplies the per-term optimum by 5.

  • Another algorithm also attains the claimed exclusive optimum

    Minor

    The text says other methods fail to attain the global optimum for f9, but Table 3 reports an average of exactly zero for GWO as well as IGWO. Table 2 lists zero as the optimum for f9.

  • Dataset d has an impossible reported MSE variation

    Minor

    MSE cannot be negative. Across 20 nonnegative accuracies with mean 6.10e-05, the sample standard deviation cannot exceed about 2.73e-04; the IGWO-SVR entry reports 5.08e-04.

5 of 5

Cites

Flagged first, then by citations

1 of the 44 references couldn’t be matched to a known paper.

Research works printed but not linked: 9

  1. Wei, 2016, Efficiency evaluation research of missile weapon system based on the ADC mode, Advances in Engineering Research, 180, 1227
  2. Yann, 2015, Deep learning, Nature, 521, 436
  3. Li, 2015, Combat effectiveness prediction model using elman feedback network, Journal of System Simulation, 27, 43
  4. Ren, 2017, Effectiveness prediction of weapon equipment system based on stacked auto-encoding, Military Operations Research and System Engineering, 31, 61
  5. Cheng, 2011, Evaluation method of operational effectiveness based on support vector machine, Systems Engineering and Electronics, 33, 1055
  6. Cui, 2012, Operational effectiveness evaluation of weapon under influence of marine environment based on v-SVR, Computer Technology and Development, 22, 20
  7. Hatta, 2018, Recent studies on optimization method of grey wolf optimizer (GWO): a review (2014–2017), Artificial Intelligence Review, 18, 1
  8. Simpson, 2015, The ant lion optimizer, Advances in Engineering Software, 8, 80
  9. Han, 2021, Related information ming of space reconnaissance equipment system index, Journal of System Simulation, 33, 2372

Cited by

Sorted by citations

Authors
Han, C., Xiong, W., Xiong, M., & Liu, Z.
Affiliations
Space Engineering University (CN)
Venue
Journal of Systems Engineering and Electronics
Published
2023-12
Volume
34
Issue
6
Pages
1626-1644
ISSN
1004-4132 (ISSN-L)
OpenAlex
W4319934199
Cited by
7
References
44
Citations by year
2026: 2 · 2025: 2 · 2024: 3
FWCI
0.55
Type
journal article
Language
en
Open access
Locations
Journal of Systems Engineering and Electronics (publishedVersion)
Topics
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Measurement and Detection Methods
  • Physical Sciences › Engineering › Aerospace Engineering › Military Defense Systems Analysis
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Reliability and Maintenance Optimization
Keywords
operational performance evaluation, support vector regression, improved grey wolf optimizer, parameter optimization, opposition-based learning, weapon system of systems, evaluation index system, combat simulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review