SGM: A Statistical Godel Machine for Risk-Controlled Recursive Self-Modification
Open accessAbstract
Recursive self-modification is increasingly central in AutoML, neural architecture search, and adaptive optimization, yet no existing framework ensures that such changes are made safely. Godel machines offer a principled safeguard by requiring formal proofs of improvement before rewriting code; however, such proofs are unattainable in stochastic, high-dimensional settings. We introduce the Statistical Godel Machine (SGM), the first statistical safety layer for recursive edits. SGM replaces proof-based requirements with statistical confidence tests (e-values, Hoeffding bounds), admitting a modification only when superiority is certified at a chosen confidence level, while allocating a global error budget to bound cumulative risk across rounds.We also propose Confirm-Triggered Harmonic Spending (CTHS), which indexes spending by confirmation events rather than rounds, concentrating the error budget on promising edits while preserving familywise validity.Experiments across supervised learning, reinforcement learning, and black-box optimization validate this role: SGM certifies genuine gains on CIFAR-100, rejects spurious improvement on ImageNet-100, and demonstrates robustness on RL and optimization benchmarks.Together, these results position SGM as foundational infrastructure for continual, risk-aware self-modification in learning systems.Code is available at: https://github.com/gravitywavelet/sgm-anon.
AI snapshot
Written from the paper’s full text.
TL;DR
Statistical Gödel Machines (SGM) replace the Gödel machine’s per-update formal proof of improvement with a conditionally valid statistical certificate and a trajectory-level error budget, bounding the probability of any non-improving commitment by δ. Controlled and synthetic experiments plus a two-attempt ImageNet-100 run show executable accept/commit/reject behavior, while exposing trade-offs among false commitments, missed improvements, and verification cost.
Key findings
- Theorem 1: if each opened confirmation satisfies the conditional validity bound and the predictable allocation is summable with total at most δ, then P(∪ₖ(Hₖ ∩ Aₖ)) ≤ δ; Corollary 1 gives a high-probability non-decreasing incumbent-utility sequence under a fixed utility and faithful state transitions.
- In the controlled numerical workload at tolerance 10⁻³, SGM had zero observed false-commit trajectories versus 76.50% for screening-only adoption, and final utility rose from 75.82% to 77.66%; the improvement cost approximately 8.9 times as many paired evaluations. Per-test confirmation also had zero observed errors and slightly higher final utility at lower cost.
- In synthetic trajectories, SGM’s observed trajectory FWER was 0.15–0.20% versus 3.35–3.95% for per-test confirmation, but SGM had lower final and cumulative utility and greater verification cost in all four comparisons; 98.7–100% of the FWER reduction came from zero-effect-only trajectories, and SGM made no strictly harmful commitments in these runs.
- In the two-attempt ImageNet-100 run, attempt 1 accepted the 0.01 branch (development gain +11.693, confirmation gain +11.948, W/L 3941/954, p = 2.00×10⁻⁴²⁷); attempt 2 was rejected (development gain +0.428, confirmation gain −0.016, W/L 401/405, p = 0.5699). Audit accuracies were 65.36% initial, 77.35% committed (+11.988 points), and 77.39% for the rejected candidate (+0.039 points above its parent); 210 training epochs took 4.09 hours, confirmation inference 31.43/31.88 seconds, and final audit 115.75 seconds.
Methods
- SGM protocol: propose and optionally screen candidates; freeze the candidate, incumbent, score, sampling rule, and certificate; reserve αₖ before revealing confirmation observations; collect valid paired evidence; commit only on a valid crossing with matching state bindings, otherwise retain the incumbent. A protected verification layer enforces the declared rule; example allocations include αₖ = δ/[k(k+1)] and αₖ = δ/(kH_K).
- Data and experiments: controlled numerical Simpson quadrature on 36 analytic functions with seven interval counts and 252 cached solver outputs, two tolerances, 2,000 trajectories of 40 rounds each; synthetic settings combining strong-effect sizes 0.10 and 0.25 with path strengths 0 and 0.2, each with 2,000 trajectories of 40 rounds; and a two-attempt ImageNet-100 ResNet-18 run with 49,977 training, 13,769 development, 49,968 confirmation, and 12,804 audit images, using 25,000 paired draws per attempt, α₁ = .025, α₂ = .05/6, and δ = .05.
- Main analysis: trajectory-level FWER, strictly harmful commitments, final and cumulative utility, and evidence-pair costs; post-hoc decomposition of zero-effect versus harmful commitments; and a conditional power diagnostic for the synthetic IID model.
Limitations
- Stated in the paper: the formal result concerns non-improvement in the certified target and does not bound catastrophic tail losses, guarantee lower total compute, ensure progress, or restore the classical Gödel machine’s global future-utility result; monotonicity requires a common utility and faithful state transitions; distribution changes require a new valid sampling argument. The ImageNet-100 run covers one initialization, two attempts, and a restricted proposer, with a certificate for a fixed finite pool and an audit that does not estimate population risk; longer independently repeated trajectories are needed. Proposition 1 further shows that under bounded likelihood ratios, expected total acceptances are finite, so infinitely many acceptances have probability zero, but this is a boundary result for generic online testing, not a self-modification-specific contribution.
- Evident from the text: the synthetic test fixes λ = 0.2, whose expected log-wealth increment is non-positive at gains 0.10 and 0.05, so weak-effect non-acceptance reflects certificate power as well as the global budget; the controlled workload is small and fully enumerated, and cached resampling does not represent repeated expensive solver execution; historical learning trajectories with unresolved certificates remain exploratory.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
18 of 25 cited sources checked · 17 supported · 0 flagged · accuracy not yet measured
- Premisecitation
Conditional Ville's inequality justifies acceptance when E_{k,n} ≥ 1/α_k at any inspected stopping time.
“Conditional Ville's inequality justifies acceptance when E k,n ≥ 1/α k at any inspected stopping time (Howard et al., 2021;Waudby-Smith & Ramdas, 2024).”
Quote in the paperSubstantial claim1 of 2 sources checked- cites Howard et al., 2021 source
Section 3.2 gives a time-uniform crossing bound for a supermartingale-based quantity at a threshold proportional to 1/α, and the discussion around Lemma 3 says sequential tests remain valid at stopping times. The paper does not state a conditional Ville inequality for E_{k,n} or justify the cited acceptance rule; its explicit testing example describes rejecting a null when a boundary is crossed. The indexed passages provide section labels but no page numbers. (LLM, conf=0.80)
Experimental — accuracy not yet measured - cites Waudby-Smith & Ramdas, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Findingown data
A two-attempt ImageNet-100 run demonstrates a certified checkpoint commitment followed by a rejected candidate trained from the adopted state.
“A two-attempt ImageNet-100 run demonstrates a certified checkpoint commitment followed by a rejected candidate trained from the adopted state.”
Quote in the paperSubstantial claim - Findingnone given
A post-hoc decomposition distinguishes reductions in zero-effect commitments from reductions in strictly harmful commitments.
“A post-hoc decomposition distinguishes reductions in zero-effect commitments from reductions in strictly harmful commitments.”
Quote in the paperNot a substantial claim - Findingown data
Both confirmation policies have zero observed errors in the numerical setting at tolerance 10^-3.
“Both confirmation policies have zero observed errors in that numerical setting.”
Quote in the paperNot a substantial claim - Findingown data
At the latter level, the randomized Neyman-Pearson power upper bound for any level-α_10 test using at most 512 observations from this model is 14.575%.
“At the latter level, the randomized Neyman-Pearson power upper bound for any level-α 10 test using at most 512 observations from this model is 14.575%.”
Quote in the paperSubstantial claim
- Findingown data
Across the four synthetic settings, SGM's observed trajectory FWER is 0.15-0.20%, versus 3.35-3.95% for per-test confirmation.
“Across these settings SGM's observed trajectory FWER is 0.15-0.20%,versus 3.35-3.95%for per-test confirmation.”
Quote in the paperSubstantial claim - Findingown data
The improvement costs approximately 8.9 times as many paired evaluations.
“The improvement costs approximately 8.9 times as many paired evaluations.”
Quote in the paperSubstantial claim - Findingown data
Approximately 98.7-100% of the synthetic FWER reduction from per-test confirmation to SGM comes from trajectories with zero-effect errors but no harmful commitment.
“Approximately 98.7-100% of the synthetic FWER reduction from per-test confirmation to SGM comes from trajectories with zeroeffect errors but no harmful commitment.”
Quote in the paperSubstantial claim - Findingown data
At numerical tolerance 10^-3, every false-commit trajectory under screening-only adoption contains a strictly harmful update.
“In contrast, at numerical tolerance 10 -3 , every false-commit trajectory under screening-only adoption contains a strictly harmful update.”
Quote in the paperSubstantial claim - Findingown data
Per-test confirmation has only zero or one harmful trajectory per 2,000 runs in each synthetic setting.
“Per-test confirmation has only zero or one harmful trajectory per 2,000 runs in each setting.”
Quote in the paperSubstantial claim - Findingown data
The FWER difference is driven primarily by zero-effect acceptances.
“The FWER difference is driven primarily by zero-effect acceptances.”
Quote in the paperSubstantial claim - Findingown data
Standard spending and SGM agree exactly in the synthetic settings.
“Standard spending and SGM again agree exactly.”
Quote in the paperNot a substantial claim - Findingown data
The displayed raw-return lower bounds remain below zero up to 19 paired seeds.
“The displayed raw-return lower bounds remain below zero up to 19 paired seeds.”
Quote in the paperNot a substantial claim - Findingown data
The controlled workload demonstrates certificate-driven configuration changes with known utility.
“The controlled workload demonstrates certificate-driven configuration changes with known utility.”
Quote in the paperSubstantial claim - Findingown data
Standard spending and SGM are exactly identical on all decomposed metrics.
“Standard spending and SGM are exactly identical on all decomposed metrics.”
Quote in the paperNot a substantial claim - Findingreasoning
The finite-pool null μ_{k,D} = p_+ - p_- ≤ 0 implies q ≤ 1/2.
“The finite-pool null µ k,D = p + -p -≤ 0 implies q ≤ 1/2.”
Quote in the paperSubstantial claim - Findingreasoning
Under g = 0, the full-sequence count W_512 is Binomial(512, 1/2).
“Under g = 0, the full-sequence count W 512 is Binomial(512, 1/2).”
Quote in the paperSubstantial claim - Findingown data
Including weak-effect-sensitive mixture components does not ensure higher finite-budget power: at g = 0.10 and k = 10, the specified mixture attains 1.698%, below the fixed bet's 4.639%.
“Conversely, including weak-effect-sensitive mixture components does not ensure higher finite-budget power: at g = 0.10 and k = 10, the specified mixture attains 1.698%, below the fixed bet's 4.639%.”
Quote in the paperSubstantial claim - Findingnone given
The upper-tail rejection rule has maximal null rejection probability at g = 0 over g ≤ 0.
“The upper-tail rejection rule also has maximal null rejection probability at g = 0 over g ≤ 0.”
Quote in the paperNot a substantial claim - Findingreasoning
Proposition 1: Under the stated assumptions, the total number of acceptances N_A satisfies E[N_A] ≤ Cδ and P(N_A = ∞) = 0.
“With unopened attempts assigned empty acceptance events, the total number of acceptances N A = k≥1 1 A k satisfies E[N A ] ≤ Cδ, P(N A = ∞) = 0.”
Quote in the paperSubstantial claim- The authors state this as an assumption: For almost every actual pre-attempt history h, a null-reference law Q_{k,h} exists such that the integral of the decision against Q_{k,h} is at most α_k(h).
- The authors state this as an assumption: A deterministic constant C < ∞, uniform over attempts and histories, satisfies the likelihood-ratio domination bound P_{k,h} ≪ Q_{k,h}, dP_{k,h}/dQ_{k,h}(z) ≤ C Q_{k,h}-almost surely.
- Findingreasoning
For any fixed g > 0, its likelihood ratio against g = 0 increases with W_512.
“For any fixed g > 0, its likelihood ratio against g = 0 increases with W 512 .”
Quote in the paperSubstantial claim - Interpretationreasoning
Under conditionally valid confirmation tests and a predictable, summable error budget, the probability of ever accepting a non-improving update is bounded by a user-specified level.
“Under conditionally valid confirmation tests and a predictable, summable error budget, the probability of ever accepting a non-improving update is bounded by a user-specified level.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Equation (1) holds for every opened confirmation.
- The authors state this as an assumption: The predictable allocation is pathwise summable with total at most δ.
- Interpretationnone given
SGM replaces the per-update formal proof requirement with a statistical improvement certificate for a specified evaluation objective.
“We introduce Statistical Gödel Machines (SGM), a framework that replaces the per-update formal proof requirement with a statistical improvement certificate for a specified evaluation objective.”
Quote in the paperSubstantial claim - Interpretationnone given
A certificate supplies a conditionally valid acceptance event: 1_{H_k} P(A_k | G_k) ≤ 1_{H_k} α_k almost surely.
“A certificate supplies a conditionally valid acceptance event: 1 H k P(A k | G k ) ≤ 1 H k α k almost surely.”
Quote in the paperSubstantial claim - Interpretationreasoning
Theorem 1: If Equation (1) holds for every opened confirmation and the predictable allocation is pathwise summable with total at most δ, then, for an adaptive sequence containing any mixture of null and improving candidates, P(∪_{k≥1} (H_k ∩ A_k)) ≤ δ.
“Theorem 1 (Trajectory-level false-commit control).If Equation (1) holds for every opened confirmation and the predictable allocation is pathwise summable with total at most δ, then, for an adaptive sequence containing any mixture of null and improving candidates,”
Quote in the paperSubstantial claim- The authors state this as an assumption: Equation (1) holds for every opened confirmation.
- The authors state this as an assumption: The predictable allocation is pathwise summable with total at most δ.
- Interpretationreasoning
Certifying a particular trained checkpoint does not certify its training rule across future random executions.
“Certifying a particular trained checkpoint therefore does not certify its training rule across future random executions.”
Quote in the paperSubstantial claim - Interpretationreasoning
SGM does not establish the original Gödel machine's full future-utility comparison or global optimality.
“It does not establish the original machine's full future-utility comparison or global optimality.”
Quote in the paperSubstantial claim - Premisecitation
High-confidence policy improvement provides probabilistic guarantees for policy updates and repeated improvement.
“High-confidence policy improvement provides probabilistic guarantees for policy updates and repeated improvement (Thomas et al., 2015).”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Thomas et al., 2015
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
If the score, evaluation distribution, or system state changes outside the certified transition, Corollary 1's premises must be reconsidered.
“If the score, evaluation distribution, or system state changes outside the certified transition, its premises must be reconsidered.”
Quote in the paperSubstantial claim - Interpretationreasoning
The practical difficulty of Gödel machines is obtaining the required model and proof for changes to complex learning systems, not an incompatibility between formal reasoning and stochastic environments.
“Its practical difficulty is obtaining the required model and proof for changes to complex learning systems, not an incompatibility between formal reasoning and stochastic environments.”
Quote in the paperSubstantial claim - Interpretationreasoning
The process E_{k,n} = sum_j w_j prod_i (1 + λ_j X_{k,i}) is a nonnegative supermartingale under the conditional null, starting at one.
“Then E k,n = j w j n i=1 (1 + λ j X k,i ) is a nonnegative supermartingale under the conditional null, starting at one.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Under the null, E[X_{k,i} | F_{k,i-1}] ≤ 0 on H_k.
- Interpretationown data
Zero events in 2,000 trajectories has a Wilson upper limit of approximately 0.192%, not zero population risk.
“Zero events in 2,000 trajectories has a Wilson upper limit of approximately 0.192%, not zero population risk.”
Quote in the paperSubstantial claim - Interpretationreasoning
After freezing the candidate, new independent uniform index draws from a fixed pool yield conditionally IID outcomes even when items have appeared before.
“After freezing the candidate, new independent uniform index draws from a fixed pool yield conditionally IID outcomes even when items have appeared before.”
Quote in the paperSubstantial claim - Interpretationreasoning
For n prespecified IID paired correctness observations, conditional on M=W+L, the null of non-positive mean difference implies W ~ Binomial(M, q) with q ≤ 1/2.
“For n prespecified IID paired correctness observations, let W and L count candidate-only and incumbent-only correct predictions.Conditional on M = W + L, the null of non-positive mean difference implies W ∼ Binomial(M, q) with q ≤ 1/2.”
Quote in the paperSubstantial claim - Interpretationreasoning
Controlling global error does not ensure affordable evidence.
“Conversely, controlling global error does not ensure affordable evidence.”
Quote in the paperSubstantial claim - Interpretationnone given
Repeatedly checking the fixed-sample bound needs an additional correction or a confidence sequence.
“Repeatedly checking this bound needs an additional correction or a confidence sequence.”
Quote in the paperSubstantial claim - Interpretationreasoning
Merely increasing the sample size after an unsuccessful test does not establish validity.
“Merely increasing the sample size after an unsuccessful test does not establish validity.”
Quote in the paperSubstantial claim - Study design and conductnone given
The experiments match rounds and per-attempt caps, not realized total expenditure.
“The experiments match rounds and per-attempt caps, not realized total expenditure.”
Quote in the paperSubstantial claim - Interpretationnone given
A fixed-sample alternative for X ∈ [-1,1] is L_n = Xn - sqrt(2 log(1/α_k)/n) at a sample size fixed before confirmation.
“A fixed-sample alternative for X ∈ [-1, 1] is L n = Xn -2 log(1/α k )/n at a sample size fixed before confirmation.”
Quote in the paperNot a substantial claim - Study design and conductnone given
A screen-selected fraction is allowed if frozen before a valid confirmation stream.
“A screen-selected fraction is also allowed if frozen before a valid confirmation stream.”
Quote in the paperNot a substantial claim - Interpretationreasoning
SGM supports self-modification within this boundary, rather than unrestricted rewriting of its own verifier.
“SGM supports self-modification within this boundary, rather than unrestricted rewriting of its own verifier.”
Quote in the paperSubstantial claim - Interpretationreasoning
The meaning of a lower FWER depends on the workload and comparator.
“Thus the meaning of a lower FWER depends on the workload and comparator.”
Quote in the paperSubstantial claim - Interpretationown data
Lower FWER here does not establish a substantial reduction in performance losses.
“Thus lower FWER here does not establish a substantial reduction in performance losses.”
Quote in the paperSubstantial claim - Interpretationreasoning
A nonzero per-attempt error rate does not imply eventual failure with probability one: positive summable levels provide an immediate counterexample.
“A nonzero per-attempt error rate does not imply eventual failure with probability one: positive summable levels provide an immediate counterexample.”
Quote in the paperSubstantial claim - Interpretationnone given
Sampling targets this finite pool's accuracy, not population generalization.
“Sampling targets this finite pool's accuracy, not population generalization.”
Quote in the paperSubstantial claim - Interpretationreasoning
Bounded sample counts alone do not imply the likelihood-ratio condition.
“Bounded sample counts alone do not imply the likelihood-ratio condition.”
Quote in the paperNot a substantial claim - Interpretationnone given
SGM changes the evidence required to authorize a modification.
“SGM changes the evidence required to authorize a modification.”
Quote in the paperSubstantial claim - Interpretationreasoning
Engineering checks support execution conditions without establishing a separate statistical contribution.
“Engineering checks support execution conditions without establishing a separate statistical contribution.”
Quote in the paperNot a substantial claim - Interpretationreasoning
A narrower limitation holds for per-attempt certification with pathwise summable levels and a uniform finite bound on the likelihood ratio of each fresh evidence transcript relative to a valid null reference.
“A narrower limitation holds for per-attempt certification with pathwise summable levels and a uniform finite bound on the likelihood ratio of each fresh evidence transcript relative to a valid null reference.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: There is a uniform finite bound on the likelihood ratio of each fresh evidence transcript relative to a valid null reference.
- Interpretationreasoning
Monotonicity additionally requires a common utility and faithful state transitions.
“Monotonicity additionally requires a common utility and faithful state transitions.”
Quote in the paperSubstantial claim - Interpretationnone given
Lifetime false-commit control does not itself imply eventual stagnation.
“Lifetime false-commit control does not itself imply eventual stagnation.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The randomized Neyman-Pearson test allocates rejection probability to the largest counts, randomizing at the boundary to attain size α.
“The randomized Neyman-Pearson test therefore allocates rejection probability to the largest counts, randomizing at the boundary to attain size α.”
Quote in the paperSubstantial claim - Interpretationreasoning
A procedure controlling only FDR need not provide the stated lifetime guarantee.
“A procedure controlling only FDR need not provide the stated lifetime guarantee.”
Quote in the paperSubstantial claim - Interpretationreasoning
The proposition does not imply that every lifetime-safe learning system must stop improving.
“Consequently, the proposition does not imply that every lifetime-safe learning system must stop improving.”
Quote in the paperNot a substantial claim - Interpretationnone given
The null-reference condition at improving histories is an additional property of the testing family, not a consequence of Equation 1 alone.
“The null-reference condition at improving histories is an additional property of the testing family, not a consequence of Equation 1 alone, whose restriction is active only at actual nulls.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Cross-attempt independence of the selected models is unnecessary.
“Cross-attempt independence of the selected models is unnecessary.”
Quote in the paperSubstantial claim - Interpretationreasoning
Previously revealed values of d_{k,j}, or of earlier candidates' scores, do not change this factorization when the fresh-index assumption holds.
“Previously revealed values of d k,j , or of earlier candidates' scores, do not change this factorization when the fresh-index assumption holds.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The new indices I_{k,1}, ..., I_{k,n} are IID uniform on {1, ..., m} conditional on H_k.
- Interpretationreasoning
Neither supports acceptance under those assumptions.
“Neither supports acceptance under those assumptions.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The historical screening-cost calculation uses a 12-pair assumption.
- Interpretationreasoning
Learning the finite pool through repeated feedback may improve that pool's score without improving external generalization.
“In particular, learning the finite pool through repeated feedback may improve that pool's score without improving external generalization.”
Quote in the paperSubstantial claim - Interpretationreasoning
If each acceptance authorizes at most one committed update, only finitely many commits occur almost surely in an infinite extension of this regime.
“If each acceptance authorizes at most one committed update, only finitely many commits occur almost surely in an infinite extension of this regime.”
Quote in the paperSubstantial claim - Interpretationreasoning
Independence of screening and confirmation implies that proposal-level acceptance equals this probability times conditional confirmation power.
“Independence of screening and confirmation implies that proposal-level acceptance equals this probability times conditional confirmation power.”
Quote in the paperNot a substantial claim - Interpretationreasoning
For N = 512, the bound 2^512 δ is far too large to predict the number of practical accepts or a stopping round.
“For N = 512, the bound 2 512 δ is far too large to predict the number of practical accepts or a stopping round.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Any such rule with conditional null-reference size at most α_k satisfies the proposition with C = 2^N.
“Thus any such rule with conditional null-reference size at most α k satisfies the proposition with C = 2 N .”
Quote in the paperSubstantial claim- The authors state this as an assumption: Every opened attempt uses at most a fixed N fresh IID observations in {-1, +1}, with conditional mean g_k ∈ [-1, 1].
- The authors state this as an assumption: For a fixed history, the null reference has mean zero, with the same auxiliary randomization as the actual test.
- Interpretationnone given
Nonpositive log drift does not preclude finite-time threshold crossing.
“Nonpositive log drift does not preclude finitetime threshold crossing.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Lifetime FWER control does not require every possible procedure to use pathwise summable per-attempt levels.
“Lifetime FWER control does not require every possible procedure to use pathwise summable per-attempt levels.”
Quote in the paperSubstantial claim - Interpretationreasoning
This establishes the acceptance bound conditional on H_k.
“This establishes the acceptance bound conditional on H k .”
Quote in the paperSubstantial claim - Interpretationnone given
For a specified objective, conditionally valid certificates and a summable budget control the probability of any non-improving commitment across adaptive updates.
“For a specified objective, conditionally valid certificates and a summable budget control the probability of any non-improving commitment across adaptive updates.”
Quote in the paperSubstantial claim - Interpretationcitation
HCPI includes an incremental algorithm, so the distinction cannot be single-step versus repeated improvement.
“HCPI includes an incremental algorithm, so the distinction cannot be single-step versus repeated improvement.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites Thomas et al., 2015
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
Under additional fixed-objective and state-transition conditions, this implies a high-probability non-decreasing incumbent sequence.
“Under additional fixed-objective and state-transition conditions, this implies a high-probability non-decreasing incumbent sequence.”
Quote in the paperNot a substantial claim
- Classified as probably not a substantial claim. 191
- May combine two claims. 158
- The quote only partly states this claim. 1
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationHyperband allocates resources through early stopping, and BOHB combines model-guided search with bandit resource allocation.
“Hyperband allocates resources through early stopping (Li et al., 2018), and BOHB combines model-guided search with bandit resource allocation (Falkner et al., 2018).”
Quote in the paperNot a substantial claim1 of 2 sources checked- cites Li et al., 2018
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Falkner et al., 2018 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSEA combines anytime-valid gates and a fixed error budget with a broader agent architecture.
“SEA (Sengupta, 2026), released in July 2026, combines anytime-valid gates and a fixed error budget with a broader agent architecture.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Sengupta, 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationPopulation Based Training adapts parameters and hyperparameters during training.
“Population Based Training adapts parameters and hyperparameters during training (Jaderberg et al., 2017).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Jaderberg et al., 2017 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSafe Harness analyzes modification generation, finite-data certification and selection, safe adoption, and opportunities after an update.
“Safe Harness (Cai et al., 2026), released in September 2026, analyzes modification generation, finite-data certification and selection, safe adoption, and opportunities after an update.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Cai et al., 2026 source
Classified as probably not a substantial claim.
PremisecitationThe Gödel machine executes a self-modification when a formal proof establishes its usefulness relative to continuing the current process, including its proof search.
“The Gödel machine gives a principled answer: execute a self-modification when a formal proof establishes its usefulness relative to continuing the current process, including its proof search (Schmidhuber, 2003).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Schmidhuber, 2003 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSEA distinguishes established statistical components from their endogenous, distribution-shifting composition, whose safety it leaves unproved.
“SEA distinguishes established statistical components from their endogenous, distribution-shifting composition, whose safety it leaves unproved.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Sengupta, 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationGödel machines motivate proof-justified self-modification; Darwin Gödel Machines use empirical evaluation to guide code-modifying search.
“Gödel machines motivate proof-justified self-modification (Schmidhuber, 2003); Darwin Gödel Machines use empirical evaluation to guide code-modifying search (Zhang et al., 2025).”
Quote in the paperNot a substantial claim2 of 2 sources checkedMay combine two claims.
PremisecitationOnline family-wise error control handles potentially unbounded hypothesis sequences; confidence sequences and betting methods support optional stopping.
“Online family-wise error control handles potentially unbounded hypothesis sequences (Tian & Ramdas, 2019); confidence sequences and betting methods support optional stopping (Howard et al., 2021;Waudby-Smith & Ramdas, 2024).”
Quote in the paperSubstantial claim1 of 3 sources checked- cites Tian & Ramdas, 2019 source
- cites Howard et al., 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites Waudby-Smith & Ramdas, 2024
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSafe Harness derives finite-run guarantees and separates generation from certification bottlenecks.
“It derives finite-run guarantees and separates generation from certification bottlenecks.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Cai et al., 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFeng et al. (2021) formulate automatic algorithmic change protocols for evolving medical prediction models, using prospective data and comparisons to approved models.
“Feng et al. (2021) formulate automatic algorithmic change protocols for evolving medical prediction models, using prospective data and comparisons to approved models.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Feng et al. (2021) source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFeng et al. identify declining approval power from alpha-wealth depletion.
“They also identify declining approval power from alpha-wealth depletion.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Feng et al. (2021) source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationFeng et al. (2021) study noninferiority-induced deterioration (bio-creep), beneficial approvals, and online bad-approval counts and ratios.
“They study noninferiority-induced deterioration (bio-creep), beneficial approvals, and online bad-approval counts and ratios.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Feng et al. (2021) source
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationSEA's SGM-CS controller explicitly builds on SGM and reuses its confirm-triggered spending.
“Its SGM-CS controller explicitly builds on SGM and reuses its confirm-triggered spending.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Sengupta, 2026 source
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationFeng et al.'s infinite-window count guarantee bounds the probability of any bad approval for their error definition, not merely a per-update guarantee.
“Their infinite-window count guarantee bounds the probability of any bad approval for their error definition; it is not merely a per-update guarantee.”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Feng et al. (2021) source
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationcitationIt sharpens the connection to the alpha-death discussion of Feng et al. (2021).
“It sharpens the connection to the alpha-death discussion of Feng et al. (2021).”
Quote in the paperNot a substantial claim1 of 1 sources checked- cites Feng et al. (2021) source
May combine two claims.
InterpretationreasoningRepeated approval under that target does not imply the monotonicity corollary and can accumulate losses.
“Repeated approval under that target does not imply the monotonicity corollary and can accumulate losses, motivating comparisons with approved benchmarks as in Feng et al. (2021).”
Quote in the paperSubstantial claim1 of 1 sources checked- cites Feng et al. (2021) source
Classified as probably not a substantial claim.
Findingown dataControlled numerical and synthetic experiments quantify erroneous commitments, retained improvements, and verification cost.
“Controlled numerical and synthetic experiments quantify erroneous commitments, retained improvements, and verification cost.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Findingnone givenThe controlled numerical workload provides known utility differences and complete state transitions.
“A controlled numerical workload provides known utility differences and complete state transitions.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAt 10^-4, screening-only adoption has no observed errors and the same final utility as SGM, which still costs more.
“At 10 -4 , screening-only adoption has no observed errors and the same final utility as SGM, which still costs more.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataSGM has lower final and cumulative utility and greater verification cost than per-test confirmation in all four synthetic comparisons.
“However, SGM has lower final and cumulative utility and greater verification cost in all four comparisons.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAt gain g = 0.10, the original test has power 28.738% at α = .05, but only 4.639% at α_10 = .05/110, with unchanged inspection times and a 512-sample cap.
“At gain g = 0.10, the original test has power 28.738% at α = .05,but only 4.639% at α 10 = .05/110,with unchanged inspection times and a 512-sample cap.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataPer-test confirmation also has zero observed errors and slightly higher final utility at lower cost.
“Per-test confirmation also has zero observed errors and slightly higher final utility at lower cost.”
Quote in the paperNot a substantial claimMay combine two claims. The quote only partly states this claim.
Findingown dataAt tolerance 10^-3, SGM has zero observed false-commit trajectories compared with 76.50% for screening-only adoption, and final utility rises from 75.82% to 77.66%.
“At tolerance 10 -3 , SGM has zero observed false-commit trajectories compared with 76.50% for screening-only adoption, and final utility rises from 75.82% to 77.66%.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
FindingreasoningThe full table supplies exact utility differences for diagnostic evaluation, while each decision sees sampled paired correctness differences in {-1,0,1}.
“Thus the full table supplies exact utility differences for diagnostic evaluation, while each decision sees sampled paired correctness differences in {-1, 0, 1}.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataSGM makes no strictly harmful commitments in these synthetic runs; per-test confirmation averages zero or 0.0005 strictly harmful commitments per trajectory, depending on the setting.
“SGM makes no strictly harmful commitments in these runs; per-test confirmation averages zero or 0.0005 strictly harmful commitments per trajectory, depending on the setting.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataPer-test FWER does not empirically exceed 5% in these settings; this observation does not supply it with a general trajectory guarantee.
“Per-test FWER does not empirically exceed 5% in these settings; this observation does not supply it with a general trajectory guarantee.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenThe synthetic test fixes betting fraction λ = 0.2, whose expected log-wealth increment is non-positive at gains 0.10 and 0.05.
“Moreover, the synthetic test fixes betting fraction λ = 0.2, whose expected log-wealth increment is non-positive at gains 0.10 and 0.05.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAt strong effect 0.25 and path strength 0, SGM accepts 5.029 beneficial updates per trajectory versus 5.358 for per-test confirmation, using 7,135 versus 5,960 evidence pairs.
“For example, at strong effect 0.25 and path strength 0, SGM accepts 5.029 beneficial updates per trajectory versus 5.358 for per-test confirmation, using 7,135 versus 5,960 evidence pairs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe ImageNet-100 record reports a 40-proposal trajectory with adopted changes at rounds 6 and 21.
“The ImageNet-100 record reports a 40-proposal trajectory with adopted changes at rounds 6 and 21 (Figure 2).”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone givenThe numerical workload at tolerance 10^-3 has 76.5% harmful trajectories under screening-only adoption and zero observed errors under either confirmation policy.
“The numerical workload at tolerance 10 -3 has 76.5% harmful trajectories under screening-only adoption and zero observed errors under either confirmation policy.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingnone givenThe attempt-1 binomial tail underflows in ordinary floating-point evaluation; its nonzero value 2.004275 × 10^-427 is recomputed with high-precision arithmetic, without modifying the original raw result.
“The attempt-1 binomial tail underflows in ordinary floating-point evaluation; its nonzero value 2.004275 × 10 -427 is recomputed with highprecision arithmetic, without modifying the original raw result.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThis is one accepted update followed by a rejected candidate generated from the adopted state.
“This is one accepted update followed by a rejected candidate generated from the adopted state.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe historical CIFAR-10 paired means are positive, but the corresponding displayed bounds do not certify improvement.
“The historical CIFAR-10 paired means are positive, but the corresponding displayed bounds do not certify improvement.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Findingown dataFigure 6 contrasts positive paired effects with conservative raw-return bounds; neither displayed bound crosses zero over the plotted sample sizes.
“Figure 6 contrasts positive paired effects with conservative raw-return bounds; neither displayed bound crosses zero over the plotted sample sizes.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingnone givenP2 adds an actual learning-state transition and a subsequent decision, but only for one initialization, two attempts, and a restricted proposer.
“P2 adds an actual learning-state transition and a subsequent decision, but only for one initialization, two attempts, and a restricted proposer.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe rejected candidate scores 77.39%, only 0.039 points above its parent.
“The rejected candidate scores 77.39%,only 0.039 points above its parent.”
Quote in the paperNot a substantial claimMay combine two claims.
FindingreasoningThe terminal incumbent remains the first committed model, with consumed allocation 1/30.
“The terminal incumbent remains the first committed model, with consumed allocation 1/30.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe run demonstrates acceptance of an improvement relative to a simple parent and refusal to commit on insufficient evidence despite a positive development gain.
“The run demonstrates acceptance of an improvement relative to a simple parent and refusal to commit on insufficient evidence despite a positive development gain.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataThe initial and committed models score 65.36% and 77.35%, respectively (+11.988 points).
“The initial and committed models score 65.36% and 77.35%, respectively (+11.988 points).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataAt a = 0.25, b = 0, the paired SGM-minus-per-test differences are -0.3285 positive accepts (descriptive 95% normal interval [-0.3549, -0.3021]), -0.048875 final utility ([-0.053535,-0.044215]), and +1174.56 evidence pairs ([1145.10,1204.02]).
“At a = 0.25, b = 0, the paired SGM-minus-per-test differences are -0.3285positive accepts (descriptive 95% normal interval [-0.3549, -0.3021]), -0.048875 final utility ([-0.053535,-0.044215]), and +1174.56evidence pairs ([1145.10,1204.02]).”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataAll seven outcomes and measured costs appear in Appendix A: 210 training epochs take 4.09 hours of logged training/development time on one RTX 4090D, confirmation inference takes 31.43/31.88 seconds, and the final audit takes 115.75 seconds.
“All seven outcomes and measured costs appear in Appendix A: 210 training epochs take 4.09 hours of logged training/development time on one RTX 4090D, confirmation inference takes 31.43/31.88seconds, and the final audit takes 115.75 seconds.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataAttempt 1 accepts the 0.01 branch; a separate state transition installs its full checkpoint as the incumbent.
“Attempt 1 accepts the 0.01 branch (Table 3); a separate state transition installs its full checkpoint as the incumbent.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Findingown dataAll displayed lower bounds are negative, so neither case is certified by these bounds.
“All displayed lower bounds are negative, so neither case is certified by these bounds.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataRound 2 selects the 0.001 branch, but confirmation does not pass.
“Round 2 selects the 0.001 branch, but confirmation does not pass.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenThe disjoint data roles contain 49,977 training, 13,769 development, 49,968 confirmation, and 12,804 audit images after duplicate screening and quarantine.
“The disjoint data roles contain 49,977 training, 13,769 development, 49,968 confirmation, and 12,804 audit images after duplicate screening and quarantine.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenFor the IID two-point diagnostic at g = .10 and k = 10, original power is 4.639%, the specified mixture yields 1.698%, and the 512-sample upper bound is 14.575%.
“For the IID two-point diagnostic at g = .10and k = 10, original power is 4.639%, the specified mixture yields 1.698%, and the 512-sample upper bound is 14.575%.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
FindingreasoningAt g = 0.10, the four-observation screen passes with probability Pr{Binomial(4, .55)≥ 3} = 4(.55)^3(.45)+ (.55)^4 = 0.39098125.
“At g = 0.10, the four-observation screen passes with probability Pr{Binomial(4, .55)≥ 3} = 4(.55) 3 (.45)+ (.55) 4 = 0.39098125.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Findingnone givenThe historical screening-cost calculation is 2136/12960 = 0.1648.
“The historical screening-cost calculation is 2136/12960 = 0.1648.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
FindingreasoningAn image can recur within or across attempts.
“An image can recur within or across attempts.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataThe fixed-sample Hoeffding lower bound is -0.337/-0.255 under α_2 = 0.1/(2H_40), or -0.309/-0.229 under inverse-square spending.
“The fixed-sample Hoeffding lower bound is -0.337/ -0.255 under α 2 = 0.1/(2H 40 ), or -0.309/ -0.229 under inverse-square spending.”
Quote in the paperNot a substantial claim- The authors state this as an assumption: The historical screening-cost calculation uses a 12-pair assumption.
Classified as probably not a substantial claim.
FindingreasoningFor measurable sets B_1,...,B_n, the joint law factors as the product of empirical averages.
“For measurable sets B 1 , . . ., B n , the joint law factors as P n i=1 {X k,i ∈ B i } | H k = n i=1 1 m m j=1 1{d k,j ∈ B i }.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The new indices I_{k,1}, ..., I_{k,n} are IID uniform on {1, ..., m} conditional on H_k.
Classified as probably not a substantial claim.
FindingreasoningConditional on H_k, the vector d_{k,j} = r(c_k,z_j) - r(s_k,z_j), j=1,...,m, is fixed, including entries the algorithm has not computed.
“Conditional on H k , the vector d k,j = r(c k , z j ) -r(s k , z j ), j = 1, . . ., m, is fixed, including entries the algorithm has not computed.”
Quote in the paperSubstantial claimMay combine two claims.
Findingnone givenFor the fixed bet, proposal-level acceptance is 11.236% at α = .05 and 1.814% at k = 10.
“For the fixed bet, it is 11.236% at α = .05and 1.814% at k = 10.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
FindingreasoningGiven M = W+L > 0, W | (M,H_k) ∼ Binomial(M,q), q = p_+/(p_+ + p_-).
“Given M = W + L > 0, W | (M, H k ) ∼ Binomial(M, q), q = p + p + + p - .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingnone givenIts 120 training epochs are separate from P2's 210 epochs.
“Its 120 training epochs are separate from P2's 210 epochs.”
Quote in the paperNot a substantial claimMay combine two claims.
Findingown dataFor CIFAR-100, a matched 30-pair record yields a mean gain of 1.538 percentage points; the earlier 5.51-point difference uses a different baseline comparison and is not this paired effect.
“For CIFAR-100, a matched 30-pair record yields a mean gain of 1.538 percentage points; the earlier 5.51-point difference uses a different baseline comparison and is not this paired effect.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataThe displayed Hoeffding and empirical-Bernstein lower bounds for P5 remain negative over the plotted sample sizes.
“Displayed Hoeffding and empirical-Bernstein lower bounds for P5 remain negative over the plotted sample sizes.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenThe learning run establishes an executable accept-commit-continue-reject chain.
“The learning run establishes an executable accept-commit-continue-reject chain.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataP2 is a revised development design, chosen after an earlier r2 experiment using a scheduled-learning-rate baseline and three ten-epoch candidates produced no positive development gain.
“P2 is a revised development design, chosen after an earlier r2 experiment using a scheduled-learning-rate baseline and three ten-epoch candidates produced no positive development gain.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataAttempts 1 and 2 contain 19,661 and 19,624 distinct images respectively; all 25,000 draws per attempt enter the fixed-sample test.
“Attempts 1 and 2 contain 19,661 and 19,624 distinct images respectively; all 25,000 draws per attempt enter the fixed-sample test.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Findingown dataThe displayed raw-return lower bounds remain below zero up to 19 paired seeds. This graphic therefore does not demonstrate certification by these bounds; any separately reported acceptance requires its own identified estimand and test.
“The displayed raw-return lower bounds remain below zero up to 19 paired seeds.This graphic therefore does not demonstrate certification by these bounds; any separately reported acceptance requires its own identified estimand and test.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Findingown dataAll three curves rise near one at iteration 6. Point estimates alone do not establish acceptance; the corresponding confirmation bounds and decision records are required.
“All three curves rise near one at iteration 6. Point estimates alone do not establish acceptance; the corresponding confirmation bounds and decision records are required.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe short finite-pool ImageNet-100 demonstration establishes executable adoption, not long-horizon error calibration; historical learning trajectories remain exploratory.
“This short finite-pool demonstration establishes executable adoption, not long-horizon error calibration; historical learning trajectories remain exploratory.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningFor a fixed utility and faithful state transitions, SGM gives a high-probability non-decreasing incumbent-utility sequence, without guaranteeing that improvements will be found.
“For a fixed utility and faithful state transitions, this also gives a high-probability non-decreasing incumbent-utility sequence, without guaranteeing that improvements will be found.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationnone givenSGM separates adaptive proposal generation from confirmation and permits a modification only when its evidence satisfies a trajectory-level risk allocation.
“SGM separates adaptive proposal generation from confirmation and permits a modification only when its evidence satisfies a trajectory-level risk allocation.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenSGM is independent of the proposer: it does not improve the search algorithm by itself.
“SGM is independent of the proposer: it does not improve the search algorithm by itself.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationThe October 2025 SGM preprint proposed statistical improvement certificates and a global error budget for recursive edits.
“The October 2025 SGM preprint (Wu et al., 2025) proposed statistical improvement certificates and a global error budget for recursive edits.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Wu et al., 2025
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThese scope choices do not establish a new statistical primitive or superiority over Feng et al.'s methods.
“These scope choices do not establish a new statistical primitive or superiority over their methods.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenThe SGM certificate tolerates a controlled probability of error and is conditional on a sampling model.
“The certificate tolerates a controlled probability of error and is conditional on a sampling model.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenThe observation unit must match the object being adopted.
“The observation unit must match the object being adopted (Table 1).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
InterpretationreasoningThis focus complements SEA's broader architecture and Safe Harness's feasibility analysis; it does not establish superiority over either framework or guarantee future improvement opportunities.
“This focus complements SEA's broader architecture and Safe Harness's feasibility analysis; it does not establish superiority over either framework or guarantee future improvement opportunities.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe true-null event H_k is G_k-measurable: the selected states determine their utility difference even though it is unknown to the algorithm.
“The true-null event H k is G k -measurable: the selected states determine their utility difference even though it is unknown to the algorithm.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenSGM specifies strict improvement in a declared scalar utility against the actual incumbent and an explicit certificate-to-state execution protocol, with checkpoint and training-procedure targets distinguished.
“SGM instead specifies strict improvement in a declared scalar utility against the actual incumbent and an explicit certificate-to-state execution protocol, with checkpoint and training-procedure targets distinguished.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThe adoption decision for a frozen candidate requires fresh confirmation before installing it and before subsequent proposals are trained from that adopted state.
“The adoption decision concerns the resulting frozen candidate: fresh confirmation must justify installing it before subsequent proposals”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenFor a frozen checkpoint, randomness can come from sampled evaluation examples.
“For a frozen checkpoint, randomness can come from sampled evaluation examples.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningCorollary 1 guarantees neither an acceptance nor a globally optimal state.
“This corollary guarantees neither an acceptance nor a globally optimal state.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningFor a frozen training procedure, a paired observation compares scores from fresh random executions under fixed data and execution rules.
“For a frozen training procedure, a paired observation instead compares scores from fresh random executions under fixed data and execution rules.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningReplacing the training procedure for future random executions is a different decision from adopting a frozen candidate, requiring evidence about the procedure's expected utility.
“Replacing the training procedure for future random executions is a different decision, requiring evidence about the procedure's expected utility.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationown dataSGM provides a practical protocol for evidence-based self-improvement, while exposing the trade-off between erroneous commitments, missed improvements, and verification cost.
“SGM provides a practical protocol for evidence-based self-improvement, while exposing the trade-off between erroneous commitments, missed improvements, and verification cost.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningComparisons require matching the target, evidence access, and total error budget; the paper claims no empirical superiority over HCPI, SEA, or Safe Harness.
“Comparisons require matching the target, evidence access, and total error budget; we claim no empirical superiority over HCPI, SEA, or Safe Harness.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenAdopting a candidate also requires that the incumbent and candidate actually match the objects to which the certificate refers.
“Adopting a candidate also requires that the incumbent and candidate actually match the objects to which the certificate refers.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenConditional IID sampling from P, after fixing the candidate, is a sufficient model for fresh paired observations.
“Conditional IID sampling from P , after fixing the candidate, is a sufficient model.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningSGM studies a statistical relaxation of the proof obligation: a candidate is evaluated against the incumbent on a specified objective, and acceptance requires a statistical certificate.
“SGM studies a statistical relaxation of this proof obligation.A candidate is evaluated against the incumbent on a specified objective, and acceptance requires a statistical certificate.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenAlthough a checkpoint can often be restored, resources already spent and opportunities encountered on a different search path may not be recovered.
“Although a checkpoint can often be restored, resources already spent and opportunities encountered on a different search path may not be recovered.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenPersistent state changes do not invalidate these foundations, and the trajectory bound is a conditional composition of them.
“Persistent state changes do not invalidate these foundations, and the trajectory bound is a conditional composition of them.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Study design and conductnone givenThe chosen object, distribution of random inputs, and scoring rule must be fixed before confirmation.
“The chosen object, distribution of random inputs, and scoring rule must be fixed before confirmation.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningA procedure certificate concerns expected utility over fresh random executions, not every resulting checkpoint.
“A procedure certificate concerns expected utility over those executions, not every resulting checkpoint.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningAccepted updates can change subsequent proposals, training states, and resource use.
“Accepted updates can change subsequent proposals, training states, and resource use.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningCorollary 1: Suppose the only state changes are s_{k+1}=c_k after acceptance and s_{k+1}=s_k otherwise, and the same U is used throughout. With probability at least 1-δ, every accepted change strictly increases U and the incumbent-utility sequence is non-decreasing.
“Corollary 1 (Fixed-objective self-improvement).Suppose the only state changes are s k+1 = c k after acceptance and s k+1 = s k otherwise, and the same U is used throughout.With probability at least 1 -δ, every accepted change strictly increases U and the incumbent-utility sequence is non-decreasing.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenThe target may concern training-seed performance on a fixed dataset; this must not be confused with a fixed model's performance on new examples.
“The target may instead concern trainingseed performance on a fixed dataset; this must not be confused with a fixed model's performance on new examples.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThese consequences motivate controlling erroneous commitments across the trajectory.
“These consequences motivate controlling erroneous commitments across the trajectory.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenMore general sampling is possible only with a justified conditional-validity argument.
“More general sampling is possible only with a justified conditional-validity argument.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe present treatment makes the declared utility, candidate-selection history, conditional certificate validity, and actual adopted state explicit.
“The present treatment makes the declared utility, candidate-selection history, conditional certificate validity, and actual adopted state explicit.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenConfirm-Triggered Harmonic Spending (CTHS) is an optional allocation instance, rather than a prerequisite for the framework.
“Confirm-Triggered Harmonic Spending (CTHS) is an optional allocation instance, rather than a prerequisite for the framework.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningCorollary 1 does not compare the entire future search process with the counterfactual process following a different decision.
“It does not compare the entire future search process with the counterfactual process following a different decision.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductnone givenα_k is chosen before confirmation observations, and sum α_k ≤ δ almost surely.
“Here α k is chosen before confirmation observations, and k α k ≤ δ almost surely.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenThese methods can supply candidates to SGM; adoption additionally requires confirmation that remains conditionally valid after adaptive selection.
“These methods can supply candidates to SGM; adoption additionally requires confirmation that remains conditionally valid after adaptive selection.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
InterpretationreasoningRejecting every proposal trivially avoids false commitments, so risk must be evaluated together with acceptance power and verification cost.
“Rejecting every proposal trivially avoids false commitments, so risk must be evaluated together with acceptance power and verification cost.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThe historical multi-seed experiments illustrate the procedure-level evaluation setting, but their unresolved certificates are not evidence of a validated procedure-level commitment.
“The historical multi-seed experiments illustrate the latter evaluation setting, but their unresolved certificates are not evidence of a validated procedure-level commitment.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenSGM organizes these established tools into an adoption protocol.
“SGM organizes these established tools into an adoption protocol.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationnone givenThe present treatment gives a trajectory guarantee under predictable, summable error allocation and studies risk, retained improvement, and evidence cost in controlled trajectories.
“It gives a trajectory guarantee under predictable, summable error allocation and studies risk, retained improvement, and evidence cost in controlled trajectories.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenThe probabilistic argument of SGM composes established inference tools.
“The probabilistic argument composes established inference tools.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThus p = Pr{Binomial(M, 1/2) ≥ W} is a valid conditional p-value, with p=1 if M=0.
“Thus p = Pr{Binomial(M, 1/2) ≥ W } is a valid conditional p-value, with p = 1 if M = 0.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataControlling non-improving commitments can sacrifice useful updates, and these results do not establish a universal utility benefit.
“Controlling non-improving commitments can sacrifice useful updates, and these results do not establish a universal utility benefit.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThis test is evaluated once at the fixed n; it is not an anytime-valid test.
“This test is evaluated once at the fixed n; it is not an anytime-valid test.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenThe evaluator, confirmation-data access, certificate implementation, and persistent budget accounting form a protected verification layer: candidate changes cannot bypass or rewrite them.
“The evaluator, confirmation-data access, certificate implementation, and persistent budget accounting form a protected verification layer: candidate changes cannot bypass or rewrite them.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThis is a sufficient bound for this test, not a universal sample-complexity lower bound.
“This is a sufficient bound for this test, not a universal sample-complexity lower bound.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThese post-hoc calculations are not new trajectory experiments or a causal decomposition of terminal utility differences.
“These post-hoc calculations are not new trajectory experiments or a causal decomposition of terminal utility differences; Appendix E.1 provides the calculation and additional diagnostics.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe modifiable state comprises the candidate model, training procedure, or task code evaluated by the protocol.
“The modifiable state comprises the candidate model, training procedure, or task code evaluated by the protocol.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenContinue proposal generation from the actual resulting state.
“Continue proposal generation from the actual resulting state.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningIndependent uniform draws with replacement from a frozen finite pool justify this model for that pool's accuracy difference, including repeated images, but not for an external population.
“Independent uniform draws with replacement from a frozen finite pool justify this model for that pool's accuracy difference, including repeated images, but not for an external population.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenVerification, screening, candidate construction, and lost improvement opportunities must all enter an empirical cost assessment.
“Verification, screening, candidate construction, and lost improvement opportunities must all enter an empirical cost assessment.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningAppendix H proves finite-pool reuse for the finite-pool target; it does not justify recounting an old sample or external generalization.
“Appendix H proves this for the finite-pool target; it does not justify recounting an old sample or external generalization.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningReporting raw paired differences alongside transformed statistics is necessary.
“Reporting raw paired differences alongside transformed statistics is necessary.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe small, fully enumerated workload makes this a mechanism benchmark: a policy given the entire table could directly select the best configuration.
“The small, fully enumerated workload makes this a mechanism benchmark: a policy given the entire table could directly select the best configuration.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenUnaccepted positive proposals are counted on each policy's own path, not as counterfactual terminal regret.
“Unaccepted positive proposals are counted on each policy's own path, not as counterfactual terminal regret.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenPositive scaling of a raw bounded difference preserves its mean's sign.
“Positive scaling of a raw bounded difference preserves its mean's sign.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductown dataTask IDs are sampled independently with replacement from the declared finite workload; this is not evaluation on unseen natural tasks.
“Task IDs are sampled independently with replacement from this declared finite workload; this is not evaluation on unseen natural tasks.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenThis simple proposer supplies an execution test, not a competitive training claim.
“This simple proposer supplies an execution test, not a competitive training claim.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenCTHS has no universal power advantage.
“CTHS has no universal power advantage.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThese cases demonstrate a conditional risk-power-cost trade-off, not universal efficiency or an empirical advantage of global spending over per-test confirmation.
“These cases demonstrate a conditional risk-power-cost trade-off, not universal efficiency or an empirical advantage of global spending over per-test confirmation.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningA change to the evaluation target or verification layer requires a separate validity argument and consistent risk accounting; a certificate issued under the old rule does not authorize that change.
“A change to the evaluation target or verification layer requires a separate validity argument and consistent risk accounting; a certificate issued under the old rule does not authorize that change.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenProtocol step 5: Record the outcome and consumed allocation, including failed or interrupted attempts.
“Record the outcome and consumed allocation, including failed or interrupted attempts.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningChoosing the largest wealth retrospectively without a valid mixture is not this procedure.
“Choosing the largest wealth retrospectively without a valid mixture is not this procedure.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningWeak-effect acceptance reflects this certificate's power properties as well as the risk allocation; low power cannot be attributed to the global budget alone.
“Weak-effect acceptance therefore reflects this certificate's power properties as well as the risk allocation; low power cannot be attributed to the global budget alone.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningIndexing actual confirmation attempts is ordinary spending indexed by actual tests.
“Indexing actual confirmation attempts is ordinary spending indexed by actual tests.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenCached resampling and logical function-call counts do not represent repeated expensive solver execution or deployment of an AI Scientist.
“Cached resampling and logical function-call counts do not represent repeated expensive solver execution or deployment of an AI Scientist.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenMultiple candidates in one outer round require separate attempts or a justified joint testing rule.
“Multiple candidates in one outer round require separate attempts or a justified joint testing rule.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenProtocol step 3: Open a uniquely indexed confirmation attempt and durably reserve α_k before revealing its observations.
“Open a uniquely indexed confirmation attempt and durably reserve α k before revealing its observations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenFor a declared maximum K, α_k = δ/(k H_K), H_K = sum_{j=1}^K 1/j, is another valid choice.
“For a declared maximum K, α k = δ/(kH K ), H K = K j=1 1/j, is another.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenUniform δ/K is a transparent reference for a fixed horizon; a summable schedule supports an unknown horizon.
“Uniform δ/K is a transparent reference for a fixed horizon; a summable schedule supports an unknown horizon.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenThe original CTHS instance 6δ/(π^2 k^2) is summable but is not a finite harmonic schedule.
“The original CTHS instance 6δ/(π 2 k 2 ) is summable but is not a finite harmonic schedule.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningA certificate for E[g(∆)] > 0 must be described as such; it cannot automatically support Corollary 1 for raw accuracy or return.
“A certificate for E[g(∆)] > 0 must be described as such; it cannot automatically support Corollary 1 for raw accuracy or return.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenProtocol step 4: Collect valid paired evidence and inspect it according to the specified stopping rule; commit only on a valid crossing and matching state bindings; otherwise retain the incumbent.
“Collect valid paired evidence and inspect it according to the specified stopping rule.Commit only on a valid crossing and matching state bindings; otherwise retain the incumbent.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenThese conditions connect the statistical decision to execution; ordinary transactions and hashes can implement them.
“These conditions connect the statistical decision to execution; ordinary transactions and hashes can implement them.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenClipping or applying a nonlinear transformation generally does not preserve the mean's sign.
“Clipping or applying a nonlinear transformation generally does not.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningBoth certificate efficiency and the level-sample-cap constraint limit acceptance.
“Thus, both certificate efficiency and the level-sample-cap constraint limit acceptance.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenFor an unbounded sequence, α_k = δ/[k(k+1)] is one valid choice.
“For an unbounded sequence, α k = δ/[k(k + 1)] is one valid choice.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductreasoningAcceptance requires p ≤ α_k.
“Acceptance requires p ≤ α k .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenA restart must not silently reset the test index.
“A restart must not silently reset the test index.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenUnder conditional IID bounded observations with μ_k > 0, a fixed-sample Hoeffding test accepts with probability at least 1-β whenever n > (2/μ_k^2) log(1/α_k) + log(1/β)/2.
“Under conditional IID bounded observations with µ k > 0, a fixed-sample Hoeffding test accepts with probability at least 1 -β whenever n > 2 µ 2 k log(1/α k ) + log(1/β) 2 .”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenPolicies share exogenous proposal moves and sampled task IDs; after decisions diverge, their incumbents and candidates can differ.
“Policies share exogenous proposal moves and sampled task IDs; after decisions diverge, their incumbents and candidates can differ.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThe artifacts make this short execution trace auditable; they do not establish that a rejected candidate is harmful or that all possible future modifications satisfy the sampling assumptions.
“The artifacts make this short execution trace auditable; they do not establish that a rejected candidate is harmful or that all possible future modifications satisfy the sampling assumptions.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenPairing uses the original shared exogenous inputs across 2,000 trajectories.
“Pairing uses the original shared exogenous inputs across 2,000 trajectories.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningOne trajectory does not estimate long-horizon FWER, certify a training procedure across seeds, or show an advantage over equivalent standard spending.
“One trajectory does not estimate long-horizon FWER, certify a training procedure across seeds, or show an advantage over equivalent standard spending.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenThe conditional test controls non-positive gain under the stated observation model.
“The conditional test controls non-positive gain under the stated observation model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationown dataThese components have different timing boundaries and are not a complete research-cost total.
“These components have different timing boundaries and are not a complete research-cost total.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAudit predictions are stored for every model and image.
“Audit predictions are stored for every model and image.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenAfter the rejected second attempt, a terminal record preserves that same incumbent and records both consumed allocations.
“After the rejected second attempt, a terminal record preserves that same incumbent and records both consumed allocations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenConfidence intervals for event rates are marginal 95% Wilson intervals across the 2,000 trajectories within each setting.
“Confidence intervals for event rates are marginal 95% Wilson intervals across the 2,000 trajectories within each setting.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenIndependent local checks reproduce labels, hashes, accuracies, paired counts and terminal bindings.
“Independent local checks reproduce labels, hashes, accuracies, paired counts and terminal bindings.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe first accepted checkpoint is copied into the incumbent together with optimizer state; its hash is checked before round-2 training.
“The first accepted checkpoint is copied into the incumbent together with optimizer state; its hash is checked before round-2 training.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenIt does not establish that the second candidate is harmful; per-test confirmation at .05 would also reject it.
“It does not establish that the second candidate is harmful; per-test confirmation at .05 would also reject it.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningLogical evidence counts are not complete measured execution costs, and equal per-attempt caps do not imply equal realized total budgets.
“Logical evidence counts are not complete measured execution costs, and equal perattempt caps do not imply equal realized total budgets.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe predeclared two-attempt cap allows no sample extension or third attempt.
“The predeclared two-attempt cap allows no sample extension or third attempt.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenThe simulation measures local non-improvement, not a formal comparison of entire counterfactual future utilities.
“The simulation measures local non-improvement, not a formal comparison of entire counterfactual future utilities.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenEach attempt fixes the candidate, sample size, exact paired-binomial test, and allocation before drawing 25,000 pairs independently with replacement from the common confirmation pool.
“Each attempt fixes the candidate, sample size, exact paired-binomial test, and allocation before drawing 25,000 pairs independently with replacement from the common confirmation pool.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe framework is useful only within that declared model; it cannot repair a misspecified objective, contaminated confirmation data, or a changed evaluator by budget accounting alone.
“The framework is useful only within that declared model; it cannot repair a misspecified objective, contaminated confirmation data, or a changed evaluator by budget accounting alone.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningIncreasing evidence capacity or using justified shared evidence can fall outside this regime; restarting the risk budget without a separate argument does not preserve the original lifetime guarantee.
“Increasing evidence capacity or using justified shared evidence can fall outside this regime; restarting the risk budget without a separate argument does not preserve the original lifetime guarantee.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAll four settings, including weaker-effect cases, are retained.
“All four settings, including weaker-effect cases, are retained.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningProposition 1 shows that the expected total number of acceptances is finite, hence infinitely many acceptances have probability zero.
“Proposition 1 shows that the expected total number of acceptances is finite, hence infinitely many acceptances have probability zero.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningDistribution changes require a new valid sampling argument; partitioning time into epochs and assigning budgets does not itself solve distribution shift.
“Distribution changes require a new valid sampling argument; partitioning time into epochs and assigning budgets does not itself solve distribution shift.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenNo betting parameter was changed in this post-hoc analysis.
“No betting parameter was changed in this post-hoc analysis.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductown dataThe committed weight hash matches the round-2 parent, and frozen training code and execution records document optimizer inheritance.
“The committed weight hash matches the round-2 parent, and frozen training code and execution records document optimizer inheritance.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe argument applies to generic online testing and is included as a boundary result, not a self-modification-specific contribution.
“The argument applies to generic online testing and is included as a boundary result, not a self-modification-specific contribution.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataThese descriptive audit outcomes do not change the incumbent or select another model.
“These descriptive audit outcomes do not change the incumbent or select another model.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningIt does not bound catastrophic tail losses, guarantee lower total compute, ensure progress, or restore the classical Gödel machine's global future-utility result.
“It does not bound catastrophic tail losses, guarantee lower total compute, ensure progress, or restore the classical Gödel machine's global future-utility result.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe formal result concerns non-improvement in the certified target.
“The formal result concerns non-improvement in the certified target.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThis graphic therefore does not demonstrate certification by these bounds; any separately reported acceptance requires its own identified estimand and test.
“This graphic therefore does not demonstrate certification by these bounds; any separately reported acceptance requires its own identified estimand and test.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenIts certificate concerns a fixed finite pool, and its held-out audit does not estimate population risk.
“Its certificate concerns a fixed finite pool, and its held-out audit does not estimate population risk.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningCounts of unaccepted positive proposals do not quantify recoverable future utility; repeated opportunities and changed paths prevent that interpretation.
“Counts of unaccepted positive proposals do not quantify recoverable future utility; repeated opportunities and changed paths prevent that interpretation.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThese are marginal descriptive intervals, not simultaneous claims across settings.
“These are marginal descriptive intervals, not simultaneous claims across settings.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenCross-policy shared draws provide paired comparisons, not additional independent trajectories.
“Cross-policy shared draws provide paired comparisons, not additional independent trajectories.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataThe post-hoc decomposition analysis uses the stored numerical and synthetic trajectories from Sections 5.1-5.2, with no new simulation or tuning.
“This analysis uses the stored numerical and synthetic trajectories from Sections 5.1-5.2, with no new simulation or tuning.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThese are descriptive intervals, not simultaneous coverage across all settings.
“These are descriptive intervals, not simultaneous coverage across all settings.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningIts contrast with the synthetic decomposition concerns different workloads and comparators, not an identified causal effect of changing error composition.
“Its contrast with the synthetic decomposition concerns different workloads and comparators, not an identified causal effect of changing error composition.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThese observations do not by themselves validate the confirmation test or the subsequent state transition.
“These observations do not by themselves validate the confirmation test or the subsequent state transition.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenStored records include frozen manifests and hashes, effective configurations, all epoch logs and branch checkpoints, confirmation reservations, sampled indices, raw paired outcomes, and both decisions.
“Stored records include frozen manifests and hashes, effective configurations, all epoch logs and branch checkpoints, confirmation reservations, sampled indices, raw paired outcomes, and both decisions.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenA successful empirical demonstration must show both that the gate can accept useful updates and that its evidence model remains valid as the system evolves.
“A successful empirical demonstration must show both that the gate can accept useful updates and that its evidence model remains valid as the system evolves.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningScreening, finite sampling caps, and the chosen test contribute to weak-effect non-acceptance alongside the global allocation.
“Screening, finite sampling caps, and the chosen test therefore contribute to weak-effect non-acceptance alongside the global allocation.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Interpretationnone givenThe unbounded accumulated synthetic utility is a decision-process diagnostic, not the bounded task-success score of the numerical workload.
“This unbounded accumulated synthetic utility is a decision-process diagnostic, not the bounded task-success score of the numerical workload.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe initial model and all six branches are fixed before a one-time census of the 12,804 audit images.
“The initial model and all six branches are fixed before a one-time census of the 12,804 audit images.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningA trajectory can contain both zero-effect and harmful commits, so their separate marginal probabilities must not be added.
“A trajectory can contain both zero-effect and harmful commits, so their separate marginal probabilities must not be added.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataNon-positive log drift does not invalidate Type-I control or rule out a finite-sample crossing, but this test does not provide eventual acceptance for every positive effect.
“Non-positive log drift does not invalidate Type-I control or rule out a finite-sample crossing, but this test does not provide eventual acceptance for every positive effect.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningIts 12-pair assumption differs from the 10-seed trajectory description, so this ratio is an accounting illustration rather than a reconciled end-to-end cost measurement.
“Its 12-pair assumption differs from the 10-seed trajectory description, so this ratio is an accounting illustration rather than a reconciled end-to-end cost measurement.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe four-observation screen requires a strictly positive sample mean and uses observations independent of confirmation.
“The four-observation screen requires a strictly positive sample mean and uses observations independent of confirmation.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningIts power upper-bounds any level-α rule using at most 512 observations from the same IID model: such a rule can be represented as a test on the full 512-observation sequence.
“Its power upper-bounds any level-α rule using at most 512 observations from the same IID model: such a rule can be represented as a test on the full 512-observation sequence.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThese calculations do not establish an optimal certifier for learning tasks.
“These calculations do not establish an optimal certifier for learning tasks.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningCaching a deterministic prediction for a newly drawn index does not change the sampling law; simply recounting a previous sample without drawing new indices does.
“Caching a deterministic prediction for a newly drawn index does not change the sampling law; simply recounting a previous sample without drawing new indices does.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataNeither adding small betting fractions nor changing a schedule guarantees a better utility-risk-cost trade-off.
“Neither adding small betting fractions nor changing a schedule guarantees a better utility-risk-cost trade-off.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenThe mixture was fixed before this diagnostic was calculated; it was not selected after comparing diagnostic outcomes and does not replace the original fixed-bet results.
“The mixture was fixed before this diagnostic was calculated; it was not selected after comparing diagnostic outcomes and does not replace the original fixed-bet results.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenUniform domination of the actual conditional law is essential.
“Uniform domination of the actual conditional law is also essential.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThis bound does not impose the original inspection schedule or quantify early-stopping savings; it is neither a general-task sample-complexity bound nor a claim of optimal expected evidence cost.
“This bound does not impose the original inspection schedule or quantify early-stopping savings; it is neither a general-task sample-complexity bound nor a claim of optimal expected evidence cost.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationown dataIt does not estimate the learning run's stopping behavior: P2 has a predeclared two-attempt cap, and the synthetic experiments have 40 rounds.
“It does not estimate the learning run's stopping behavior: P2 has a predeclared two-attempt cap, and the synthetic experiments have 40 rounds.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenA noninferiority rule with tolerance ε > 0 tests H_0: μ ≤ -ε and can approve small deteriorations.
“A noninferiority rule with tolerance ε > 0 instead tests H 0 : µ ≤ -ε and can approve small deteriorations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe script analyze_confirmation_power.py computes the diagnostic and checks probability conservation, null acceptance no greater than the allocated level, positive-effect power below the Neyman-Pearson bound, agreement with a direct binomial sum for a single inspection, and monotonicity in the testing level.
“The script analyze_confirmation_power.pycomputes the diagnostic and checks probability conservation, null acceptance no greater than the allocated level, positive-effect power below the Neyman-Pearson bound, agreement with a direct binomial sum for a single inspection, and monotonicity in the testing level.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningAveraging over any additional conditioning on D preserves the bound in Equation 1, with the estimand explicitly defined as μ_{k,D}.
“Averaging over any additional conditioning on D preserves the bound in Equation 1, with the estimand explicitly defined as µ k,D .”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThis calculation does not establish a non-positive true effect, identify the historical test, or exclude another pre-specified valid certificate.
“This calculation does not establish a non-positive true effect, identify the historical test, or exclude another pre-specified valid certificate.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningThe guarantee is deliberately restricted to the former.
“The guarantee is deliberately restricted to the former.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningScreening, certificate efficiency, and the level-sample-cap constraint each affect acceptance in this fixed-candidate model.
“Thus screening, certificate efficiency, and the level-sample-cap constraint each affect acceptance in this fixed-candidate model.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenP2 uses fresh OS-backed uniform index draws with replacement and includes every draw in its fixed-sample count.
“P2 uses fresh OS-backed uniform index draws with replacement and includes every draw in its fixed-sample count.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenA sequential A/B gate can use the same evidence tools; its stopping validity alone does not specify the target or cross-update error control of a persistent adoption protocol.
“Likewise, a sequential A/B gate can use the same evidence tools; its stopping validity alone does not specify the target or cross-update error control of a persistent adoption protocol.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThe conclusion gives almost-sure eventual cessation of acceptances, not a known finite cutoff at which each test has exactly zero power.
“The conclusion gives almost-sure eventual cessation of acceptances, not a known finite cutoff at which each test has exactly zero power.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe proof uses conditional change of measure and summability and remains unchanged for generic online testing.
“The proof uses conditional change of measure and summability and remains unchanged for generic online testing.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationreasoningThese are alternative scientific objectives, not power improvements under an unchanged objective.
“These are alternative scientific objectives, not power improvements under an unchanged objective.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThis statement allows shrinking positive effects and does not rely on a positive lower bound on improvement size.
“This statement allows shrinking positive effects and does not rely on a positive lower bound on improvement size.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningThis is inference with respect to repeated index sampling on a fixed population of items, not inference from newly sampled external examples.
“This is inference with respect to repeated index sampling on a fixed population of items, not inference from newly sampled external examples.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenThese settings match the synthetic workload in Section 5.2.
“These settings match the synthetic workload in Section 5.2.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThe paper already includes an alternative mixture with a negative result.
“Thus the paper already includes an alternative mixture with a negative result.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenIt does not assert that proposals or learning computations cease.
“It also does not assert that proposals or learning computations cease.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductreasoningIt neither changes the original experimental thresholds nor constitutes new learning-trajectory evidence.
“It neither changes the original experimental thresholds nor constitutes new learning-trajectory evidence.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Premisenone givenOnline FDR controls an expected proportion of erroneous rejections, whereas lifetime FWER controls the probability of any such error.
“Online FDR controls an expected proportion of erroneous rejections, whereas lifetime FWER controls the probability of any such error.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenIt holds the candidate effect and confirmation index fixed and does not identify the causal contribution of these factors to adaptive trajectory utility.
“It holds the candidate effect and confirmation index fixed and does not identify the causal contribution of these factors to adaptive trajectory utility.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe analysis script analyze_commit_composition.py reads the saved arrays, checks their recorded hashes, verifies the disjoint identity and numerical event counts, and produces complete per-policy metrics and paired descriptive intervals without rerunning the simulations.
“The analysis script analyze_commit_composition.pyreads the saved arrays, checks their recorded hashes, verifies the disjoint identity and numerical event counts, and produces complete per-policy metrics and paired descriptive intervals without rerunning the simulations.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationnone givenA fixed sample cap need not bound likelihood ratios in other observation models, and bounded observations alone are insufficient.
“A fixed sample cap need not bound likelihood ratios in other observation models, and bounded observations alone are insufficient.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenChanging the evidence-sharing mechanism, observation model, or allocation method requires checking the assumptions anew; none is automatically a valid remedy.
“Changing the evidence-sharing mechanism, observation model, or allocation method requires checking the assumptions anew; none is automatically a valid remedy.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenCTHS names an optional instance; it has no universal power advantage over other valid allocations.
“CTHS names an optional instance; it has no universal power advantage over other valid allocations.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningStochastic scoring requires its own execution-randomness assumptions.
“Stochastic scoring requires its own execution-randomness assumptions.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThis argument fails if the candidate is chosen after inspecting the current indices, if sampling or item retention depends on observed performance without adjustment, if the model or score changes during confirmation, or if samples are drawn without replacement but analyzed using this IID model.
“This argument fails if the candidate is chosen after inspecting the current indices, if sampling or item retention depends on observed performance without adjustment, if the model or score changes during confirmation, or if samples are drawn without replacement but analyzed using this IID model.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningCandidates may depend on past history, but not on the current confirmation indices or outcomes after freezing.
“Candidates may depend on past history, but not on the current confirmation indices or outcomes after freezing.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataThe corresponding 14.575% upper bound shows room for certificate improvement, but also rules out high power from changing the certificate alone under this level, sample cap, and observation model.
“The corresponding 14.575% upper bound shows room for certificate improvement, but also rules out high power from changing the certificate alone under this level, sample cap, and observation model.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
Interpretationnone givenAn inadequate prospective design may justify changing resources or the study plan before new confirmation; it does not justify relabeling previously rejected candidates as accepted.
“An inadequate prospective design may justify changing resources or the study plan before new confirmation; it does not justify relabeling previously rejected candidates as accepted.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Interpretationnone givenA decline of per-attempt power to zero alone would not establish the conclusion below: probabilities can tend to zero while remaining nonsummable.
“In particular, a decline of per-attempt power to zero alone would not establish the conclusion below: probabilities can tend to zero while remaining nonsummable.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenDecreasing summable schedules support an unbounded number of attempts but assign less evidence tolerance to late tests.
“Decreasing summable schedules support an unbounded number of attempts but assign less evidence tolerance to late tests.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThis is a discrete probability recursion evaluated numerically, with floating-point rounding but no Monte Carlo sampling error.
“This is a discrete probability recursion evaluated numerically, with floating-point rounding but no Monte Carlo sampling error.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningThe 40-round learning observations are retained as exploratory evidence; matching original paired observations, the declared test, and the actual next incumbent is necessary to establish a certificate-driven learning trajectory.
“The 40-round learning observations are retained as exploratory evidence; matching original paired observations, the declared test, and the actual next incumbent is necessary to establish a certificate-driven learning trajectory.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThe upper-tail binomial p-value evaluated at 1/2 is conditionally super-uniform under the null; take p = 1 when M = 0.
“The upper-tail binomial p-value evaluated at 1/2 is therefore conditionally super-uniform under the null; take p = 1 when M = 0.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenEach P2 confirmation rule and binding is frozen before its own confirmation data are revealed; the second proposal round uses the actual first committed state.
“Each P2 confirmation rule and binding is frozen before its own confirmation data are revealed; the second proposal round uses the actual first committed state.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenEvidence-cost comparisons use the controlled workload, where the number of observations and decision rules are specified consistently.
“Evidence-cost comparisons use the controlled workload, where the number of observations and decision rules are specified consistently.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Interpretationnone givenBroader practical value remains to be established beyond this short finite-pool demonstration.
“Broader practical value remains to be established beyond this short finite-pool demonstration.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenAll seven models fixed before final audit. Dev. and audit accuracies are percentages. Round-2 branches start from the actual round-1 committed model. The audit is a census of 12,804 held-out images and is not used for model selection.
“All seven models fixed before final audit.Dev. and audit accuracies are percentages.Round-2 branches start from the actual round-1 committed model.The audit is a census of 12,804 held-out images and is not used for model selection.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningSGM's conditional composition can use a suitable valid policy certificate, but does not establish those off-policy conditions by itself.
“SGM's conditional composition can use a suitable valid policy certificate, but does not establish those off-policy conditions by itself.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataThese comparisons distinguish the effect of continued training from the candidate change without making a proposer-strength claim.
“These comparisons distinguish the effect of continued training from the candidate change without making a proposer-strength claim.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThat earlier trajectory stopped without opening confirmation and is retained as a negative development result.
“That earlier trajectory stopped without opening confirmation and is retained as a negative development result.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationown dataCIFAR-10 and LunarLander positive-effect observations require separating reported adoption from valid certification: the supplied graphics do not support their acceptance labels with the displayed bounds. We therefore do not count these cases as additional certified improvements.
“CIFAR-10 and LunarLander positive-effect observations also require separating reported adoption from valid certification: the supplied graphics do not support their acceptance labels with the displayed bounds.We therefore do not count these cases as additional certified improvements.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
InterpretationreasoningThe following historical observations illustrate the distinction between positive estimates and statistical certification. These cases are exploratory; the controlled workload and P2 provide the certificate-driven evaluations.
“The following historical observations illustrate the distinction between positive estimates and statistical certification.These cases are exploratory; the controlled workload and P2 provide the certificate-driven evaluations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenNew index draws use operating-system-backed uniform sampling with replacement.
“New index draws use operating-system-backed uniform sampling with replacement.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThese annotations are not independently re-certified here; the current protocol requires a strict positive certificate for its specified estimand.
“These annotations are not independently re-certified here; the current protocol requires a strict positive certificate for its specified estimand.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
InterpretationreasoningA familywise comparison must use the same error event and total level; repeated per-invocation guarantees alone should not be relabeled as a fixed lifetime guarantee.
“A familywise comparison must use the same error event and total level; repeated per-invocation guarantees alone should not be relabeled as a fixed lifetime guarantee.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing), S4 (based).
Study design and conductnone givenThe four roles are disjoint in content SHA-256.
“The four roles are disjoint in content SHA-256.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe confirmation role is stored in two shards of 24,984 images but is sampled as one pool of 49,968.
“The confirmation role is stored in two shards of 24,984 images but is sampled as one pool of 49,968.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe audit model list and descriptive comparisons are frozen after confirmation, before audit predictions are opened.
“The audit model list and descriptive comparisons are frozen after confirmation, before audit predictions are opened.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningP2 is not presented as an ex-ante design untouched by prior development observations.
“P2 is not presented as an ex-ante design untouched by prior development observations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe frozen r2 partition removes 77 quarantined images without replacement; a perceptual-hash review additionally inspected flagged cross-role pairs.
“The frozen r2 partition removes 77 quarantined images without replacement; a perceptual-hash review additionally inspected flagged cross-role pairs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
InterpretationreasoningThe resulting protocol makes empirical adoption executable while exposing its assumptions, missed improvements, and evidence costs.
“The resulting protocol makes empirical adoption executable while exposing its assumptions, missed improvements, and evidence costs.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThe protocol comparison above is not such an experiment, and no superiority claim is inferred from it.
“The protocol comparison above is not such an experiment, and no superiority claim is inferred from it.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationnone givenThis is a documented duplicate-screening procedure, not a proof that all semantic near-duplicates have been eliminated.
“This is a documented duplicate-screening procedure, not a proof that all semantic near-duplicates have been eliminated.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Interpretationnone givenStatistical Gödel Machines provide a statistical relaxation of proof-justified self-improvement.
“Statistical Gödel Machines provide a statistical relaxation of proof-justified self-improvement.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningThe same frozen image can appear in both attempts under the declared finite-pool sampling model.
“The same frozen image can appear in both attempts under the declared finite-pool sampling model.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Interpretationown dataBecause the cumulative curve increases at every displayed iteration, this panel should not be read as a measured ledger of confirmation-only spending.
“Because the cumulative curve increases at every displayed iteration, this panel should not be read as a measured ledger of confirmation-only spending.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable).
Interpretationnone givenThe error decomposition shows why FWER must be interpreted alongside error severity, useful acceptances, and evidence cost.
“The error decomposition shows why FWER must be interpreted alongside error severity, useful acceptances, and evidence cost.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenAppendix F gives a conditional check of the historical round-21 bound. The trajectory is not used to estimate false-commit risk or to validate the global guarantee.
“Appendix F gives a conditional check of the historical round-21 bound.The trajectory is not used to estimate false-commit risk or to validate the global guarantee.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenNo confidence interval or new acceptance test is attached to this descriptive audit.
“No confidence interval or new acceptance test is attached to this descriptive audit.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenEqual per-attempt caps do not match realized total costs.
“Equal per-attempt caps do not match realized total costs.”
Quote in the paperNot a substantial claim
The authors state this as an assumption.
The protected verification layer executes the declared rule correctly.
“The guarantees assume that this layer executes the declared rule correctly.”
The authors state this as an assumption.
Equation (1) holds for every opened confirmation.
“If Equation (1) holds for every opened confirmation and the predictable allocation is pathwise summable with total at most δ, then, for an adaptive sequence containing any mixture of null and improving candidates,”
The authors state this as an assumption.
The predictable allocation is pathwise summable with total at most δ.
“If Equation (1) holds for every opened confirmation and the predictable allocation is pathwise summable with total at most δ, then, for an adaptive sequence containing any mixture of null and improving candidates,”
The authors state this as an assumption.
Under the null, E[X_{k,i} | F_{k,i-1}] ≤ 0 on H_k.
“Under the null, assume E[X k,i | F k,i-1 ] ≤ 0 on H k .”
The authors state this as an assumption.
There is a uniform finite bound on the likelihood ratio of each fresh evidence transcript relative to a valid null reference.
“A narrower limitation holds for per-attempt certification with pathwise summable levels and a uniform finite bound on the likelihood ratio of each fresh evidence transcript relative to a valid null reference.”
The authors state this as an assumption.
The synthetic IID model X_i ∈ {-1, +1} with Pr_g(X_i = +1) = (1 + g)/2 holds for the conditional power diagnostic.
“This separate post-hoc calculation uses the synthetic IID model X i ∈ {-1, +1} with Pr g (X i = +1) = (1 + g)/2, so E g [X i ] = g.”
The authors state this as an assumption.
For almost every actual pre-attempt history h, a null-reference law Q_{k,h} exists such that the integral of the decision against Q_{k,h} is at most α_k(h).
“For almost every actual pre-attempt history h, assume a null-reference law Q k,h exists such that a k (h, z) Q k,h (dz) ≤ α k (h).”
The authors state this as an assumption.
A deterministic constant C < ∞, uniform over attempts and histories, satisfies the likelihood-ratio domination bound P_{k,h} ≪ Q_{k,h}, dP_{k,h}/dQ_{k,h}(z) ≤ C Q_{k,h}-almost surely.
“Assume also that a deterministic constant C < ∞, uniform over attempts and these histories, satisfies P k,h ≪ Q k,h , dP k,h dQ k,h (z) ≤ C Q k,h -almost surely.”
The authors state this as an assumption.
Every opened attempt uses at most a fixed N fresh IID observations in {-1, +1}, with conditional mean g_k ∈ [-1, 1].
“Suppose every opened attempt uses at most a fixed N fresh IID observations in {-1, +1}, with conditional mean g k ∈ [-1, 1].”
The authors state this as an assumption.
For a fixed history, the null reference has mean zero, with the same auxiliary randomization as the actual test.
“For a fixed history take the null reference to have mean zero, with the same auxiliary randomization as the actual test.”
The authors state this as an assumption.
The new indices I_{k,1}, ..., I_{k,n} are IID uniform on {1, ..., m} conditional on H_k.
“Assume the new indices I k,1 , . . ., I k,n are IID uniform on {1, . . ., m} conditional on H k .”
The authors state this as an assumption.
The plotted round-21 mean 0.606 is the mean of IID observations in [-1, 1], the sample size is fixed at 10 or 12, this is the second confirmation attempt, and the total budget is 0.1.
“For a conditional diagnostic, suppose the plotted round-21 mean 0.606 is the mean of IID observations in [-1, 1], the sample size is fixed at 10 or 12, this is the second confirmation attempt, and the total budget is 0.1.”
The authors state this as an assumption.
The historical screening-cost calculation uses a 12-pair assumption.
“Its 12-pair assumption differs from the 10-seed trajectory description, so this ratio is an accounting illustration rather than a reconciled end-to-end cost measurement.”
Cites
Hyperband: A Novel Bandit-Based Approach to Hyperparameter OptimizationPreprint
Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06560 · Cited by 1,368 · cited 1× · see citing works
“Hyperband allocates resources through early stopping (Li et al., 2018), and BOHB combines model-guided search with bandit resource allocation (Falkner et al., 2018).”
BOHB: Robust and Efficient Hyperparameter Optimization at Scale
Falkner, S., Klein, A., & Hutter, F. (2018) · arXiv · doi:10.48550/arxiv.1807.01774 · Cited by 551 · cited 1× · see citing works
“Hyperband allocates resources through early stopping (Li et al., 2018), and BOHB combines model-guided search with bandit resource allocation (Falkner et al., 2018).”
Estimating means of bounded random variables by betting
Waudby-Smith, I., & Ramdas, A. (2020) · Journal of The Royal Statistical Society Series B-statistical Methodology · doi:10.1093/jrsssb/qkad009 · Cited by 337 · see citing works
Population Based Training of Neural Networks
Jaderberg, M., Dalibard, V., Osindero, S., Czarnecki, W. M., Donahue, J., Razavi, A., Vinyals, O., Green, T., Dunning, I., Simonyan, K., et al. (2017) · arXiv · doi:10.48550/arxiv.1711.09846 · Cited by 284 · cited 1× · see citing works
“Population Based Training adapts parameters and hyperparameters during training (Jaderberg et al., 2017).”
Time-uniform, nonparametric, nonasymptotic confidence sequences
Howard, S. R., Ramdas, A., McAuliffe, J., & Sekhon, J. (2021) · The Annals of Statistics · doi:10.1214/20-aos1991 · Cited by 190 · cited 2× · see citing works
“Online family-wise error control handles potentially unbounded hypothesis sequences (Tian & Ramdas, 2019); confidence sequences and betting methods support optional stopping (Howard et al., 2021; Waudby-Smith & Ramdas, 2024).”
Feng, J., Emerson, S., & Simon, N. (2021) · Biometrics · doi:10.1111/biom.13379 · Cited by 26 · cited 4× · see citing works
“It sharpens the connection to the alpha-death discussion of Feng et al. (2021).”
InfluentialSchmidhuber, J. (2003) · arXiv · doi:10.48550/arxiv.cs/0309048 · Cited by 24 · cited 2× · see citing works
“The Gödel machine gives a principled answer: execute a self-modification when a formal proof establishes its usefulness relative to continuing the current process, including its proof search (Schmidhuber, 2003).”
Online control of the familywise error rate
Tian, J., & Ramdas, A. (2019) · arXiv · doi:10.48550/arxiv.1910.04900 · Cited by 18
Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents
Zhang, J., Hu, S., Lu, C., Lange, R., & Clune, J. (2025) · arXiv · doi:10.48550/arxiv.2505.22954 · Cited by 5 · see citing works
Group sequential crossover trial designs with strong control of the familywise error rate
Grayling, M. J., Wason, J., & Mander, A. (2017) · Sequential Analysis · doi:10.1080/07474946.2018.1466528 · Cited by 3 · cited 13×
“Online FDR controls an expected proportion of erroneous rejections, whereas lifetime FWER controls the probability of any such error.”
InfluentialSelf-Evolving Agents with Anytime-Valid Certificates
Sengupta, B. (2026) · arXiv · doi:10.48550/arxiv.2607.00871 · cited 5×
“SEA (Sengupta, 2026) Agent architecture using anytime-valid gates; SGM-CS explicitly builds on SGM. Broadens the agent loop; the safety of its endogenous distribution-shifting composition is left unproved.”
InfluentialA Theory of Reliable Self-Evolution for Agent Harnesses
Cai, Q., Zhang, Y., Nie, J., Ran, M., Zheng, H., Song, J., Tian, X., Guo, Y., & Xue, W. (2026) · arXiv · doi:10.48550/arxiv.2609.08175 · cited 4×
“Safe Harness (Cai et al., 2026), released in September 2026, analyzes modification generation, finite-data certification and selection, safe adoption, and opportunities after an update.”
Influential
Flagged first, then by citations
2 of the 14 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Philip Thomas, Georgios Theocharous, and Mohammad Ghavamzadeh. High confidence policy improvement. In Proceedings of the 32nd International Conference on Machine Learning, 2015. URL https://proceedings.mlr.press/v37/thomas15.html.
- Xuening Wu, Shenqin Yin, Yanlan Kang, Xinhang Zhang, Qianya Xu, Zeping Chen, and Wenqiang Zhang. SGM: A statistical Godel machine for risk-controlled recursive self-modification. arXiv preprint arXiv:2510.10232, 2025. URL https://arxiv.org/abs/2510.10232.
Cited by
Sorted by citations
Self-Evolving Agents with Anytime-Valid Certificates
Sengupta, B. (2026) · arXiv · doi:10.48550/arxiv.2607.00871
“…with familywise control under Ville’s inequality [Ramdas et al., 2023], the confirm-triggered harmonic spending of the Statistical Gödel Machine [Wu et al., 2025], the time-uniform confidence sequence of Howard et al. [2021], and the abstention semantics of safe policy improvement [Thomas et…”
InfluentialSun, X., Zeng, Y., She, Z., & Wang, X. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.33180
“To address this, recent works gate self-modification with statistical tests, utilizing global error budgets (Wu et al., 2025; Sengupta, 2026), sequential testing (Shawn, 2026), or sealed audits (Guo et al., 2026; Mittal & Gupta, 2026).”
Tang, H., Ma, Y., Li, P.-Y., et al. (2026) · doi:10.48550/arxiv.2609.13406 · Cited by 1
“The nearest prior formal work fixes one design rather than comparing designs: the Gödel machine is anchored and proof-gated by construction, and statistical variants add a risk budget to that gate (Wu et al., 2025).”
SAHOO: Safeguarded Alignment for High-Order Optimization Objectives in Recursive Self-Improvement
Sahoo, S., Chadha, A., Jain, V., & Chaudhary, D. (2026) · arXiv
“The alignment and safety challenge in self improvement systems has received increasing attention Yin et al. (2025); Zhang et al. (2025).”
Post-Audit Mirage: Identifiability Limits of Offline Agent Verification
Behera, A., & Payal, P. (2026) · doi:10.48550/arxiv.2610.31130
“Recent systems instead budget evidence across edits (SGM), test candidates sequentially (PACE), compose controllers (SEA), add an exogenous audit (SEAL), or benchmark evolutionary trajectories (SEA-Eval) [10, 14, 33, 34, 38].”
A Kinetic Theory of the Gated Self-Evolving LLM Agent
Wang, H.-P. (2026) · doi:10.48550/arxiv.2610.03243
“Reflexion (Shinn et al., 2023) and the G¨odel Agent (Yin et al., 2025) update task/meta policies by self-critique without external statistical control; our certificate is what that line lacks.”
G\"odel Forest: Balancing Search Depth and Breadth for Data-Centric Recursive Self-Improvement
Zhao, Z.-Q., Meng, F.-Q., Lu, H.-C., et al. (2026) · doi:10.48550/arxiv.2609.36675
Self-citationPaired Seed Evaluation: Statistical Reliability for Learning-Based Simulators
Sharma, U. (2025) · arXiv.org · doi:10.48550/arxiv.2512.24145
“In learning-based simulators, paired seeds have occasionally appeared as implementation details in specific contexts[15, 16], but have not been studied as a general evaluation design choice.”
Recursive self-improvement of AI research agents
Srikanth, D., Zhao, B., Xu, D., Wu, Y., & Jiang, Z. (2026) · arXiv · doi:10.48550/arxiv.2609.26457
- Authors
- Wu, X., Yin, S., Kang, Y., Zhang, X., Xu, Q., Chen, Z., & Zhang, W.
- Affiliations
- Fudan University (CN)
- University of California San Diego (US)
- Tongji University (CN)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2025
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2510.10232
- OpenAlex
- W4417100123
- Cited by
- 0
- References
- 14
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- recursive self-modification, E-value, Hoeffding bound, automated machine learning, neural architecture search, reinforcement learning, black-box optimization, continual learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-24.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-24.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review