Hollow-LLM Attack: Computationally Trivial Weights in Zero-Knowledge Verification of LLM Inference

Classified as probably not a substantial claim.

The proof certifies that the served output is consistent with some fixed private weights under the public architecture, without revealing those weights.

“Overall, the proof certifies that the served output is consistent with some fixed private weights under the public architecture, without revealing those weights.”

• cites [6] sourceSupported
• cites [7] Qu, 2025, zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM InferenceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

zkLLM proves a 13B-parameter model in under 15 minutes.

“zkLLM proves a 13B-parameter model in under 15 minutes”

• cites [6] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

TEEs are inherently hardware-dependent and may be vulnerable to microarchitectural attacks.

“TEE are inherently hardware-dependent and may be vulnerable to microarchitectural attacks [38], [39], [40], [41], [42], [43].”

• cites [38] sourceNot checked — no full text of the source
• cites [39] Moghimi, 2023, Downfall: Exploiting speculative data gatheringNot checked yet

  Not checked in the free run.

• cites [40] Van Schaik, 2020Not checked yet

  Not checked in the free run.

• cites [41] Van Bulck, Sgx-step: An open-source framework for precise dissection and practical exploitation of intel sgx enclavesNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [42] sourcePartly supported

  The Intel SGX section describes SGX as a processor architecture extension whose enclaves are created and maintained by the processor. The paper also reports microarchitectural attacks that can expose secrets in SGX enclaves. It supports SGX as an example, but does not establish that TEEs generally are inherently hardware-dependent. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites [43] sourcePartly supported

  The Introduction describes SGX as a CPU-based TEE whose use of shared resources can expose it to side-channel attacks. The Attack Vectors section identifies hardware caches that create such channels. The paper supports the vulnerability claim for SGX, but does not establish that all TEEs are inherently hardware-dependent. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

By construction, hollow serve time equals inner serve time and is much less than outer serve time, so the provider saves serve-time cost.

“By construction, T serve hollow = T serve(Ain) ≪ T serve(Aout), so the provider saves serve-time cost.”

• AssumptionThe authors state this as an assumption: Stochastic decoding, if used, is made reproducible by a public, committed seed that fixes the sampling trajectory.

May combine two claims.

In Attack B, because softmax is taken over key positions and replication only happens along the feature dimension, the score matrices and resulting attention weights are identical across all replicated blocks and coincide with the inner model's attention weights.

“the resulting attention weights softmax(·) are identical across all s and coincide with the inner model's attention weights.”

May combine two claims.

For RoPE under Attack B, if the inner head uses rotation matrix Rt at position t, the outer model applies Rt independently in each block, and attention scores and probabilities are identical across blocks and equal to those of Ain.

“If the inner head uses rotation matrix Rt at position t, the outer model applies Rt independently in each block. Attention scores and probabilities are then identical across blocks and equal to those of Ain.”

May combine two claims.

Verification runs the fixed circuit for the outer architecture and is lightweight relative to proving.

“Verification runs the fixed circuit for Aout and is lightweight relative to proving (Section 5).”

May combine two claims.

For learned absolute embeddings, Attack A reuses the same table for t <= Lin, and added layers do not change pt.

“Attack A. The outer model reuses the same table for t ≤ Lin; added layers do not change pt.”

May combine two claims.

Zeros remain zeros under elementwise nonlinearities and replicated coordinates remain exactly equal.

“Thus zeros remain zeros under ϕ, and replicated coordinates remain exactly equal.”

May combine two claims.

For any elementwise nonlinearity phi used in the FFN, phi(0)=0 and phi(Rz)=R phi(z) for any vector z and replication factor m.

“Then ϕ(0) = 0, ϕ(Rz) = Rϕ(z) because ϕ is applied coordinatewise.”

May combine two claims.

In Attack B, for both single-block routing and averaging output projections, the attention sublayer output matches the inner attention output exactly for replicated inputs, and the softmax distributions over positions are unchanged.

“In both cases, the attention sublayer output matches the inner attention output exactly for replicated inputs, and the softmax distributions over positions are unchanged.”

May combine two claims.

For sinusoidal encodings under Attack A, since dmodel,out = dmodel,in, pout_t = pin_t for t <= Lin, and extra layers leave positions unchanged.

“Attack A. Since dmodel,out = dmodel,in, we have pout t = pin t for t ≤ Lin, and extra layers leave positions unchanged.”

May combine two claims.

The effects of Attack A and Attack B differ but are both strictly above the baseline.

“Their effects differ but are both strictly above the baseline.”

May combine two claims.

For sinusoidal encodings under Attack B, with dmodel,out = m dmodel,in and m > 1, the circuit enforces p'_t = g(t, dmodel,out) while the replicate construction needs p'_t = R g(t, dmodel,in); in general these differ, so the witness cannot satisfy both and the mismatch makes Attack B ZK-detectable.

“With dmodel,out = mdmodel,in and m > 1, the circuit enforces p′t = g(t, dmodel,out), while we would need p′t = Rg(t, dmodel,in) for the replicate construction. In general g(t, dmodel,out) ̸= Rg(t, dmodel,in), so the witness cannot satisfy both the circuit equation and the desired replicated hidden state. The resulting mismatch at the first layer makes Attack B ZK-detectable”

• AssumptionRests on an assumption: The verifier's circuit is locked to the declared outer architecture and checks exact arithmetic relations.

May combine two claims.

Zero-knowledge LLM inference proofs certify equation-level correctness (output consistent with committed private weights under the declared architecture) but do not bind or attest to the computational effort expended to produce the output.

“we show that it does not bind the effort expended to produce the output.”

• Also stated in: § 1. Introduction · plain

  “it verifies the correctness of the equations that define the model size and architecture, but not the effort expended to calculate the result.”

• Also stated in: § 2.2. Zero-knowledge LLM inference · plain

  “acceptance establishes equation-level correctness of the claimed inference relative to the declared statement, but does not by itself certify how much computation or execution effort was expended to produce that output.”

May combine two claims.

In Attack A, the served model is embedded as a prefix of the declared layers, and all remaining blocks are configured to behave as algebraic identities.

“The served model is embedded as a prefix of the declared layers, and all remaining blocks are configured to behave as algebraic identities.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Extreme outer/inner gaps may create detectable quality shifts or behavioral anomalies.

“Extreme outer/inner gaps may create detectable quality shifts or behavioral anomalies.”

Classified as probably not a substantial claim.

Since transformations operate only on Aout and not on the runtime semantics of Ain, evaluating correctness does not require a large or instruction-heavy model.

“Since transformations operate only on Aout and not on the runtime semantics of Ain, evaluating correctness does not require a large or instruction-heavy model.”

May combine two claims.

The attack exposes large-model proving complexity while avoiding the execution overhead of a large model.

“the attack exposes large-model proving complexity while avoiding the execution overhead of a large model.”

Classified as probably not a substantial claim.

This guarantee is commitment-relative: it ties the output to the committed private weights under the public architecture, rather than to any external reference model or public weight hash.

“This guarantee is commitment-relative: it ties the output to the committed private weights under the public architecture, rather than to any external reference model or public weight hash.”

May combine two claims.

Attack B inflates the declared model dimension by replicating the hidden state across m coordinate blocks, arranges attention and FFN weights in block-diagonal/repeated form to forbid cross-block mixing, and collapses the replicas at the output so that logits match those of Ain exactly.

“Attack B mirrors the same template along the width axis: it inflates the declared model dimension by replicating the hidden state across m coordinate blocks, arranges attention and FFN weights in block-diagonal / repeated form to forbid cross-block mixing, and then collapses the replicas at the output so that logits match those of Ain exactly (Figure 5).”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Both Attack A and Attack B modify only the shape of the public circuit and are insensitive to prompt semantics, token distribution, or linguistic variation.

“both Attack A and Attack B modify only the shape of the public circuit and are insensitive to prompt semantics, token distribution, or linguistic variation.”

May combine two claims.

In Attack B with an untied decoder, because logits are identical, the softmax outputs and next-token distribution are unchanged, and the verifier sees exactly the same equations as in an honest run of Ain.

“Therefore, the softmax outputs and the next-token distribution are unchanged, and the verifier sees exactly the same equations as in an honest run of Ain.”

May combine two claims.

In a hollow LLM deployment, all public interfaces remain intact; the layer layout, tensor shapes, and stated parameter count match the declared model, so both the proof and associated metadata appear legitimate.

“In this hollow LLM deployment, all public interfaces remain intact. The layer layout, tensor shapes, and stated parameter count all match the declared model, so both the proof and associated metadata appear legitimate.”

Classified as probably not a substantial claim. May combine two claims.

Using a small model enables complete end-to-end proving while keeping the methodology controlled and fully reproducible.

“Using a small model enables complete end-to-end proving while keeping the methodology controlled and fully reproducible.”

Classified as probably not a substantial claim.

In benign optimization scenarios, the model preserves its advertised structure and behavior and is simply executed more efficiently.

“in benign optimization scenarios, the model preserves its advertised structure and behavior and is simply executed more efficiently.”

Probably not a substantial claim: may not pass S4 (based).

Typically the outer architecture strictly expands the inner architecture (e.g., deeper or wider), enabling algebraic alignment between the outer equations and a cheaper computation path.

“Typically Aout strictly expands Ain (e.g., deeper or wider), enabling algebraic alignment between the outer equations and a cheaper computation path.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

A declared architecture that vastly exceeds what users expect for a given service or device may break immersion and raise suspicion.

“A declared architecture that vastly exceeds what users expect for a given service or device may break immersion and raise suspicion.”

• AssumptionRests on an assumption: Users cannot directly observe the true model size or workload.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

The serve/prove separation enabled by the Hollow-LLM attack is agnostic to the underlying model family and is expected to hold across a wide range of scale ratios.

“This serve/prove separation enabled by the Hollow-LLM attack is agnostic to the underlying model family and is expected to hold across a wide range of scale ratios”

Classified as probably not a substantial claim.

A zero-knowledge proof of inference certifies membership in an NP relation: there exist private weights such that the output is the result of executing the public model architecture on the input.

“a zero-knowledge proof of inference certifies membership in a nondeterministic polynomial-time (NP) relation: it proves that there exist private weights such that the output is the result of executing the public model architecture on the input.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Zero-knowledge proof technology remains fundamentally sound as a cryptographic primitive.

“which remains fundamentally sound as a cryptographic primitive.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The Hollow-LLM attack does not imply any flaw in zero-knowledge proof technology.

“the Hollow-LLM attack does not imply any flaw in zero-knowledge proof technology”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Adding sparsity-related subproofs for each parameter matrix introduces substantial overhead and undermines system scalability.

“adding sparsity-related subproofs for each parameter matrix introduces substantial overhead and undermines system scalability.”

May combine two claims. The quote only partly states this claim.

Learned encodings and RoPE remain compatible with both attacks after simple copying or replication, whereas deterministic sinusoidal encodings still support Attack A but can rule out Attack B.

“Learned tables and RoPE-style schemes can be aligned with both attacks by copying or replicating positional vectors in a way that respects Section 4.1.”

May combine two claims.

Consequently, the computation follows that of the smaller inner model while the verifier continues to endorse the claimed outer model.

“Consequently, the computation follows that of the smaller inner model while the verifier continues to endorse the claimed outer model.”

May combine two claims.

The 1/m rescaling in Attack B must happen at the last LayerNorm or immediately after the final residual block; scaling at the beginning would reintroduce unscaled contributions through residual adds and sums would no longer match the inner representation.

“It is important that this 1/m rescaling happens at the last LayerNorm (or immediately after the final residual block).”

May combine two claims.

These expansions act locally and compose without interference.

“confirming that these expansions act locally and compose without interference.”

May combine two claims.

LayerNorm replicate invariance is the core algebraic fact that makes Attack B possible, allowing width inflation by factor m while preserving normalized activations that drive attention and FFN computation.

“This LayerNorm replicate invariance is the core algebraic fact that makes Attack B possible: we can inflate the width by a factor m while preserving the normalized activations that drive attention and FFN computation.”

May combine two claims.

A zero-knowledge proof of inference does not attest to the algorithmic path taken to obtain the result or how much computation was necessary to get the result.

“However, the proof does not attest to the algorithmic path taken to obtain that result or how much computation was necessary to get the result.”

May combine two claims.

Ghost weights can be chosen so that the declared layers either perform negligible computation or reuse the computation of the much smaller inner model.

“These ghost weights are chosen so that the declared layers either perform negligible computation or reuse the computation of the much smaller Inner Model.”

May combine two claims.

The system is no longer faithfully executing the advertised workload; it only mimics the outputs of the large model.

“the system is no longer faithfully executing the advertised workload; it only mimics the outputs of the large model.”

May combine two claims.

The cost of generating a ZKP for LLM inference is order-of-magnitude times that of inference serving and can be scheduled asynchronously (21.8s for GPT-2 using 32 threads).

“the cost of generating a ZKP for LLM inference is order-of-magnitude times that of inference serving and can be scheduled asynchronously (21.8s for GPT-2 using 32 threads [7]).”

• cites [7] Qu, 2025, zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM InferenceNot checked yet

  Not checked in the free run.

May combine two claims.

This creates strong incentives to exaggerate model scale and challenges emerging zkML narratives that equate proof validity with faithful execution effort.

“This creates strong incentives to exaggerate model scale and challenges emerging zkML narratives that equate proof validity with faithful execution effort.”

May combine two claims.

In Attack B with a tied decoder, rescaling the final hidden representation by 1/m right before applying the tied decoder preserves tying and makes the logits exactly equal to those of the inner model.

“so tying is preserved and the logits are exactly equal to those of the inner model.”

May combine two claims.

In practice, inference results may be returned without real-time proof for non-interactive ZK proofs; when a call is selected for audit, the provider can reconstruct a witness for the already-served transcript and produce a proof off the critical path.

“In practice, inference results may be returned without real-time proof for non-interactive ZK proofs ( [7]): when a specific call is selected for audit, the provider can reconstruct a witness for the already-served transcript (by re-running the deterministic computation or using cached intermediates) and produces a proof π off the critical path.”

• cites [7] Qu, 2025, zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM InferenceNot checked yet

  Not checked in the free run.

May combine two claims.

This regime avoids the overhead of attaching a ZK proof to every response, while still enabling post-hoc audits that deter blatant cheating.

“This regime avoids the overhead of attaching a ZK proof to every response, while still enabling post-hoc audits that deter blatant cheating.”

May combine two claims.

Attack A increases the declared depth and optionally the FFN width while keeping the effective computation depth Lin equal to that of the inner model Ain.

“Attack A increases the declared depth and, optionally, the FFN width of the model while keeping the effective computation depth Lin equal to that of the inner model Ain.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

TEEs rely on specialized hardware and trust assumptions and do not provide public verifiability.

“TEEs rely on specialized hardware and trust assumptions and do not provide public verifiability.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Challenge-based audits, ablation tests, and diversity probes could stress-test the model's response surface and make low-effort deployments more risky.

“Collectively, these techniques could stress-test the model's response surface and make low-effort deployments more risky.”

Classified as probably not a substantial claim.

An attacker may still be able to design ghost weights that satisfy non-triviality constraints while preserving the same low-effort computation path.

“An attacker may still be able to design ghost weights that satisfy non-triviality constraints while preserving the same low-effort computation path.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

A proof that no alternative ghost-weight constructions exist would be required, yet such a guarantee may be generally difficult to achieve.

“A proof that no alternative ghost-weight constructions exist would be required, yet such a guarantee may be generally difficult to achieve.”

Probably not a substantial claim: may not pass S2 (checkable).

The findings challenge emerging narratives around zkML-based assurance.

“challenge emerging narratives around zkML-based assurance.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

TEEs lack public verifiability, depend on proprietary hardware, and remain vulnerable to side channels or firmware-level exploits.

“TEEs still inherit limitations: they lack public verifiability, depend on proprietary hardware, and remain vulnerable to side channels or firmware-level exploits.”

Probably not a substantial claim: may not pass S3 (load-bearing).

zkGPT verifies GPT-2 inference in roughly 25 seconds.

“zkGPT verifies GPT-2 inference in roughly 25 seconds.”

• cites [7] Qu, 2025, zkGPT: An Efficient Non-interactive Zero-knowledge Proof Framework for LLM InferenceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

In scenarios that require strong trustless guarantees or open verification, ZK-proof-based approaches provide a more robust, although higher-latency, alternative.

“In scenarios that require strong trustless guarantees or open verification, ZK-proof–based approaches provide a more robust, although higher-latency, alternative.”