A Theory for Emergence of Complex Skills in Language Models

The emergence of skills arises from training on related tasks that were implicitly solved while solving next-word prediction in the training dataset.

“One might thus posit, with some justification from the above-mentioned studies, that the emergence of skills arises from training on related tasks that were implicitly solved while solving next-word prediction in the training dataset.This is indeed our starting point.”

• cites Wei et al., 2022 sourcePartly supported

  Section 5.2 (p. 7) discusses how a language-modeling objective may facilitate downstream behavior, and Section 5.6 (p. 11) proposes investigating links between emergent tasks and similar training data. But the paper calls the cause of emergence an open question; it does not say that skills emerge because related tasks were implicitly solved during next-word prediction. (LLM, conf=0.82)

  Experimental — accuracy not yet measured
• cites Srivastava et al., 2022 sourcePartly supported

  The Introduction (section 1; page not provided) describes text continuation as a general capability, and section 3.4.2 reports that prerequisite skills can become apparent before a target capability. The paper does not establish the proposed explanation that emerging skills arise because related tasks were implicitly solved during next-word-prediction training. (LLM, conf=0.83)

  Experimental — accuracy not yet measured
• cites Schaeffer et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Under condition (8) and uniform distribution on skills and text pieces in the skill-cluster, irrespective of the details of GEN and CLOZE, for every subset Y of text pieces that contains at least θ fraction of text pieces, at least 1-α fraction of skills have at most βθkN1/N2 edges to Y.

“at least 1 -α fraction of skills have at most βθkN 1 /N 2 edges to Y (in other words, at most β times the number of edges a skill would be expected to have to text-pieces in Y ).”

• AssumptionThe authors state this as an assumption: Each text-piece requires exactly k skills for some k, and this set was drawn by iid sampling from an underlying measure on the set of skills.
• AssumptionThe authors state this as an assumption: The skill-graph has random edges.
• AssumptionThe authors state this as an assumption: In the Section 5 analysis, the test-stream consists of a single degree-k skill cluster, and the skills and text-pieces are uniformly distributed.

If k' > k, the performance curve for θ, k' lies below that for θ, k.

“If k ′ > k then the performance curve of θ, k ′ lies below that for k, θ.”

Corollary 11: In the same setting as Theorem 8, for integer k' ∈ [2, 1/θ], the conclusion of Theorem 8 holds for α, β pairs satisfying inequality (9).

“In the same setting as Theorem 8, for integer k ′ ∈ [2, 1/θ] the conclusion of that theorem holds also for α, β pairs satisfying”

If H(k'θ) < k'H(θ), the emergence curve from Corollary 11 dominates that derived from Theorem 8.

“Furthermore, if H(k ′ θ) < k ′ H(θ) the emergence curve from this expression dominates that derived from Theorem 8.”

If θ' < θ, the performance curve for θ', k lies below that for θ, k.

“If θ ′ < θ then the performance curve for θ ′ , k lies below that for θ, k.”

For a fixed prediction loss θ, using higher k implies better emergence of skills.

“Theorem 8 shows that for a fixed prediction loss θ, using higher k implies better emergence of skills.”

If the error per average text-piece is δ, then the error per average k'-piece is k'δ.

“Clearly, if the error of the model per average text-piece was δ, then the error per average b-piece is k ′ δ.”

The emergence curves shift down noticeably as k increases.

“The emergence curves shift down noticeably (i.e., imply emergence of more skills) as we increase k.”

When N and D are increased by a factor of 10, excess entropy reduces by roughly 10^0.28 ≈ 2.

“For example when N, D are increased by a factor of 10 it reduces by roughly (10) 0.28 ≈ 2.”

If the average cross-entropy loss for the text-pieces is δ, then Y consists of at most 2δ fraction of text pieces.

“If the average cross-entropy loss for the text-pieces is δ we conclude Y consists of at most 2δ fraction of text pieces.”

The lemma statements in Sections 5 and 5.1.1 hold in the general setting with measures, with cardinalities replaced by measure.

“In this setup, our lemma statements (e.g., Lemma 15 as well as the ones in Sections 5 and 5.1.1)hold --the claim is the same but with cardinalities replaced by measure!”

Lemma 12: For the skill-graph of k'-tuples, for any α, β > 0, β > 1, αβ < 1 satisfying (12), at least 1-α fraction of k'-tuples of skills have at most αβθθN^{1/βθ} fraction of their edges connected to Y.

“Then for any α, β > 0, β > 1, αβ < 1 satisfying (12) there are at least 1 -α fraction of k ′ -tuples of skills that have at most αβθθN 1 βθ fraction of their edges connected to Y .”

Lemma 15: For every positive integer k and θ ∈ [0,1], there are α, β > 0 with αβ ≤ 1 such that with probability almost 1, for every Y ⊆ V1 of size θN1, at least (1-α) fraction of vertices in V2 have at most βθD edges going to Y, where D = kN1/N2.

“For every positive integer k and θ ∈ [0, 1] there are α, β > 0 such that αβ ≤ 1 and the following holds with probability almost 1.For every Y ⊆ V 1 of size θN 1 , there are at least (1 -α) fraction of vertices in V 2 each of which has at most βθD edges going to Y , where D = kN 1 /N 2 is the expected degree of a node in V 2 .”

The relation k • E_t[loss(t)] = E_s[failure rate of statistical task τ_s] is incorrect.

“k • E t [loss(t)] = E s [failure rate of statistical task τ s ]. (Incorrect!)(7)”

Corollary 13: When model M1 with loss δ is scaled up so that M2 has loss δ/k', the performance curve inferred for k'-tuples of skills using M2 is identical to the curve inferred for individual skills on M1.

“When the model M 1 with loss δ is scaled up (e.g., as per equation ( 2)) so that the new model M 2 has loss δ/k ′ , then the performance curve inferred by our method for k ′ -tuples of skills using M 2 is identical to the curve inferred for individual skills on model M 1 .”

Competence at complex skills, which involve t-tuples of basic skills, emerges essentially at very similar scaling as competence on the elementary skills themselves.

“(c) A key example of slingshot generalization, that competence at”

• AssumptionThe authors state this as an assumption: LLM Scaling Laws, empirical relationships describing reduction in language modeling loss with model scale, hold and are assumed by the theory.
• AssumptionThe authors state this as an assumption: There exists a set of ground-truth skills, and text-pieces are generated by an unknown process that takes random tuples of skills and converts them into a text-piece whose comprehension requires those skills.
• AssumptionThe authors state this as an assumption: Understanding of a text-piece is testable by simple multiple-choice cloze questions inserted by another unknown process into the text at test time.
• AssumptionThe authors state this as an assumption: The pre-trained model's average multiclass prediction loss on cloze questions closely tracks, within a small multiplicative factor like 1.1, the excess cross-entropy of the model on classical next-word prediction.
• AssumptionThe authors state this as an assumption: Each text-piece requires exactly k skills for some k, and this set was drawn by iid sampling from an underlying measure on the set of skills.
• AssumptionThe authors state this as an assumption: The skill-graph has random edges.

The Scaling Laws imply a strong form of inductive bias that allows the pre-trained model to learn very efficiently (slingshot generalization).

“Mathematical analysis showing that the Scaling Laws imply a strong form of inductive bias that allows the pre-trained model to learn very efficiently.We informally call this slingshot generalization since, naively viewed, it appears to give competence levels at skills that violate usual generalization theory.”

• AssumptionThe authors state this as an assumption: LLM Scaling Laws, empirical relationships describing reduction in language modeling loss with model scale, hold and are assumed by the theory.

A statistical framework relates cross-entropy loss of LLMs to competence on the basic skills that underlie language tasks.

“(a) A statistical framework that relates cross-entropy loss of LLMs to competence on the basic skills that underlie language tasks.”

Reductions in excess entropy lead to improvements in model capabilities.

“Section 3 will argue that reductions in excess entropy lead to improvements in model capabilities.”

• Also stated in: § 3 (Mis)understanding, Excess entropy, and Cloze Questions · hedged

  “Thus reductions in excess entropy (which happen naturally due to scaling) will tend to squeeze out such errors.”

The A term in the scaling law (2) captures the entropy of language.

“In (2) the A term captures the entropy of language 4 .No model, however good, can achieve lower cross-entropy loss than A for large corpora.”

• AssumptionRests on an assumption: There is a ground-truth human distribution for generating the next word, which assigns probability to the next word given previous words.

The model's mistakes on downstream tasks (its misunderstandings) are captured by the excess entropy.

“We argue now that the model's mistakes on downstream tasks (i.e., its misunderstandings) are captured by the excess entropy, which as noted in Section 2.1 reduces by a constant factor each time the model is scaled up by an order of magnitude”

• AssumptionThe authors state this as an assumption: The pre-trained model's average multiclass prediction loss on cloze questions closely tracks, within a small multiplicative factor like 1.1, the excess cross-entropy of the model on classical next-word prediction.

k'-tuples that include more frequent skills will tend to emerge faster.

“Thus essentially the calculation says that k ′ -tuples that include more frequent skills will tend to emerge faster.”

This notion of competence does not capture out-of-distribution generalization.

“Note this notion of competence also does not capture out-of-distribution generalization (i.e.predict well when the distribution of text pieces changes).”

In our framework, proficiency in combinations of skills arises just as naturally as proficiency in individual skills, and need not require seeing examples of all or most combinations in training.

“One concrete example of this inductive bias is that in our framework proficiency in combinations of skills arises just as naturally as proficiency in the individual skills themselves, and need not require seeing examples of all (or even most) of these combinations in the training set.”

• AssumptionThe authors state this as an assumption: Normal language, down to short paragraph level, already utilizes multiple skills, mixed up randomly.

Reducing θ by a factor 2 (roughly one order of scaling up) raises competence on 2k'-tuples to at least the same level as what it was on k'-tuples before scaling.

“Theorem 14 and Corollary 13 implies that the effect of reducing θ by a factor 2 (which as per scaling laws corresponds to roughly one order of scaling up in model parameters) has the effect of raising competence on 2k ′ -tuples to at least the same level as what it was on k ′ -tuples before scaling.”

• AssumptionThe authors state this as an assumption: A Chinchilla-like scaling law holds: 10x up-scaling leads to a factor 2 reduction in excess entropy.

First signs of improvement on downstream tasks appear once the model's loss drops below some threshold.

“In terms of the emergence phenomenon, this corresponds to first signs of improvement on downstream tasks once the model's loss drops below some threshold.”

A larger model trained with 10^13 tokens would display proficiency in most 8-tuples of skills, despite not having seen most combinations in training.

“Then a larger model trained with 10 trillion tokens (10 13 ) -closer to the size of corpora used in training today's models-would display proficiency in most 8-tuples of skills, despite never not having seen most of those combinations in training”

• AssumptionThe authors state this as an assumption: A Chinchilla-like scaling law holds: 10x up-scaling leads to a factor 2 reduction in excess entropy.

If the model displays competency on even 10% of the k'-tuples of skills, then it must have acquired competence in k'-tuples that were not seen during training.

“Therefore if the model displays competency on even 10% of the k ′ -tuples of skills then it must have somehow acquired competence in k ′ -tuples that were not seen during training.”

Overall reduction in loss will drive emergence within individual clusters.

“Thus overall reduction in loss will drive emergence within individual clusters.”

• AssumptionRests on an assumption: The overall loss is the weighted sum of measure of text in the individual clusters.

After scaling by k orders of magnitude, performance on k'-tuples of skills is as good as performance on individual skills before scaling.

“After scaling by k orders of magnitude (i.e., increasing dataset size to c k ′ D tokens, where in the Chinchilla framework c is around 10) the performance on k ′ -tuples of skills is as good as what the performance was on individual skills before the scaling.”

• AssumptionThe authors state this as an assumption: A Chinchilla-like scaling law holds: 10x up-scaling leads to a factor 2 reduction in excess entropy.

No model, however good, can achieve lower cross-entropy loss than A for large corpora.

“No model, however good, can achieve lower cross-entropy loss than A for large corpora.”

If the scaling law is the same for both types of text, more complex text will be more effective at inducing skills.

“If the scaling law is same for both types of text (i.e., how θ reduces from scaling) our theorem predicts that more complex text will be more effective at inducing skills.”

A model that has not learned pronoun disambiguation will display huge excess entropy at many places in surrounding text.

“Given the frequency of ambiguous pronouns in usual English, one concludes that a model that has not learned pronoun disambiguation will display huge excess entropy at many places in surrounding text.”

More frequent skills tend to reach competence level quicker than less frequent skills.

“More frequent skills tend to reach competence level quicker than less frequent skills: This effect is hidden in the proof of Theorem 14.”

Better performance curves can be derived via a tensorization argument than by the above method.

“While the above method yields performance curves, better curves can be derived via a tensorization argument.”

Our analysis is allowed to assume an upper bound on the test loss, but the text-pieces on which this loss occurs form an arbitrary subset that depends upon the graph structure.

“In other words, our analysis is allowed to assume an upper bound on the test loss, but the text-pieces on which this loss occurs form an arbitrary subset that depends upon the graph structure.”

Reductions in excess entropy will tend to squeeze out errors such as pronoun disambiguation mistakes.

“Thus reductions in excess entropy (which happen naturally due to scaling) will tend to squeeze out such errors.”

In our framework, the language model also seems to defy paucity of stimulus with respect to skill combinations.

“In our framework, the language model (whose learning is initialized using Gaussian noise) also seems to defy paucity of stimulus with respect to skill combinations.”

• AssumptionThe authors state this as an assumption: As the model and data set size tend to infinity in tandem, the model will perfectly learn the language distribution.

Since the model saw part of the test stream corresponding to training data, it has picked some information about the skill cluster.

“Since the model”