Categorizing Variants of Goodhart's Law
Open accessAbstract
There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an extent that further optimization is ineffective or harmful, and is sometimes termed Goodhart's Law. This class of failure is often poorly understood, partly because terminology for discussing them is ambiguous, and partly because discussion using this ambiguous terminology ignores distinctions between different failure modes of this general type. This paper expands on an earlier discussion by Garrabrant, which notes there are "(at least) four different mechanisms" that relate to Goodhart's Law. This paper is intended to explore these mechanisms further, and specify more clearly how they occur. This discussion should be helpful in better understanding these types of failures in economic regulation, in public policy, in machine learning, and in Artificial Intelligence alignment. The importance of Goodhart effects depends on the amount of power directed towards optimizing the proxy, and so the increased optimization power offered by artificial intelligence makes it especially critical for that field.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Approval and star-rating measures become examples of Goodhart's Law.
“Star ratings and approval metrics then become exemplars of Goodhart's Law (the principle that when a measure becomes a target, it ceases to be a reliable measure).23”
From When a Human‐in‐the‐Loop Amplifies the Risk of Misalignment: Considerations for Research Ethics Oversight (2026) · cites this paper as [23]
“Varieties of Goodhart-like Phenomena” (p. 1) defines a Goodhart effect as optimization breaking the relationship between a goal and its proxy, supporting the parenthetical explanation. The paper does not identify star ratings or approval metrics as examples, so it does not substantiate that specific application. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 2
- [3][3] Jeffery Rodamar, U.S. Department of Education There ought to be a law! Campbell v. Goodhart Significance, Vol. 15, 2018. doi: 10.1111/j.17409713.2018.01205.x - · DOI
- [6][6] Donald T. Campbell 'Assessing the impact of planned social change Evaluation and Program Planning. 2 (1): 6790. doi:10.1016/0149-7189(79)90048X - · DOI
Cited by
Sorted by citations
How to Design AI for Social Good: Seven Essential Factors
Floridi, L., Cowls, J., King, T. C., & Taddeo, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00213-5 · Cited by 430 · see citing works
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101 · see citing works
“Applying strong optimization pressure for an imperfect proxy measure for a goal tends to cause poor performance on the underlying target goal [Hoskin, 1996, Manheim and Garrabrant, 2018, Gao et al., 2022].”
BackgroundThe alignment problem from a deep learning perspective
Ngo, R., Chan, L., & Mindermann, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00626 · Cited by 68 · see citing works
“One underlying reason for this difficulty is Goodhart’s law: when a measure becomes a target, it ceases to be a good measure [Manheim and Garrabrant, 2018].”
BackgroundDetecting anomalous citation groups in journal networks
Kojaku, S., Livan, G., & Masuda, N. (2021) · Scientific Reports · doi:10.1038/s41598-021-93572-3 · Cited by 53 · see citing works
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Dalrymple, D. "., Skalse, J., Bengio, Y., Russell, S., Tegmark, M., Seshia, S., Omohundro, S., Szegedy, C., Goldhaber, B., Ammann, N., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.06624 · Cited by 12 · see citing works
SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents
Zhao, B., Srikanth, D., Wu, Y., & Jiang, Z. (2026) · arXiv · doi:10.48550/arxiv.2605.21384 · Cited by 1 · see citing works
Taylor, E. (2026) · Ethics & Human Research · doi:10.1002/eahr.70024
At Equal Inference Cost, Multi-Agent Structure Does Not Beat a Single Frozen Agent
Dylan, D., Brennan, A., Murphy, C., O'Sullivan, N., Kelly, C., & Walsh, S. (2026) · arXiv · doi:10.48550/arxiv.2609.04217
Evaluating and Guarding Citation Faithfulness in Agentic Scientific Synthesis
Goo, T., Kim, J., Han, K., Jo, G., Kim, J.-S., & Kim, T.-H. (2026) · arXiv · doi:10.48550/arxiv.2607.20527
- Authors
- Manheim, D., & Garrabrant, S.
- ORCID
- Manheim, David
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- arXiv
- 1803.04585
- OpenAlex
- W2793512604
- Cited by
- 73
- References
- 0
- Citations by year
- 2026: 9 · 2025: 2 · 2024: 7 · 2023: 32 · 2022: 3 · 2021: 4 · 2020: 9 · 2019: 3 · 2018: 4
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Economic Theory and Institutions
- Social Sciences › Business, Management and Accounting › Strategy and Management › Regulation and Compliance Studies
- Keywords
- Goodhart's law, proxy metrics, proxy optimization, AI alignment, economic regulation, public policy, machine learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
