The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Measuring and Bounding Experimenter Demand
Open accessAbstract
We propose a technique for assessing robustness to demand effects of findings from experiments and surveys. The core idea is that by deliberately inducing demand in a structured way we can bound its influence. We present a model in which participants respond to their beliefs about the researcher’s objectives. Bounds are obtained by manipulating those beliefs with “demand treatments.” We apply the method to 11 classic tasks, and estimate bounds averaging 0.13 standard deviations, suggesting that typical demand effects are probably modest. We also show how to compute demand-robust treatment effects and how to structurally estimate the model. (JEL C83, C90, D83, D91)
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2012) · American Economic Review · doi:10.1257/aer.102.7.i · Cited by 6,810
Gender Differences in Preferences
Croson, R., & Gneezy, U. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.2.448 · Cited by 5,694
Behavioral Study of obedience.
Milgram, S. (1963) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0040525 · Cited by 4,918
Identification and estimation of local average treatment effects
Imbens, G. W., & Angrist, J. D. (1994) · Econometrica · doi:10.2307/2951620 · Cited by 4,101
Orne, M. T. (1962) · American Psychologist · doi:10.1037/h0043424 · Cited by 4,041
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?
Levitt, S. D., & List, J. A. (2007) · Journal of Economic Perspectives · doi:10.1257/jep.21.2.153 · Cited by 2,668
Harrison, G. W., & List, J. A. (2004) · Journal of Economic Literature · doi:10.1257/0022051043004577 · Cited by 2,099
Hauser, D. J., & Schwarz, N. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0578-z · Cited by 1,739
Preferences, property rights, and anonymity in bargaining games.
Hoffman, E., McCabe, K., Shachat, K., & Smith, V. (1994) · Games and Economic Behavior · doi:10.1006/game.1994.1056 · Cited by 1,674
Experimental methods: Between-subject and within-subject design
Charness, G., Gneezy, U., & Kuhn, M. A. (2012) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2011.08.009 · Cited by 1,469
The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework
Camerer, C. F., & Hogarth, R. M. (1999) · Journal of Risk and Uncertainty · doi:10.1023/a:1007850605129 · Cited by 1,428
Do People Mean What They Say? Implications for Subjective Survey Data
Bertrand, M., & Mullainathan, S. (2001) · American Economic Review · doi:10.1257/aer.91.2.67 · Cited by 1,391
Experimenter demand effects in economic experiments
Zizzo, D. J. (2010) · Experimental Economics · doi:10.1007/s10683-009-9230-z · Cited by 1,194
Testing for Altruism and Social Pressure in Charitable Giving
DellaVigna, S., List, J. A., & Malmendier, U. (2012) · The Quarterly Journal of Economics · doi:10.1093/qje/qjr050 · Cited by 1,080
Lab Experiments Are a Major Source of Knowledge in the Social Sciences
Falk, A., & Heckman, J. J. (2009) · Science · doi:10.1126/science.1168244 · Cited by 1,033
Small, D. A., Loewenstein, G., & Slovic, P. (2007) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.01.005 · Cited by 984
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014) · American Journal of Political Science · doi:10.1111/ajps.12081 · Cited by 951
On the Interpretation of Giving in Dictator Games
List, J. A. (2007) · Journal of Political Economy · doi:10.1086/519249 · Cited by 918
Confidence Intervals for Partially Identified Parameters
Imbens, G. W., & Manski, C. F. (2004) · Econometrica · doi:10.1111/j.1468-0262.2004.00555.x · Cited by 660
Dictator game giving: altruism or artefact?
Bardsley, N. (2008) · Experimental Economics · doi:10.1007/s10683-007-9172-2 · Cited by 627
Dictator game giving: Rules of fairness versus acts of kindness.
Bolton, G. E., Katok, E., & Zwick, R. (1998) · International Journal of Game Theory · doi:10.1007/s001820050072 · Cited by 480
Evaluating Behaviorally Motivated Policy: Experimental Evidence from the Lightbulb Market
Allcott, H., & Taubinsky, D. (2015) · American Economic Review · doi:10.1257/aer.20131564 · Cited by 427
Experimental Economics From the Vantage‐point of Behavioural Economics
Loewenstein, G. (1999) · The Economic Journal · doi:10.1111/1468-0297.00400 · Cited by 421
What Motivates Effort? Evidence and Expert Forecasts
DellaVigna, S., & Pope, D. (2018) · The Review of Economic Studies · doi:10.1093/restud/rdx033 · Cited by 392
List, J. A. (2006) · Journal of Political Economy · doi:10.1086/498587 · Cited by 392
On expectations and the monetary stakes in ultimatum games
Hoffman, E., McCabe, K. A., & Smith, V. L. (1996) · International Journal of Game Theory · doi:10.1007/bf02425259 · Cited by 356
Dellavigna, S., List, J. A., Malmendier, U., & Rao, G. (2017) · The Review of Economic Studies · doi:10.1093/restud/rdw056 · Cited by 285
Examining the Role of Social Isolation on Stated Preferences
List, J. A., Berrens, R. P., Bohara, A. K., & Kerkvliet, J. (2004) · American Economic Review · doi:10.1257/0002828041464614 · Cited by 190
Selective Trials: A Principal-Agent Approach to Randomized Controlled Experiments
Chassang, S., Miquel, G. P. I., & Snowberg, E. (2012) · American Economic Review · doi:10.1257/aer.102.4.1279 · Cited by 180
“Do the right thing:” The effects of moral suasion on cooperation
Dal Bó, E., & Dal Bó, P. (2014) · Journal of Public Economics · doi:10.1016/j.jpubeco.2014.05.002 · Cited by 157
Fischhoff, B. (2010) · Wiley Interdisciplinary Reviews Cognitive Science · doi:10.1002/wcs.65 · Cited by 143
Scaling for Economists: Lessons from the Non-Adherence Problem in the Medical Literature
Al‐Ubaydli, O., List, J. A., LoRe, D., & Suskind, D. L. (2017) · The Journal of Economic Perspectives · doi:10.1257/jep.31.4.125 · Cited by 97
The white-man effect: How foreigner presence affects behavior in experiments
Cilliers, J., Dube, O., & Siddiqi, B. M. (2015) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2015.03.015 · Cited by 96
Big experimenter is watching you! Anonymity and prosocial behavior in the laboratory
Barmettler, F., Fehr, E., & Zehnder, C. (2011) · Games and Economic Behavior · doi:10.1016/j.geb.2011.09.003 · Cited by 64
Group status, minorities and trust
Tsutsui, K., & Zizzo, D. J. (2013) · Experimental Economics · doi:10.1007/s10683-013-9364-x · Cited by 52
Experiments on Decisions under Uncertainty: A Theoretical Framework
Shmaya, E., & Yariv, L. (2016) · American Economic Review · doi:10.1257/aer.20120978 · Cited by 46
A simple stress test of experimenter demand effects
Fleming, P., & Zizzo, D. J. (2014) · Theory and Decision · doi:10.1007/s11238-014-9419-2 · Cited by 25
How does communication affect beliefs in one-shot games with complete information?
Ellingsen, T., Östling, R., & Wengström, E. (2017) · Games and Economic Behavior · doi:10.1016/j.geb.2017.11.004 · Cited by 11
Cited by
Coppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
MTurk Research: Review and Recommendations
Aguinis, H., Villamor, I., & Ramani, R. S. (2021) · Journal of Management · doi:10.1177/0149206320969787 · Cited by 984
Demand Effects in Survey Experiments: An Empirical Assessment
MUMMOLO, J., & PETERSON, E. (2019) · American Political Science Review · doi:10.1017/s0003055418000837 · Cited by 548
Designing Information Provision Experiments
Haaland, I., Roth, C., & Wohlfart, J. (2023) · Journal of Economic Literature · doi:10.1257/jel.20211658 · Cited by 541
Self-citationWomen's Empowerment in Action: Evidence from a Randomized Control Trial in Africa
Bandiera, O., Buehren, N., Burgess, R., Goldstein, M., Gulesci, S., Rasul, I., & Sulaiman, M. (2020) · American Economic Journal: Applied Economics · doi:10.1257/app.20170416 · Cited by 311
Metadata
- Authors
- de Quidt, J., Haushofer, J., & Roth, C.
- Venue
- American Economic Review
- Publisher
- American Economic Association
- Published
- 2018-11-01
- Volume
- 108
- Issue
- 11
- Pages
- 3266-3302
- ISSN
- 0002-8282 (ISSN-L)
- OpenAlex
- W2622015926
- Cited by
- 568
- References
- 39
- FWCI
- 90.77
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.