The DOI matches its Crossref record; 327 of the 327 references it cites were found in a registry or an index.
Experimental practices in economics: A challenge for psychologists?: (683322011-032)
Abstract
This target article is concerned with the implications of the surprisingly different experimental practices in economics and in areas of psychology relevant to both economists and psychologists, such as behavioral decision making. We consider four features of experimentation in economics, namely, script enactment, repeated trials, performance-based monetary payments, and the proscription against deception, and compare them to experimental practices in psychology, primarily in the area of behavioral decision making. Whereas economists bring a precisely defined "script" to experiments for participants to enact, psychologists often do not provide such a script, leaving participants to infer what choices the situation affords. By often using repeated experimental trials, economists allow participants to learn about the task and the environment; psychologists typically do not. Economists generally pay participants on the basis of clearly defined performance criteria; psychologists usually pay a flat fee or grant a fixed amount of course credit. Economists virtually never deceive participants; psychologists, especially in some areas of inquiry, often do. We argue that experimental standards in economics are regulatory in that they allow for little variation between the experimental practices of individual researchers. The experimental standards in psychology, by contrast, are comparatively laissez-faire. We believe that the wider range of experimental practices in psychology reflects a lack of procedural regularity that may contribute to the variability of empirical findings in the research fields under consideration. We conclude with a call for more research on the consequences of methodological preferences, such as the use on monetary payments, and propose a "do-it-both-ways" rule regarding the enactment of scripts, repetition of trials, and performance-based monetary payments. We also argue, on pragmatic grounds, that the default practice should be not to deceive participants.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,395
The Structure of Scientific Revolutions
Kuhn, T. S., & Schlegel, R. (1963) · Physics Today · doi:10.1063/1.3050879 · Cited by 35,538
Nonparametric Statistics for the Behavioral Sciences
Guitton, H., & Siegel, S. (1958) · Revue économique · doi:10.2307/3498751 · Cited by 35,245
Kytle, J., & Bandura, A. (1978) · Contemporary Sociology · doi:10.2307/2065952 · Cited by 25,422
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497
A Theory of Cognitive Dissonance
(2017) · Macat Library eBooks · doi:10.4324/9781912282432 · Cited by 18,122
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116
Advances in prospect theory: Cumulative representation of uncertainty
Tversky, A., & Kahneman, D. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122574 · Cited by 15,467
Cited by
The neural basis of economic decision-making in the Ultimatum Game.
Sanfey, A. G., Rilling, J. K., Aronson, J. A., Nystrom, L. E., & Cohen, J. D. (2003) · Science · doi:10.1126/science.1082976 · Cited by 3,375
What Do Laboratory Experiments Measuring Social Preferences Reveal About the Real World?
Levitt, S. D., & List, J. A. (2007) · Journal of Economic Perspectives · doi:10.1257/jep.21.2.153 · Cited by 2,668
Investor Psychology and Asset Pricing
(2001) · The Journal of Finance · doi:10.1111/0022-1082.00379 · Cited by 2,357
The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework
Camerer, C. F., Hogarth, R. M., Budescu, D. V., & Eckel, C. (1999) · Elicitation of Preferences · doi:10.1007/978-94-017-1406-8_2 · Cited by 1,545
Decisions from Experience and the Effect of Rare Events in Risky Choice
Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004) · Psychological Science · doi:10.1111/j.0956-7976.2004.00715.x · Cited by 1,225
Self-citationExperimenter demand effects in economic experiments
Zizzo, D. J. (2010) · Experimental Economics · doi:10.1007/s10683-009-9230-z · Cited by 1,194
Leadership, creativity, and innovation: A critical review and practical recommendations
Hughes, D. J., Lee, A., Tian, A. W., Newman, A., & Legood, A. (2018) · The Leadership Quarterly · doi:10.1016/j.leaqua.2018.03.001 · Cited by 1,080
Constructivist and Ecological Rationality in Economics
Smith, V. L. (2003) · American Economic Review · doi:10.1257/000282803322156954 · Cited by 960
Johnson, N. D., & Mislin, A. A. (2011) · Journal of Economic Psychology · doi:10.1016/j.joep.2011.05.007 · Cited by 955
Dunning, D. (2011) · Advances in Experimental Social Psychology · doi:10.1016/b978-0-12-385522-0.00005-6 · Cited by 806
Loss Aversion Under Prospect Theory: A Parameter-Free Measurement
Abdellaoui, M., Bleichrodt, H., & Paraschiv, C. (2007) · Management Science · doi:10.1287/mnsc.1070.0711 · Cited by 783
The priority heuristic: Making choices without trade-offs.
Brandstätter, E., Gigerenzer, G., & Hertwig, R. (2006) · Psychological Review · doi:10.1037/0033-295x.113.2.409 · Cited by 745
Self-citationPersonality and prosocial behavior: A theoretical framework and meta-analysis.
Thielmann, I., Spadaro, G., & Balliet, D. (2020) · Psychological Bulletin · doi:10.1037/bul0000217 · Cited by 689
The Methodology of Experimental Economics
Guala, F. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511614651 · Cited by 582
Promises and lies: Restoring violated trust
Schweitzer, M. E., Hershey, J. C., & Bradlow, E. T. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.05.005 · Cited by 529
Marx, S. M., Weber, E. U., Orlove, B. S., Leiserowitz, A., Krantz, D. H., Roncoli, C., & Phillips, J. (2007) · Global Environmental Change · doi:10.1016/j.gloenvcha.2006.10.004 · Cited by 499
Cumulative prospect theory's functional menagerie
Stott, H. P. (2006) · Journal of Risk and Uncertainty · doi:10.1007/s11166-006-8289-6 · Cited by 482
On Adaptation, Maximization, and Reinforcement Learning Among Cognitive Strategies.
Erev, I., & Barron, G. (2005) · Psychological Review · doi:10.1037/0033-295x.112.4.912 · Cited by 475
On doing better science: From thrill of discovery to policy implications
Antonakis, J. (2017) · The Leadership Quarterly · doi:10.1016/j.leaqua.2017.01.006 · Cited by 475
Aspects of endowment: A query theory of value construction.
Johnson, E. J., Häubl, G., & Keinan, A. (2007) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.33.3.461 · Cited by 472
Age differences in risky choice: a meta‐analysis
Mata, R., Josef, A. K., Samanez‐Larkin, G. R., & Hertwig, R. (2011) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.2011.06200.x · Cited by 426
Self-citationBeyond Collective Entities: Multilevel Research on Organizational Routines and Capabilities
Salvato, C., & Rerup, C. (2011) · Journal of Management · doi:10.1177/0149206310371691 · Cited by 419
The Role of Representative Design in an Ecological Approach to Cognition.
Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.6.959 · Cited by 408
Self-citationEliciting Willingness to Pay Without Bias: Evidence from a Field Experiment
Blumenschein, K., Blomquist, G. C., Johannesson, M., Horn, N., & Freeman, P. (2008) · The Economic Journal · doi:10.1111/j.1468-0297.2007.02106.x · Cited by 377
Krueger, J. I., & Funder, D. C. (2004) · Behavioral and Brain Sciences · doi:10.1017/s0140525x04000081 · Cited by 372
Teaching Bayesian reasoning in less than two hours.
Sedlmeier, P., & Gigerenzer, G. (2001) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.130.3.380 · Cited by 366
Audience effects on moralistic punishment
Kurzban, R., DeScioli, P., & O'Brien, E. (2007) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2006.06.001 · Cited by 360
The effects of (incentivized) belief elicitation in public goods experiments
Gächter, S., & Renner, E. (2010) · Experimental Economics · doi:10.1007/s10683-010-9246-4 · Cited by 359
On doing relevant and rigorous experiments: Review and recommendations
Lonati, S., Quiroga, B. F., Zehnder, C., & Antonakis, J. (2018) · Journal of Operations Management · doi:10.1016/j.jom.2018.10.003 · Cited by 354
A re‐examination of probability matching and rational choice
Shanks, D. R., Tunney, R. J., & McCarthy, J. D. (2002) · Journal of Behavioral Decision Making · doi:10.1002/bdm.413 · Cited by 322
Madden, G. J., Raiff, B. R., Lagorio, C. H., Begotka, A. M., Mueller, A. M., Hehli, D. J., & Wegener, A. A. (2004) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.12.4.251 · Cited by 314
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500005611 · Cited by 304
The cognitive basis of model-based reasoning in science
Nersessian, N. J. (2002) · The Cognitive Basis of Science · doi:10.1017/cbo9780511613517.008 · Cited by 313
Ever Failed, Try Again, Succeed Better: Results from a Randomized Educational Intervention on Grit
Alan, S., Boneva, T., & Ertac, S. (2019) · The Quarterly Journal of Economics · doi:10.1093/qje/qjz006 · Cited by 313
Podsakoff, P. M., & Podsakoff, N. P. (2019) · The Leadership Quarterly · doi:10.1016/j.leaqua.2018.11.002 · Cited by 312
Framing decisions: Hypothetical and real
Kühberger, A., Schulte-Mecklenbeck, M., & Perner, J. (2002) · Organizational Behavior and Human Decision Processes · doi:10.1016/s0749-5978(02)00021-3 · Cited by 304
Belief elicitation in experiments: is there a hedging problem?
Blanco, M., Engelmann, D., Koch, A. K., & Normann, H.-T. (2010) · Experimental Economics · doi:10.1007/s10683-010-9249-1 · Cited by 297
Viewpoint: On the generalizability of lab behaviour to the field
Levitt, S. D., & List, J. A. (2007) · Canadian Journal of Economics/Revue canadienne d'économique · doi:10.1111/j.1365-2966.2007.00412.x · Cited by 289
The contributions of oxytocin and vasopressin pathway genes to human behavior
Ebstein, R. P., Knafo, A., Mankuta, D., Chew, S. H., & Lai, P. S. (2012) · Hormones and Behavior · doi:10.1016/j.yhbeh.2011.12.014 · Cited by 283
The genetic architecture of economic and political preferences
Benjamin, D. J., Cesarini, D., van der Loos, M. J. H. M., Dawes, C. T., Koellinger, P. D., Magnusson, P. K. E., Chabris, C. F., Conley, D., Laibson, D., Johannesson, M., et al. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1120666109 · Cited by 279
The Cognitive Basis of Science
(2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511613517 · Cited by 276
A meta-analytic review of two modes of learning and the description-experience gap.
Wulff, D. U., Mergenthaler-Canseco, M., & Hertwig, R. (2018) · Psychological Bulletin · doi:10.1037/bul0000115 · Cited by 271
Self-citationOn the External Validity of Social Preference Games: A Systematic Lab-Field Study
Galizzi, M. M., & Navarro-Martinez, D. (2019) · Management Science · doi:10.1287/mnsc.2017.2908 · Cited by 270
Reading people’s minds from emotion expressions in interdependent decision making.
de Melo, C. M., Carnevale, P. J., Read, S. J., & Gratch, J. (2014) · Journal of Personality and Social Psychology · doi:10.1037/a0034251 · Cited by 260
Artificiality: The tension between internal and external validity in economic experiments
Schram, A. (2005) · Journal of Economic Methodology · doi:10.1080/13501780500086081 · Cited by 260
A parametric analysis of prospect theory’s functionals for the general population
Booij, A. S., van Praag, B. M. S., & van de Kuilen, G. (2010) · Theory and Decision · doi:10.1007/s11238-009-9144-4 · Cited by 258
Incentivizing High Quality Crowdwork
Ho, C.-J., Slivkins, A., Suri, S., & Vaughan, J. W. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741102 · Cited by 146
Decision making and learning while taking sequential risks.
Pleskac, T. J. (2008) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.34.1.167 · Cited by 121
Effects of Financial Incentives on the Breakdown of Mutual Trust
Parco, J. E., Rapoport, A., & Stein, W. E. (2002) · Psychological Science · doi:10.1111/1467-9280.00454 · Cited by 74
Social influence in adolescence as a double-edged sword
Molleman, L., Ciranka, S. K., & van den Bos, W. (2021) · doi:10.31234/osf.io/gcbdf · Cited by 60
Schulze, C., & Hertwig, R. (2021) · Cognition · doi:10.1016/j.cognition.2020.104580 · Cited by 36
Self-citationThe influence of prior belief on scientific thinking
Evans, J. S. B. T. (2002) · The Cognitive Basis of Science · doi:10.1017/cbo9780511613517.011 · Cited by 22
Risk of injury in moral dilemmas with autonomous vehicles
de Melo, C. M., Marsella, S., & Gratch, J. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2020.572529 · Cited by 17
Optimal design of experiments to identify latent behavioral types
Balietti, S., Klein, B., & Riedl, C. (2021) · Experimental Economics · doi:10.1007/s10683-020-09680-w · Cited by 18
Metadata
- Authors
- Hertwig, R., & Ortmann, A.
- Venue
- PsycEXTRA Dataset
- Published
- 2000
- Volume
- 24
- Issue
- 3
- Pages
- 383-403; discussion 403
- OpenAlex
- W2122028729
- Cited by
- 668
- References
- 327
- Type
- dataset
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.