The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
How effective is nudging? A quantitative review on the effect sizes and limits of empirical nudging studies
Abstract
Abstract Changes in the choice architecture, so-called nudges, have been employed in a variety of contexts to alter people's behavior. Although nudging has gained a widespread popularity, the effect sizes of its influences vary considerably across studies. In addition, nudges have proven to be ineffective or even backfire in selected studies which raises the question whether, and under which conditions, nudges are effective. Therefore, we conduct a quantitative review on nudging with 100 primary publications including 317 effect sizes from different research areas. We derive four key results. (1) A morphological box on nudging based on eight dimensions, (2) an assessment of the effectiveness of different nudging interventions, (3) a categorization of the relative importance of the application context and the nudge category, and (4) a comparison of nudging and digital nudging. Thereby, we shed light on the (in)effectiveness of nudging and we show how the findings of the past can be used for future research. Practitioners, especially government officials, can use the results to review and adjust their policy making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. (1989) · Technometrics · doi:10.1080/00401706.1989.10488618 · Cited by 99,167
Statistical power analysis for the behavioral sciences
(1990) · Computers, Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,269
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 1×
“For instance, they found that the way a decision is framed influences the outcome (Tversky and Kahneman, 1981) or that the availability of recalling any particular information determines the expected probability of its occurrence (Tversky and Kahneman, 1973).”
Background(2011) · Cited by 14,317
Nudge: improving decisions about health, wealth, and happiness
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0977 · Cited by 11,572
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984
Status quo bias in decision making
Samuelson, W., & Zeckhauser, R. (1988) · Journal of Risk and Uncertainty · doi:10.1007/bf00055564 · Cited by 5,897
Nudge: Improving Decisions About Health, Wealth and Happiness
Adkisson, R. V. (2008) · The Social Science Journal · doi:10.1016/j.soscij.2008.09.003 · Cited by 5,543
A perspective on judgment and choice: mapping bounded rationality.
Kahneman, D. (2003) · American Psychologist · doi:10.1037/0003-066x.58.9.697 · Cited by 5,540
Social norms and energy conservation
Allcott, H. (2011) · Journal of Public Economics · doi:10.1016/j.jpubeco.2011.03.003 · Cited by 2,802
A review of intervention studies aimed at household energy conservation
Abrahamse, W., Steg, L., Vlek, C., & Rothengatter, T. (2005) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2005.08.002 · Cited by 2,668
Meta-analysis and the science of research synthesis
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018) · Nature · doi:10.1038/nature25753 · Cited by 1,709 · cited 2×
“This problem of incomplete data reporting is common in many primary publications (Gurevitch et al., 2018).”
BackgroundAdvances in Behavioral Economics
(2004) · Princeton University Press eBooks · doi:10.1515/9781400829118 · Cited by 1,696
Reconstructing the Giant: On the Importance of Rigour in Documenting the Literature Search Process
Brocke, J. V., Simons, A., Niehaves, B., Reimer, K., Plattfaut, R., & Cleven, A. (2009) · Alexandria (UniSG) (University of St.Gallen) · Cited by 1,383
Pickering, C., & Byrne, J. (2014) · Higher Education Research & Development · doi:10.1080/07294360.2013.841651 · Cited by 1,334
How Many Studies Do You Need?: A Primer on Statistical Power for Meta-Analysis
Valentine, J. C., Pigott, T. D., & Rothstein, H. R. (2010) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998609346961 · Cited by 1,294
Nudging Farmers to Use Fertilizer: Theory and Experimental Evidence from Kenya
Duflo, E., Kremer, M., & Robinson, J. (2011) · American Economic Review · doi:10.1257/aer.101.6.2350 · Cited by 1,170
A method for taxonomy development and its application in information systems
Nickerson, R. C., Varshney, U., & Muntermann, J. (2013) · European Journal of Information Systems · doi:10.1057/ejis.2012.26 · Cited by 1,143
Can There Ever Be Too Many Options? A Meta-Analytic Review of Choice Overload
Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2010) · Journal of Consumer Research · doi:10.1086/651235 · Cited by 1,070 · cited 1×
“In a related meta-analysis on choice overload, the publications with high effect sizes were published in journals, while low effect sizes mainly remain unpublished data (Scheibehenne et al., 2010).”
BackgroundWheat From Chaff: Meta-Analysis As Quantitative Literature Review
Stanley, T. D. (2001) · Journal of Economic Perspectives · doi:10.1257/jep.15.3.131 · Cited by 948
Beyond nudges: Tools of a choice architecture
Johnson, E. J., Shu, S. B., Dellaert, B. G. C., Fox, C., Goldstein, D. G., Häubl, G., Larrick, R. P., Payne, J. W., Peters, E., Schkade, D., et al. (2012) · Marketing Letters · doi:10.1007/s11002-012-9186-1 · Cited by 881
Should Governments Invest More in Nudging?
Benartzi, S., Beshears, J., Milkman, K. L., Sunstein, C. R., Thaler, R. H., Shankar, M., Tucker-Ray, W., Congdon, W. J., & Galing, S. (2017) · Psychological Science · doi:10.1177/0956797617702501 · Cited by 861
Proceedings of the 18th European conference on information systems (ECIS2010)
Tervonen, T., Brock, D. B., Graeff, D. P., & Hillege, H. L. (2010) · Cited by 836
Sunstein, C. R. (2014) · Journal of Consumer Policy · doi:10.1007/s10603-014-9273-1 · Cited by 788
Behavioral Economics: Past, Present, Future
Camerer, C. F., & Loewenstein, G. F. (2003) · CaltechAUTHORS (California Institute of Technology) · Cited by 733
Costa, D. L., & Kahn, M. E. (2013) · Journal of the European Economic Association · doi:10.1111/jeea.12011 · Cited by 656
Weinmann, M., Schneider, C., & Brocke, J. V. (2016) · Business & Information Systems Engineering · doi:10.1007/s12599-016-0453-1 · Cited by 553
Bucher, T., Collins, C., Rollo, M. E., McCaffrey, T. A., De Vlieger, N., Van der Bend, D., Truby, H., & Perez-Cueto, F. J. A. (2016) · British Journal of Nutrition · doi:10.1017/s0007114516001653 · Cited by 528
Which Healthy Eating Nudges Work Best? A Meta-Analysis of Field Experiments
Cadario, R., & Chandon, P. (2020) · Marketing Science · doi:10.1287/mksc.2018.1128 · Cited by 504
Arno, A., & Thomas, S. (2016) · BMC Public Health · doi:10.1186/s12889-016-3272-x · Cited by 495
New business models for electric cars—A holistic approach
Kley, F., Lerch, C., & Dallinger, D. (2011) · Energy Policy · doi:10.1016/j.enpol.2011.03.036 · Cited by 414
TESTING ENFORCEMENT STRATEGIES IN THE FIELD: THREAT, MORAL APPEAL AND SOCIAL INFORMATION
Fellner, G., Sausgruber, R., & Traxler, C. (2013) · Journal of the European Economic Association · doi:10.1111/jeea.12013 · Cited by 370
A Review and Taxonomy of Choice Architecture Techniques
Münscher, R., Vetter, M., & Scheuerle, T. (2016) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1897 · Cited by 348
Your Location has been Shared 5,398 Times!: A Field Study on Mobile App Privacy Nudging
Almuhimedi, H., Schaub, F., Sadeh, N., Adjerid, I., Acquisti, A., Gluck, J., Cranor, L. F., & Agarwal, Y. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702210 · Cited by 322
Wilson, A. L., Buckley, E., Buckley, J. D., & Bogomolova, S. (2016) · Food Quality and Preference · doi:10.1016/j.foodqual.2016.02.009 · Cited by 308
SUNSTEIN, C. R. (2017) · Behavioural Public Policy · doi:10.1017/bpp.2016.3 · Cited by 302
Demarque, C., Charalambides, L., Hilton, D. J., & Waroquier, L. (2015) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2015.06.008 · Cited by 254
Do You Have a Voting Plan?: Implementation Intentions, Voter Turnout, and Organic Plan Making
Nickerson, D. W., & Rogers, T. (2010) · Psychological Science · doi:10.1177/0956797609359326 · Cited by 244
American attitudes toward nudges
Jung, J. Y., & Mellers, B. A. (2016) · Judgment and Decision Making · doi:10.1017/s1930297500007592 · Cited by 241
Don't Forget to Vote: Text Message Reminders as a Mobilization Tool
Dale, A., & Strauss, A. (2009) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00401.x · Cited by 225
Is There a Right Way to Nudge? The Practice and Ethics of Choice Architecture
Selinger, E., & Whyte, K. (2011) · Sociology Compass · doi:10.1111/j.1751-9020.2011.00413.x · Cited by 214
Adam, A., & Jensen, J. D. (2016) · BMC Public Health · doi:10.1186/s12889-016-3985-x · Cited by 199
Experimental pretesting of hand-washing interventions in a natural setting
Judah, G., Aunger, R., Schmidt, W.-P., Michie, S., Granger, S., & Curtis, V. (2009) · American Journal of Public Health · doi:10.2105/ajph.2009.164160 · Cited by 196
When should the Ask be a Nudge? The Effect of Default Amounts on Charitable Donations
Goswami, I., & Urminsky, O. (2016) · Journal of Marketing Research · doi:10.1509/jmr.15.0001 · Cited by 188
The hidden costs of nudging: Experimental evidence from reminders in fundraising
Damgaard, M. T., & Gravert, C. (2018) · Journal of Public Economics · doi:10.1016/j.jpubeco.2017.11.005 · Cited by 183
Smarter Lunchrooms Can Address New School Lunchroom Guidelines and Childhood Obesity
Hanks, A. S., Just, D. R., & Wansink, B. (2013) · The Journal of Pediatrics · doi:10.1016/j.jpeds.2012.12.031 · Cited by 169
Skov, L. R., Lourenço, S., Hansen, G. L., Mikkelsen, B. E., & Schofield, C. (2012) · Obesity Reviews · doi:10.1111/j.1467-789x.2012.01054.x · Cited by 168
From mechanism to virtue: Evaluating Nudge theory
Kosters, M., & Van der Heijden, J. (2015) · Evaluation · doi:10.1177/1356389015590218 · Cited by 161
Default Options In Advance Directives Influence How Patients Set Goals For End-Of-Life Care
Halpern, S. D., Loewenstein, G. F., Volpp, K. G. M., Cooney, E. L., Vranas, K. C., Quill, C. M., McKenzie, M. S., Harhay, M. O., Gabler, N. B., Silva, T., et al. (2013) · Health Affairs · doi:10.1377/hlthaff.2012.0895 · Cited by 158
Steffel, M., Williams, E. F., & Pogacar, R. (2016) · Journal of Marketing Research · doi:10.1509/jmr.14.0421 · Cited by 155
Book Review: Inside the Nudge Unit: How Small Changes can Make a Big Difference
Mahmoud, O. (2016) · International Journal of Market Research · doi:10.2501/ijmr-2016-010 · Cited by 152
How does happiness relate to economic behaviour? A review of the literature
Lane, T. (2017) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2017.04.001 · Cited by 148
Alós‐Ferrer, C., Hügelschäfer, S., & Li, J. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00169 · Cited by 145
Digital Nudging: Altering User Behavior in Digital Environments
Mirsch, T., Lehrer, C., & Jung, R. (2017) · Journal of the Association for Information Systems · Cited by 139
Slim by Design: Serving Healthy Foods First in Buffet Lines Improves Overall Meal Selection
Wansink, B. C., & Hanks, A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0077055 · Cited by 130
Cohen, J. F. W., Richardson, S. A., Cluggish, S. A., Parker, E., Catalano, P. J., & Rimm, E. B. (2015) · JAMA Pediatrics · doi:10.1001/jamapediatrics.2014.3805 · Cited by 123
Sudarshan, A. (2016) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2016.12.015 · Cited by 120
Friis, R. V. H., Skov, L. R., Olsen, A., Appleton, K. M., Saulais, L., Dinnella, C., Hartwell, H., Depezay, L., Monteleone, E., Giboreau, A., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0176028 · Cited by 119
Altmann, S., & Traxler, C. (2014) · European Economic Review · doi:10.1016/j.euroecorev.2014.07.007 · Cited by 113
Active Decisions and Prosocial Behaviour: A Field Experiment on Blood Donation
Stutzer, A., Goette, L. F., & Zehnder, M. (2011) · The Economic Journal · doi:10.1111/j.1468-0297.2011.02477.x · Cited by 85
Virtual reality experiments in economics
Innocenti, A. (2017) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2017.06.001 · Cited by 82
Painting the fence: Social norms as economic incentives to non-automotive travel behavior
Riggs, W. (2017) · Travel Behaviour and Society · doi:10.1016/j.tbs.2016.11.004 · Cited by 80 · cited 1×
“As an example, some studies used financial incentives (Riggs, 2017), but still call their approach a “nudge”.”
ResultPush, don’t nudge: Behavioral spillovers and policy instruments
D’Adda, G., Capraro, V., & Tavoni, M. (2016) · Economics Letters · doi:10.1016/j.econlet.2017.02.029 · Cited by 79
Hilton, D., Charalambides, L., Demarque, C., Waroquier, L., & Charles Raux, A. (2014) · Journal of Economic Psychology · doi:10.1016/j.joep.2014.02.007 · Cited by 68
A Traffic-Light Label Intervention and Dietary Choices in College Cafeterias
Seward, M. W., Block, J. P., & Chatterjee, A. (2016) · American Journal of Public Health · doi:10.2105/ajph.2016.303301 · Cited by 67
Nudging legally: On the checks and balances of behavioral regulation
Alemanno, A., & Spina, A. (2014) · International Journal of Constitutional Law · doi:10.1093/icon/mou033 · Cited by 64
The effect of computer reminders on GPs’ prescribing behaviour: A cluster-randomised trial
Martens, J. D., van der Weijden, T., Severens, J. L., de Clercq, P. A., Bruijn‐Geraets, D. P. D., Kester, A. D. M., & Winkens, R. A. G. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.04.005 · Cited by 63
Would an Energy Conservation Nudge in Hotels Encourage Hotel Guests to Conserve?
Chang, H. “., Huh, C., & Lee, M. J. (2015) · Cornell Hospitality Quarterly · doi:10.1177/1938965515588132 · Cited by 58
Social norms and rank-based nudging: Changing willingness to pay for healthy food.
Aldrovandi, S., Brown, G. D. A., & Wood, A. M. (2015) · Journal of Experimental Psychology Applied · doi:10.1037/xap0000048 · Cited by 56
Nudging to prevent the purchase of incompatible digital products online: An experimental study
Esposito, G., Hernández, P., van Bavel, R., & Vila, J. E. (2017) · PLoS ONE · doi:10.1371/journal.pone.0173333 · Cited by 56
Social comparison nudges—Guessing the norm increases charitable giving
Bartke, S., Friedl, A., Gelhaar, F., & Reh, L. (2016) · Economics Letters · doi:10.1016/j.econlet.2016.12.023 · Cited by 54
‘Nudge’ interventions for improving children's dietary behaviors in the home: A systematic review
Lycett, K., Miller, A. R., Knox, A., Dunn, S., Kerr, J. A., Sung, V., & Wake, M. A. (2017) · Obesity Medicine · doi:10.1016/j.obmed.2017.06.001 · Cited by 53
The application of defaults to optimize parents' health-based choices for children
Loeb, K. L., Radnitz, C. L., Keller, K., Schwartz, M. B., Marcus, S. M., Pierson, R. N., Shannon, M. C., & DeLaurentis, D. (2017) · Appetite · doi:10.1016/j.appet.2017.02.039 · Cited by 52
Dogruel, L., Joeckel, S., & Jessica Vitak, B. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.08.035 · Cited by 51
Agarwal, S. D., Rengarajan, S., Sing, T. F., & Yang, Y. (2016) · Energy Economics · doi:10.1016/j.eneco.2016.10.014 · Cited by 51
Nudging charitable giving: Three field experiments
Zarghamee, H., Messer, K. D., Fooks, J. R., Schulze, W. D., Wu, S., & Yan, J. (2016) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2016.04.008 · Cited by 48
Kressel, L. M., Chapman, G. B., & Leventhal, E. A. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0204-6 · Cited by 42
Johnson, E. J., & Goldstein, D. (2003) · Science · doi:10.1126/science.1091721 · Cited by 35
Guérassimoff, G., & Thomas, J. (2015) · Energy and Buildings · doi:10.1016/j.enbuild.2015.06.080 · Cited by 35
Which Healthy Eating Nudges Work Best? A Meta-Analysis of Field ExperimentsPreprint
Cadario, R., & Chandon, P. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3090829 · Cited by 34
The digital nudge in social security administration
Gregor, S. D., & Archer, B. J. (2016) · International Social Security Review · doi:10.1111/issr.12111 · Cited by 32
Increasing Hand Washing Compliance With a Simple Visual Cue
Ford, E. W., Boyer, B. T., Menachemi, N., & Huerta, T. R. (2013) · American Journal of Public Health · doi:10.2105/ajph.2013.301477 · Cited by 30
The excessive choice effect meets the market: A field experiment on craft beer choice
Malone, T., & Lusk, J. L. (2017) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2017.01.008 · Cited by 29
Jung, D., & Dorner, V. (2017) · Lecture notes in information systems and organisation · doi:10.1007/978-3-319-67431-5_18 · Cited by 29
Nudging People to Pay CO2 offsets - the effect of Anchors in Flight Booking Processes.
Székely, N., Weinmann, M., & Brocke, J. V. (2016) · European Conference on Information Systems · Cited by 27
Baek, Y. M., Bae, Y., Jeong, I., Kim, E., & Rhee, J. W. (2014) · The Social Science Journal · doi:10.1016/j.soscij.2014.07.002 · Cited by 26
When message tailoring backfires: The role of initial attitudes in affect–cognition matching
See, Y., Valenti, G., Ho, A. Y. Y., & Tan, M. S. Q. (2013) · European Journal of Social Psychology · doi:10.1002/ejsp.1967 · Cited by 25
IMPROVING RESIDENTIAL ENERGY CONSUMPTION AT LARGE USING PERSUASIVE SYSTEMS
Graml, T., Loock, C.-M., Baeriswyl, M., & Staake, T. (2011) · Journal of the Association for Information Systems · Cited by 24
Nudging and shoving: Using in-game cues to guide player exertion in exergames
Schneider, A. L. J., & Graham, T. C. N. (2017) · Entertainment Computing · doi:10.1016/j.entcom.2017.01.002 · Cited by 22
Huang, Y., White, C. N., Xia, H., & Wang, Y. (2016) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2016.11.003 · Cited by 21
Djurica, D., & Figl, K. (2017) · WU Research · Cited by 19
Khern-am-nuai, W., Yang, W., & Li, N. (2017) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2017.071 · Cited by 18
Improving Digital Nudging Using Attentive User Interfaces: Theory Development and Experiment Design
Hummel, D., Toreini, P., & Maedche, A. (2018) · Design Science Research in Information Systems and Technology · Cited by 17
Self-citationThe disconnection between privacy notices and information disclosure: an online experiment
Rodríguez-Priego, N., van Bavel, R., & Monteleone, S. (2016) · Economia Politica · doi:10.1007/s40888-016-0040-4 · Cited by 13
Nudging transparent behavioural science and policy
Maynard, O., & Munafo, M. R. (2018) · Behavioural Public Policy · doi:10.1017/bpp.2018.10 · Cited by 8
The Dark Side of Positive Social Influence
Liu, C., Agarwal, R., & Gao, G. (2016) · International Conference on Information Systems · Cited by 7
Cited by
Mertens, S., Herberz, M., Hahnel, U. J. J., & Brosch, T. (2022) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2107346118 · Cited by 738
RCTs to Scale: Comprehensive Evidence From Two Nudge Units
DellaVigna, S., & Linos, E. (2022) · Econometrica · doi:10.3982/ecta18709 · Cited by 480
The ethics of nudging: An overview
Schmidt, A. T., & Engelen, B. (2020) · Philosophy Compass · doi:10.1111/phc3.12658 · Cited by 328
Free will without consciousness?
Mudrik, L., Arie, I. G., Amir, Y., Shir, Y., Hieronymi, P., Maoz, U., O'Connor, T., Schurger, A., Vargas, M., Vierkant, T., et al. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.03.005 · Cited by 54
Metadata
- Authors
- Hummel, D., & Maedche, A.
- Venue
- Journal of Behavioral and Experimental Economics
- Publisher
- Elsevier BV
- Published
- 2019-06
- Volume
- 80
- Pages
- 47-58
- ISSN
- 2214-8043 (ISSN-L)
- OpenAlex
- W2921959179
- Cited by
- 543
- References
- 96
- FWCI
- 37.46
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.