The DOI matches its Crossref record; 72 of the 73 references it cites were found in a registry or an index.
An algorithm for identifying nonsystematic delay-discounting data.
Open accessAbstract
Several [corrected] discounting studies have use the R2 measure to identify data [corrected] with poor fits to a mathematical discounting model as nonsystematic data to be eliminated [corrected] Data from three previous delay-discounting studies (six separate groups, with a total of 161 individuals) were used to demonstrate why using R2 to assess the fits of discounting data is problematic. A significant, positive correlation between discounting rate parameter and R2 was found in most groups, showing that R2 is more stringent as a measure of fit for low discounting rates than for high discounting rates. Furthermore, it is suggested that identifying nonsystematic data based on any measure of fit to a mathematical discounting model may be problematic because it confounds discounting rate comparison with the issue of discounting model assessment. Therefore, a model-free method to identify nonsystematic data is needed. An algorithm for identifying nonsystematic data is presented that is based on the expectation of a monotonically decreasing discounting function. This algorithm identified 13 cases out of the 161 reanalyzed data sets as nonsystematic. These nonsystematic data are presented, along with examples of data not identified as nonsystematic. This algorithm, or modifications of it, may be useful in a variety of human and nonhuman animal discounting studies (e.g., delay discounting, probability discounting) as an alternative to the R2 measure for identifying nonsystematic data. The algorithm may be used in empirical investigations to improve discounting methodology, and may be used to identify outliers to be removed from analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Data that did not meet validity criteria were treated as missing in analyses.”
From Subtyping cognitive profiles in Autism Spectrum Disorder using a Functional Random Forest algorithm (2018) · cites this paper as [76]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Time Discounting and Time Preference: A Critical Review
Frederick, S., Loewenstein, G., & O’Donoghue, T. (2002) · Journal of Economic Literature · doi:10.1257/002205102320161311 · Cited by 6,106
Model Selection and Multimodel Inference
(2003) · Technometrics · doi:10.1198/tech.2003.s146 · Cited by 5,509
Delay of gratification in children.
Mischel, W., Shoda, Y., & Rodriguez, M. L. (1989) · Science · doi:10.1126/science.2658056 · Cited by 3,397
Specious reward: A behavioral theory of impulsiveness and impulse control.
Ainslie, G. (1975) · Psychological Bulletin · doi:10.1037/h0076860 · Cited by 2,708
Anomalies in Intertemporal Choice: Evidence and an Interpretation
Loewenstein, G., & Prelec, D. (1992) · The Quarterly Journal of Economics · doi:10.2307/2118482 · Cited by 2,305
Some empirical evidence on dynamic inconsistency
Thaler, R. H. (1981) · Economics Letters · doi:10.1016/0165-1765(81)90067-7 · Cited by 2,282
Methods for dealing with reaction time outliers.
Ratcliff, R. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.3.510 · Cited by 2,142
Motulsky, H. J., & Christopoulos, A. (2004) · Cited by 1,972
An adjusting procedure for studying delayed reinforcement.
Commons, M. L., Mazur, J., Nevin, J. A., & Rachlin, H. C. (1987) · doi:10.4324/9781315825502-11 · Cited by 1,842
Impulsivity and cigarette smoking: delay discounting in current, never, and ex-smokers
Bickel, W. K., Odum, A. L., & Madden, G. J. (1999) · Psychopharmacology · doi:10.1007/pl00005490 · Cited by 1,433
Self-citationSUBJECTIVE PROBABILITY AND DELAY
Rachlin, H., Raineri, A., & Cross, D. (1991) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1991.55-233 · Cited by 1,341
COMMITMENT, CHOICE AND SELF‐CONTROL1
Rachlin, H., & Green, L. (1972) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1972.17-15 · Cited by 1,204
Discounting of Delayed Rewards: A Life-Span Comparison
Green, L., Fry, A. F., & Myerson, J. (1994) · Psychological Science · doi:10.1111/j.1467-9280.1994.tb00610.x · Cited by 1,165
Ainslie, G. W. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9781139164191 · Cited by 995
WITHIN‐SUBJECT COMPARISON OF REAL AND HYPOTHETICAL MONEY REWARDS IN DELAY DISCOUNTING
Johnson, M. W., & Bickel, W. K. (2002) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.2002.77-129 · Cited by 991
Self-citationDiscount rates inferred from decisions: an experimental study
Benzion, U., Israel, H., Rapoport, A., & Yagil, J. (1987) · Management Science · doi:10.1287/mnsc.35.3.270 · Cited by 967
Madden, G. J., Petry, N. M., Badger, G. J., & Bickel, W. K. (1997) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.5.3.256 · Cited by 911
Self-citationHow persuasive is a good fit? A comment on theory testing.
Roberts, S., & Pashler, H. (2000) · Psychological Review · doi:10.1037/0033-295x.107.2.358 · Cited by 879
DISCOUNTING OF DELAYED REWARDS: MODELS OF INDIVIDUAL CHOICE
Myerson, J., & Green, L. (1995) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1995.64-263 · Cited by 843
DELAY OR PROBABILITY DISCOUNTING IN A MODEL OF IMPULSIVE BEHAVIOR: EFFECT OF ALCOHOL
Richards, J. B., Zhang, L., Mitchell, S. H., & de Wit, H. (1999) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1999.71-121 · Cited by 813
Research on self-control: An integrating framework
Logue, A. W. (1988) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00053978 · Cited by 792
Handbook of Nonlinear Regression Models.
Giddings, D. R., & Ratkowsky, D. A. (1991) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347928 · Cited by 777
A review of delay-discounting research with humans: relations to drug use and gambling
Reynolds, B. (2006) · Behavioural Pharmacology · doi:10.1097/fbp.0b013e3280115f99 · Cited by 711
Frames of mind in intertemporal choice
Loewenstein, G. (1988) · Management Science · doi:10.1287/mnsc.34.2.200 · Cited by 667
Delay discounting of real and hypothetical rewards.
Madden, G. J., Begotka, A. M., Raiff, B. R., & Kastern, L. L. (2003) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.11.2.139 · Cited by 660
Barkley, R. A., Edwards, G., Laneri, M., Fletcher, K., & Metevia, L. (2001) · Journal of Abnormal Child Psychology · doi:10.1023/a:1012233310098 · Cited by 638
Loewenstein, G. F., & Elster, J. (1992) · Medical Entomology and Zoology · Cited by 635
Baker, F., Johnson, M. W., & Bickel, W. K. (2003) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.112.3.382 · Cited by 602
Self-citationPathological gambling severity is associated with impulsivity in a delay discounting procedure
Alessi, S. M., & Petry, N. M. (2003) · Behavioural Processes · doi:10.1016/s0376-6357(03)00150-5 · Cited by 545
CHOICE AND DELAY OF REINFORCEMENT1
Chung, S., & Herrnstein, R. J. (1967) · Journal of the Experimental Analysis of Behavior · doi:10.1901/jeab.1967.10-67 · Cited by 540
Cross-Cultural Comparisons of Discounting Delayed and Probabilistic Rewards
Du, W., Green, L., & Myerson, J. (2002) · The Psychological Record · doi:10.1007/bf03395199 · Cited by 514
Rate of temporal discounting decreases with amount of reward
Green, L., Myerson, J., & McFadden, E. (1997) · Memory & Cognition · doi:10.3758/bf03211314 · Cited by 508
Temporal discounting in choice between delayed rewards: The role of age and income.
Green, L., Myerson, J., Lichtman, D., Rosen, S., & Fry, A. F. (1996) · Psychology and Aging · doi:10.1037//0882-7974.11.1.79 · Cited by 505
An Introduction to Model Selection
Zucchini, W. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1276 · Cited by 466
TEMPORAL DISCOUNTING: BASIC RESEARCH AND THE ANALYSIS OF SOCIALLY IMPORTANT BEHAVIOR
Critchfield, T. S., & Kollins, S. H. (2001) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.2001.34-101 · Cited by 439
Krishnan-Sarin, S., Reynolds, B., Duhig, A. M., Smith, A., Liss, T., McFetridge, A., Cavallo, D. A., Carroll, K. M., & Potenza, M. N. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2006.09.006 · Cited by 436
Pitt, M. A., & Myung, I. J. (2002) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(02)01964-2 · Cited by 419
Temporal discounting and preference reversals in choice between delayed outcomes
Green, L., Fristoe, N. M., & Myerson, J. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03213979 · Cited by 418
Diergaarde, L., Pattij, T., Poortvliet, I., Hogenboom, F., de Vries, W., Schoffelmeer, A. N. M., & De Vries, T. J. (2008) · Biological Psychiatry · doi:10.1016/j.biopsych.2007.07.011 · Cited by 391
Key Concepts in Model Selection: Performance and Generalizability
Forster, M. R. (2000) · Journal of Mathematical Psychology · doi:10.1006/jmps.1999.1284 · Cited by 390
Perry, J. L., Larson, E. B., German, J. P., Madden, G. J., & Carroll, M. E. (2005) · Psychopharmacology · doi:10.1007/s00213-004-1994-4 · Cited by 388
Effects of THC on Behavioral Measures of Impulsivity in Humans
McDonald, J. M., Schleifer, L., Richards, J. B., & de Wit, H. (2003) · Neuropsychopharmacology · doi:10.1038/sj.npp.1300176 · Cited by 386
Measuring state changes in human delay discounting: an experiential discounting task
Reynolds, B. A., & Schiffbauer, R. (2004) · Behavioural Processes · doi:10.1016/j.beproc.2004.06.003 · Cited by 368
The effect of delay and of intervening events on reinforcement value.
Commons, M. L. (1987) · L. Erlbaum Associates Inc. eBooks · Cited by 359
Lagorio, C. H., & Madden, G. J. (2005) · Behavioural Processes · doi:10.1016/j.beproc.2005.02.003 · Cited by 359
Preference reversal and self control: choice as a function of reward amount and delay
Green, L., Fisher, E. B., Perlow, S., & Sherman, L. (1981) · Cited by 335
Delay discounting predicts postpartum relapse to cigarette smoking among pregnant women.
Yoon, J. H., Higgins, S. T., Heil, S. H., Sugarbaker, R. J., Thomas, C. S., & Badger, G. J. (2007) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.15.2.186 · Cited by 322
Madden, G. J., Raiff, B. R., Lagorio, C. H., Begotka, A. M., Mueller, A. M., Hehli, D. J., & Wegener, A. A. (2004) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.12.4.251 · Cited by 314
Exponential versus hyperbolic discounting of delayed outcomes: Risk and waiting time.
GREEN, L., & MYERSON, J. (1996) · American Zoologist · doi:10.1093/icb/36.4.496 · Cited by 294
Winstanley, C. A., Dalley, J. W., Theobald, D. E. H., & Robbins, T. W. (2003) · Psychopharmacology · doi:10.1007/s00213-003-1546-3 · Cited by 261
Three-month stability of delay and probability discounting measures.
Ohmura, Y., Takahashi, T., Kitamura, N., & Wehr, P. (2006) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.14.3.318 · Cited by 258
Green, L., Myerson, J., & Ostaszewski, P. (1999) · Behavioural Processes · doi:10.1016/s0376-6357(99)00021-2 · Cited by 258
Reynolds, B., Richards, J. B., & de Wit, H. (2006) · Pharmacology Biochemistry and Behavior · doi:10.1016/j.pbb.2006.01.007 · Cited by 247
Reward discounting as a measure of impulsive behavior in a psychiatric outpatient population.
Crean, J., de Wit, H., & Richards, J. (2000) · Experimental and clinical psychopharmacology · doi:10.1037//1064-1297.8.2.155 · Cited by 218
Effects of sleep deprivation on impulsive behaviors in men and women
Acheson, A., Richards, J. B., & de Wit, H. (2007) · Physiology & Behavior · doi:10.1016/j.physbeh.2007.03.020 · Cited by 211
Delay discounting predicts cigarette smoking in a laboratory model of abstinence reinforcement
Dallery, J., & Raiff, B. R. (2007) · Psychopharmacology · doi:10.1007/s00213-006-0627-5 · Cited by 197
Mitchell, S. H. (2004) · Nicotine & Tobacco Research · doi:10.1080/14622200412331296002 · Cited by 197
Reliability of a Measure of Temporal Discounting
Simpson, C. A., & Vuchinich, R. E. (2000) · The Psychological Record · doi:10.1007/bf03395339 · Cited by 181
Outcome Signs, Question Frames and Discount Rates
Shelley, M. K. (1993) · Management Science · doi:10.1287/mnsc.39.7.806 · Cited by 177
Comparison between two measures of delay discounting in smokers.
Epstein, L. H., Richards, J. B., Saad, F. G., Paluch, R. A., Roemmich, J. N., & Lerman, C. E. (2003) · Experimental and Clinical Psychopharmacology · doi:10.1037/1064-1297.11.2.131 · Cited by 123
Hamidovic, A., Kang, U. J., & de Wit, H. (2008) · Journal of Clinical Psychopharmacology · doi:10.1097/jcp.0b013e3181602fab · Cited by 97
Is the delay discounting paradigm useful in understanding social anxiety?
Rounds, J., Beck, J. G., & Grant, D. M. (2006) · Behaviour Research and Therapy · doi:10.1016/j.brat.2006.06.007 · Cited by 84
Measurement of delay discounting using trial-by-trial consequences
Lane, S. D., Cherek, D. R., Pietras, C. J., & Tcheremissine, O. V. (2003) · Behavioural Processes · doi:10.1016/s0376-6357(03)00143-8 · Cited by 80
Concave utility, transaction costs, and risk in measuring discounting of delayed rewards.
Kirby, K. N., & Santiesteban, M. (2003) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.29.1.66 · Cited by 80
Functional parameters of delay discounting assessment tasks: Order of presentation
Robles, E., & Vargas, P. A. (2007) · Behavioural Processes · doi:10.1016/j.beproc.2007.02.014 · Cited by 72
Self-control and impulsiveness involving aversive events.
Deluty, M. Z. (1978) · Journal of Experimental Psychology Animal Behavior Processes · doi:10.1037/0097-7403.4.3.250 · Cited by 69
Delay discounting: A fundamental behavioral process of drug dependence.
Bickel, W. K., & Johnson, M. W. (2003) · Cited by 60
Self-citationAre some motives more linked to suicide proneness than others?
Holden, R. R., Kerr, P. S., Mendonca, J. D., & Velamoor, V. R. (1998) · Journal of Clinical Psychology · doi:10.1002/(sici)1097-4679(199808)54:5 · Cited by 59
Dimensions of Deception in Personality Assessment: The Example of the MMPI-2
Nichols, D. S., & Greene, R. L. (1997) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6802_3 · Cited by 56
Experimental Analysis of Behavior
(2021) · Springer eBooks · doi:10.1007/978-3-319-19650-3_301644 · Cited by 56
A comparison of two algorithms in computerized temporal discounting procedures
Kowal, B. P., Yi, R., Erisman, A. C., & Bickel, W. K. (2007) · Behavioural Processes · doi:10.1016/j.beproc.2007.02.005 · Cited by 31
Self-citationTemporal discounting predicts individual competitive success in a human analogue of group foraging
Critchfield, T. S., & Atteberry, T. (2003) · Behavioural Processes · doi:10.1016/s0376-6357(03)00129-3 · Cited by 29
Cited by
Sorted by citations
Peters, J., & Büchel, C. (2010) · Neuron · doi:10.1016/j.neuron.2010.03.026 · Cited by 871
A 5-trial adjusting delay discounting task: Accurate discount rates in less than one minute.
Koffarnus, M. N., & Bickel, W. K. (2014) · Experimental and Clinical Psychopharmacology · doi:10.1037/a0035973 · Cited by 411
Self-citationIdentification and management of nonsystematic purchase task data: Toward best practice.
Stein, J. S., Koffarnus, M. N., Snider, S. E., Quisenberry, A. J., & Bickel, W. K. (2015) · Experimental and Clinical Psychopharmacology · doi:10.1037/pha0000020 · Cited by 281
Self-citationPercent body fat is related to delay and probability discounting for food in humans
Rasmussen, E. B., Lawyer, S. R., & Reilly, W. (2009) · Behavioural Processes · doi:10.1016/j.beproc.2009.09.001 · Cited by 286
Unstuck in time: episodic future thinking reduces delay discounting and cigarette smoking
Stein, J. S., Wilson, A. G., Koffarnus, M. N., Daniel, T. O., Epstein, L. H., & Bickel, W. K. (2016) · Psychopharmacology · doi:10.1007/s00213-016-4410-y · Cited by 267
Self-citation
Metadata
- Authors
- Johnson, M. W., & Bickel, W. K.
- Affiliations
- Johns Hopkins University (US)
- University of Arkansas for Medical Sciences (US)
- Venue
- Experimental and Clinical Psychopharmacology
- Published
- 2008
- Volume
- 16
- Issue
- 3
- Pages
- 264-274
- ISSN
- 1064-1297 (ISSN-L)
- 1936-2293
- PMC
- PMC2765051
- OpenAlex
- W2092110116
- Cited by
- 555
- References
- 73
- Citations by year
- 2026: 28 · 2025: 33 · 2024: 35 · 2023: 37 · 2022: 39 · 2021: 40 · 2020: 81 · 2019: 34 · 2018: 32 · 2017: 40 · 2016: 31 · 2015: 36 · 2014: 28 · 2013: 18 · 2012: 11
- FWCI
- 6.51
- Type
- journal article
- Language
- en
- Update notices
- correction (2008-08-25)
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Experimental and Clinical Psychopharmacology (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institute on Drug Abuse (R01 DA 11692; T32 DA07242)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- delay discounting, discount rate, model fit assessment, outlier detection, probability discounting
- MeSH terms
- Algorithms, Animals, Behavior, Humans, Motivation, Reinforcement, Psychology, Smoking, Time Factors, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
