Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r.
Abstract
In order to facilitate comparisons across follow-up studies that have used different measures of effect size, we provide a table of effect size equivalencies for the three most common measures: ROC area (AUC), Cohen's d, and r. We outline why AUC is the preferred measure of predictive or diagnostic accuracy in forensic psychology or psychiatry, and we urge researchers and practitioners to use numbers rather than verbal labels to characterize effect sizes.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The magnitude of the significant overall effects for school absenteeism ranged from small (r=0.099 for low IQ/learning difficulties) to large (r=0.553 for having a negative school attitude) based on Rice and Harris (2005) criteria.
“The magnitude of the effects of these risk domains ranged from small (i.e., r= 0.099 for “low IQ/learning difficulties”) to large (i.e., r= 0.553 for “having a negative school attitude”) based on the criteria of Rice and Harris (2005) for interpreting effect sizes.”
From Risk Factors for School Absenteeism and Dropout: A Meta-Analytic Review (2019) · cites this paper as Rice and Harris (2005)
The AUC of 0.62 for the DASH in distinguishing persistent offenders is a moderate effect size.
“The AUC of 0.62 is also a moderate effect size [14], but is less than that for deadly violence (but this difference was not statistically significant).”
From The Domestic Abuse, Stalking and Harassment and Honour-Based Violence (DASH) Risk Assessment Instrument in Predicting Deadly or Persistent Domestic Abuse (2025) · cites this paper as [14]
An AUC of 0.67 for deadly violence is regarded as a moderate effect size.
“The AUC of 0.67 for deadly violence is regarded as a “moderate” effect size [14].”
From The Domestic Abuse, Stalking and Harassment and Honour-Based Violence (DASH) Risk Assessment Instrument in Predicting Deadly or Persistent Domestic Abuse (2025) · cites this paper as [14]
AUC values of 0.56, 0.64, and 0.71 indicate small, medium, and large effects.
“AUC values of 0.56, 0.64, and 0.71 indicate small, medium, and large effects (Rice & Harris, 2005).”
From Strategies for monitoring mentoring relationship quality to predict early program dropout (2022) · cites this paper as Rice & Harris, 2005
Cites
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · see citing works
Interpreting the magnitudes of correlation coefficients.
Hemphill, J. F. (2003) · American Psychologist · doi:10.1037/0003-066x.58.1.78 · Cited by 1,237 · see citing works
A common language effect size statistic
McGraw, K. O., & Wong, S. P. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.361 · Cited by 934 · cited 1× · see citing works
“AUC also has a plain language interpretation because it is conceptually and mathematically similar to the common language effect size or CL ( McGraw & Wong, 1992; see also Delaney & Vargha, 2002).”
Flagged first, then by citations
Cited by
Sorted by citations
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063 · see citing works
“The CL gives the probability that a user picked at random from the high users group will have a higher average improvement than a user picked at random from the low users group [38,39].”
BackgroundHanson, R., & Morton-Bourgon, K. E. (2009) · Psychological Assessment · doi:10.1037/a0014421 · Cited by 1,034 · see citing works
“Although the effect sizes for most of the measures were moderate to large by conventional standards (Cohen, 1988; Rice & Harris, 2005), there are no direct procedures for translating the observed effect sizes into statements concerning the normative value or utility of specific measures (e.g., is a…”
BackgroundMann, R. E., Hanson, R. K., & Thornton, D. (2010) · Sexual Abuse · doi:10.1177/1079063210366039 · Cited by 960 · see citing works
“Both d and AUC are based on similar statistical models (Swets, 1986), and the transformations between these metrics are well documented: for example, d of 0.20 = AUC of 0.556, d of 0.50 = AUC of 0.639, d of 0.80 = AUC of 0.714; Rice & Harris, 2005).”
BackgroundInfluentialViolent offenders: Appraising and managing risk, 2nd ed.
Quinsey, V., Harris, G., Rice, M., & Cormier, C. A. (2006) · American Psychological Association eBooks · doi:10.1037/11367-000 · Cited by 720 · see citing works
Self-citationThe efficacy of violence prediction: a meta-analytic comparison of nine risk assessment tools.
Yang, M., Wong, S. C. P., & Coid, J. (2010) · Psychological bulletin · doi:10.1037/a0020473 · Cited by 708 · see citing works
“For eight studies that reported only the AUC values, a direct conversion of the AUC value to the d value was carried out on the basis of the table of M. E. Rice and Harris (2005).”
BackgroundMethodInfluentialRisk Factors for School Absenteeism and Dropout: A Meta-Analytic Review
Gubbels, J., van der Put, C. E., & Assink, M. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01072-5 · Cited by 596 · see citing works
“The magnitude of the effects of these risk domains ranged from small (i.e., r= 0.099 for “low IQ/learning difficulties”) to large (i.e., r= 0.553 for “having a negative school attitude”) based on the criteria of Rice and Harris (2005) for interpreting effect sizes.”
MethodInfluentialMachine learning prediction in cardiovascular diseases: a meta-analysis
Krittanawong, C., Virk, H. H., Bangalore, S., Wang, Z., Johnson, K. W., Pinotti, R., Zhang, H.-J., Kaplin, S. L., Narasimhan, B., Kitai, T., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-72685-1 · Cited by 447 · see citing works
A probability-based measure of effect size: Robustness to base rates and other factors.
Ruscio, J. (2008) · Psychological Methods · doi:10.1037/1082-989x.13.1.19 · Cited by 398 · see citing works
“Rice and Harris (2005) presented tables of equivalent d, r pb , and CL values under the assumptions of normality and equal variances, and they provided conversion formulas that apply in this case.”
MethodImproving the Predictive Accuracy of Static-99 and Static-2002 With Older Sex Offenders
Helmus, L., Thornton, D., Hanson, R. K., & Babchishin, K. (2012) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063211409951 · Cited by 349 · see citing works
Eliciting cues to deception and truth: What matters are the questions asked
Vrij, A., & Granhag, P. (2012) · doi:10.1016/j.jarmac.2012.02.004 · Cited by 348
“He gave as an example the difference in height between 15- and 16-year-old girls (Cohen, 1988; Rice & Harris, 2005).”
BackgroundRisk factors for child neglect: A meta-analytic review.
Mulder, T., Kuiper, K., van der Put, C. E., Stams, G., & Assink, M. (2018) · International Journal of Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.01.006 · Cited by 314 · see citing works
Risk Assessment for Juvenile Justice: A Meta-Analysis
Schwalbe, C. (2007) · Law and Human Behavior · doi:10.1007/s10979-006-9071-7 · Cited by 310 · see citing works
“b Conversion to r based on Rice and Harris (2005) .”
BackgroundMethodInfluentialChildhood forecasting of a small segment of the population with large economic burden
Caspi, A., Houts, R., Belsky, D. W., Harrington, H., Hogan, S., Ramrakha, S., Poulton, R., & Moffitt, T. (2016) · Nature Human Behaviour · doi:10.1038/s41562-016-0005 · Cited by 302 · see citing works
The intergenerational transmission of child maltreatment: A three-level meta-analysis
Assink, M., Spruit, A., Schuts, M., Lindauer, R., van der Put, C. E., & Stams, G.-J. J. M. (2018) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.07.037 · Cited by 275 · see citing works
“Our meta-analysis yielded a signi fi cant summary e ff ect of r = 0.289, which is medium according to Rice and Harris ’ (2005) criteria for interpreting e ff ect sizes.”
MethodInfluentialRisk Assessment With Young Offenders
Olver, M., Stockdale, K. C., & Wormith, J. (2009) · Criminal Justice and Behavior · doi:10.1177/0093854809331457 · Cited by 266 · see citing works
“When only an AUC statistic from Receiver Operating Characteristic (ROC) analyses was provided, the tables provided in Rice and Harris (2005) were used to convert an ROC value into the equivalent rpb. 3 If correlations with continuous recidivism measures (e.g., total convictions) were reported, we…”
MethodOlver, M., Stockdale, K. C., & Wormith, J. (2014) · Psychological Assessment · doi:10.1037/a0035080 · Cited by 264 · see citing works
Risk Assessment in Criminal Sentencing
Monahan, J., & Skeem, J. L. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-092945 · Cited by 256 · see citing works
Kessler, R., van Loo, H. V., Wardenaar, K., Bossarte, R., Brenner, L., Cai, T., Ebert, D., Hwang, I., Li, J.-L., de Jonge, P., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2015.198 · Cited by 247 · see citing works
Bowers, A. J., Sprott, R., & Taff, S. A. (2013) · The High School journal/The High school journal · doi:10.1353/hsj.2013.0000 · Cited by 243 · see citing works
“While outside the scope of the present study, we encourage future research in this domain to examine the use of continuous indicator outcomes, such as ROC “Area Under the Curve” (AUC) analysis (Swets, 1988), which is amenable to effect size comparisons such as meta-analysis (Rice & Harris, 2005).”
MethodAre the Major Risk/Need Factors Predictive of Both Female and Male Reoffending?
Andrews, D. A., Guzzo, L., Raynor, P., Rowe, R., Rettinger, L. J., Brews, A., & Wormith, J. (2012) · International journal of offender therapy and comparative criminology · doi:10.1177/0306624x10395716 · Cited by 233 · see citing works
“AUC is less sensitive to variations in the base rate of recidivism (Rice & Harris, 2005).”
BackgroundHelmus, L., Hanson, R. K., Thornton, D., Babchishin, K., & Harris, A. (2012) · Criminal Justice and Behavior · doi:10.1177/0093854812443648 · Cited by 218 · see citing works
“The most commonly recommended statistic for reporting predictive accuracy is the area under the receiver operating characteristic curve (AUC for ROC; Mossman, 1994; Rice & Harris, 2005; Swets, Dawes, & Monahan, 2000).”
BackgroundRisk, Race, and Recidivism: Predictive Bias and Disparate Impact
Skeem, J., & Lowenkamp, C. T. (2016) · Criminology · doi:10.1111/1745-9125.12123 · Cited by 217 · see citing works
“Minimum AUCs of .56, .64, and .71 correspond to “small,” “medium,” and “large” effect sizes, respectively (see Rice and Harris, 2005).”
Bowers, A. J., & Zhou, X. (2019) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1080/10824669.2018.1523734 · Cited by 244 · see citing works
“…& Campbell, 1993) and has more recently been applied to domains such as law, education, and youth development to understand the accuracy of at-risk indicators and predictors (Bowers, Sprott, & Taff, 2013; Knowles, 2015; Olver, Stockdale, & Wormith, 2009; Rice & Harris, 2005; Vivo & Franco, 2008).”
MethodOsteopontin is a marker for cancer aggressiveness and patient survival
Weber, G., Lett, G. S., & Haubein, N. C. (2010) · British Journal of Cancer · doi:10.1038/sj.bjc.6605834 · Cited by 201
“…feature used to quantify this characteristic is the area under the curve, which can be interpreted here as the probability that for two randomly chosen samples, the one with the higher Osteopontin rank will also have a higher rank for the clinical variable in question (Rice and Harris, 2005).”
Firm, fair, and caring officer-offender relationships protect against supervision failure.
Kennealy, P., Skeem, J., Manchak, S. M., & Louden, J. E. (2012) · Law and Human Behavior · doi:10.1037/h0093935 · Cited by 189 · see citing works
“In this study, DRI-R total scores had an AUC value of .68, CI [.58, .78], p Ͻ .01, which is considered a medium effect size according to Cohen's standards (see Rice & Harris, 2005).”
MethodPredictive validity performance indicators in violence risk assessment: a methodological primer.
Singh, J. P. (2013) · Behavioral sciences & the law · doi:10.1002/bsl.2052 · Cited by 189 · see citing works
Asscher, J., van Vugt, E., Stams, G., Deković, M., Eichelsheim, V., & Yousfi, S. (2011) · Journal of Child Psychology and Psychiatry and Allied Disciplines · doi:10.1111/j.1469-7610.2011.02412.x · Cited by 186 · see citing works
“Those statistics for which no formula was presented, e.g., AUCs, were transformed with the help of the table provided by Rice and Harris (2005).”
MethodHawes, S., Boccaccini, M., & Murrie, D. (2013) · Psychological Assessment · doi:10.1037/a0030391 · Cited by 185 · see citing works
Handbook of violence risk assessment.
Otto, R., & Douglas, K. (2010) · Routledge eBooks · doi:10.4324/9780203843666 · Cited by 181 · see citing works
Guo, W., & Caliskan, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462536 · Cited by 180 · see citing works
“g the effect size in the null hypothesis, in case biased associations did not exist. According to Cohen’s effect size metric, d>j0:5 jand d>j0:8 jare medium and large effect sizes, respectively [38]. In a neural language model, each stimulus sfrom WEAT contained in n s input sentences has at most n s different C…”
MethodThe Average Predictive Validity of Intimate Partner Violence Risk Assessment Instruments
Messing, J., & Thaller, J. (2013) · Journal of Interpersonal Violence · doi:10.1177/0886260512468250 · Cited by 180 · see citing works
“Of these nine, five of the correctly administered risk assessments were used in tests of the ODARA.”
BackgroundMethodInfluentialHilton, N., Harris, G., Rice, M., Houghton, R. E., & Eke, A. W. (2008) · Law and Human Behavior · doi:10.1007/s10979-007-9088-6 · Cited by 174 · see citing works
“The ROC area for the predictive ability of DVRAG scores in Sample 1 was .71, SE = .03, corresponding to a Cohen’s d = .80 (Rice & Harris, 2005), and represented a statistically significant improvement (p .05, 1-tailed) over ODARA scores alone.”
Self-citationThe psychological assessment of risk for crime and violence.
Hanson, R. K. (2009) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0015726 · Cited by 172 · see citing works
Gray, N., Fitzgerald, S., Taylor, J., Macculloch, M., & Snowden, R. (2007) · Psychological Assessment · doi:10.1037/1040-3590.19.4.474 · Cited by 165
“Using standard conventions, we considered AUCs .56 as small effects, AUCs .64 as medium effects, and AUCs .71 as large effects (Rice & Harris, 2005).”
MethodDai, C., Zhang, H., Arens, E., & Lian, Z. (2017) · doi:10.1016/j.buildenv.2016.12.005 · Cited by 164
Robbé, M. D. V., de Vogel, V., & de Spa, E. (2011) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2011.600232 · Cited by 163 · see citing works
“In or- der to assess the predictive validity for violent recidivism, Receiver Operating Characteristics (ROC; Mossman, 1994; Rice & Harris, 1995, 2005) analyses were carried out with recidivism as outcome measure for the HCR-20 and nonrecidivism as outcome measure for the SAPROF.”
Walters, G. (2020) · Trauma, Violence, & Abuse · doi:10.1177/1524838020906513 · Cited by 152 · see citing works
Hartung, T., Friedrich, M., Johansen, C., Wittchen, H., Faller, H., Koch, U., Brähler, E., Härter, M., Keller, M., Schulz, H., et al. (2017) · Cancer · doi:10.1002/cncr.30846 · Cited by 150 · see citing works
Schmidt, F., Campbell, M., & Houlding, C. M. (2011) · doi:10.1177/1541204010371793 · Cited by 150
“Rice and Harris (2005) described AUC values of .556 as small, .639 as moderate, and .714 as large predictive validity effect sizes.”
BackgroundBrown, L., & Kuss, D. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17124566 · Cited by 149 · see citing works
“01) [66] between SC change scores and MWB change scores: participants’ increase in perceived SC following abstinence was positively related with their increase in perceived MWB following abstinence.”
BackgroundInfluentialBaglivio, M. T., & Jackowski, K. (2013) · Youth Violence and Juvenile Justice · doi:10.1177/1541204012440107 · Cited by 147 · see citing works
“50s is fairly modest and considered ‘‘small’’ in magnitude of effect size (Rice and Harris, 2005).”
BackgroundInfluentialRice, M., Harris, G., & Lang, C. (2013) · Psychological Assessment · doi:10.1037/a0032878 · Cited by 143 · see citing works
Self-citationLuong, D., & Wormith, J. (2011) · doi:10.1177/0093854811421596 · Cited by 143
“Receiver operating characteristics are often preferred over correlations because of their relative insensitivity to differences in the base rate of the criterion variable: in this case, recidivism (Rice & Harris, 1995, 2005).”
BackgroundThe assessment of risk to recidivate among a juvenile offending population
Baglivio, M. T. (2009) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2009.09.008 · Cited by 142 · see citing works
The Estimate of Risk of Adolescent Sexual Offense Recidivism (ERASOR): Preliminary Psychometric Data
Worling, J. R. (2004) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/107906320401600305 · Cited by 133
Performance of recidivism risk assessment instruments in U.S. correctional settings.
Desmarais, S. L., Johnson, K. L., & Singh, J. P. (2016) · Psychological services · doi:10.1037/ser0000075 · Cited by 132 · see citing works
“…1.50–2.99 .10–.19 Good .50–.79 .24–.36 .64–.71 3.00–4.99 .20–.29 Excellent > .80 .37–1.00 .71–1.00 > 5.00 .30–1.00 Notes: Benchmarks were anchored to Cohen’s d (1988) and based upon the calculations of Rice and Harris (2005) for AUC values and Chen, Cohen, and Chen (2010) for the odds ratios.”
InfluentialRobbé, M. D. V., de Vogel, V., Douglas, K., & Nijman, H. (2015) · Law and Human Behavior · doi:10.1037/lhb0000089 · Cited by 132 · see citing works
Predicting the incidence of non-suicidal self-injury in college students
Kiekens, G., Hasking, P., Claes, L., Boyes, M., Mortier, P., Auerbach, R. P., Cuijpers, P., Demyttenaere, K., Green, J. G., Kessler, R., et al. (2019) · European psychiatry · doi:10.1016/j.eurpsy.2019.04.002 · Cited by 131 · see citing works
“Resulting Area Under the Curve (AUC) values close to .56, .64, and .71 are considered, respectively, as small, moderate, and large effects [60].”
BackgroundOlver, M., Stockdale, K. C., & Wong, S. C. P. (2012) · Law and Human Behavior · doi:10.1037/h0093927 · Cited by 129 · see citing works
Jordan, N., Glutting, J., Ramineni, C., & Watkins, M. W. (2010) · School Psychology Review · doi:10.1080/02796015.2010.12087772 · Cited by 129 · see citing works
Self-declared Throwaway Accounts on Reddit
Ammari, T., Schoenebeck, S. Y., & Romero, D. M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359237 · Cited by 102 · see citing works
Lichtenberg, P. A., Sugarman, M. A., Paulson, D., Ficker, L. J., & Rahman-Filipiak, A. (2016) · Clinical Gerontologist · doi:10.1080/07317115.2015.1101632 · Cited by 127 · see citing works
“An AUC of .65 corresponds to a Cohen’s d value of about 0.54, in the range of what is considered to be a “medium” effect size for discriminating between groups (Rice & Harris, 2005).”
BackgroundInfluentialRisk Assessment in Intimate Partner Violence: A Systematic Review of Contemporary Approaches
Nicholls, T. L., Pritchard, M. M., Reeves, K. A., & Hilterman, E. L. B. (2013) · Partner Abuse · doi:10.1891/1946-6560.4.1.76 · Cited by 137 · see citing works
“…VRAG; see Tables 1–7), our review suggests that the measures generally perform in the moderate to good range according to statistical standards (e.g., r .30; Cohen, 1988; AUC .60; Cicchetti & Sparrow, 1981; Rice & Harris, 2005; generally for a risk assessment perspective, see Otto & Douglas, 2010).”
BackgroundSalgado, J. (2018) · The European Journal of Psychology Applied to Legal Context · doi:10.5093/ejpalc2018a5 · Cited by 126 · see citing works
“In order to facilitate comparisons across follow-up studies that report different ES indexes, Rice and Harris (2005) developed a table of the equivalences among AUC and d, and rpb. Rice and Harris (2005) Table includes the AUC values from .50 to 1 and the respective values of z, d, and rpb.”
BackgroundMethodInfluentialPredictors of First Lifetime Onset of Major Depressive Disorder in Young Adulthood
Klein, D., Glenn, C. R., Kosty, D. B., Seeley, J., Rohde, P., & Lewinsohn, P. (2012) · Journal of Abnormal Psychology · doi:10.1037/a0029567 · Cited by 126 · see citing works
“The AUC values of .56, .64, and .71 correspond to small, medium, and large effects, respectively (Rice & Harris, 2005).”
Dinga, R., Marquand, A. F., Veltman, D. J., Beekman, A. T. F., Schoevers, R. A., van Hemert, A. M., Penninx, B. W. J. H., & Schmaal, L. (2018) · Translational Psychiatry · doi:10.1038/s41398-018-0289-1 · Cited by 125 · see citing works
Primer on Risk Assessment and the Statistics Used to Evaluate Its Accuracy
Helmus, L., & Babchishin, K. (2017) · Criminal Justice and Behavior · doi:10.1177/0093854816678898 · Cited by 125 · see citing works
Lutz, W., Schwartz, B., Hofmann, S., Fisher, A., Husen, K., & Rubel, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-25953-0 · Cited by 124 · see citing works
Predicting violent reconvictions using the HCR-20
Gray, N., Taylor, J., & Snowden, R. (2008) · British Journal of Psychiatry · doi:10.1192/bjp.bp.107.044065 · Cited by 124
Simulating computer adaptive testing with the Mood and Anxiety Symptom Questionnaire.
Flens, G., Smits, N., Carlier, I., van Hemert, A., & de Beurs, E. (2016) · Psychological Assessment · doi:10.1037/pas0000240 · Cited by 121 · see citing works
“We evaluated the AUC values using the guideline proposed by Rice and Harris (2005): .56 (cid:2) small effect, .64 (cid:2) medium effect, .71 (cid:2) large effect; a higher effect meaning a higher predictive utility for the scale.”
MethodWormith, J. S., Hogg, S. M., & Guzzo, L. (2012) · doi:10.1177/0093854812455741 · Cited by 120
Risk factors and protective factors: a two-sided dynamic approach to violence risk assessment
Robbé, M. D. V., de Vogel, V., & Douglas, K. (2013) · doi:10.1080/14789949.2013.818162 · Cited by 118
“To assess the predictive validity for violent recidivism, receiver operating characteristics (ROC; Mossman, 1994; Rice & Harris, 1995, 2005) analyses were carried out with recidivism at different follow-up times as outcome measure for the HCR-20 and non-recidivism as outcome measure for the SAPROF.”
MethodTan, J., Cheng, K., & Siah, R. C. J. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.14025 · Cited by 116 · see citing works
Predictive modelling and analytics for diabetes using a machine learning approach
Kaur, H., & Kumari, V. (2020) · Applied Computing and Informatics · doi:10.1016/j.aci.2018.12.004 · Cited by 115 · see citing works
The Impact of Forced Answering and Reactance on Answering Behavior in Online Surveys
Sischka, P., Décieux, J. P., Mergener, A., Neufang, K. M., & Schmidt, A. F. (2020) · Social science computer review · doi:10.1177/0894439320907067 · Cited by 114 · see citing works
“…operating characteristics (ROC) analyses (Swets, 1986) that are independent from base rates to correct for deflated associations due to floor effects in criterion base rates ( AUC s of .50, .65, .75, and .90 correspond to unbiased r pb s of 0, .26, .43, and .67, respectively; Rice & Harris, 2005).”
BackgroundPredictive Validity of Risk Assessments in Juvenile Offenders
Hilterman, E., Nicholls, T. L., & van Nieuwenhuizen, C. (2014) · Assessment (Odessa, Fla.) · doi:10.1177/1073191113498113 · Cited by 114 · see citing works
“AUCs between .56 and .64 are considered small effects, AUCs above .64 are medium effects, and AUCs greater than .71 are described as large effects (Rice & Harris, 2005).”
BackgroundViljoen, J. L., Mordell, S., & Beneteau, J. L. (2012) · Law and Human Behavior · doi:10.1037/h0093938 · Cited by 113 · see citing works
Viljoen, J. L., Cochrane, D. M., & Jonnson, M. R. (2018) · Law and Human Behavior · doi:10.1037/lhb0000280 · Cited by 112 · see citing works
Liu, Y., Yang, M., Ramsay, M. S., Li, X., & Coid, J. (2011) · Journal of Quantitative Criminology · doi:10.1007/s10940-011-9137-7 · Cited by 112 · see citing works
“There is still no consensus as to the proper interpretation of AUC estimates for predictive validity, although different interpretations have been suggested by different researchers ( Rice and Harris 1995b; Sjo ¨stedt and Grann 2002).”
BackgroundMethodInfluentialSingh, J. P., Desmarais, S. L., & van Dorn, R. V. (2013) · Behavioral sciences & the law · doi:10.1002/bsl.2053 · Cited by 112 · see citing works
Ping, P., Qin, W.-H., Xu, Y., Miyajima, C., & Takeda, K. (2019) · IEEE Access · doi:10.1109/access.2019.2920489 · Cited by 107 · see citing works
Trait Repetitive Negative Thinking: A Brief Transdiagnostic Assessment
McEvoy, P. M., Thibodeau, M. A., & Asmundson, G. J. G. (2014) · Journal of Experimental Psychopathology · doi:10.5127/jep.037813 · Cited by 104 · see citing works
“Receiver-operator characteristic analyses using RTQ-10 total scores resulted in a large (Rice & Harris, 2005) area under the curve (.86, SE =.01) when classifying individuals as belonging to either the undergraduate or clinical sample.”
MethodAdler, D. A., Ben-Zeev, D., Tseng, V. W. S., Kane, J., Brian, R., Campbell, A. T., Hauser, M., Scherer, E. A., & Choudhury, T. (2020) · JMIR mHealth and uHealth · doi:10.2196/19962 · Cited by 104 · see citing works
“We chose the effect size, a metric traditionally used to measure the strength of a treatment in an RCT, to identify the most differentiated features within NR30 [54].”
BackgroundMethodCharacterizing the Value of Actuarial Violence Risk Assessments
Harris, G., & Rice, M. (2007) · Criminal Justice and Behavior · doi:10.1177/0093854807307029 · Cited by 103 · see citing works
“Predictive accuracy might be characterized in various ways (e.g., percentage correct, odds ratios, Cohen’s d, r), but we recommended area under the relative operating characteristic2 (ROC area; Mossman, 1994; Rice & Harris, 1995, 2005).”
BackgroundSelf-citationBrankley, A. E., Babchishin, K., & Hanson, R. K. (2019) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063219871572 · Cited by 100 · see citing works
“For AUC values, Cohen’s labels describe .56 as a small effect, .64 as a moderate effect, and .71 as a large effect (Rice & Harris, 2005).”
MethodShort- and long-term recidivism prediction of the PCL-R and the effects of age: a 24-year follow-up.
Olver, M., & Wong, S. C. P. (2015) · Personality disorders · doi:10.1037/per0000095 · Cited by 100 · see citing works
Reindl, M., Tiller, C., Holzknecht, M., Lechner, I., Beck, A., Plappert, D., Gorzala, M., Pamminger, M., Mayr, A., Klug, G., et al. (2019) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.119.009404 · Cited by 99 · see citing works
Machine learning and modeling: Data, validation, communication challenges
Naqa, I. E., Ruan, D., Valdes, G., Dekker, A., McNutt, T., Ge, Y., Wu, Q., Oh, J. H., Thor, M., Smith, W. P., et al. (2018) · Medical Physics (Lancaster) · doi:10.1002/mp.12811 · Cited by 99 · see citing works
“7), does occur often and should be cautiously addressed in scientific publications and presentations.(51,52)”
Tully, R., Chou, S.-N., & Browne, K. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.12.002 · Cited by 98 · see citing works
Babchishin, K., & Helmus, L. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0627-7 · Cited by 97 · see citing works
“AUCs are more robust to base rates than correlations, are more widely recommended (Rice & Harris, 2005), and are easily computed in many statistical packages, such as SPSS (see BAppendix^).”
BackgroundMethodInfluentialDevelopment and Adoption of an Adaptive Learning System: Reflections and Lessons Learned
Khosravi, H., Sadiq, S., & Gašević, D. (2020) · Technical Symposium on Computer Science Education · doi:10.1145/3328778.3366900 · Cited by 96 · see citing works
“6 which is still considered a medium to large effect size [44].”
BackgroundVana, K. D., Silva, G. E., & Goldberg, R. (2013) · Research in Nursing and Health · doi:10.1002/nur.21512 · Cited by 96 · see citing works
Smartphone technology can measure postural stability and discriminate fall risk in older adults.
Hsieh, K. L., Roach, K. L., Wajda, D., & Sosnoff, J. (2019) · Gait & Posture · doi:10.1016/j.gaitpost.2018.10.005 · Cited by 95 · see citing works
Interviewing to detect deception
Vrij, A. (2014) · European Psychologist · doi:10.1027/1016-9040/a000201 · Cited by 94 · see citing works
“He gave as an example the difference in height between 15- and 16-yearold girls (Cohen, 1988; Rice & Harris, 2005).”
BackgroundBabchishin, K., Nunes, K., & Hermann, C. A. (2013) · Archives of Sexual Behavior · doi:10.1007/s10508-012-0022-8 · Cited by 92 · see citing works
Implicit Measurement of Sexual Associations in Child Sex Abusers
Brown, A. S., Gray, N., & Snowden, R. (2009) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063209332234 · Cited by 91 · see citing works
Sources of Variability in Estimates of Predictive Validity
Andrews, D. A., Bonta, J., Wormith, J., Guzzo, L., Brews, A., Rettinger, J., & Rowe, R. (2011) · Criminal Justice and Behavior · doi:10.1177/0093854811401990 · Cited by 88 · see citing works
“The corresponding AUC estimates based on the Rice and Harris (2005) conversion tables were .63, .71, and .77.”
BackgroundMethodInfluentialScreening for Depression Prevention: Identifying Adolescent Girls at High Risk for Future Depression
Seeley, J., Stice, E., & Rohde, P. (2009) · Journal of Abnormal Psychology · doi:10.1037/a0014741 · Cited by 88 · see citing works
“In terms of effect size corresponding to Cohen’s d, AUC values of .56, .64, and .71 represent small, medium, and large effects, respectively (Rice & Harris, 2005).”
BackgroundShort-Term Assessment of Risk and Treatability (START): systematic review and meta-analysis.
O'Shea, L., & Dickens, G. (2014) · Psychological Assessment · doi:10.1037/a0036794 · Cited by 87 · see citing works
Hilton, N., Harris, G., & Rice, M. (2009) · American Psychological Association eBooks · doi:10.1037/12066-000 · Cited by 87 · see citing works
Self-citationEisenberg, M., van Horn, J. V., Dekker, J., Assink, M., van der Put, C. E., Hendriks, J., & Stams, G. (2019) · Criminal Justice and Behavior · doi:10.1177/0093854819826109 · Cited by 85 · see citing works
Main, A. R., Webb, E. B., Goyne, K. W., & Mengel, D. C. (2018) · Ecological Applications · doi:10.1002/eap.1723 · Cited by 105 · see citing works
van der Put, C. E., Assink, M., & van Solinge, N. F. B. (2017) · International Journal of Child Abuse & Neglect · doi:10.1016/j.chiabu.2017.09.016 · Cited by 85 · see citing works
Gardner, B., Boccaccini, M., Bitting, B. S., & Edens, J. (2015) · Psychological Assessment · doi:10.1037/pas0000065 · Cited by 85 · see citing works
Mossman, D. (2013) · Behavioral sciences & the law · doi:10.1002/bsl.2050 · Cited by 84 · see citing works
Predicting domestic violence: A meta-analysis on the predictive validity of risk assessment tools
van der Put, C. E., Gubbels, J., & Assink, M. (2019) · Aggression and Violent Behavior · doi:10.1016/j.avb.2019.03.008 · Cited by 83 · see citing works
Even Highly Correlated Measures Can Add Incrementally to Predicting Recidivism Among Sex Offenders
Babchishin, K. M., Hanson, R. K., & Helmus, L. M. (2012) · Assessment · doi:10.1177/1073191112458312 · Cited by 83 · see citing works
“The Area Under the Receiver Operating Characteristic Curves (AUC ROC) is the most common method of assessing discrimination (Pintea & Moldovan, 2009; Rice & Harris, 2005; Swets, Dawes, & Monahan, 2000).”
MethodAssessing Protective Factors for Sexually Violent Offending With the SAPROF
Robbé, M. D. V., de Vogel, V., Koster, K., & Bogaerts, S. (2015) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063214550168 · Cited by 82
“…validity for violent and sexually violent recidivism of each tool individually and of the combined HCR-SAPROF index and the combined SVR-SAPROF index, Receiver Operating Characteristics (ROC; Mossman, 1994; Rice & Harris, 2005) analyses were conducted resulting in Area Under the Curve (AUC) values.”
MethodYahav-Dovrat, A., Saban, M., Merhav, G., Lankri, I., Abergel, E., Eran, A., Tanne, D., Nogueira, R., & Sivan-Hoffmann, R. (2020) · American Journal of Neuroradiology · doi:10.3174/ajnr.a6923 · Cited by 81 · see citing works
“70 and higher would be considered strong effects.(15) The level of significance for all statistical analyses was 5%.”
BackgroundRichmond-Rakerd, L., Caspi, A., Arseneault, L., Baldwin, J., Danese, A., Houts, R., Matthews, T., Wertz, J., & Moffitt, T. (2019) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.18060740 · Cited by 80 · see citing works
“82, indicating a large effect (25) that requires out-of-sample replication (Supplementary Text)).”
- Authors
- Rice, M. E., & Harris, G. T.
- Affiliations
- Waypoint Centre for Mental Health Care (CA)
- Venue
- Law and Human Behavior
- Published
- 2005-10
- Volume
- 29
- Issue
- 5
- Pages
- 615-620
- ISSN
- 0147-7307 (ISSN-L)
- 1573-661X
- OpenAlex
- W2060774914
- Cited by
- 1,608
- References
- 0
- Citations by year
- 2026: 60 · 2025: 84 · 2024: 90 · 2023: 121 · 2022: 151 · 2021: 150 · 2020: 174 · 2019: 137 · 2018: 112 · 2017: 80 · 2016: 88 · 2015: 65 · 2014: 64 · 2013: 58 · 2012: 51
- FWCI
- 11.55
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Law and Human Behavior (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- effect size, Cohen's d, diagnostic accuracy, prediction accuracy, forensic psychology, follow-up study
- MeSH terms
- Forecasting, Humans, ROC Curve, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
