The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Comparing effect sizes in follow-up studies: ROC Area, Cohen's d, and r.
Abstract
In order to facilitate comparisons across follow-up studies that have used different measures of effect size, we provide a table of effect size equivalencies for the three most common measures: ROC area (AUC), Cohen's d, and r. We outline why AUC is the preferred measure of predictive or diagnostic accuracy in forensic psychology or psychiatry, and we urge researchers and practitioners to use numbers rather than verbal labels to characterize effect sizes.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The magnitude of the effects of these risk domains ranged from small (i.e., r= 0.099 for “low IQ/learning difficulties”) to large (i.e., r= 0.553 for “having a negative school attitude”) based on the criteria of Rice and Harris (2005) for interpreting effect sizes.”
From Risk Factors for School Absenteeism and Dropout: A Meta-Analytic Review (2019) · cites this paper as Rice and Harris (2005)
In the citing paper
“The AUC of 0.62 is also a moderate effect size [14], but is less than that for deadly violence (but this difference was not statistically significant).”
From The Domestic Abuse, Stalking and Harassment and Honour-Based Violence (DASH) Risk Assessment Instrument in Predicting Deadly or Persistent Domestic Abuse (2025) · cites this paper as [14]
In the citing paper
“The AUC of 0.67 for deadly violence is regarded as a “moderate” effect size [14].”
From The Domestic Abuse, Stalking and Harassment and Honour-Based Violence (DASH) Risk Assessment Instrument in Predicting Deadly or Persistent Domestic Abuse (2025) · cites this paper as [14]
In the citing paper
“AUC values of 0.56, 0.64, and 0.71 indicate small, medium, and large effects (Rice & Harris, 2005).”
From Strategies for monitoring mentoring relationship quality to predict early program dropout (2022) · cites this paper as Rice & Harris, 2005
Citations
Cites
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Interpreting the magnitudes of correlation coefficients.
Hemphill, J. F. (2003) · American Psychologist · doi:10.1037/0003-066x.58.1.78 · Cited by 1,237
A common language effect size statistic
McGraw, K. O., & Wong, S. P. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.361 · Cited by 934 · cited 1×
“AUC also has a plain language interpretation because it is conceptually and mathematically similar to the common language effect size or CL ( McGraw & Wong, 1992; see also Delaney & Vargha, 2002).”
Flagged first, then by citations
Research works printed but not linked: 13
- Berlin, F. S., Galbreath, N. W., Geary, B., & McGlone, G. (2003). The use of actuarials at civil commitment hearings to predict the likelihood of future sexual violence. Sexual Abuse: A Journal of Research and Treatment, 15, 377–382. · DOI
- Cohen, J. (1969). Statistical power analysis for the behavioral sciences. New York: Academic Press.
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
- Cohen, J. (1992). A power primer. Psychological Bulletin, 122, 155–159. · DOI · Paper page
- Delaney, H. D., & Vargha, A. (2002). Comparing several robust tests of stochastic equality with ordinally scaled variables and small to moderate sized samples. Psychological Methods, 7, 485–503. · DOI
- Harris, G. T., & Rice, M. E. (2003). Actuarial assessment of risk among sex offenders. In R. A. Prentky, E. S. Janus, & M. C. Seto (Eds.), Understanding and managing sexually coercive behavior, Vol. 989 (pp. 198–210). New York: Annals of the New York Academy of Sciences. · DOI
- Mossman, D. (1994). Assessing predictions of violence being accurate about accuracy. Journal of Consulting and Clinical Psychology, 62, 783–792. · DOI
- Pearson, E. S., & Hartley, H. O. (Eds.). (1954). Biometrika tables for statisticians, Vol. 1 (1st ed.). Cambridge: Cambridge University Press.
- Rice, M. E., & Harris, G. T. (1995). Violent recidivism: Assessing predictive validity. Journal of Consulting and Clinical Psychology, 63, 737–748. · DOI
- Rosenthal, R. (1990). How are we doing in soft psychology? American Psychologist, 45, 775–777. · DOI
- Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74, 166–169. · DOI
- Swets, J. A. (1986). Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. Psychological Bulletin, 99, 100–117. · DOI
- Swets, J. A., Dawes, R. M., & Monahan, J. (2000). Psychological science can improve diagnostic decisions. Psychological Science in the Public Interest: A Journal of the American Psychological Society, 1, 1–26. · DOI
Cited by
Sorted by citations
Inkster, B., Sarda, S., & Subramanian, V. (2018) · doi:10.2196/preprints.12106 · Cited by 1,063
“The CL gives the probability that a user picked at random from the high users group will have a higher average improvement than a user picked at random from the low users group [38,39].”
BackgroundHanson, R., & Morton-Bourgon, K. E. (2009) · Psychological Assessment · doi:10.1037/a0014421 · Cited by 1,034
“Although the effect sizes for most of the measures were moderate to large by conventional standards (Cohen, 1988; Rice & Harris, 2005), there are no direct procedures for translating the observed effect sizes into statements concerning the normative value or utility of specific measures (e.g., is a…”
BackgroundMann, R. E., Hanson, R. K., & Thornton, D. (2010) · Sexual Abuse · doi:10.1177/1079063210366039 · Cited by 1,044
“Both d and AUC are based on similar statistical models (Swets, 1986), and the transformations between these metrics are well documented: for example, d of 0.20 = AUC of 0.556, d of 0.50 = AUC of 0.639, d of 0.80 = AUC of 0.714; Rice & Harris, 2005).”
BackgroundInfluentialViolent offenders: Appraising and managing risk (2nd ed.).
Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (2006) · American Psychological Association eBooks · doi:10.1037/11367-000 · Cited by 680
Self-citationThe efficacy of violence prediction: a meta-analytic comparison of nine risk assessment tools.
Yang, M., Wong, S. C. P., & Coid, J. (2010) · Psychological Bulletin · doi:10.1037/a0020473 · Cited by 525
“For eight studies that reported only the AUC values, a direct conversion of the AUC value to the d value was carried out on the basis of the table of M. E. Rice and Harris (2005).”
BackgroundMethodInfluentialRisk Factors for School Absenteeism and Dropout: A Meta-Analytic Review
Gubbels, J., van der Put, C. E., & Assink, M. (2019) · Journal of Youth and Adolescence · doi:10.1007/s10964-019-01072-5 · Cited by 596
“The magnitude of the effects of these risk domains ranged from small (i.e., r= 0.099 for “low IQ/learning difficulties”) to large (i.e., r= 0.553 for “having a negative school attitude”) based on the criteria of Rice and Harris (2005) for interpreting effect sizes.”
MethodInfluentialMachine learning prediction in cardiovascular diseases: a meta-analysis
Krittanawong, C., Virk, H. H., Bangalore, S., Wang, Z., Johnson, K. W., Pinotti, R., Zhang, H.-J., Kaplin, S. L., Narasimhan, B., Kitai, T., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-72685-1 · Cited by 447
A probability-based measure of effect size: Robustness to base rates and other factors.
Ruscio, J. (2008) · Psychological Methods · doi:10.1037/1082-989x.13.1.19 · Cited by 398
“Rice and Harris (2005) presented tables of equivalent d, r pb , and CL values under the assumptions of normality and equal variances, and they provided conversion formulas that apply in this case.”
MethodImproving the Predictive Accuracy of Static-99 and Static-2002 With Older Sex Offenders
Helmus, L., Thornton, D., Hanson, R. K., & Babchishin, K. (2012) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063211409951 · Cited by 349
Eliciting cues to deception and truth: What matters are the questions asked.
Vrij, A., & Granhag, P. A. (2012) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2012.02.004 · Cited by 344
“He gave as an example the difference in height between 15- and 16-year-old girls (Cohen, 1988; Rice & Harris, 2005).”
BackgroundRISK, RACE, AND RECIDIVISM: PREDICTIVE BIAS AND DISPARATE IMPACT*
SKEEM, J. L., & LOWENKAMP, C. T. (2016) · Criminology · doi:10.1111/1745-9125.12123 · Cited by 317
“Minimum AUCs of .56, .64, and .71 correspond to “small,” “medium,” and “large” effect sizes, respectively (see Rice and Harris, 2005).”
Risk factors for child neglect: A meta-analytic review
Mulder, T. M., Kuiper, K. C., van der Put, C. E., Stams, G. J. J. M., & Assink, M. (2018) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.01.006 · Cited by 437
Risk Assessment for Juvenile Justice: A Meta-Analysis
Schwalbe, C. (2007) · Law and Human Behavior · doi:10.1007/s10979-006-9071-7 · Cited by 310
“b Conversion to r based on Rice and Harris (2005) .”
BackgroundMethodInfluentialChildhood forecasting of a small segment of the population with large economic burden
Caspi, A., Houts, R., Belsky, D. W., Harrington, H., Hogan, S., Ramrakha, S., Poulton, R., & Moffitt, T. (2016) · Nature Human Behaviour · doi:10.1038/s41562-016-0005 · Cited by 302
The intergenerational transmission of child maltreatment: A three-level meta-analysis
Assink, M., Spruit, A., Schuts, M., Lindauer, R., van der Put, C. E., & Stams, G.-J. J. M. (2018) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2018.07.037 · Cited by 275
“Our meta-analysis yielded a signi fi cant summary e ff ect of r = 0.289, which is medium according to Rice and Harris ’ (2005) criteria for interpreting e ff ect sizes.”
MethodInfluentialRisk Assessment With Young Offenders
Olver, M., Stockdale, K. C., & Wormith, J. (2009) · Criminal Justice and Behavior · doi:10.1177/0093854809331457 · Cited by 266
“When only an AUC statistic from Receiver Operating Characteristic (ROC) analyses was provided, the tables provided in Rice and Harris (2005) were used to convert an ROC value into the equivalent rpb. 3 If correlations with continuous recidivism measures (e.g., total convictions) were reported, we…”
MethodOlver, M., Stockdale, K. C., & Wormith, J. (2014) · Psychological Assessment · doi:10.1037/a0035080 · Cited by 264
Risk Assessment in Criminal Sentencing
Monahan, J., & Skeem, J. L. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-092945 · Cited by 256
Kessler, R. C., van Loo, H. M., Wardenaar, K. J., Bossarte, R. M., Brenner, L. A., Cai, T., Ebert, D. D., Hwang, I., Li, J. J., de Jonge, P., et al. (2016) · Molecular Psychiatry · doi:10.1038/mp.2015.198 · Cited by 263
Bowers, A. J., Sprott, R., & Taff, S. A. (2013) · The High School journal/The High school journal · doi:10.1353/hsj.2013.0000 · Cited by 243
“While outside the scope of the present study, we encourage future research in this domain to examine the use of continuous indicator outcomes, such as ROC “Area Under the Curve” (AUC) analysis (Swets, 1988), which is amenable to effect size comparisons such as meta-analysis (Rice & Harris, 2005).”
MethodAre the Major Risk/Need Factors Predictive of Both Female and Male Reoffending?
Andrews, D. A., Guzzo, L., Raynor, P., Rowe, R., Rettinger, L. J., Brews, A., & Wormith, J. (2012) · International journal of offender therapy and comparative criminology · doi:10.1177/0306624x10395716 · Cited by 233
“AUC is less sensitive to variations in the base rate of recidivism (Rice & Harris, 2005).”
BackgroundHelmus, L., Hanson, R. K., Thornton, D., Babchishin, K., & Harris, A. (2012) · Criminal Justice and Behavior · doi:10.1177/0093854812443648 · Cited by 218
“The most commonly recommended statistic for reporting predictive accuracy is the area under the receiver operating characteristic curve (AUC for ROC; Mossman, 1994; Rice & Harris, 2005; Swets, Dawes, & Monahan, 2000).”
BackgroundBowers, A. J., & Zhou, X. (2019) · Journal of Education for Students Placed at Risk (JESPAR) · doi:10.1080/10824669.2018.1523734 · Cited by 260
“…& Campbell, 1993) and has more recently been applied to domains such as law, education, and youth development to understand the accuracy of at-risk indicators and predictors (Bowers, Sprott, & Taff, 2013; Knowles, 2015; Olver, Stockdale, & Wormith, 2009; Rice & Harris, 2005; Vivo & Franco, 2008).”
MethodOsteopontin is a marker for cancer aggressiveness and patient survival
Weber, G., Lett, G. S., & Haubein, N. C. (2010) · British Journal of Cancer · doi:10.1038/sj.bjc.6605834 · Cited by 201
“…feature used to quantify this characteristic is the area under the curve, which can be interpreted here as the probability that for two randomly chosen samples, the one with the higher Osteopontin rank will also have a higher rank for the clinical variable in question (Rice and Harris, 2005).”
Firm, fair, and caring officer-offender relationships protect against supervision failure.
Kennealy, P., Skeem, J., Manchak, S. M., & Louden, J. E. (2012) · Law and Human Behavior · doi:10.1037/h0093935 · Cited by 189
“In this study, DRI-R total scores had an AUC value of .68, CI [.58, .78], p Ͻ .01, which is considered a medium effect size according to Cohen's standards (see Rice & Harris, 2005).”
MethodPredictive Validity Performance Indicators in Violence Risk Assessment: A Methodological Primer
Singh, J. P. (2013) · Behavioral Sciences & the Law · doi:10.1002/bsl.2052 · Cited by 205
Asscher, J. J., van Vugt, E., Stams, G. J. J. M., Deković, M., Eichelsheim, V. I., & Yousfi, S. (2011) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2011.02412.x · Cited by 193
“Those statistics for which no formula was presented, e.g., AUCs, were transformed with the help of the table provided by Rice and Harris (2005).”
MethodHawes, S., Boccaccini, M., & Murrie, D. (2013) · Psychological Assessment · doi:10.1037/a0030391 · Cited by 185
Handbook of violence risk assessment.
Otto, R., & Douglas, K. (2010) · Routledge eBooks · doi:10.4324/9780203843666 · Cited by 181
Guo, W., & Caliskan, A. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462536 · Cited by 180
“g the effect size in the null hypothesis, in case biased associations did not exist. According to Cohen’s effect size metric, d>j0:5 jand d>j0:8 jare medium and large effect sizes, respectively [38]. In a neural language model, each stimulus sfrom WEAT contained in n s input sentences has at most n s different C…”
MethodThe Average Predictive Validity of Intimate Partner Violence Risk Assessment Instruments
Messing, J., & Thaller, J. (2013) · Journal of Interpersonal Violence · doi:10.1177/0886260512468250 · Cited by 180
“Of these nine, five of the correctly administered risk assessments were used in tests of the ODARA.”
BackgroundMethodInfluentialHilton, N., Harris, G., Rice, M., Houghton, R. E., & Eke, A. W. (2008) · Law and Human Behavior · doi:10.1007/s10979-007-9088-6 · Cited by 174
“The ROC area for the predictive ability of DVRAG scores in Sample 1 was .71, SE = .03, corresponding to a Cohen’s d = .80 (Rice & Harris, 2005), and represented a statistically significant improvement (p .05, 1-tailed) over ODARA scores alone.”
Self-citationThe psychological assessment of risk for crime and violence.
Hanson, R. K. (2009) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0015726 · Cited by 172
Gray, N., Fitzgerald, S., Taylor, J., Macculloch, M., & Snowden, R. (2007) · Psychological Assessment · doi:10.1037/1040-3590.19.4.474 · Cited by 165
“Using standard conventions, we considered AUCs .56 as small effects, AUCs .64 as medium effects, and AUCs .71 as large effects (Rice & Harris, 2005).”
MethodDai, C., Zhang, H., Arens, E., & Lian, Z. (2017) · doi:10.1016/j.buildenv.2016.12.005 · Cited by 164
Robbé, M. D. V., de Vogel, V., & de Spa, E. (2011) · International Journal of Forensic Mental Health · doi:10.1080/14999013.2011.600232 · Cited by 163
“In or- der to assess the predictive validity for violent recidivism, Receiver Operating Characteristics (ROC; Mossman, 1994; Rice & Harris, 1995, 2005) analyses were carried out with recidivism as outcome measure for the HCR-20 and nonrecidivism as outcome measure for the SAPROF.”
Walters, G. (2020) · Trauma, Violence, & Abuse · doi:10.1177/1524838020906513 · Cited by 152
Hartung, T., Friedrich, M., Johansen, C., Wittchen, H., Faller, H., Koch, U., Brähler, E., Härter, M., Keller, M., Schulz, H., et al. (2017) · Cancer · doi:10.1002/cncr.30846 · Cited by 150
SCHMIDT, F. H., Campbell, M. A., & Houlding, C. (2010) · Youth Violence and Juvenile Justice · doi:10.1177/1541204010371793 · Cited by 166
“Rice and Harris (2005) described AUC values of .556 as small, .639 as moderate, and .714 as large predictive validity effect sizes.”
BackgroundBrown, L., & Kuss, D. J. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17124566 · Cited by 201
“01) [66] between SC change scores and MWB change scores: participants’ increase in perceived SC following abstinence was positively related with their increase in perceived MWB following abstinence.”
BackgroundInfluentialBaglivio, M. T., & Jackowski, K. (2013) · Youth Violence and Juvenile Justice · doi:10.1177/1541204012440107 · Cited by 147
“50s is fairly modest and considered ‘‘small’’ in magnitude of effect size (Rice and Harris, 2005).”
BackgroundInfluentialRice, M., Harris, G., & Lang, C. (2013) · Psychological Assessment · doi:10.1037/a0032878 · Cited by 143
Self-citationLuong, D., & Wormith, J. S. (2011) · Criminal Justice and Behavior · doi:10.1177/0093854811421596 · Cited by 149
“Receiver operating characteristics are often preferred over correlations because of their relative insensitivity to differences in the base rate of the criterion variable: in this case, recidivism (Rice & Harris, 1995, 2005).”
BackgroundThe assessment of risk to recidivate among a juvenile offending population
Baglivio, M. T. (2009) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2009.09.008 · Cited by 131
The Estimate of Risk of Adolescent Sexual Offense Recidivism (ERASOR): Preliminary Psychometric Data
Worling, J. R. (2004) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/107906320401600305 · Cited by 133
Performance of recidivism risk assessment instruments in U.S. correctional settings.
Desmarais, S. L., Johnson, K. L., & Singh, J. P. (2016) · Psychological services · doi:10.1037/ser0000075 · Cited by 132
“…1.50–2.99 .10–.19 Good .50–.79 .24–.36 .64–.71 3.00–4.99 .20–.29 Excellent > .80 .37–1.00 .71–1.00 > 5.00 .30–1.00 Notes: Benchmarks were anchored to Cohen’s d (1988) and based upon the calculations of Rice and Harris (2005) for AUC values and Chen, Cohen, and Chen (2010) for the odds ratios.”
InfluentialRobbé, M. D. V., de Vogel, V., Douglas, K., & Nijman, H. (2015) · Law and Human Behavior · doi:10.1037/lhb0000089 · Cited by 132
Predicting the incidence of non-suicidal self-injury in college students
Kiekens, G., Hasking, P. A., Claes, L., Boyes, M. E., Mortier, P., Auerbach, R. P., Cuijpers, P. J., Demyttenaere, K., Green, J. G., Kessler, R. C., et al. (2019) · European Psychiatry · doi:10.1016/j.eurpsy.2019.04.002 · Cited by 189
“Resulting Area Under the Curve (AUC) values close to .56, .64, and .71 are considered, respectively, as small, moderate, and large effects [60].”
BackgroundOlver, M., Stockdale, K. C., & Wong, S. C. P. (2012) · Law and Human Behavior · doi:10.1037/h0093927 · Cited by 129
Jordan, N., Glutting, J., Ramineni, C., & Watkins, M. W. (2010) · School Psychology Review · doi:10.1080/02796015.2010.12087772 · Cited by 129
Self-declared Throwaway Accounts on Reddit
Ammari, T., Schoenebeck, S. Y., & Romero, D. M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359237 · Cited by 102
Lichtenberg, P. A., Sugarman, M. A., Paulson, D., Ficker, L. J., & Rahman-Filipiak, A. (2016) · Clinical Gerontologist · doi:10.1080/07317115.2015.1101632 · Cited by 127
“An AUC of .65 corresponds to a Cohen’s d value of about 0.54, in the range of what is considered to be a “medium” effect size for discriminating between groups (Rice & Harris, 2005).”
BackgroundInfluentialRisk Assessment in Intimate Partner Violence: A Systematic Review of Contemporary Approaches
Nicholls, T. L., Pritchard, M. M., Reeves, K. A., & Hilterman, E. L. B. (2013) · Partner Abuse · doi:10.1891/1946-6560.4.1.76 · Cited by 137
“…VRAG; see Tables 1–7), our review suggests that the measures generally perform in the moderate to good range according to statistical standards (e.g., r .30; Cohen, 1988; AUC .60; Cicchetti & Sparrow, 1981; Rice & Harris, 2005; generally for a risk assessment perspective, see Otto & Douglas, 2010).”
BackgroundSalgado, J. (2018) · The European Journal of Psychology Applied to Legal Context · doi:10.5093/ejpalc2018a5 · Cited by 126
“In order to facilitate comparisons across follow-up studies that report different ES indexes, Rice and Harris (2005) developed a table of the equivalences among AUC and d, and rpb. Rice and Harris (2005) Table includes the AUC values from .50 to 1 and the respective values of z, d, and rpb.”
BackgroundMethodInfluentialPredictors of First Lifetime Onset of Major Depressive Disorder in Young Adulthood
Klein, D., Glenn, C. R., Kosty, D. B., Seeley, J., Rohde, P., & Lewinsohn, P. (2012) · Journal of Abnormal Psychology · doi:10.1037/a0029567 · Cited by 126
“The AUC values of .56, .64, and .71 correspond to small, medium, and large effects, respectively (Rice & Harris, 2005).”
Dinga, R., Marquand, A. F., Veltman, D. J., Beekman, A. T. F., Schoevers, R. A., van Hemert, A. M., Penninx, B. W. J. H., & Schmaal, L. (2018) · Translational Psychiatry · doi:10.1038/s41398-018-0289-1 · Cited by 125
Primer on Risk Assessment and the Statistics Used to Evaluate Its Accuracy
Helmus, L. M., & Babchishin, K. M. (2016) · Criminal Justice and Behavior · doi:10.1177/0093854816678898 · Cited by 154
Lutz, W., Schwartz, B., Hofmann, S. G., Fisher, A. J., Husen, K., & Rubel, J. A. (2018) · Scientific Reports · doi:10.1038/s41598-018-25953-0 · Cited by 145
Predicting violent reconvictions using the HCR-20
Gray, N., Taylor, J., & Snowden, R. (2008) · British Journal of Psychiatry · doi:10.1192/bjp.bp.107.044065 · Cited by 124
Simulating computer adaptive testing with the Mood and Anxiety Symptom Questionnaire.
Flens, G., Smits, N., Carlier, I., van Hemert, A., & de Beurs, E. (2016) · Psychological Assessment · doi:10.1037/pas0000240 · Cited by 121
“We evaluated the AUC values using the guideline proposed by Rice and Harris (2005): .56 (cid:2) small effect, .64 (cid:2) medium effect, .71 (cid:2) large effect; a higher effect meaning a higher predictive utility for the scale.”
MethodWormith, J. S., Hogg, S. M., & Guzzo, L. (2012) · doi:10.1177/0093854812455741 · Cited by 120
Risk factors and protective factors: a two-sided dynamic approach to violence risk assessment
Robbé, M. D. V., de Vogel, V., & Douglas, K. (2013) · doi:10.1080/14789949.2013.818162 · Cited by 118
“To assess the predictive validity for violent recidivism, receiver operating characteristics (ROC; Mossman, 1994; Rice & Harris, 1995, 2005) analyses were carried out with recidivism at different follow-up times as outcome measure for the HCR-20 and non-recidivism as outcome measure for the SAPROF.”
MethodTan, J. P., Cheng, K. K. F., & Siah, C. J. R. (2019) · Journal of Advanced Nursing · doi:10.1111/jan.14025 · Cited by 158
Predictive modelling and analytics for diabetes using a machine learning approach
Kaur, H., & Kumari, V. (2018) · Applied Computing and Informatics · doi:10.1016/j.aci.2018.12.004 · Cited by 357
The Impact of Forced Answering and Reactance on Answering Behavior in Online Surveys
Sischka, P., Décieux, J. P., Mergener, A., Neufang, K. M., & Schmidt, A. F. (2020) · Social science computer review · doi:10.1177/0894439320907067 · Cited by 114
“…operating characteristics (ROC) analyses (Swets, 1986) that are independent from base rates to correct for deflated associations due to floor effects in criterion base rates ( AUC s of .50, .65, .75, and .90 correspond to unbiased r pb s of 0, .26, .43, and .67, respectively; Rice & Harris, 2005).”
BackgroundPredictive Validity of Risk Assessments in Juvenile Offenders
Hilterman, E., Nicholls, T. L., & van Nieuwenhuizen, C. (2014) · Assessment (Odessa, Fla.) · doi:10.1177/1073191113498113 · Cited by 114
“AUCs between .56 and .64 are considered small effects, AUCs above .64 are medium effects, and AUCs greater than .71 are described as large effects (Rice & Harris, 2005).”
BackgroundViljoen, J. L., Mordell, S., & Beneteau, J. L. (2012) · Law and Human Behavior · doi:10.1037/h0093938 · Cited by 113
Viljoen, J. L., Cochrane, D. M., & Jonnson, M. R. (2018) · Law and Human Behavior · doi:10.1037/lhb0000280 · Cited by 136
Liu, Y. Y., Yang, M., Ramsay, M., Li, X. S., & Coid, J. W. (2011) · Journal of Quantitative Criminology · doi:10.1007/s10940-011-9137-7 · Cited by 114
“There is still no consensus as to the proper interpretation of AUC estimates for predictive validity, although different interpretations have been suggested by different researchers ( Rice and Harris 1995b; Sjo ¨stedt and Grann 2002).”
BackgroundMethodInfluentialSingh, J. P., Desmarais, S. L., & van Dorn, R. V. (2013) · Behavioral sciences & the law · doi:10.1002/bsl.2053 · Cited by 112
Ping, P., Qin, W., Xu, Y., Miyajima, C., & Takeda, K. (2019) · IEEE Access · doi:10.1109/access.2019.2920489 · Cited by 135
Trait Repetitive Negative Thinking: A Brief Transdiagnostic Assessment
McEvoy, P. M., Thibodeau, M. A., & Asmundson, G. J. G. (2014) · Journal of Experimental Psychopathology · doi:10.5127/jep.037813 · Cited by 104
“Receiver-operator characteristic analyses using RTQ-10 total scores resulted in a large (Rice & Harris, 2005) area under the curve (.86, SE =.01) when classifying individuals as belonging to either the undergraduate or clinical sample.”
MethodAdler, D. A., Ben‐Zeev, D., Tseng, V. W.-S., Kane, J. M., Brian, R., Campbell, A. T., Hauser, M., Scherer, E. A., & Choudhury, T. (2020) · JMIR mhealth and uhealth · doi:10.2196/19962 · Cited by 107
“We chose the effect size, a metric traditionally used to measure the strength of a treatment in an RCT, to identify the most differentiated features within NR30 [54].”
BackgroundMethodCharacterizing the Value of Actuarial Violence Risk Assessments
Harris, G., & Rice, M. (2007) · Criminal Justice and Behavior · doi:10.1177/0093854807307029 · Cited by 103
“Predictive accuracy might be characterized in various ways (e.g., percentage correct, odds ratios, Cohen’s d, r), but we recommended area under the relative operating characteristic2 (ROC area; Mossman, 1994; Rice & Harris, 1995, 2005).”
BackgroundSelf-citationBrankley, A. E., Babchishin, K. M., & Hanson, R. K. (2019) · Sexual Abuse · doi:10.1177/1079063219871572 · Cited by 122
“For AUC values, Cohen’s labels describe .56 as a small effect, .64 as a moderate effect, and .71 as a large effect (Rice & Harris, 2005).”
MethodShort- and long-term recidivism prediction of the PCL-R and the effects of age: a 24-year follow-up.
Olver, M., & Wong, S. C. P. (2015) · Personality disorders · doi:10.1037/per0000095 · Cited by 100
Reindl, M., Tiller, C., Holzknecht, M., Lechner, I., Beck, A., Plappert, D., Gorzala, M., Pamminger, M., Mayr, A., Klug, G., et al. (2019) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.119.009404 · Cited by 112
Machine learning and modeling: Data, validation, communication challenges
Naqa, I. E., Ruan, D., Valdés, G., Dekker, A. L. A. J., McNutt, T., Ge, Y., Wu, Q., Oh, J. H., Thor, M., Smith, W. P., et al. (2018) · Medical Physics · doi:10.1002/mp.12811 · Cited by 112
“7), does occur often and should be cautiously addressed in scientific publications and presentations.(51,52)”
Tully, R., Chou, S.-N., & Browne, K. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.12.002 · Cited by 98
Babchishin, K., & Helmus, L. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0627-7 · Cited by 97
“AUCs are more robust to base rates than correlations, are more widely recommended (Rice & Harris, 2005), and are easily computed in many statistical packages, such as SPSS (see BAppendix^).”
BackgroundMethodInfluentialDevelopment and Adoption of an Adaptive Learning System
Khosravi, H., Sadiq, S. W., & Gašević, D. (2020) · ACM Technical Symposium on Computer Science Education (SIGCSE) · doi:10.1145/3328778.3366900 · Cited by 123
“6 which is still considered a medium to large effect size [44].”
BackgroundVana, K. D., Silva, G. E., & Goldberg, R. (2013) · Research in Nursing and Health · doi:10.1002/nur.21512 · Cited by 96
Smartphone technology can measure postural stability and discriminate fall risk in older adults.
Hsieh, K. L., Roach, K. L., Wajda, D., & Sosnoff, J. (2019) · Gait & Posture · doi:10.1016/j.gaitpost.2018.10.005 · Cited by 95
Interviewing to Detect Deception
Vrij, A. (2014) · European Psychologist · doi:10.1027/1016-9040/a000201 · Cited by 92
“He gave as an example the difference in height between 15- and 16-yearold girls (Cohen, 1988; Rice & Harris, 2005).”
BackgroundBabchishin, K., Nunes, K., & Hermann, C. A. (2013) · Archives of Sexual Behavior · doi:10.1007/s10508-012-0022-8 · Cited by 92
Implicit Measurement of Sexual Associations in Child Sex Abusers
Brown, A. S., Gray, N., & Snowden, R. (2009) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063209332234 · Cited by 91
Sources of Variability in Estimates of Predictive Validity
Andrews, D. A., Bonta, J., Wormith, J., Guzzo, L., Brews, A., Rettinger, J., & Rowe, R. (2011) · Criminal Justice and Behavior · doi:10.1177/0093854811401990 · Cited by 88
“The corresponding AUC estimates based on the Rice and Harris (2005) conversion tables were .63, .71, and .77.”
BackgroundMethodInfluentialScreening for Depression Prevention: Identifying Adolescent Girls at High Risk for Future Depression
Seeley, J., Stice, E., & Rohde, P. (2009) · Journal of Abnormal Psychology · doi:10.1037/a0014741 · Cited by 88
“In terms of effect size corresponding to Cohen’s d, AUC values of .56, .64, and .71 represent small, medium, and large effects, respectively (Rice & Harris, 2005).”
BackgroundShort-Term Assessment of Risk and Treatability (START): systematic review and meta-analysis.
O'Shea, L., & Dickens, G. (2014) · Psychological Assessment · doi:10.1037/a0036794 · Cited by 87
Hilton, N. Z., Harris, G. T., & Rice, M. E. (2010) · American Psychological Association eBooks · doi:10.1037/12066-000 · Cited by 94
Self-citationEisenberg, M. J., van Horn, J. E., Dekker, J. M., Assink, M., van der Put, C. E., Hendriks, J., & Stams, G. J. J. M. (2019) · Criminal Justice and Behavior · doi:10.1177/0093854819826109 · Cited by 101
Main, A. R., Webb, E. B., Goyne, K. W., & Mengel, D. C. (2018) · Ecological Applications · doi:10.1002/eap.1723 · Cited by 111
van der Put, C. E., Assink, M., & van Solinge, N. F. B. (2017) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2017.09.016 · Cited by 138
Gardner, B., Boccaccini, M., Bitting, B. S., & Edens, J. (2015) · Psychological Assessment · doi:10.1037/pas0000065 · Cited by 85
Mossman, D. (2013) · Behavioral sciences & the law · doi:10.1002/bsl.2050 · Cited by 84
Predicting domestic violence: A meta-analysis on the predictive validity of risk assessment tools
van der Put, C. E., Gubbels, J., & Assink, M. (2019) · Aggression and Violent Behavior · doi:10.1016/j.avb.2019.03.008 · Cited by 83
Even Highly Correlated Measures Can Add Incrementally to Predicting Recidivism Among Sex Offenders
Babchishin, K. M., Hanson, R. K., & Helmus, L. M. (2012) · Assessment · doi:10.1177/1073191112458312 · Cited by 83
“The Area Under the Receiver Operating Characteristic Curves (AUC ROC) is the most common method of assessing discrimination (Pintea & Moldovan, 2009; Rice & Harris, 2005; Swets, Dawes, & Monahan, 2000).”
MethodAssessing Protective Factors for Sexually Violent Offending With the SAPROF
Robbé, M. D. V., de Vogel, V., Koster, K., & Bogaerts, S. (2015) · Sexual abuse. A Journal of Research and Treatment · doi:10.1177/1079063214550168 · Cited by 82
“…validity for violent and sexually violent recidivism of each tool individually and of the combined HCR-SAPROF index and the combined SVR-SAPROF index, Receiver Operating Characteristics (ROC; Mossman, 1994; Rice & Harris, 2005) analyses were conducted resulting in Area Under the Curve (AUC) values.”
MethodYahav-Dovrat, A., Saban, M., Merhav, G., Lankri, I., Abergel, E., Eran, A., Tanné, D., Nogueira, R. G., & Sivan-Hoffmann, R. (2020) · American Journal of Neuroradiology · doi:10.3174/ajnr.a6923 · Cited by 118
“70 and higher would be considered strong effects.(15) The level of significance for all statistical analyses was 5%.”
BackgroundRichmond-Rakerd, L., Caspi, A., Arseneault, L., Baldwin, J., Danese, A., Houts, R., Matthews, T., Wertz, J., & Moffitt, T. (2019) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.18060740 · Cited by 80
“82, indicating a large effect (25) that requires out-of-sample replication (Supplementary Text)).”
Metadata
- Authors
- Rice, M. E., & Harris, G. T.
- Affiliations
- Waypoint Centre for Mental Health Care (CA)
- Venue
- Law and Human Behavior
- Published
- 2005-10
- Volume
- 29
- Issue
- 5
- Pages
- 615-620
- ISSN
- 0147-7307 (ISSN-L)
- 1573-661X
- OpenAlex
- W2060774914
- Cited by
- 1,608
- References
- 0
- Citations by year
- 2026: 60 · 2025: 84 · 2024: 90 · 2023: 121 · 2022: 151 · 2021: 150 · 2020: 174 · 2019: 137 · 2018: 112 · 2017: 80 · 2016: 88 · 2015: 65 · 2014: 64 · 2013: 58 · 2012: 51
- FWCI
- 11.55
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Law and Human Behavior (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- effect size, Cohen's d, diagnostic accuracy, prediction accuracy, forensic psychology, follow-up study
- MeSH terms
- Forecasting, Humans, ROC Curve, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
