The DOI matches its Crossref record; 48 of the 48 references it cites were found in a registry or an index.
A Measurement Feedback System (MFS) Is Necessary to Improve Mental Health Outcomes
Open accessAbstract
This article takes the position that mental health (MH) services for youths are unlikely to improve without a system of measurement that is administered frequently, is concurrent with treatment, and provides feedback. The system, which I characterize as a measurement feedback system (MFS), should include clinical processes (mediators), contexts (moderators), outcomes, and feedback to clinicians and supervisors. In spite of the routine call to collect and use outcome data in realworld treatment, progress has been painstakingly slow. 1Y3 For example, Garland and colleagues 4 found that even when outcome assessments were required, more than 90% of the clinicians surveyed used their own judgment and paid little heed to the data. A more recent national survey of MH service organizations serving children and families indicated that almost 75% reported collecting some standardized outcome data. 5 However, just collecting data on an annual basis will not result in improvement. MEASUREMENT IS NOT ENOUGH Feedback from clients and families naturally occurs in treatment, but it is highly filtered, biased, and subject to distortions caused by the use of cognitive heuristics and schemas. 6 This informal and flawed feedback needs to be supplemented by an MFS that uses valid, reliable, and standardized measures. This system is central to quality improvement, professional development, as well as enhancing accountability. Feedback has been successfully applied outside MH for several decades. 7,8 However the application of a fully implemented MFS is in its infancy in MH. An MFS has been shown to improve outcomes in adult MH, especially for those clients who were either not
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
Ægisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., et al. (2006) · The Counseling Psychologist · doi:10.1177/0011000005285875 · Cited by 737 · cited 1×
“Feedback from clients and families naturally occurs in treatment, but it is highly filtered, biased, and subject to distortions caused by the use of cognitive heuristics and schemas.(6) This informal and flawed feedback needs to be supplemented by an MFS that uses valid, reliable,”
Creative Teaching: Collaborative Discussion as Disciplined Improvisation
Sawyer, R. K. (2004) · Educational Researcher · doi:10.3102/0013189x033002012 · Cited by 674
Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., Green, P., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0148-5 · Cited by 608
Is it time for clinicians to routinely track patient outcome? A meta-analysis
Lambert, M. J., Whipple, J. L., Hawkins, E. J., Vermeersch, D. A., Nielsen, S. L., & Smart, D. W. (2003) · Clinical Psychology: Science and Practice · doi:10.1093/clipsy.bpg025 · Cited by 608
Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress.
Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996) · American Psychologist · doi:10.1037/0003-066x.51.10.1059 · Cited by 573
Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006) · American Psychologist · doi:10.1037/0003-066x.61.7.671 · Cited by 560
Hunsley, J., & Mash, E. J. (2007) · Annual Review of Clinical Psychology · doi:10.1146/annurev.clinpsy.3.022806.091419 · Cited by 528
Wampold, B. E., & Brown, G. S. (. (2005) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.73.5.914 · Cited by 476
The effectiveness of continuing medical education: eight research reports
Stein, L. S. (1981) · Academic Medicine · doi:10.1097/00001888-198102000-00004 · Cited by 472
Effectiveness of continuing medical education.
Marinopoulos, S. S., Dorman, T., Ratanawongsa, N., Wilson, L. M., Ashar, B. H., Magaziner, J. L., Miller, R. G., Thomas, P. A., Prokopowicz, G. P., Qayyum, R., et al. (2007) · PubMed · Cited by 453
Meta‐Analysis of Therapist Effects in Psychotherapy Outcome Studies
Crits-Christoph, P., Baranackie, K., Kurcias, J., Beck, A., Carroll, K., Perry, K., Luborsky, L., McLellan, A., Woody, G., Thompson, L., et al. (1991) · Psychotherapy Research · doi:10.1080/10503309112331335511 · Cited by 433
Evidence-Based Assessment of Child and Adolescent Disorders: Issues and Challenges
Mash, E. J., & Hunsley, J. (2005) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3403_1 · Cited by 379
Kim, D.-M., Wampold, B. E., & Bolt, D. M. (2006) · Psychotherapy Research · doi:10.1080/10503300500264911 · Cited by 330
Measurement and psychotherapy: Evidence-based practice and practice-based evidence
Margison, F. R., Barkham, M., Evans, C., McGrath, G., Clark, J. M., Audin, K., & Connell, J. (2000) · British Journal of Psychiatry · doi:10.1192/bjp.177.2.123 · Cited by 326
The Use of Outcome Measures by Psychologists in Clinical Practice.
Hatfield, D. R., & Ogles, B. M. (2004) · Professional Psychology: Research and Practice · doi:10.1037/0735-7028.35.5.485 · Cited by 305 · cited 1×
“large time burden, insufficient resources, low clinical usefulness, confidentiality, potential misuse, low scientific merit, and value differences.(2,12,13) There are five not so discernible barriers to the adoption of an MFS that are particularly important.”
Clinicians and outcome measurement: What's the use?
Garland, A. F., Kruse, M., & Aarons, G. A. (2003) · The Journal of Behavioral Health Services & Research · doi:10.1007/bf02287427 · Cited by 298
Feedback to clinicians: Theory, research, and practice
Sapyta, J., Riemer, M., & Bickman, L. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20107 · Cited by 253
Self-citationTherapist effects in outpatient psychotherapy: A three-level growth curve approach.
Lutz, W., Leon, S. C., Martinovich, Z., Lyons, J. S., & Stiles, W. B. (2007) · Journal of Counseling Psychology · doi:10.1037/0022-0167.54.1.32 · Cited by 233
The therapeutic effects of providing patient progress information to therapists and patients
Hawkins, E. J., Lambert, M. J., Vermeersch, D. A., Slade, K. L., & Tuttle, K. C. (2004) · Psychotherapy Research · doi:10.1093/ptr/kph027 · Cited by 230 · cited 1×
“especially for those clients who were either not improving or deteriorating while in therapy.(9) It has rarely been applied in children_s MH.”
A Theoretical Model of Common Process Factors in Youth and Family Therapy
Karver, M. S., Handelsman, J. B., Fields, S., & Bickman, L. (2005) · Mental Health Services Research · doi:10.1007/s11020-005-1964-4 · Cited by 194
Self-citationBarber, J. P., Sharpless, B. A., Klostermann, S., & McCarthy, K. S. (2007) · Professional Psychology: Research and Practice · doi:10.1037/0735-7028.38.5.493 · Cited by 172
The local clinical scientist: A bridge between science and practice.
Stricker, G., & Trierweiler, S. J. (1995) · American Psychologist · doi:10.1037//0003-066x.50.12.995 · Cited by 161
Schoenwald, S. K., Chapman, J. E., Kelleher, K., Hoagwood, K. E., Landsverk, J., Stevens, J., Glisson, C., Rolls-Reutz, J., & The Research Network on Youth Mental Health (2008) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0147-6 · Cited by 149 · cited 1×
“serving children and families indicated that almost 75% reported collecting some standardized outcome data.(5) However, just collecting data on an annual basis will not result in improvement.”
Psychological practice and managed care: Results of the CAPP practitioner survey.
Phelps, R., Eisman, E. J., & Kohout, J. L. (1998) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.29.1.31 · Cited by 145
Chorpita, B. F., Bernstein, A., & Daleiden, E. L. (2007) · Administration and Policy in Mental Health and Mental Health Services Research · doi:10.1007/s10488-007-0151-x · Cited by 126
Aggressive behaviour in the high‐secure forensic setting: the perceptions of patients
Meehan, T., McINTOSH, W., & Bergen, H. A. (2006) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/j.1365-2850.2006.00906.x · Cited by 125
Theories Related to Changing Clinician Practice
Riemer, M., Rosof-Williams, J., & Bickman, L. (2005) · Child and Adolescent Psychiatric Clinics of North America · doi:10.1016/j.chc.2004.05.002 · Cited by 103
Self-citationMartin, A., & LESLIE, D. L. (2003) · American Journal of Psychiatry · doi:10.1176/appi.ajp.160.4.757 · Cited by 102
Learning to Teach: The Acquisition and Maintenance of Teaching Skills
Rose, D., & Church, R. (1998) · Journal of Behavioral Education · doi:10.1023/a:1022860606825 · Cited by 87
Routine Outcome Measurement: A Survey of UK Child and Adolescent Mental Health Services
Johnston, C., & Gowers, S. (2005) · Child and Adolescent Mental Health · doi:10.1111/j.1475-3588.2005.00357.x · Cited by 74
Routine measurement of outcomes in Australia's public sector mental health services
Pirkis, J., Burgess, P. M., Dickson, R., Clarke, A., Jones-Ellis, D., & Coombes, T. (2005) · Australia and New Zealand Health Policy · doi:10.1071/hp050208 · Cited by 73
Introduction of routine outcome measures: staff reactions and issues for consideration
Meehan, T., MCCOMBES, S., Hatzipetrou, L., & CATCHPOOLE, R. (2006) · Journal of Psychiatric and Mental Health Nursing · doi:10.1111/j.1365-2850.2006.00985.x · Cited by 61
Factors Associated with Competence in Cognitive Therapists
Brosan, L., Reynolds, S. A., & Moore, R. G. (2006) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465806003304 · Cited by 60
Should I Pack My Umbrella? Clinical Versus Statistical Prediction of Mental Health Decisions
Ægisdóttir, S., Spengler, P. M., & White, M. J. (2006) · The Counseling Psychologist · doi:10.1177/0011000006286696 · Cited by 51
A Common Factors Approach to Improving Mental Health Services
Bickman, L. (2005) · Mental Health Services Research · doi:10.1007/s11020-005-1961-7 · Cited by 38
Self-citationThe local clinical scientist: A bridge between science and practice.
Stricker, G., & Trierweiler, S. J. (2006) · Training and Education in Professional Psychology · doi:10.1037/1931-3918.s.1.37 · Cited by 38
Turning Child and Adolescent Mental-Health Services into Learning Organizations
Birleson, P. (1999) · Clinical Child Psychology and Psychiatry · doi:10.1177/1359104599004002011 · Cited by 31
Consumer measurement systems for child and adolescent mental health
Bickman, L., Nurcombe, B., Townsend, C., Belle, M., Schut, L. J. A., & Karver, M. S. (1999) · Cited by 28
Self-citationInteractions of gender and race–a problem for teachers? A review of the emerging literature
Haw, K. F. (1991) · Educational Research · doi:10.1080/0013188910330102 · Cited by 25
Michael, K. D., Huelsman, T. J., & Crowley, S. L. (2005) · Journal of Child and Family Studies · doi:10.1007/s10826-005-5050-8 · Cited by 23
Evaluating Program Environments
Conrad, K. J. (1989) · Medical Entomology and Zoology · Cited by 22
Rejoinder to commentaries by Stephen Soldz and Paul Crits-Christoph on therapist effects
Elkin, I., Falconnier, L., Martinovich, Z., & Mahoney, C. (2006) · Psychotherapy Research · doi:10.1080/10503300500282186 · Cited by 15
Program personnel: The missing ingredient in describing the program environment
Peterson, K. A., & Bickman, L. (1988) · New Directions for Program Evaluation · doi:10.1002/ev.1498 · Cited by 12
Self-citationGlisson, C. A., Landsverk, J. A., Schoenwald, S. K., Kelleher, K. J., Hoagwood, K., Mayberg, S. W., & Green, P. W. (2008) · Administration and Policy in Mental Health and Mental Health Services Research · Cited by 9
Pay for performance in primary and specialty behavioral health care: Two "concept" proposals.
Bachman, J. W. (2006) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.37.4.384 · Cited by 4
Reichhardt, T. (2005) · Nature · doi:10.1038/4351024a · Cited by 4
Cited by
Rebooting psychotherapy research and practice to reduce the burden of mental illness
Kazdin, A. E., & Blase, S. L. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393527 · Cited by 1,254
Implementing Measurement-Based Care in Behavioral Health: A Review
Lewis, C. C., Boyd, M., Puspitasari, A., Navarro, E., Howard, J., Kassab, H., Hoffman, M., Scott, K., Lyon, A., Douglas, S., et al. (2019) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.3329 · Cited by 605
Using Measurement-Based Care to Enhance Any Treatment
Scott, K., & Lewis, C. C. (2015) · Cognitive and Behavioral Practice · doi:10.1016/j.cbpra.2014.01.010 · Cited by 461
“…into treatment has been referred to in the published literature as progress monitoring, outcome monitoring, measurement-based care, or the use of feedback systems (Bickman, 2008; Lambert, Harmon, Slade, Whipple, & Hawkins, 2005; Saggese, 2005; Trivedi et al., 2006; Valenstein et al., 2009).”
BackgroundWandersman, A., Chien, V. H., & Katz, J. (2012) · American Journal of Community Psychology · doi:10.1007/s10464-012-9509-7 · Cited by 422
“Monitoring methods include use of decision support data systems (Fixsen et al. 2009) or measurement feedback systems (Bickman 2008) Conduct an outcome evaluation (GTO step 8) An outcome evaluation is conducted to determine the extent to which QA/QI desired outcomes have been attained.”
de Jong, K., Conijn, J. M., Gallagher, R. A. V., Reshetnikova, A. S., Heij, M., & Lutz, M. C. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102002 · Cited by 417
Metadata
- Authors
- Hamilton, J. D., & Bickman, L.
- Venue
- Journal of the American Academy of Child & Adolescent Psychiatry
- Publisher
- Elsevier BV
- Published
- 2008-10
- Volume
- 47
- Issue
- 10
- Pages
- 1114-1119
- ISSN
- 0890-8567 (ISSN-L)
- OpenAlex
- W2091313449
- Cited by
- 270
- References
- 48
- FWCI
- 15.80
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.