Using Focus Groups and Rasch Item Response Theory to Improve Instrument Development
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Item response theory (IRT) is not widely used in counseling psychology research, despite its considerable advantages for instrument development. Focus groups help create a broad and representative item pool that is more likely to tap the full range of the latent dimension, thereby capitalizing on IRT strengths. We provide suggestions for using these tools, with an empirical example, the Everyday Multicultural Competencies/Revised Scale of Ethnocultural Empathy (EMC/RSEE). Rasch IRT methods were used to (a) analyze response format performance and collapse poorly performing categories, (b) evaluate differential item functioning sex bias, and (c) select items to maximize subscale sensitivity and bandwidth. Traditional classical test theory (CTT) subscales composed of items with the highest factor loadings were compared with subscales based on IRT criteria. Compared with CTT subscales, IRT subscales used in the EMC/RSEE demonstrated generally higher correlations with other variables of interest, and superior sensitivity to change over time and to group differences.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Infit and outfit values under .60 or over 1.40 indicate that the patterns of responses were unexpected (i.e., response patterns appear to be influenced by factors other than the construct underlying the measure; Mallinckrodt et al., 2016).”
From The development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF). (2019) · cites this paper as Mallinckrodt et al., 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238 · cited 1×
“In 1976, Bordin’s seminal paper prompted further interest, yet in the 10-year period from 1977-1986, only 31 peer-reviewed articles appeared on the therapeutic alliance. A perceptible change began when the Working Alliance Inventory (WAI) was introduced in a book chapter (Horvath & Greenberg, 1986), and later in a jou…”
BackgroundScale Development Research: A Content Analysis and Recommendations for Best Practices
Worthington, R. L., & Whittaker, T. A. (2006) · The Counseling Psychologist · doi:10.1177/0011000006288127 · Cited by 4,087 · cited 15×
“Thus, approximately 6.25 subjects for each item presented, plus 375 additional holdout subjects for CFA, represent a minimum sample size.”
BackgroundMethodInfluentialIdentifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865 · cited 2×
“Findings suggest that failure to correct for this problem can obscure otherwise significant results in both regression and experimental studies (Maniaci & Rogge, 2014), as well as factor analyses (Meade & Craig, 2012).”
BackgroundResultDevelopment and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520 · cited 9×
“A perceptible change began when the Working Alliance Inventory (WAI) was introduced in a book chapter (Horvath & Greenberg, 1986), and later in a journal article (Horvath & Greenberg, 1989), thereby joining several other measures of the psychotherapy relationship.”
BackgroundMethodInfluentialPreston, C. C., & Colman, A. M. (2000) · Acta Psychologica · doi:10.1016/s0001-6918(99)00050-5 · Cited by 2,106
Best practices for missing data management in counseling psychology
Schlomer, G. L., Bauman, S., & Card, N. A. (2010) · Journal of Counseling Psychology · doi:10.1037/a0018082 · Cited by 1,835 · cited 1×
“Following the recommendations of Worthington and Whittaker (2006), an item was assigned to Factor 2 if its loading was >.”
MethodHarrington, D. (2008) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195339888.001.0001 · Cited by 1,105
Caring about carelessness: Participant inattention and its effects on research
Maniaci, M. R., & Rogge, R. D. (2014) · Journal of Research in Personality · doi:10.1016/j.jrp.2013.09.008 · Cited by 852
The Scale of Ethnocultural Empathy: Development, validation, and reliability.
Wang, Y.-W., Davidson, M. M., Yakushko, O. F., Savoy, H. B., Tan, J. A., & Bleier, J. K. (2003) · Journal of Counseling Psychology · doi:10.1037/0022-0167.50.2.221 · Cited by 489 · cited 2×
“Worthington and Whittaker (2006) recommended using each item as an observed indicator of its latent factor.”
Background
Research works printed but not linked: 58
- Andrich D., 2002, Journal of Applied Measurement, 3, 325
- Bandalos D. J., 2001, New developments and techniques in structural equation modeling, 269
- 10.3758/s13428-011-0130-8 · DOI
- Bond T. G., 2007, Applying the Rasch model: Fundamental measurement in the human sciences
- 10.1037/a0023350 · DOI · Paper page
- Carey M. A., 2012, Focus group research
- 10.1177/1558689810382916 · DOI · Paper page
- 10.1080/00049537408255223 · DOI · Paper page
- 10.1037/h0040957 · DOI · Paper page
- 10.1177/1548051808318000 · DOI
- de Ayala R. J., 2009, The theory and practice of item response theory
- 10.1037/a0020940 · DOI
- DeVellis R. F., 2003, Scale development: Theory and applications
- 10.1080/10503300902894422 · DOI
- Embretson S. E., 2000, Item response theory for psychologists
- 10.1037/1082-989X.4.3.272 · DOI · Paper page
- Flick W., 1998, An introduction to qualitative research
- 10.1037/1040-3590.7.3.286 · DOI · Paper page
- 10.1177/014662168100500203 · doi:10.1177/014662168100500203
- 10.1037/0022-0167.45.1.30 · doi:10.1037/0022-0167.45.1.30 · Paper page
- 10.1080/07481756.2000.12069007 · DOI · Paper page
- 10.1037/0022-006X.56.4.483 · DOI
- Gorsuch R. L., 1983, Factor analysis
- Greenson R. R., 1967, Technique and practice of psychoanalysis
- Hambleton R. K., 2010, Item response theory: Principles and applications
- 10.1037/0022-0167.52.2.224 · DOI · Paper page
- 10.1177/0011000099273004 · DOI · Paper page
- Horvath A. O., 1986, The psychotherapeutic process: A research handbook, 529
- 10.1037/0022-0167.36.2.223 · doi:10.1037/0022-0167.36.2.223 · Paper page
- 10.1037/13754-003 · doi:10.1037/13754-003
- 10.1016/j.ijintrel.2005.01.012 · DOI
- 10.1007/s11136-012-0122-x · DOI · Paper page
- Kline R. B., 2011, Principles and practice of structural equation modeling
- Krueger R. A., 2009, Focus groups: A practical guide for applied research
- 10.1007/s11336-010-9199-8 · DOI
- Linacre J. M., 1998, Journal of Outcome Measurement, 2, 266
- Linacre J. M., 1999, Journal of Outcome Measurement, 3, 103
- Linacre J. M., 2004, Rasch Measurement Transactions, 18, 959
- 10.1037/0022-0167.49.3.355 · doi:10.1037/0022-0167.49.3.355 · Paper page
- 10.1027/1614-2241.4.2.73 · DOI · Paper page
- 10.1037/a0026549 · DOI
- 10.1037/a0035214 · DOI
- Mallinckrodt B., Psychotherapy Research
- 10.1037/a0029855 · DOI · Paper page
- 10.1177/0265532211430367 · DOI
- 10.4135/9781412993913 · DOI · Paper page
- 10.1080/01411920903018182 · DOI
- 10.2307/2943979 · doi:10.2307/2943979
- 10.1016/B978-0-12-590241-0.50006-X · DOI · Paper page
- 10.1037/0033-3204.38.4.443 · DOI · Paper page
- 10.1037/1082-989X.8.2.164 · DOI · Paper page
- Revelle W. (2013). The “new psychometrics”—Item response theory. In Revelle W. (Ed.), The personality project (pp. 235-238). Retrieved from http://www.personality-project.org/r/book/Chapter8.pdf
- 10.1037/1082-989X.1.1.81 · DOI
- 10.1007/BF02293705 · DOI · Paper page
- 10.1177/014662168601000104 · doi:10.1177/014662168601000104
- 10.1037/0022-0167.45.1.46 · DOI
- 10.1177/0013164414528209 · DOI · Paper page
- 10.1111/j.1745-3992.1997.tb00606.x · DOI · Paper page
Cited by
Coercive Control in the Courtroom: the Legal Abuse Scale (LAS)
Gutowski, E. R., & Goodman, L. A. (2023) · Journal of Family Violence · doi:10.1007/s10896-022-00408-3 · Cited by 57
“1 3 unidimensionality is an important assumption of RA and a multi-scale measure does not necessarily indicate that a measure is multidimensional (Mallinckrodt et al., 2016), we first examined a principal components analysis (PCA) of the residuals with all 14 items in a single model.”
BackgroundMethodResultInfluentialThe development and evaluation of a brief form of the Normative Male Alexithymia Scale (NMAS-BF).
Levant, R. F., & Parent, M. C. (2019) · Journal of Counseling Psychology · doi:10.1037/cou0000312 · Cited by 12
Metadata
- Authors
- Mallinckrodt, B., Miles, J. R., & Recabarren, D. A.
- Affiliations
- University of Tennessee, Knoxville, TN, USA
- Venue
- The Counseling Psychologist
- Publisher
- SAGE Publications
- Published
- 2015-08-05
- Volume
- 44
- Issue
- 2
- Pages
- 146-194
- ISSN
- 0011-0000
- 1552-3861
- Cited by
- 31
- References
- 10
- Language
- en
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using Focus Groups and Rasch Item Response Theory to Improve Instrument Development. https://app.cochat.ai/papers/10.1177/0011000015596437
- The paper
- Mallinckrodt, B., Miles, J. R., & Recabarren, D. A. (2016). Using Focus Groups and Rasch Item Response Theory to Improve Instrument Development. The Counseling Psychologist. https://doi.org/10.1177/0011000015596437
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.