Health outcomes assessment in vulnerable populations: measurement challenges and recommendations
Abstract
With growing recognition that some population subgroups are particularly vulnerable to receiving suboptimal health care and achieving poor health outcomes, innovative techniques are required for collecting and evaluating health outcomes data. Research is also needed to better understand the causal pathways linking vulnerability with health outcomes. This article focuses on patients with a chronic illness (cancer) who also have low literacy and/or poor English language skills. We summarize the association among literacy, language, ethnicity, and health outcomes; describe innovative technologies to enhance communication; and discuss the advantages of using psychometric measurement models in health outcomes assessment. Results from our ongoing research projects are presented, including the development of an audiovisual computer-based testing platform for self-administration of questionnaires. Such innovative multimedia technologies allow patients with limited or even no reading ability to participate in outcomes assessment and have the potential to be incorporated into a clinical setting with minimal burden on staff and patients. Appropriate methods are also needed to evaluate measurement equivalence across diverse patient groups, that is, the extent to which items in a questionnaire perform similarly across groups. Item response theory measurement models provide a strategy for differentiating between measurement bias and real differences that may exist between groups. Recommendations for clinical practice and research are offered specifically to address medically underserved and vulnerable populations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Among the validation studies included in this review, use of modern psychometric validation techniques such as item response theory (IRT) models was not considered.
“Finally, among the validation studies, use of modern psychometric validation techniques such as item response theory (IRT) models was not considered.57”
From A systematic review of the demoralization syndrome in individuals with progressive disease and cancer: a decade of research (2015) · cites this paper as [57]
Cites
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
Probabilistic Models for Some Intelligence and Attainment Tests
Psychology, M. (1961) · Information and Control · doi:10.1016/s0019-9958(61)80061-2 · Cited by 6,936 · see citing works
The functional assessment of cancer therapy scale: development and validation of the general measure
Cella, D. F., Tulsky, D. S., Gray, G., Sarafian, B., Linn, E., Bonomi, A., Silberman, M., Yellen, S. B., Winicour, P., & Brannon, J. (1993) · Journal of Clinical Oncology · doi:10.1200/jco.1993.11.3.570 · Cited by 6,020 · see citing works
Self-citationFundamentals of item response theory
(1992) · Choice Reviews Online · doi:10.5860/choice.29-4185 · Cited by 3,417 · see citing works
A Rating Formulation for Ordered Response Categories
Andrich, D. (1978) · Psychometrika · doi:10.1007/bf02293814 · Cited by 2,984 · see citing works
ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES1
Samejima, F. (1968) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1968.tb00153.x · Cited by 2,642 · see citing works
Handbook of Modern Item Response Theory
van der Linden, W. J., & Hambleton, R. K. (1997) · Springer eBooks · doi:10.1007/978-1-4757-2691-6 · Cited by 2,323 · see citing works
Measuring Health-Related Quality of Life
Guyatt, G., Feeny, D., & Patrick, D. (1993) · Annals of Internal Medicine · doi:10.7326/0003-4819-118-8-199304150-00009 · Cited by 2,168 · see citing works
Turner, C. F., Ku, L. C., Rogers, S., Lindberg, L., Pleck, J. H., & Sonenstein, F. L. (1998) · Science · doi:10.1126/science.280.5365.867 · Cited by 2,048 · see citing works
(1968) · Addison-Wesley eBooks · Cited by 2,001 · see citing works
Fiscella, K. A., Franks, P. J., Gold, M. R., & Clancy, C. M. (2000) · JAMA · doi:10.1001/jama.283.19.2579 · Cited by 1,186 · see citing works
Relationship of Functional Health Literacy to Patients' Knowledge of Their Chronic Disease
Williams, M. V., Baker, D. W., Parker, R. M., & Nurss, J. R. (1998) · Archives of Internal Medicine · doi:10.1001/archinte.158.2.166 · Cited by 1,159 · see citing works
Inadequate Functional Health Literacy Among Patients at Two Public Hospitals
Williams, M. V. (1995) · JAMA · doi:10.1001/jama.1995.03530210031026 · Cited by 1,089 · see citing works
Rating Scale Analysis: Rasch Measurement
Wolins, L., Wright, B. D., & Masters, G. N. (1983) · Journal of the American Statistical Association · doi:10.2307/2288670 · Cited by 1,033 · see citing works
Item Response Theory and Health Outcomes Measurement in the 21st Century
Hays, R. D., Morales, L. S., & Reise, S. P. (2000) · Medical Care · doi:10.1097/00005650-200009002-00007 · Cited by 868 · see citing works
The relationship of patient reading ability to self-reported health and use of health services.
Baker, D. W., Parker, R. M., Williams, M. V., Clark, W. S., & Nurss, J. R. (1997) · American Journal of Public Health · doi:10.2105/ajph.87.6.1027 · Cited by 720 · see citing works
Ellwood, P. M. (1988) · New England Journal of Medicine · doi:10.1056/nejm198806093182327 · Cited by 698 · see citing works
Shame and health literacy: the unspoken connection
Parikh, N. S., Parker, R. M., Nurss, J. R., Baker, D. W., & Williams, M. V. (1996) · Patient Education and Counseling · doi:10.1016/0738-3991(95)00787-3 · Cited by 684 · see citing works
Lannin, D. R. (1998) · JAMA · doi:10.1001/jama.279.22.1801 · Cited by 669 · see citing works
Adherence to combination antiretroviral therapies in HIV patients of low health literacy
Kalichman, S. C., Ramachandran, B., & Catz, S. L. (1999) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1999.00334.x · Cited by 631 · see citing works
Best Test Design. Rasch Measurement.
Wright, B. D., & Stone, M. H. (1979) · Cited by 612 · see citing works
Cervical cancer screening: who is not screened and why?
Harlan, L. C., Bernstein, A. B., & Kessler, L. G. (1991) · American Journal of Public Health · doi:10.2105/ajph.81.7.885 · Cited by 485 · see citing works
Solis, J. M., Marks, G. N., Garcia, M., & Shelton, D. L. (1990) · American Journal of Public Health · doi:10.2105/ajph.80.suppl.11 · Cited by 478 · see citing works
The health care experience of patients with low literacy
Baker, D. W. (1996) · Archives of Family Medicine · doi:10.1001/archfami.5.6.329 · Cited by 446 · see citing works
Racial and Ethnic Differences in Advanced-Stage Prostate Cancer: the Prostate Cancer Outcomes Study
Hoffman, R. M., Gilliland, F. D., Eley, J. W., Harlan, L. C., Stephenson, R. A., Stanford, J. L., Albertson, P. C., Hamilton, A. S., Hunt, W. C., & Potosky, A. L. (2001) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/93.5.388 · Cited by 440 · see citing works
The Status of Health Assessment 1994
Ware, J. E. (1995) · Annual Review of Public Health · doi:10.1146/annurev.pu.16.050195.001551 · Cited by 405 · see citing works
Limited English Proficiency and Latinos’ Use of Physician Services
Derose, K., & Baker, D. (2000) · Medical Care Research and Review · doi:10.1177/107755870005700105 · Cited by 376 · see citing works
Bennett, C. L., Ferreira, M. R., Davis, T. C., Kaplan, J., Weinberger, M., Kuzel, T. M., Seday, M. A., & Sartor, O. A. (1998) · Journal of Clinical Oncology · doi:10.1200/jco.1998.16.9.3101 · Cited by 376 · see citing works
Institute of Medicine (1999) · National Academies Press eBooks · doi:10.17226/6377 · Cited by 367 · see citing works
The impact of culture on the cognitive structure of illness
Angel, R. J., & Thoits, P. A. (1987) · Culture Medicine and Psychiatry · doi:10.1007/bf00048494 · Cited by 343 · see citing works
Velikova, G., Wright, E. P., Smith, A. B., Cull, A., Gould, A., Forman, D., Perren, T., Stead, M., Brown, J., & Selby, P. J. (1999) · Journal of Clinical Oncology · doi:10.1200/jco.1999.17.3.998 · Cited by 339 · see citing works
Quality of Life in Cancer: Definition, Purpose, and Method of Measurement
Cella, D. F., & Tulsky, D. S. (1993) · Cancer Investigation · doi:10.3109/07357909309024860 · Cited by 332 · see citing works
Self-citationHaley, S. M., McHorney, C. A., & Ware, J. E. (1994) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(94)90215-1 · Cited by 287 · see citing works
Weiss, B. D., Hart, G., McGee, D., & D'Estelle, S. (1992) · PubMed · Cited by 282 · see citing works
Cella, D. F., Hernández, L., Bonomi, A. E., Corona, M., Vaquero, M., Shiomoto, G., & Báez, L. (1998) · Medical Care · doi:10.1097/00005650-199809000-00012 · Cited by 238 · see citing works
Self-citationWeiss, B. D. (1998) · PubMed · Cited by 238 · see citing works
Comprehension of Discharge Instructions by Patients in an Urban Emergency Department
Spandorfer, J. M., Karras, D. J., Hughes, L., et al. (1995) · Annals of Emergency Medicine · doi:10.1016/s0196-0644(95)70358-6 · Cited by 212 · see citing works
Language of interview: relevance for research of southwest Hispanics.
Kirkman‐Liff, B. L., & Mondragón, D. (1991) · American Journal of Public Health · doi:10.2105/ajph.81.11.1399 · Cited by 178 · see citing works
Cella, D. F., & Chang, C. (2000) · Medical Care · doi:10.1097/00005650-200009002-00010 · Cited by 169 · see citing works
Self-citationThompson, I. M., Tangen, C. M., Tolcher, A. W., Crawford, E. D., Eisenberger, M. A., & Moinpour, C. M. (2001) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/93.3.219 · Cited by 166 · see citing works
Raczek, A. E., Ware, J. E., Bjørner, J. B., Gandek, B., Haley, S. M., Aaronson, N. K., Apolone, G., Bech, P., Brazier, J. E., Bullinger, M., et al. (1998) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(98)00112-7 · Cited by 162 · see citing works
Cella, D. F. (1992) · Journal of Palliative Care · doi:10.1177/082585979200800303 · Cited by 156 · see citing works
Self-citationIntroduction to evaluation of interactive health communication applications
Eng, T. R. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(98)00107-x · Cited by 148 · see citing works
(1994) · PubMed · Cited by 132 · see citing works
Buxton, J. A., White, M., & Osoba, D. (1998) · Quality of Life Research · doi:10.1023/a:1008826408328 · Cited by 124 · see citing works
Literacy skills and communication methods of low-income older persons
Weiss, B. D., Reed, R. L., & Kligman, E. W. (1995) · Patient Education and Counseling · doi:10.1016/0738-3991(95)00710-h · Cited by 116 · see citing works
Tarlov, A. R. (1983) · New England Journal of Medicine · doi:10.1056/nejm198305193082028 · Cited by 105 · see citing works
Functional illiteracy among emergency department patients: A preliminary study
Jolly, B. T., Scott, J. L., Feied, C. F., et al. (1993) · Annals of Emergency Medicine · doi:10.1016/s0196-0644(05)81944-4 · Cited by 101 · see citing works
Pouwer, F., Snoek, F. J., van der Ploeg, H. M., Heine, R. J., & Brand, A. N. (1997) · Quality of Life Research · doi:10.1023/a:1008832821181 · Cited by 100 · see citing works
Language, Sociodemographics, and Health Care Use of Hispanic Adults
Schur, C. L., & Albers, L. A. (1996) · Journal of Health Care for the Poor and Underserved · doi:10.1353/hpu.2010.0024 · Cited by 98 · see citing works
Outcome Assessment 70 Years Later: Are We Ready?
Schroeder, S. A. (1987) · New England Journal of Medicine · doi:10.1056/nejm198701153160309 · Cited by 89 · see citing works
Can Patients Use an Automated Questionnaire to Define Their Current Health Status?
Roizen, M. F., Coalson, D. W., Hayward, R. S., Schmittner, J. P., Thisted, R. A., Apfelbaum, J. L., Stocking, C. B., Cassel, C. K., Pompei, P., Ford, D. E., et al. (1992) · Medical Care · doi:10.1097/00005650-199205001-00007 · Cited by 83 · see citing works
Readability of printed sources of diet and health information
Dollahite, J., Thompson, C., & McNew, R. W. (1996) · Patient Education and Counseling · doi:10.1016/0738-3991(95)00829-2 · Cited by 71 · see citing works
Hayes, R. P., & Baker, D. W. (1998) · Medical Care · doi:10.1097/00005650-199802000-00011 · Cited by 70 · see citing works
A comparison of computer and questionnaire methods of history-taking in a genito-urinary clinic
Robinson, R., & West, R. (1992) · Psychology and Health · doi:10.1080/08870449208402024 · Cited by 65 · see citing works
Baker, D. W. (1999) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1999.00342.x · Cited by 61 · see citing works
Are Touchscreen Computer Surveys Acceptable to Medical Oncology Patients?
Newell, S., Girgis, A., Sanson-Fisher, R. W., & Stewart, J. (1997) · Journal of Psychosocial Oncology · doi:10.1300/j077v15n02_03 · Cited by 57 · see citing works
Computer-assisted self-interviewing: a multimedia approach to dietary assessment
Kohlmeier, L., Mendez, M., McDuffie, J. R., et al. (1997) · American Journal of Clinical Nutrition · doi:10.1093/ajcn/65.4.1275s · Cited by 57 · see citing works
Cross-cultural instrument equating: Current research and future directions
Cella, D. F., Lloyd, S., & Wright, B. (1996) · Cited by 53 · see citing works
Self-citationBreast Cancer among Hispanics, American Indians and Non-Hispanic Whites in New Mexico
EDISON, M., Becker, T. M., Wiggins, C. L., et al. (1994) · International Journal of Epidemiology · doi:10.1093/ije/23.2.231 · Cited by 50 · see citing works
Glazer, H. R., Kirk, L. M., & Bosler, F. E. (1996) · Patient Education and Counseling · doi:10.1016/0738-3991(95)00834-9 · Cited by 44 · see citing works
Some latent trait models and their uses in inferring an examinee's ability
Birnbaum, A. (1968) · Medical Entomology and Zoology · Cited by 44 · see citing works
Cunningham, W. E., Hays, R. D., Burton, T. M., & Kington, R. S. (2000) · Journal of Health Care for the Poor and Underserved · doi:10.1353/hpu.2010.0638 · Cited by 39 · see citing works
Test-retest reliability in psychiatric patients of the sf-36 health survey
Burke, J. D., Burke, K. C., Baker, J. H., & Hillis, A. (1996) · International Journal of Methods in Psychiatric Research · Cited by 29 · see citing works
User Preferences for Computer Administration of Quality of Life Instruments
Crawley, J. A., Kleinman, L., & Dominitz, J. A. (2000) · Drug Information Journal · doi:10.1177/009286150003400119 · Cited by 26 · see citing works
Sigle, J. M., & Porzsolt, F. (1996) · Cancer Treatment Reviews · doi:10.1016/s0305-7372(96)90067-5 · Cited by 26 · see citing works
Pain assessment with interactive computer animation
Swanston, M. T., Abraham, C., Macrae, W. A., et al. (1993) · Pain · doi:10.1016/0304-3959(93)90231-d · Cited by 24 · see citing works
A Comparison of the Rasch Separate Calibration and between-Fit Methods of Detecting Item Bias
Smith, R. M. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056003003 · Cited by 23 · see citing works
Data collection strategies for patient-reported information
Cella, D. F., & Lloyd, S. R. (1994) · Quality Management in Health Care · doi:10.1097/00019514-199402040-00006 · Cited by 17 · see citing works
Self-citationComputer-assisted quality of life assessment for clinical trials.
Lenert, L. A., & Hornberger, J. C. (1996) · PubMed · Cited by 14 · see citing works
Reported pap test use by Hispanic women in Connecticut and Long Island.
Polednak, A. P. (1996) · PubMed · Cited by 9 · see citing works
A Hierarchical Model of Functional Performance in Rehabilitation Medicine
Ludlow, L. H., Haley, S. M., & Gans, B. M. (1992) · Evaluation & the Health Professions · doi:10.1177/016327879201500104 · Cited by 7 · see citing works
Successful strategies for recruiting African Americans to prevention trials
Ryan, D. H., Kennedy, B. M., Smith, L. L., Tucker, E. W., Melancon, L. E., Phillips, B. H., Lassale, C. C., & Bray, G. A. (1998) · Cancer · doi:10.1002/(sici)1097-0142(19981015)83:8+ · Cited by 2 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Robinson, S., Kissane, D. W., Brooker, J., & Burney, S. (2015) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2014.07.008 · Cited by 401 · see citing works
Cancer survivorship and aging: Moving the science forward
Bellizzi, K. M., Mustian, K. M., Palesh, O. G., & Diefenbach, M. (2008) · Cancer · doi:10.1002/cncr.23942 · Cited by 67 · see citing works
- Authors
- Hahn, E. A., & Cella, D.
- Affiliations
- Northwestern University (US)
- Venue
- Archives of Physical Medicine and Rehabilitation
- Publisher
- Elsevier BV
- Published
- 2003-04
- Volume
- 84
- Issue
- 4 Suppl 2
- Pages
- S35-S42
- ISSN
- 0003-9993 (ISSN-L)
- OpenAlex
- W2059667061
- Cited by
- 133
- References
- 74
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 1 · 2022: 4 · 2021: 1 · 2020: 6 · 2019: 7 · 2018: 6 · 2017: 3 · 2016: 4 · 2015: 6 · 2014: 5 · 2013: 6 · 2012: 8
- FWCI
- 22.80
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Archives of Physical Medicine and Rehabilitation (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Agency for Healthcare Research and Quality (R01 HS10333)
- National Cancer Institute (R01 CA61679, R01CA061679)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Health Sciences › Health Professions › General Health Professions › Interpreting and Communication in Healthcare
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- health outcome measurement, vulnerable populations, low health literacy, limited English proficiency, cancer patients, ethnicity, measurement invariance, item response theory, self-administered questionnaire, medically underserved populations, health communication technology
- MeSH terms
- Activities of Daily Living, Chronic Disease, Communication Barriers, Educational Status, Hispanic or Latino, Humans, Neoplasms, Psychometrics, Quality of Life, Surveys and Questionnaires, United States, Reproducibility of Results, Bias, Outcome Assessment, Health Care, Sickness Impact Profile, Cultural Diversity, Multimedia, Vulnerable Populations
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
