CoChat
Sign Up
Cited by 16Delphi Technique in ResearchHealthcare Quality and ManagementHealth Education and Validation

Establishing nursing‐sensitive quality indicators for pediatric oncology: An international mixed methods Delphi study

Open access
Sullivan, Courtney E., Day, Sara W., Ivankova, Nataliya, Markaki, Adelais, Patrician, Patricia A., Landier, Wendy·2023·Journal of Nursing Scholarship
DOI 10.1111/jnu.12798PMC PMC9946155
Open this paper in CoChat — free account

Abstract

Introduction: Nursing‐sensitive indicators (NSIs) measure factors influencing nursing care quality and patient outcomes. Established NSIs reflect general and select specialty nursing practices. However, a core set of NSIs for international pediatric oncology nursing practice does not currently exist. Without valid and reliable quality indicators, the impact of nursing care on children and adolescents with cancer cannot be effectively measured and improved. The purpose of this study was to develop a preliminary core set of NSIs for international pediatric oncology nursing that would be important, actionable, and feasible to measure across varied resource settings and countries. Design/Methods: A multiphase sequential mixed methods research design, intersected with a classical Delphi method, was utilized. Through purposive snowball sampling, 122 expert pediatric oncology nurses from 43 countries participated. Round One: Panelists identified five potential NSIs and constructs. Open‐ended responses were coded and categorized through descriptive content analysis and integrated into the next round. Round Two: Panelists selected their top 10 NSIs and constructs and ranked them by importance to patient care quality. Mean importance scores were calculated through reverse scoring; the top 10 NSIs and constructs were integrated into the next round. Round Three: Panelists ranked the top 10 NSIs and constructs by order of importance for this particular population, then rated each NSI/Construct for actionability and feasibility of measurement by Likert‐scale. Rounds Two and Three were analyzed using descriptive statistics. Mixed methods meta‐inferences were derived from the integration of Rounds One and Three findings. Results: Eighty‐five (70%) panelists from 38 countries completed all Delphi survey rounds. The preliminary core set of NSIs and constructs identified by the expert panel, and ranked in order of importance, were as follows: safe chemotherapy administration and handling, infection prevention/control, pediatric oncology nursing orientation program, early warning score system/recognition of patient deterioration, chemotherapy/biotherapy education/course, pain assessment/management, symptom assessment/management, patient and family education, palliative/end of life care, and continuing nursing education/competency. All NSIs and constructs were rated as actionable; all but palliative/end of life care were rated as feasible to measure. Each of the 10 NSIs and constructs were nominated in Round One by at least one expert panelist from low‐ and middle‐income and high‐income countries, and at least one panelist from the Americas. Conclusion: Preliminary core NSIs and constructs provide insight into common attributes of international pediatric oncology nursing practice that are important, actionable, and feasible for quality measurement. Clinical Relevance: NSIs have the potential to drive quality improvement, guide comparison with other institutions, promote knowledge‐sharing, and advance pediatric oncology nursing outcomes around the world. These NSIs and constructs may also be relevant to other pediatric and adult oncology settings.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/jnu.12798

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 38

  1. Agulnik A, Cárdenas A, Carrillo AK, Bulsara P, Garza M, Alfonso Carreras Y, Alvarado M, Calderón P, Díaz R, de León C, Del Real C, Huitz T, Martínez A, Miralda S, Montalvo E, Negrín O, Osuna A, Perez Fermin CK, Pineda E, ... Rodriguez-Galindo. (2021). Clinical and organizational risk factors for mortality during deterioration events among pediatric oncology patients in Latin America: A multicenter prospective cohort. Cancer, 127(10), 1668-1678. 10.1002/cncr.33411 [PubMed: 33524166] · DOI
  2. Agulnik A, Forbes PW, Stenquist N, Rodriguez-Galindo C, & Kleinman M (2016). Validation of a pediatric early warning score in hospitalized pediatric oncology and hematopoietic stem cell transplant patients. Pediatric Critical Care Medicine, 17(4), e146-e153. 10.1097/ PCC.0000000000000662 [PubMed: 26914628]
  3. Agulnik A, Méndez Aceituno A, Mora Robles LN, Forbes PW, Soberanis Vasquez DJ, Mack R, Antillon -Klussmann F, Kleinman M, & Rodriguez -Galindo C (2017). Validation of a pediatric early warning system for hospitalized pediatric oncology patients in a resource -limited setting. Cancer, 123(24), 4903-4913. 10.1002/cncr.30951 [PubMed: 28881451] · DOI
  4. Aiken LH, Cimiotti JP, Sloane DM, Smith HL, Flynn L, & Neff DF (2011). Effects of nurse staffing and nurse education on patient deaths in hospitals with different nurse work environments. Medical Care, 49(12), 1047-1053. 10.1097/MLR.0b013e3182330b6e [PubMed: 21945978] · DOI
  5. Arias AV, Garza M, Murthy S, Cardenas A, Diaz F, Montalvo E, Nielsen KR, Kortz T, Sharara -Chami R, Friedrich P, McArthur J, & Agulnik A (2020). Quality and capacity indicators for hospitalized pediatric oncology patients with critical illness: A modified Delphi consensus. Cancer Medicine, 9(19), 6984-6995. 10.1002/cam4.3351 [PubMed: 32777172] · DOI
  6. Armes J, Wagland R, Finnegan-John J, Richardson A, Corner J, & Griffiths P (2014). Development and testing of the patient-reported chemotherapy indicators of symptoms and experience: Patientreported outcome and process indicators sensitive to the quality of nursing care in ambulatory chemotherapy settings. Cancer Nursing, 37(3), E52-E60. 10.1097/NCC.0b013e3182980420 [PubMed: 24141376] · DOI
  7. Baxter AL, Watcha MF, Baxter WV, Leong T, & Wyatt MM (2011). Development and validation of a pictorial nausea rating scale for children. Pediatrics, 127(6), e1542-e1549. 10.1542/peds.2010-1410 [PubMed: 21624874] · DOI
  8. Bradley NM, Robinson PD, Greenberg ML, Barr RD, Klassen AF, Chan YL, & Greenberg CM (2013a). Measuring the quality of a childhood cancer care delivery system: Assessing stakeholder agreement. Value in Health, 16(4), 639-646. 10.1016/j.jval.2013.02.016 [PubMed: 23796299] · DOI
  9. Bradley NM, Robinson PD, Greenberg ML, Barr RD, Klassen AF, Chan YL, & Greenberg CM (2013b). Measuring the quality of a childhood cancer care delivery system: Quality indicator development. Value in Health, 16(4), 647-654. 10.1016/j.jval.2013.03.1627 [PubMed: 23796300] · DOI
  10. Cheung RB, Aiken LH, Clarke SP, & Sloane DM (2008). Nursing care and patient outcomes: International evidence. Enfermería Clínica, 18(1), 35-40. 10.1016/s1130-8621(08)70691-0 [PubMed: 18218265] · DOI
  11. Connor JA, Larson C, Baird J, & Hickey PA (2016). Use of a pediatric cardiovascular nursing consortium for development and evaluation of quality measures: The C4-MNP Experience. Journal of Pediatric Nursing, 31(5), 471-477. 10.1016/j.pedn.2016.04.010 [PubMed: 27368931] · DOI
  12. Conway JE (1998). Evolution of the species 'expert nurse'. An examination of the practical knowledge held by expert nurses. Journal of Clinical Nursing, 7(1), 75-82. 10.1046/ j.1365-2702.1998.00127.x [PubMed: 9510711]
  13. Donabedian A (1988). Quality Assessment and Assurance: Unity of Purpose, Diversity of Means. Inquiry, 25(1), 173-192. http://www.jstor.org/stable/29771941 [PubMed: 2966122]
  14. Downing J, Namisango E, & Harding R (2018). Outcome measurement in paediatric palliative care: Lessons from the past and future developments. Annals of Palliative Medicine, 7(Suppl 3), S151S163. https://apm.amegroups.com/article/view/19946. https://doi.org/10.21037/apm.2018.04.02 [PubMed: 30180720] · DOI
  15. Fessele K, Yendro S, & Mallory G (2014). Setting the bar: Developing quality measures and education programs to define evidence-based, patient-centered, high-quality care. Clinical Journal of Oncology Nursing, 18, Suppl., 7-11. 10.1188/14.CJON.S2.7-11 · DOI
  16. Given BA, & Sherwood PR (2005). Nursing sensitive patient outcomes-A white paper. Oncology Nursing Forum, 32(4), 773-784. 10.1188/05.ONF.773-784 [PubMed: 15990907] · DOI
  17. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, & Conde JG (2009). Research Electronic Data Capture (REDCap)-A metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics, 42(2), 377-381. 10.1016/j.jbi.2008.08.010 [PubMed: 18929686] · DOI · Paper page
  18. Hockenberry M, & Landier W (2019). Symptom assessment during childhood cancer treatment. Journal of Pediatric Oncology Nursing, 36(4), 242-243. 10.1177/1043454219852611 [PubMed: 31307319] · DOI
  19. Jackson-Rose J, Del Monte J, Groman A, Dial LS, Atwell L, Graham J, O'Neil Semler R, O'Sullivan M, Truini-Pittman L, Cunningham TA, Roman-Fischetti L, Costantinou E, Rimkus C, Banavage AJ, Dietz B, Colussi CJ, Catania K, Wasko M, Schreffler KA, West C, . . . Rice RD. (2017). Chemotherapy extravasation: Establishing a national benchmark for incidence among cancer centers. Clinical Journal of Oncology Nursing, 21(4), 438-445. 10.1188/17.CJON.438-445 [PubMed: 28738039] · DOI
  20. Johnston EE, Molina J, Martinez I, Dionne -Odom JN, Currie ER, Crowl T, Butterworth L, Chamberlain LJ, Bhatia S, & Rosenberg AR (2020). Bereaved parents' views on end -of -life care for children with cancer: Quality marker implications. Cancer, 126(14), 3352-3359. 10.1002/ cncr.32935 [PubMed: 32383817]
  21. Keeney S, Hasson F, & McKenna H (2011). The Delphi technique in nursing and health research Wiley-Blackwell.
  22. Lam CG, Howard SC, Bouffet E, & Pritchard-Jones K (2019). Science and health for all children with cancer. Science, 363(6432), 1182-1186. 10.1126/science.aaw4892 [PubMed: 30872518] · DOI
  23. Landier W, Leonard M, & Ruccione KS (2013). Children's Oncology Group's 2013 blueprint for research: Nursing discipline. Pediatric Blood and Cancer, 60(6), 1031-1036. 10.1002/pbc.24415 [PubMed: 23255369] · DOI
  24. Looper K, Winchester K, Robinson D, Price A, Langley R, Martin G, Jones S, Holloway J, Rosenberg S, & Flake S (2016). Best practices for chemotherapy administration in pediatric oncology: Quality and safety process improvements (2015). Journal of Pediatric Oncology Nursing, 33(3), 165-172. 10.1177/1043454215610490 [PubMed: 26668214] · DOI
  25. Montalvo I (2007). The National Database of Nursing Quality Indicators(TM)(NDNQI ® ). Online Journal of Issues in Nursing, 12(3), 2.
  26. Morrissey L, Lurvey M, Sullivan C, Challinor J, Forbes PW, Abramovitz L, Afungchwi GM, Hollis R, & Day S (2019). Disparities in the delivery of pediatric oncology nursing care by country income classification: International survey results. Pediatric Blood and Cancer, 66(6), e27663. 10.1002/pbc.27663 [PubMed: 30786168] · DOI
  27. Mukkada S, Smith CK, Aguilar D, Sykes A, Tang L, Dolendo M, & Caniza MA (2018). Evaluation of a fever -management algorithm in a pediatric cancer center in a low -resource setting. Pediatric Blood and Cancer, 65(2), e26790. 10.1002/pbc.26790 · DOI
  28. Nagoya Y, Miyashita M, Irie W, Yotani N, & Shiwaku H (2020). Development of a proxy quality-oflife rating scale for the end-of-life care of pediatric cancer patients evaluated from a nurse's perspective. Journal of Palliative Medicine, 23(1), 82-89. 10.1089/jpm.2018.0598 [PubMed: 31355699] · DOI
  29. Pfaff K, & Markaki A (2017). Compassionate collaborative care: An integrative review of quality indicators in end-of-life care. BMC Palliative Care, 16(1), 65. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5709969/pdf/12904\_2017\_Article\_246.pdf. https://doi.org/10.1186/s12904-017-0246-4 [PubMed: 29191185] · DOI
  30. Plano Clark V, & Ivankova NV (2016). Mixed methods research: A guide to the field (Vol. 3) Sage Publications.
  31. Rodgers CC, Hooke MC, Taylor OA, Koerner KM, Mitby PA, Moore IM, Scheurer ME, Hockenberry MJ, & Pan W (2019). Childhood cancer symptom cluster: Leukemia and health-related quality of life. Oncology Nursing Forum, 46(2), 228-237. 10.1188/19.ONF.228-237 [PubMed: 30767969] · DOI
  32. Seow H, Snyder CF, Mularski RA, Shugarman LR, Kutner JS, Lorenz KA, Wu AW, & Dy SM (2009). A framework for assessing quality indicators for cancer care at the end of life. Journal of Pain and Symptom Management, 38(6), 903-912. 10.1016/j.jpainsymman.2009.04.024 [PubMed: 19775860] · DOI
  33. Start R, Matlock AM, Brown D, Aronow H, & Soban L (2018). Realizing momentum and synergy: Benchmarking meaningful ambulatory care nurse-sensitive indicators. Nursing Economics, 36(5), 246-251.
  34. Sullivan CE (2021). Developing a core set of nursing-sensitive indicators for international pediatric oncology nursing practice (Publication No. 28414564) [Doctoral dissertation, The University of Alabama at Birmingham]. ProQuest Dissertations and Theses Global
  35. Sullivan CE, Morrissey L, Day SW, Chen Y, Shirey M, & Landier W (2020). Predictors of hospitals' nonachievement of baseline nursing standards for pediatric oncology. Cancer Nursing, 43(4), E197-E206. 10.1097/NCC.0000000000000688 [PubMed: 30932923] · DOI
  36. Teichman J, Punnett A, & Gupta S (2017). Development of quality metrics to evaluate pediatric hematologic oncology care in the outpatient setting. Journal of Pediatric Hematology/Oncology, 39(2), 90-96. 10.1097/MPH.0000000000000656 [PubMed: 27467373] · DOI
  37. Tomlinson D, Dupuis LL, Gibson P, Johnston DL, Portwine C, Baggott C, Zupanec S, Watson J, Spiegler B, Kuczynski S, Macartney G, & Sung L (2014). Initial development of the Symptom Screening in Pediatrics Tool (SSPedi). Supportive Care in Cancer, 22(1), 71-75. 10.1007/s00520-013-1945-x [PubMed: 23995756] · DOI
  38. Wilson S, Hauck Y, Bremner A, & Finn J (2012). Quality nursing care in Australian paediatric hospitals: A Delphi approach to identifying indicators. Journal of Clinical Nursing, 21(11-12), 1594-1605. 10.1111/j.1365-2702.2011.04004.x [PubMed: 22416966] · DOI

Cited by

Sorted by citations

Authors
Sullivan, C. E., Day, S. W., Ivankova, N., Markaki, A., Patrician, P. A., & Landier, W.
Affiliations
  • University of Alabama at Birmingham (US)
  • University of Tennessee Health Science Center (US)
Corresponding author
Sullivan, Courtney E.
ORCID
Venue
Journal of Nursing Scholarship
Publisher
Wiley
Published
2022-07-05
Volume
55
Issue
1
Pages
388-400
ISSN
  • 0363-2792 (ISSN-L)
  • 1527-6546
  • 1547-5069
DOI
10.1111/jnu.12798
PMC
PMC9946155
OpenAlex
W4283824216
Cited by
16
References
0
Citations by year
2026: 5 · 2025: 4 · 2024: 3 · 2023: 4
FWCI
3.96
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • Journal of Nursing Scholarship (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • Oncology Nursing Foundation
  • American Cancer Society (131662‐DSCN‐18‐067‐01‐SCN)
Topics
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
  • Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
  • Social Sciences › Education › Health Education and Validation
Keywords
nursing-sensitive quality indicators, pediatric oncology nursing, children and adolescents with cancer, nursing care quality, infection control, early warning scores, pain assessment and management, patient and family education, palliative and end-of-life care, continuing nursing education, nurse orientation, international nursing practice, low- and middle-income countries, Delphi consensus
MeSH terms
Adolescent, Adult, Child, Delphi Technique, Humans, Neoplasms, Nursing Care, Pediatric Nursing, Quality Indicators, Health Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review