Cited by 20Health Policy Implementation ScienceHealthcare Quality and ManagementHealth Education and Validation

Reliability and validity of a Spanish-language measure assessing clinical capacity to sustain Paediatric Early Warning Systems (PEWS) in resource-limited hospitals

Open access
Agulnik, Asya, Malone, Sara, Puerto-Torres, Maria, Gonzalez-Ruiz, Alejandra, Vedaraju, Yuvanesh, Wang, Huiqi, Graetz, Dylan, Prewitt, Kim +17 more·2021·BMJ Open

Abstract

Background: Paediatric Early Warning Systems (PEWSs) improve identification of deterioration, however, their sustainability has not been studied. Sustainability is critical to maximise impact of interventions like PEWS, particularly in low-resource settings. This study establishes the reliability and validity of a Spanish-language Clinical Sustainability Assessment Tool (CSAT) to assess clinical capacity to sustain interventions in resource-limited hospitals. Methods: Participants included PEWS implementation leadership teams of 29 paediatric cancer centres in Latin America involved in a collaborative to implement PEWS. The CSAT, a sustainability assessment tool validated in high-resource settings, was translated into Spanish and distributed to participants as an anonymous electronic survey. Psychometric, confirmatory factor analysis (CFA), and multivariate analyses were preformed to assess reliability, structure and initial validity. Focus groups were conducted after participants reviewed CSAT reports to assess their interpretation and utility. Results: The CSAT survey achieved an 80% response rate (n=169) with a mean score of 4.4 (of 5; 3.8–4.8 among centres). The CSAT had good reliability with an average internal consistency of 0.77 (95% CI 0.71 to 0.81); and CFAs supported the seven-domain structure. CSAT results were associated with respondents’ perceptions of the evidence for PEWS, its implementation and use in their centre, and their assessment of the hospital culture and implementation climate. The mean CSAT score was higher among respondents at centres with longer time using PEWS (p<0.001). Focus group participants noted the CSAT report helped assess their centre’s clinical capacity to sustain PEWS and provided constructive feedback for improvement. Conclusions: We present information supporting the reliability and validity of the CSAT tool, the first Spanish-language instrument to assess clinical capacity to sustain evidence-based interventions in hospitals of variable resource levels. This assessment demonstrates a high capacity to sustain PEWS in these resource-limited centres with improvement over time from PEWS implementation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjopen-2021-053116

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • An adaptation of the PSAT was developed that focuses on clinical programs and practices.

    “To assess these settings, an adaptation of the PSAT was developed that focuses on clinical programs and practices.”

    From Assessing the sustainability capacity of evidence-based programs in community and health settings (2022) · cites this paper as [35]

    The Background and Methods (CSAT adaptation; page not provided) describe the Clinical Sustainability Assessment Tool as a measure for sustaining clinical practices and report adapting it into Spanish. The paper cites work on the PSAT when describing the CSAT’s domains, but our copy does not explicitly say the CSAT was developed as an adaptation of the PSAT; that relationship is not fully supported. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • The Clinical Sustainability Assessment Tool (CSAT) had been previously developed with established reliability and validity.

    “Both measures had been previously developed with established reliability and validity, and have been widely used across a variety of contexts and settings.”

    From Lowering the burden: Shorter versions of the Program Sustainability Assessment Tool (PSAT) and Clinical Sustainability Assessment Tool (CSAT) (2024) · cites this paper as [9]

    The Methods (CSAT adaptation; page not provided) state that the CSAT had previously been developed and tested, with excellent internal consistency. They describe only trends toward discriminant validity, however, so the claim that its validity was already established overstates the evidence. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • The Clinical Sustainability Assessment Tool has been translated for use in other languages and has been demonstrated as reliable in both domestic and global settings.

    “The Clinical Sustainability Assessment Tool has also been translated for use in other languages and has been demonstrated as reliable in both domestic and global settings (35).”
  • The PSAT and CSAT have been completed online over 13,000 times to measure sustainability capacity across a variety of settings.

    “Together, the PSAT and CSAT have been completed online over 13,000 times to measure the sustainability capacity of programs and practices across a variety of settings, including public health [10, 13], clinical care [9, 14], behavioral health [15], and education [16].”
  • The CSAT contains seven domains with five items each, for a total of 35 items.

    “The CSAT contains seven different domains with five items in each domain for a total of 35 items.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

2 of the 41 references couldn’t be matched to a known paper.

Research works printed but not linked: 3

  1. [12]Agulnik A, Garza M, Gonzalez-Ruiz A. Successful implementation of a pediatric early warning system (PEWS) in 10 Resource-LimitedPediatric oncology centers in Latin America and the Caribbean. Pediatr Blood Cancer 2019; 66: s512– 3.
  2. [26]Malone S, Prewitt K, Luke D. Assessing clinical sustainability: a new, user-friendly tool for evaluating real-world practices. American Evaluation Association’s annual conference. Minneapolis, MN, 2019.
  3. [27]Luke D. The clinical Susatinability assessment tool (CSAT): assessing sustainability in clinical medicine settings.. Paper presented at: 11th Annual Conference on the Science of Dissemination and Implementation in Health. Washington, D.C, 2018.

Cited by

Sorted by citations

Authors
Agulnik, A., Malone, S., Puerto-Torres, M., Gonzalez-Ruiz, A., Vedaraju, Y., Wang, H., Graetz, D., Prewitt, K., Villegas, C., Cardenas-Aguierre, A., Acuna, C., Arana, A. E., Díaz, R., Espinoza, S., Guerrero, K., Martínez, A., Mendez, A., Montalvo, E., Soberanis, D., Torelli, A., Quelal, J., Villanueva, E., Devidas, M., Luke, D., & McKay, V.
Affiliations
  • St. Jude Children's Research Hospital (US)
  • Washington University in St. Louis (US)
  • Instituto Nacional de Enfermedades Neoplásicas (PE)
  • SOLCA Núcleo de Quito (EC)
  • Hospital Luis Calvo Mackenna (CL)
Corresponding author
Agulnik, Asya
Venue
BMJ Open
Publisher
BMJ
Published
2021-10
Volume
11
Issue
10
Pages
e053116
ISSN
2044-6055 (ISSN-L)
OpenAlex
W3207525512
Cited by
20
References
41
Citations by year
2026: 1 · 2025: 1 · 2024: 4 · 2023: 6 · 2022: 8
FWCI
5.06
Type
journal article
Language
en
Open access
Locations
  • BMJ Open (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
American Lebanese Syrian Associated Charities (n/a)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Health Professions › Health Information Management › Healthcare Quality and Management
  • Social Sciences › Education › Health Education and Validation
Keywords
paediatric early warning systems, Spanish-language instrument, psychometric validation, resource-limited hospitals, implementation climate, hospital culture, clinical deterioration, resource-limited settings
MeSH terms
Child, Hospitals, Humans, Language, Neoplasms, Reproducibility of Results

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review