The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index; none of the 5 sources its claims cite is checked for support yet.
The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence
Open accessAbstract
BACKGROUND: Recently there has been a significant increase in the number of systematic reviews addressing questions of prevalence. Key features of a systematic review include the creation of an a priori protocol, clear inclusion criteria, a structured and systematic search process, critical appraisal of studies, and a formal process of data extraction followed by methods to synthesize, or combine, this data. Currently there exists no standard method for conducting critical appraisal of studies in systematic reviews of prevalence data. METHODS: A working group was created to assess current critical appraisal tools for studies reporting prevalence data and develop a new tool for these studies in systematic reviews of prevalence. Following the development of this tool it was piloted amongst an experienced group of sixteen healthcare researchers. RESULTS: The results of the pilot found that this tool was a valid approach to assessing the methodological quality of studies reporting prevalence data to be included in systematic reviews. Participants found the tool acceptable and easy to use. Some comments were provided which helped refine the criteria. CONCLUSION: The results of this pilot study found that this tool was well-accepted by users and further refinements have been made to the tool based on their feedback. We now put forward this tool for use by authors conducting prevalence systematic reviews.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper describes development and pilot testing of a 10-item critical appraisal tool for studies reporting prevalence data in systematic reviews. In a pilot with 16 experienced healthcare researchers, the tool was rated acceptable, easy to use, timely, and valid by all but one participant, though further psychometric testing was recommended.
Key findings
- The working group found no standard method for critically appraising prevalence studies and developed a tool addressing representative sampling, recruitment, sample size, description of subjects/setting, data coverage, objective measurement, reliability, statistical analysis, confounding, and subpopulation criteria.
- Among 16 pilot participants, mean ease of use was 3.63 on a 5-point scale, with 75% rating it 4 (“easy”); mean acceptability was 4.33, with all participants rating 4 or 5; mean timeliness was 3.94, with 88% rating 4 (“acceptable”).
- All participants except 1 viewed the tool as a valid quality appraisal checklist for prevalence data.
- Feedback led to changes in question order and supporting information, but no changes were made to the individual questions; the tool was to be incorporated into the next Joanna Briggs Institute systematic review package.
Methods
- A Joanna Briggs Institute working group searched for systematic reviews of prevalence data and existing critical appraisal tools, then developed a new tool based on identified criteria and their own knowledge/research; the tool was internally trialed and refined.
- The tool was piloted at the 2013 Joanna Briggs Institute convention in Adelaide in October; attendees appraised a cross-sectional study using the new tool and completed a short survey on face validity, ease of use, acceptability, timeliness, and areas for improvement.
- Sixteen workshop participants completed the task and survey; 13 had an academic/research background, 2 had a health background, and 1 had both, with average research experience of 11 years (range 2.5–30 years).
Limitations
- The paper states the tool needs further testing in a larger-scale study to assess other clinimetric properties, particularly construct validity and inter-rater reliability.
- Evident from the text: the pilot assessed only face validity, ease of use, acceptability, and timeliness, with no formal inter-rater reliability or construct validity testing.
- Evident from the text: the pilot used a small convenience sample of 16 workshop attendees, mostly with academic/research backgrounds, which may limit generalizability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 5 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The group identified that the major area where prevalence reviews were disparate was in their conduct of critical appraisal or quality assessment of included studies.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our tool also has the advantage of being simple, easy and quick as shown during the pilot testing.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“We have developed this tool as we did not feel that any of the current checklists identified from our search sufficiently addressed important quality issues in prevalence studies.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot test of this tool amongst a group of experienced healthcare professionals and researchers found that this tool had face validity and high levels of acceptability, ease of use and timeliness to complete.”
- Also stated in: § Discussion · plain
“Our tool also has the advantage of being simple, easy and quick as shown during the pilot testing.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Also stated in: § Discussion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants found the tool acceptable and easy to use.”
- Also stated in: § 10. · plain
“participants providing a rating of 4, corresponding to 'easy' .”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Abstract
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Also stated in: § 10. · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A number of tools were identified including the Joanna Briggs Institute's Descriptive/Case series critical appraisal tool.”
- Also stated in: § Developing the Tool · plain
“Critical appraisal tools from the Cochrane Collaboration and the Critical Appraisal Skills Program (CASP) were also identified.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Developing the Tool
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § Developing the Tool · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot test of this tool amongst a group of experienced healthcare professionals and researchers found that this tool had face validity and high levels of acceptability, ease of use and timeliness to complete.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of all the participants, all except 1 viewed the tool as a valid quality appraisal checklist for prevalence data (Table 4).”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10.
Strength:plain· not yet calibratedn = 16Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot test of this tool amongst a group of experienced healthcare professionals and researchers found that this tool had face validity and high levels of acceptability, ease of use and timeliness to complete.”
- Also stated in: § Discussion · plain
“Our tool also has the advantage of being simple, easy and quick as shown during the pilot testing.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Also stated in: § Discussion · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, currently there does not appear to be any formal guidance for authors wishing to conduct a review of prevalence.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Introduction
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclear- Rests on an assumption: The working group's search for systematic reviews of prevalence was comprehensive enough to conclude that no formal guidance exists.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Consequently, there is significant variability in the methods used to conduct these reviews.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: The working group's search for systematic reviews of prevalence was comprehensive enough to conclude that no formal guidance exists.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Due to this, a new tool assessing validity and quality indicators specific to issues of prevalence has been developed.”
- Also stated in: § Conclusion · plain
“Following a search of the literature a new tool has been proposed that can be used for studies reporting prevalence data, developed by a working party within the Joanna Briggs Institute.”
- Also stated in: § Implications for policy makers · plain
“This article proposes a new tool that can be used during the conduct of these types of systematic reviews to critically appraise studies to ensure that their results are valid.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: There has been substantial variability in how studies reporting prevalence data are critically appraised.
- Rests on an assumption: Systematic reviews of prevalence and incidence data are becoming increasingly important and popular.
- Also stated in: § Conclusion · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A pilot test of this tool amongst a group of experienced healthcare professionals and researchers found that this tool had face validity and high levels of acceptability, ease of use and timeliness to complete.”
- Also stated in: § Conclusion · boosted
“The results of this pilot study found that this tool was well-accepted by users and further refinements have been made to the tool based on their feedback.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § Conclusion · boosted
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“further refinements have been made to the tool based on their feedback.”
- Also stated in: § 10. · plain
“There were a number of suggestions provided for refinement and improvement of the tool.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Abstract
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes- Also stated in: § 10. · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Participants found the tool acceptable and easy to use.”
- Also stated in: § 10. · plain
“For the acceptability of the tool, the mean score was 4.33, with all participants giving either a ranking of 4 (acceptable) or 5 (very acceptable).”
- Also stated in: § Abstract · boosted
“The results of this pilot study found that this tool was well-accepted by users and further refinements have been made to the tool based on their feedback.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Abstract
Load-bearingStrength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § 10. · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the last five years, there has been a substantial increase in the number of systematic reviews addressing questions of prevalence (Figure 1).”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Introduction
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This checklist addresses critical issues of internal and external validity that must be considered when assessing validity of prevalence data that can be used across different study designs (not just cross-sectional studies but all studies that might report prevalence data).”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“Prevalence data can be sourced from various study designs, even randomized controlled trials (11);”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
What the sources say
- Not checked yetcites [11]
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results of the pilot found that this tool was a valid approach to assessing the methodological quality of studies reporting prevalence data to be included in systematic reviews.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Rests on an assumption: User acceptance and face validity of the tool indicate its validity as a quality appraisal checklist.
- Rests on an assumption: The tool's criteria cover all important aspects of methodological quality for prevalence studies.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, criteria regarding the use of an intention-to-treat analysis as often seen in critical appraisal checklists for randomized controlled trials are not a true quality indicator for questions of prevalence.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This checklist addresses critical issues of internal and external validity that must be considered when assessing validity of prevalence data that can be used across different study designs (not just cross-sectional studies but all studies that might report prevalence data).”
- Also stated in: § Implications for policy makers · plain
“This tool guides the assessment of internal and external validity of studies reporting prevalence data.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § Implications for policy makers · plain
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although many of these checklists identified important criteria, it was felt by the group that none of these tools were complete and ideal for use during assessment of quality during the systematic review process.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Developing the Tool
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIn the paper
“For example, we provide a criteria regarding sample size which is not included in the Hoy et al. checklist.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
What the sources say
- Not checked yetcites [20]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- Probably not a substantial claim. 11
- It may state two separate claims as one. 4
- The quote only partly states this claim. 6
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The average time spent working in research was 11 years, with the minimum being 2.5 years and the maximum experience being 30 years.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Results
Strength:plain· not yet calibratedn = 16Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These comments resulted in some changes in the order of the questions of the tool and the supporting information used to assist in judging criteria, although no changes were made to the individual questions.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For timeliness, the mean score was 3.94, with 88% providing a score of 4 (acceptable).”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the 16, 13 participants stated they had an academic/research background, 2 said they had a health background, and one said they had both.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Results
Strength:plain· not yet calibratedn = 16Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Currently there exists no standard method for conducting critical appraisal of studies in systematic reviews of prevalence data.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: The working group's search for critical appraisal tools was comprehensive enough to conclude that no standard method exists.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Arithmetic mean | 11.37 | 3.63 | 4.33 | 3.94 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% lower confidence limit | 6.94 | 3.24 | 4.06 | 3.63 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% upper confidence limit | 15.80 | 4.00 | 4.60 | 4.24 |”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10. Were subpopulations identified using objective criteria?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Arithmetic mean | 11.37 | 3.63 | 4.33 | 3.94 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% lower confidence limit | 6.94 | 3.24 | 4.06 | 3.63 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% upper confidence limit | 15.80 | 4.00 | 4.60 | 4.24 |”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10. Were subpopulations identified using objective criteria?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Arithmetic mean | 11.37 | 3.63 | 4.33 | 3.94 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% lower confidence limit | 6.94 | 3.24 | 4.06 | 3.63 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% upper confidence limit | 15.80 | 4.00 | 4.60 | 4.24 |”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10. Were subpopulations identified using objective criteria?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Arithmetic mean | 11.37 | 3.63 | 4.33 | 3.94 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% lower confidence limit | 6.94 | 3.24 | 4.06 | 3.63 |”
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
“| 95.00% upper confidence limit | 15.80 | 4.00 | 4.60 | 4.24 |”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10. Were subpopulations identified using objective criteria?
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
- Also stated in: § 10. Were subpopulations identified using objective criteria? · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“| Number of cases | 15 | 16 | 15 | 16 |”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10. Were subpopulations identified using objective criteria?
Tests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“| Number of cases | 15 | 16 | 15 | 16 |”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > 10. Were subpopulations identified using objective criteria?
Strength:plain· not yet calibratedn = 15Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. The quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sixteen workshop participants completed the critical appraisal task and survey.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Results
Strength:plain· not yet calibratedn = 16Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“The systematic review of evidence has been proposed and is now well-accepted as the ideal method to summarize the literature relating to a certain social or healthcare topic (2,3).”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Introduction
What the sources say
- Not checked yetcites [2]
- Not checked yetcites [3]
Strength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Policy-makers are often faced with conflicting reports of disease prevalence in the literature.
In the paper
“Some of the tools [most notably the tool refined by Hoy et al. (20)] contain similar questions to our tool but there are important differences.”
§ The development of a critical appraisal tool for use in systematic reviews addressing questions of prevalence > Discussion
What the sources say
- Not checked yetcites [20]
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper relies on this without support.
Policy-makers are often faced with conflicting reports of disease prevalence in the literature.
“However, policy-makers are often faced with conflicting reports of disease prevalence in the literature.”
The paper relies on this without support.
User acceptance and face validity of the tool indicate its validity as a quality appraisal checklist.
“The results of the pilot found that this tool was a valid approach to assessing the methodological quality of studies reporting prevalence data to be included in systematic reviews.”
“along with a short survey that was developed to establish the face validity, ease of use, acceptability and timeliness (i.e.time taken to complete) of the tool, and feedback on areas for improvement (23).”
The paper relies on this without support.
The working group's search for critical appraisal tools was comprehensive enough to conclude that no standard method exists.
“Currently there exists no standard method for conducting critical appraisal of studies in systematic reviews of prevalence data.”
“The group then searched for critical appraisal tools that have been used to assess studies reporting on prevalence data.”
The paper relies on this without support.
The tool's criteria cover all important aspects of methodological quality for prevalence studies.
“The results of the pilot found that this tool was a valid approach to assessing the methodological quality of studies reporting prevalence data to be included in systematic reviews.”
“Based on a review of these criteria and our own knowledge and research we developed a tool specifically for use in systematic reviews of prevalence data.”
The paper relies on this without support.
The working group's search for systematic reviews of prevalence was comprehensive enough to conclude that no formal guidance exists.
“However, currently there does not appear to be any formal guidance for authors wishing to conduct a review of prevalence.”
“The working party began by conducting a search for systematic reviews of prevalence data to determine how the methodological quality of studies included in these reviews were assessed.”
The paper relies on this without support.
There has been substantial variability in how studies reporting prevalence data are critically appraised.
“Until now there has been substantial variability in how studies reporting prevalence data are critically appraised.”
The paper relies on this without support.
Systematic reviews of prevalence and incidence data are becoming increasingly important and popular.
“Systematic reviews of prevalence and incidence data are becoming increasingly important as decision makers realize their usefulness in informing policy and practice.”
“Prevalence systematic reviews are becoming increasingly popular within the research community.”
The paper relies on this without support.
Many health problems are not easily diagnosed or defined, and some measures may not be capable of including or excluding appropriate levels or stages of the health problem.
“Many health problems are not easily diagnosed or defined and some measures may not be capable of including or excluding appropriate levels or stages of the health problem.”
The paper relies on this without support.
If outcomes are assessed using observer-reported or self-reported scales, the risk of over- or under-reporting is increased, and objectivity is compromised.
“If the outcomes were assessed using observer reported, or self-reported scales, the risk of over-or under-reporting is increased, and objectivity is compromised.”
The paper relies on this without support.
Validated measurement tools have a significant impact on outcome assessment validity.
“Importantly, determine if the measurement tools used were validated instruments as this has a significant impact on outcome assessment validity.”
The paper relies on this without support.
Considerable judgment is required to determine the presence of some health outcomes.
“Considerable judgment is required to determine the presence of some health outcomes.”
The paper relies on this without support.
Differing methods of analysis are based on differing assumptions about the data and how it will respond.
“differing methods of analysis are based on differing assumptions about the data and how it will respond.”
The paper relies on this without support.
Prevalence rates found in studies only provide estimates of the true prevalence of a problem in the larger population.
“Prevalence rates found in studies only provide estimates of the true prevalence of a problem in the larger population.”
The paper relies on this without support.
Since some subgroups are very small, 95% confidence intervals are usually given.
“Since some subgroups are very small, 95% confidence intervals are usually given.”
The paper relies on this without support.
Incidence and prevalence studies often draw or report findings regarding the differences between groups.
“Incidence and prevalence studies often draw or report findings regarding the differences between groups.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Each study included in the systematic review underwent a quality assessment to evaluate the research methodology employed in each study to ensure the reliability and validity of its findings. The Joanna Briggs Institute (JBI) critical appraisal tools, widely acknowledged and reliable for quality assessment, were used to investigate each study [17].”
From Malaria in pregnancy: Meta-analyses of prevalence and associated complications (2024) · cites this paper as [17]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“both have been previously validated [23, 24].”
From What are the barriers, facilitators and interventions targeting help-seeking behaviours for common mental health problems in adolescents? A systematic review (2020) · cites this paper as [23]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Two reviewers independently conducted the quality assessment using the JBI tool, with any discrepancies resolved through discussion with a third reviewer.”
From Adherence to PCSK9 Inhibitors in Clinical Practice (2026) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Two reviewers (MD and CS) independently assessed the risk of bias/ study quality of each included study with the adjusted Joanna Briggs Institute Critical Appraisal Checklist for Studies Reporting Prevalence Data (Munn et al., 2014).”
From Prevalence of attention-deficit/hyperactivity disorder in older adults: a systematic review and meta-analysis (2020) · cites this paper as Munn et al., 2014
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Tropical climates with appropriate temperature, humidity, and rainfall conditions encourage endemicity of the disease as they are conducive to the reproduction of the parasite within the anopheles’ mosquito, which is itself native to these environments [17].”
From Malaria in pregnancy: Meta-analyses of prevalence and associated complications (2024) · cites this paper as [17]
Experimental — accuracy not yet measured
In the citing paper
“we urge reviewers to refer to the JBI critical appraisal tool, [4] which has been formally evaluated and is increasingly used across these types of reviews.”
From How are systematic reviews of prevalence conducted? A methodological study (2020) · cites this paper as [4]
In the citing paper
“The quality and risk of bias of included studies was assessed using the Prevalence Critical Appraisal Instrument (PCAI) which is designed specifically for assessing prevalence studies 25 .”
From Anxiety in Informal Dementia Carers: A Meta-Analysis of Prevalence (2020) · cites this paper as [25]
In the citing paper
“Environmental factors can have a substantial impact on the prevalence or incidence of a condition so it is important that authors define the context or specific setting relevant to their review question [15, 29].”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [29]
In the citing paper
“These types of reviews enable governments, policy makers, health professionals and the general population to inform the development and delivery of health services and evaluate changes and trends in diseases over time [15, 29].”
From What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences (2018) · cites this paper as [29]
Citations
Cites
Two new Later Stone Age sites from the Final Pleistocene in the Falémé Valley, eastern Senegal
Ndiaye, M., Lespez, L., Tribolo, C., Rassé, M., Hadjas, I., Davidoux, S., Huysecom, É., & Douze, K. (2024) · Monash University Research Portal (Monash University) · doi:10.3929/ethz-b-000667478 · Cited by 24,214
Vandenbroucke, J. P., von Elm, E., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040297 · Cited by 8,613 · cited 2×
“(STROBE) Checklist (18) 22 Targeted towards reporting”
BackgroundVandenbroucke, J. P., Elm, E. V., Altman, D. G., Gøtzsche, P. C., Mulrow, C. D., Pocock, S. J., Poole, C., Schlesselman, J. J., Egger, M., & initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010-w1 · Cited by 3,543
Hoy, D., Brooks, P., Woolf, A., Blyth, F., March, L., Bain, C., Baker, P., Smith, E., & Buchbinder, R. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.11.014 · Cited by 2,555
Practical Issues in Calculating the Sample Size for Prevalence Studies
Naing, L., & Winn, T. (2006) · Cited by 1,381
McGrath, J., Saha, S., Welham, J., El Saadi, O., MacCauley, C., & Chant, D. (2004) · BMC Medicine · doi:10.1186/1741-7015-2-13 · Cited by 1,072
Biostatistics: A Foundation for Analysis in the Health Sciences.
Marriott, F. H. C., & Daniel, W. W. (1984) · Biometrics · doi:10.2307/2530782 · Cited by 760
Bahekar, A., Singh, S., Saha, S. A., Molnár, J., & Arora, R. (2007) · American Heart Journal · doi:10.1016/j.ahj.2007.06.037 · Cited by 699
Critical appraisal of the health research literature: prevalence or incidence of a health problem.
Loney, P., Chambers, L. W., BENNETT, K. J., Roberts, J. G., & Stratford, P. W. (1998) · PubMed · Cited by 550
Munn, Z., Tufănaru, C., & Aromataris, E. (2014) · AJN American Journal of Nursing · doi:10.1097/01.naj.0000451683.66447.89 · Cited by 543
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- The Joanna Briggs Institute. Joanna Briggs Institute Reviewers' Manual: 2011 edition. Adelaide: The Joanna Briggs Institute; 2011.
- Munn Z, Moola S, Lisy K, Riitano D. The Systematic Review of Prevalence and Incidence Data The Joanna Briggs Institute Reviewer's Manual 2014. Australia: The Joanna Briggs Institute; 2014.
Cited by
Sorted by citations
Munn, Z., Moola, S., Lisy, K., Riitano, D., & Tufanaru, C. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000054 · Cited by 3,131
“Recently, there has been a substantial increase in the number of published systematic reviews of prevalence and incidence data.(8) Across these reviews, there is considerable variation in terms of the approaches to inclusion criteria, critical appraisal, data extraction, and data synthesis.”
BackgroundMethodSelf-citationBozdag, G., Mumusoglu, S., Zengin, D., Karabulut, E., & Yildiz, B. O. (2016) · Human Reproduction · doi:10.1093/humrep/dew218 · Cited by 1,526
“This appraisal tool was inspired by recent protocols produced by Munn et al. (2014) and Hoy et al. (2012).”
MethodSavoldi, A., Carrara, E., Graham, D. Y., Conti, M., & Tacconelli, E. (2018) · Gastroenterology · doi:10.1053/j.gastro.2018.07.007 · Cited by 1,323
“) independently, using an adapted version of the tool proposed by the Joanna Briggs Institute for critical appraisal of prevalence studies (Supplementary Table 1).(19) A score ranging from 0 to 8 points was attributed to each study (>5 points: high quality, 4-5 points: medium quality, <4 points: low quality).”
MethodMunn, Z., Stern, C., Aromataris, E., Lockwood, C., & Jordan, Z. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0468-4 · Cited by 1,164
“Environmental factors can have a substantial impact on the prevalence or incidence of a condition so it is important that authors define the context or specific setting relevant to their review question [15, 29].”
BackgroundSelf-citationPopova, S., Lange, S., Probst, C., Gmel, G., & Rehm, J. (2017) · The Lancet Global Health · doi:10.1016/s2214-109x(17)30021-9 · Cited by 937
Metadata
- Authors
- Munn, Z., Moola, S., Riitano, D., & Lisy, K.
- Affiliations
- The University of Adelaide (AU)
- Venue
- International Journal of Health Policy and Management
- Publisher
- Maad Rayan Publishing Company
- Published
- 2014
- Volume
- 3
- Issue
- 3
- Pages
- 123-128
- ISSN
- 2322-5939 (ISSN-L)
- PMC
- PMC4154549
- OpenAlex
- W2167191881
- Semantic Scholar
- 834402d2ae3882ab962ec84179a5857fc9bd890b
- Cited by
- 1,479
- References
- 23
- Citations by year
- 2026: 145 · 2025: 166 · 2024: 177 · 2023: 193 · 2022: 220 · 2021: 172 · 2020: 151 · 2019: 93 · 2018: 73 · 2017: 37 · 2016: 39 · 2015: 11 · 2014: 2
- FWCI
- 8.12
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- International Journal of Health Policy and Management (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Research Information System of Ardabil University of Medical Sciences (Ardabil University of Medical Sciences) (acceptedVersion)
- SSRN Electronic Journal (submittedVersion)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- critical appraisal tools, prevalence, systematic reviews, methodological quality assessment, prevalence data, tool development, pilot study
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
