CoChat

Best-Evidence Synthesis: Why Less Is More

Slavin, Robert E. · 1987 · Educational Researcher
Cited by 44

The DOI matches its Crossref record.

Abstract

In his response to my proposal for an alternative to meta-analytic and traditional reviews (Slavin, 1986), Bruce Joyce has made the exact misinterpretation of what I am proposing that I most feared readers would make. Therefore I am delighted to have an opportunity to clarify what I meant by and what I did not mean by it. One thing I did not mean to suggest was that reviewers establish a priori inclusion criteria before searching the literature. Rather, my position is very close to that which Joyce proposes: Find everything, then organize it carefully. The first step in conducting a best-evidence synthesis is to conduct a broad-scale preliminary literature search in order to fully understand the meaning and quality of the evidence. Only then am I suggesting formulating inclusion criteria. Also, I am proposing that the evidence excluded be described in some detail, so that the reader, to use Joyce's term, can enter the thought processes of the investigator and understand the consequences of whatever inclusion criteria were applied. Actually, I would have no objection if a reviewer obtained every study on a particular subject and then partitioned the evidence according to well-defended criteria. What mostly differentiates bestevidence synthesis from metaanalysis is not issues relating to inclusion, but rather the requirement is a best-evidence synthesis that the studies on which the argument depends be identified and described in enough detail to allow the reader to understand what the body of research really investigated and what it found. Very few meta-analyses discuss more than one or two of the dozens or hundreds of studies whose results they synthesize. The issue of a priori inclusion comes up only because it is a practical impossibility to discuss everything in a l rge literature, so we must have some means of deciding what is worth discussing for a given purpose. Every review makes one or more principal conclusions. What is he nature and quality of the evidence on which these conclusions rest? The reader has a right to know.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2307/1175827

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cited by

Metadata

Authors
Slavin, R. E.
Venue
Educational Researcher
Published
1987-05
Volume
16
Issue
4
Pages
15
ISSN
0013-189X (ISSN-L)
OpenAlex
W2073386347
Cited by
44
References
0
FWCI
2.42
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.