CoChat
Sign Up
Cited by 222Health Policy Implementation ScienceEvaluation and Performance AssessmentAdvanced Causal Inference Techniques

Study Designs for Effectiveness and Translation Research

Mercer, Shawna L., DeVinney, Barbara J., Fine, Lawrence J., Green, Lawrence W., Dougherty, Denise·2007·American Journal of Preventive Medicine
DOI 10.1016/j.amepre.2007.04.005
Open this paper in CoChat — free account

Abstract

Background Practitioners and policymakers need credible evidence of effectiveness to justify allocating resources to complex, expensive health programs. Investigators, however, face challenges in designing sound effectiveness and translation research with relevance for "real-world" settings.Methods Research experts and federal and foundation funders (n= approximately 120) prepared for and participated in a symposium, held May 4-5, 2004, to weigh the strengths, limitations, and trade-offs between alternate designs for studying the effectiveness and translation of complex, multilevel health interventions.Results Symposium attendees acknowledged that research phases (hypothesis generating, efficacy, effectiveness, translation) are iterative and cyclical, not linear, since research in advanced phases may reveal unanswered questions in earlier phases. Research questions thus always need to drive the choice of study design. When randomization and experimental control are feasible, participants noted that the randomized controlled trial with individual random assignment remains the gold standard for safeguarding internal validity. Attendees highlighted trade-offs of randomized controlled trial variants, quasi-experimental designs, and natural experiments for use when randomization or experimental control or both are impossible or inadequately address external validity. Participants discussed enhancements to all designs to increase confidence in causal inference while accommodating greater external validity. Since no single study can establish causality, participants encouraged replication of studies and triangulation using different study designs. Participants also recommended participatory research approaches for building population relevance, acceptability, and usefulness.Conclusions Consideration of the study design choices, trade-offs, and enhancements discussed here can guide the design, funding, completion, and publication of appropriate policy- and practice-oriented effectiveness and translational research for complex, multilevel health interventions.

Integrity signals

Registry

Crossref · 10.1016/j.amepre.2007.04.005

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Trials of efficacy or effectiveness almost always randomize, whereas implementation studies frequently face a choice between a randomized design that offers strong internal validity but is hard to mount and an observational study with weak experimental control but possible value.

    “Efficacy or effectiveness trials nearly always use random assignment, while implementation research often requires tradeoffs between a randomized trial design that can have high internal validity but is difficult to mount, and an observational intervention study that has little experimental control but still may provide valuable information (71).”

    From An Overview of Research and Evaluation Designs for Dissemination and Implementation (2017) · cites this paper as [71]

  • Design choice is complex and depends on research questions, existing knowledge, whether generalizable or local knowledge is sought, stakeholder values and resources, and available opportunities.

    “The choice of design is a complex process, with major input regarding the research questions; the state of existing knowledge; the intention to obtain generalizable or local knowledge; the community, organizational, and funder values, expectations, and resources; and the available opportunities to conduct such research (71).”

    From An Overview of Research and Evaluation Designs for Dissemination and Implementation (2017) · cites this paper as [71]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Mercer, S. L., DeVinney, B. J., Fine, L. J., Green, L. W., & Dougherty, D.
Affiliations
  • Centers for Disease Control and Prevention (US)
  • Agency for Healthcare Research and Quality (US)
  • National Institutes of Health (US)
  • University of California, San Francisco (US)
  • Office of Extramural Research (US)
Corresponding author
Mercer, Shawna L.
ORCID
Mercer, Shawna L.
Venue
American Journal of Preventive Medicine
Publisher
Elsevier BV
Published
2007-08
Volume
33
Issue
2
Pages
139-154.e2
ISSN
0749-3797 (ISSN-L)
DOI
10.1016/j.amepre.2007.04.005
OpenAlex
W2008150938
Cited by
222
References
67
Citations by year
2026: 2 · 2025: 4 · 2024: 10 · 2022: 4 · 2021: 7 · 2020: 5 · 2019: 10 · 2018: 11 · 2017: 13 · 2016: 13 · 2015: 11 · 2014: 16 · 2013: 21 · 2012: 24
FWCI
31.59
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • American Journal of Preventive Medicine (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Evaluation and Performance Assessment
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
Keywords
research design, effectiveness research, translation research, complex health interventions, multilevel interventions, randomized controlled trials, quasi-experimental design, natural experiment, generalizability, internal validity, causal inference, replication, triangulation, participatory research, research funding, health policy
MeSH terms
Humans, Pilot Projects, Research Design, Evidence-Based Medicine, Biomedical Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review