CoChat
Sign Up
Cited by 43Health Literacy and Information AccessibilitySocial Media in Health EducationUsability and User Interface Design

Measuring the Impact of a Moving Target: Towards a Dynamic Framework for Evaluating Collaborative Adaptive Interactive Technologies

Open access
O’Grady, Laura, Witteman, Holly, Bender, Jacqueline L, Urowitz, Sara, Wiljer, David, Jadad, Alejandro R·2009·Journal of Medical Internet Research
DOI 10.2196/jmir.1058PMC PMC2762807
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Website evaluation is a key issue for researchers, organizations, and others responsible for designing, maintaining, endorsing, approving, and/or assessing the use and impact of interventions designed to influence health and health services. Traditionally, these evaluations have included elements such as content credibility, interface usability, and overall design aesthetics. With the emergence of collaborative, adaptive, and interactive ("Web 2.0") technologies such as wikis and other forms of social networking applications, these metrics may no longer be sufficient to adequately assess the quality, use or impact of a health website. Collaborative, adaptive, interactive applications support different ways for people to interact with health information on the Web, including the potential for increased user participation in the design, creation, and maintenance of such sites. OBJECTIVE: We propose a framework that addresses how to evaluate collaborative, adaptive, and interactive applications. METHODS: In this paper, we conducted a comprehensive review of a variety of databases using terminology related to this area. RESULTS: We present a review of evaluation frameworks and also propose a framework that incorporates collaborative, adaptive, and interactive technologies, grounded in evaluation theory. CONCLUSION: This framework can be applied by researchers who wish to compare Web-based interventions, non-profit organizations, and clinical groups who aim to provide health information and support about a particular health concern via the Web, and decisions about funding grants by agencies interested in the role of social networks and collaborative, adaptive, and interactive technologies technologies to improve health and the health system.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/jmir.1058

Cites

29 of 29 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Other methodological considerations for adaptive interactive technologies applied to eHealth may be appropriate to consider for mHealth interventions.

    “Other methodological considerations for adaptive interactive technologies applied to eHealth also may be appropriate to consider for mHealth interventions [107, 108].”

    From Health behavior models in the age of mobile interventions: are our theories up to the task? (2011) · cites this paper as [107]

  • The Milk Man evaluation plan is based on the Collaborative Adaptive Interactive Technology framework developed by O'Grady et al. and is designed to measure indicators for the formative, process and impact evaluation of the app across the five areas of people, content, technology, computer-mediated interaction and health systems integration.

    “The comprehensive plan is based on the Collaborative Adaptive Interactive Technology framework developed by O'Grady et al.21 and is designed to measure indicators for the formative, process and impact evaluation of the app across the five different areaspeople, content, technology, computer-mediated interaction and health systems integration.”

    From Designing evaluation plans for health promotion mHealth interventions: a case study of the <i>Milk Man</i> mobile app (2016) · cites this paper as [21]

  • The Collaborative Adaptive and Interactive Technologies framework was developed by O'Grady et al. in 2009 and organises formative, process and impact evaluation over five key areas: people, content, technology, computer-mediated interaction and health systems integration.

    “The Collaborative Adaptive and Interactive Technologies framework was developed by O'Grady et al. in 2009.21”

    From Designing evaluation plans for health promotion mHealth interventions: a case study of the <i>Milk Man</i> mobile app (2016) · cites this paper as [21]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. [5]Witteman H O’Grady L eHealth in the Era of Web 2.0. Working Papers for Virtually Informed: The Internet as (New) Health Information Source, Final Conference of the Project, Virtually Informed: The Internet in the Medical Field. 2008 1 2009 May 21 Vienna, Austria 13 40 http://www.univie.ac.at/virusss/documents/737266552.pdf
  2. [22]Stout R Web Site Stats: Tracking Hits and Analyzing Traffic 1997 Berkeley, CA Osborne McGraw-Hill
  3. [24]Eysenbach G Designing and evaluation of consumer health information Web sites Lewis D, Eysenbach G, Kukafka R, Stavri PZ, Jimison HB, editors. Consumer Health Informatics: Informing Consumers and Improving Health Care 2005 New York, NY Springer 34 61
  4. [33]Witteman H Chandrashekar S Betel L O’Grady L Sense-making and credibility of health information on the social Web: a multi-method study accessing tagging and tag clouds 2007 World Conference on e-Learning in Corporate, Government, Healthcare & Higher Education October 15-19 2007 Quebec City, QC, Canada
  5. [34]Eysenbach G Credibility of health information and digital media: new perspectives and implications for youth Metzger MJ, Flanagin AJ, editors. Digital Media, Youth, and Credibility 2008 Cambridge, MA The MIT Press 123 154 10.1162/dmal.9780262562324.123 · DOI
  6. [35]Akrich M Méadel C Rémy C Building Collectives Via the Web? Information and Mobilization on Cancer Websites. Working Papers for Virtually Informed: The Internet as (New) Health Information Source, Final Conference of the Project, Virtually Informed: The Internet in the Medical Field. 2008 1 2009 May 21 Vienna, Austria 104 120 http://www.univie.ac.at/virusss/documents/737266552.pdf
  7. [39]Eysenbach G Journal of Medical Internet 2009 May 11 Call for Papers: Medicine 2.0. 2008 http://www.jmir.org/announcement/view/12

Cited by

Sorted by citations

Related

Authors
O’Grady, L., Witteman, H., Bender, J. L., Urowitz, S., Wiljer, D., & Jadad, A. R.
Affiliations
Princess Margaret Cancer Centre (CA)
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2009-06-18
Volume
11
Issue
2
Pages
e20
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/jmir.1058
PMC
PMC2762807
OpenAlex
W2165298997
Cited by
43
References
29
Citations by year
2025: 1 · 2024: 2 · 2022: 2 · 2021: 2 · 2020: 1 · 2019: 2 · 2018: 1 · 2017: 5 · 2016: 4 · 2014: 4 · 2013: 6 · 2012: 3
FWCI
0.77
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Canadian Institutes of Health Research
  • Institute of Gender and Health
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
  • Social Sciences › Health › Social Media in Health Education
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
health websites, website evaluation, evaluation framework, Web 2.0 tools, collaborative technologies, adaptive technology, interactive technologies, wikis, social media applications, user participation, health information, evaluation theory
MeSH terms
Adaptation, Psychological, Cognition, Computer User Training, Humans, Identification, Psychological, Interpersonal Relations, Social Behavior, Technology, Internet

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review