Evaluation Frameworks for Health Virtual Communities
Abstract
Virtual communities (VCs) evaluation research looked into different aspects of evaluation such as usability, effectiveness, and impact. However, these measurements do not take into account some particularities of the Health Virtual Communities (Health VCs), such as the impact of the patient’s health condition on software usability and the effect of the Health VC on health outcomes. In this chapter, we give an overview of existing literature related to VC evaluation, and summarize the main VC evaluation limitations and challenges, particularly the need for a framework for VC evaluation. We then look into the available frameworks to evaluate information technology and information systems and their ability to VCs, and then lay out the need for a framework to evaluate Health VC.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Frameworks to evaluate the successful implementation and adoption of online communities do exist.
“It is important to note that frameworks to evaluate the successful implementation and adoption of online communities do exist [57].”
From Effectiveness of online mindfulness interventions on medical students’ mental health: a systematic review (2021) · cites this paper as [57]
Cites
Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999) · American Journal of Public Health · doi:10.2105/ajph.89.9.1322 · Cited by 7,217 · see citing works
(1985) · Cited by 6,185 · see citing works
The Virtual Community: Homesteading on the Electronic Frontier
Rheingold, H. (1994) · Cited by 5,655 · see citing works
The Technology Acceptance Model: Its past and its future in health care
Holden, R. J., & Karsh, B.-T. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.002 · Cited by 2,669 · see citing works
Hine, C. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9780857020277 · Cited by 2,436 · see citing works
(2005) · JAMA · doi:10.1001/jama.294.19.2423-c · Cited by 2,325 · see citing works
Rheingold, H. (2000) · The MIT Press eBooks · doi:10.7551/mitpress/7105.001.0001 · Cited by 2,108 · see citing works
Motivation and barriers to participation in virtual knowledge‐sharing communities of practice
Ardichvili, A., Page, V. J., & Wentling, T. L. (2003) · Journal of Knowledge Management · doi:10.1108/13673270310463626 · Cited by 1,550 · see citing works
Online Communities: Designing Usability and Supporting Sociability
Preece, J. (2000) · Cited by 1,512 · see citing works
Online Communities: Designing Usability, Supporting Sociability
Preece, J. J. (2000) · Industrial Management & Data Systems · doi:10.1108/imds.2000.100.9.459.3 · Cited by 1,390 · see citing works
Supporting trust in virtual communities
Abdul-Rahman, A., & Hailes, S. M. V. (2005) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2000.926814 · Cited by 1,341 · see citing works
Eysenbach, G., Powell, J., Englesakis, M., Rizo, C., & Stern, A. (2004) · BMJ · doi:10.1136/bmj.328.7449.1166 · Cited by 1,268 · see citing works
Overview of the Technology Acceptance Model: Origins, Developments and Future Directions
Chuttur, M. Y. (2009) · Cited by 1,039 · see citing works
Patient empowerment in theory and practice: Polysemy or cacophony?
Aujoulat, I., D’Hoore, W., & Deccache, A. (2006) · Patient Education and Counseling · doi:10.1016/j.pec.2006.09.008 · Cited by 717 · see citing works
Chau, P. Y. K., & Hu, P. J. (2002) · Journal of Management Information Systems · doi:10.1080/07421222.2002.11045699 · Cited by 707 · see citing works
Usability measurement and metrics: A consolidated model
Seffah, A., Donyaee, M., Kline, R. B., & Padda, H. (2006) · Software Quality Journal · doi:10.1007/s11219-006-7600-8 · Cited by 652 · see citing works
Aura: An Architectural Framework for User Mobility in Ubiquitous Computing Environments
Sousa, J. P., & Garlan, D. (2002) · IFIP advances in information and communication technology · doi:10.1007/978-0-387-35607-5_2 · Cited by 549 · see citing works
Sense of Virtual Community: A Conceptual Framework and Empirical Validation
Koh, J., Kim, Y., & Kim, Y. (2003) · International Journal of Electronic Commerce · doi:10.1080/10864415.2003.11044295 · Cited by 523 · see citing works
Impact of a patient-centered, computer-based health information/support system
Gustafson, D. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(98)00108-1 · Cited by 461 · see citing works
Ranmuthugala, G., Plumb, J., Cunningham, F. C., Georgiou, A., Westbrook, J. I., & Braithwaite, J. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-273 · Cited by 441 · see citing works
Getting the Seats of Your Pants Dirty: Strategies for Ethnographic Research on Virtual Communities
Paccagnella, L. (2006) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.1997.tb00065.x · Cited by 379 · see citing works
Use of communities of practice in business and health care sectors: A systematic review
Li, L. C., Grimshaw, J. M., Nielsen, C., Judd, M., Coyte, P. C., & Graham, I. D. (2009) · Implementation Science · doi:10.1186/1748-5908-4-27 · Cited by 377 · see citing works
Why communities of practice succeed and why they fail
PROBST, G. J. B., & Borzillo, S. (2008) · European Management Journal · doi:10.1016/j.emj.2008.05.003 · Cited by 358 · see citing works
Non-public and public online community participation: Needs, attitudes and behavior
Nonnecke, B., Andrews, D. C., & Preece, J. J. (2006) · Electronic Commerce Research · doi:10.1007/s10660-006-5985-x · Cited by 317 · see citing works
Sense of community and its relevance to adolescents of all ages
Pretty, G. M. H., Conroy, C., Dugay, J., Fowler, K., & Williams, D. (1996) · Journal of Community Psychology · doi:10.1002/(sici)1520-6629(199610)24:4 · Cited by 306 · see citing works
Gagnon, M. P., Orruño, E., Asua, J., Abdeljelil, A. B., & Emparanza, J. I. (2011) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2011.0066 · Cited by 301 · see citing works
Community: A Critical Response
Gusfield, J. R. (1975) · Cited by 286 · see citing works
Richardson, C. R., Buis, L. R., Janney, A. W., Goodrich, D. E., Sen, A., Hess, M. L., Mehari, K. S., Fortlage, L. A., Resnick, P. J., Zikmund-Fisher, B. J., et al. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1338 · Cited by 239 · see citing works
Glasgow, R. E., Nelson, C. C., Kearney, K. A., Reid, R., Ritzwoller, D. P., Strecher, V. J., Couper, M. P., Green, B., & Wildenhaus, K. (2007) · Journal of Medical Internet Research · doi:10.2196/jmir.9.2.e11 · Cited by 216 · see citing works
Physiological indicators for the evaluation of co-located collaborative play
Mandryk, R. L., & Inkpen, K. M. (2004) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/1031607.1031625 · Cited by 190 · see citing works
Greenhill, J. (2009) · Elsevier eBooks · doi:10.1016/b978-0-7295-3861-9.10011-0 · Cited by 161 · see citing works
A practical evaluation of Web analytics
Phippen, A., Sheppard, L., & Furnell, S. (2004) · Internet Research · doi:10.1108/10662240410555306 · Cited by 149 · see citing works
Encyclopedia of Community: From the Village to the Virtual World
Christensen, K., & Levinson, D. (2003) · Sage eBooks · doi:10.4135/9781412952583 · Cited by 139 · see citing works
The Encyclopedia of community: from the village to the virtual world
(2004) · Choice Reviews Online · doi:10.5860/choice.41-3812 · Cited by 138 · see citing works
Barley, E. A., Walters, P., Tylee, A. T., & Murray, J. (2012) · BMC Family Practice · doi:10.1186/1471-2296-13-1 · Cited by 132 · see citing works
(2021) · Advancing microelectronics · doi:10.4071/2380-7016-48.3.1 · Cited by 129 · see citing works
General practice training and virtual communities of practice - a review of the literature
Barnett, S., Jones, S. C., Bennett, S., Iverson, D., & Bonney, A. D. (2012) · BMC Family Practice · doi:10.1186/1471-2296-13-87 · Cited by 120 · see citing works
The use of guidelines to automatically verify Web accessibility
Abascal, J., Arrue, M., Fajardo, I., Garay-Vitoria, N., & Tom�s, J. (2004) · Universal Access in the Information Society · doi:10.1007/s10209-003-0069-3 · Cited by 113 · see citing works
Designing mobile technologies to support co-present collaboration
Cole, H., & Fraser, D. S. (2003) · Personal and Ubiquitous Computing · doi:10.1007/s00779-003-0249-4 · Cited by 99 · see citing works
Sharing the square: Collaborative Leisure in the City Streets
Brown, B. A. T., Chalmers, M., Bell, M., Hall, M., MacColl, I., & Rudman, P. D. (2006) · Springer eBooks · doi:10.1007/1-4020-4023-7_22 · Cited by 97 · see citing works
2011 WRITING GROUP MEMBERS*, Rooke, T. W., Hirsch, A. T., Misra, S., Sidawy, A. N., Beckman, J. A., Findeiss, L., Golzarian, J., Gornik, H. L., Halperin, J. L., et al. (2011) · Catheterization and Cardiovascular Interventions · doi:10.1002/ccd.23373 · Cited by 85 · see citing works
Examining information exchange and virtual communities: an emergent framework
Hersberger, J. A., Murray, A. L., & Rioux, K. S. (2007) · Online Information Review · doi:10.1108/14684520710747194 · Cited by 83 · see citing works
Blanchard, A., Welbourne, J. L., & Boughton, M. D. (2010) · Information Communication & Society · doi:10.1080/13691181003739633 · Cited by 78 · see citing works
Blending digital and physical spaces for ubiquitous community participation
Churchill, E. F., Girgensohn, A., Nelson, L., & Lee, A. (2004) · Communications of the ACM · doi:10.1145/966389.966413 · Cited by 74 · see citing works
Wearable communities: augmenting social networks with wearable computers
Kortuem, G., & Segall, Z. (2003) · IEEE Pervasive Computing · doi:10.1109/mprv.2003.1186728 · Cited by 71 · see citing works
Bridging the Gap: Knowledge Seeking and Sharing in a Virtual Community of Emergency Practice
Curran, J., Murphy, A. L., Abidi, S. S. R., Sinclair, D. E., & McGrath, P. J. (2009) · Evaluation & the Health Professions · doi:10.1177/0163278709338570 · Cited by 70 · see citing works
Bender, J. L., Jimenez, M. C., Ferris, L. E., Katz, J. D., & Jadad, A. R. (2012) · Supportive Care in Cancer · doi:10.1007/s00520-012-1655-9 · Cited by 69 · see citing works
Community: A Critical Response
Marsland, D., & Gusfield, J. R. (1977) · British Journal of Sociology · doi:10.2307/589432 · Cited by 62 · see citing works
Mobile virtual communities research: a synthesis of current trends and a look at future perspectives
Morr, C. E., & Kawash, J. (2007) · International Journal of Web Based Communities · doi:10.1504/ijwbc.2007.015865 · Cited by 61 · see citing works
Dorsey, E. R., Achey, M. A., Beck, C. A., Beran, D. B., Biglan, K., Boyd, C. M., Schmidt, P. N., Simone, R. B., Willis, A. W., Galifianakis, N. B., et al. (2016) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2015.0191 · Cited by 60 · see citing works
Communication That Changes Lives: Social Support Within an Online Health Community for ALS
Loane, S. S., & D'Alessandro, S. P. (2013) · Communication Quarterly · doi:10.1080/01463373.2012.752397 · Cited by 52 · see citing works
Rantanen, M., Oulasvirta, A., Blom, J., Tiitta, S., & Mäntylä, M. (2004) · Nordic Conference on Human-Computer Interaction · doi:10.1145/1028014.1028035 · Cited by 52 · see citing works
Weissman, D. E. (2000) · Cited by 51 · see citing works
Davies, J. W., Duke, A., & Sure, Y. (2003) · International Conference on Knowledge Capture · doi:10.1145/945645.945652 · Cited by 50 · see citing works
Andersson, N., & Lamothe, G. (2011) · BMC Health Services Research · doi:10.1186/1472-6963-11-s2-s15 · Cited by 50 · see citing works
Barnett, S., Jones, S. C., Bennett, S., Iverson, D. C., & Bonney, A. D. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.2555 · Cited by 48 · see citing works
Use of a virtual community as a psychosocial support system in pediatric transplantation
Bers, M. U., Beals, L. M., Chau, C., Satoh, K., Blume, E. D., DeMaso, D. R., & Gonzalez–Heydrich, J. (2010) · Pediatric Transplantation · doi:10.1111/j.1399-3046.2010.01271.x · Cited by 46 · see citing works
Policy issues relevant to evaluation of interactive health communication applications
Patrick, K. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(98)00103-2 · Cited by 42 · see citing works
Ridgeway, J. L., LeBlanc, A., Branda, M. E., Harms, R. W., Morris, M. A., Nesbitt, K., Gostout, B. S., Barkey, L. M., Sobolewski, S. M., Brodrick, E. M., et al. (2015) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-015-0762-2 · Cited by 39 · see citing works
Health care providers and purchasers and evaluation of interactive health communication applications
Jimison, H. B. (1999) · American Journal of Preventive Medicine · doi:10.1016/s0749-3797(98)00105-6 · Cited by 36 · see citing works
Virtual community health promotion.
Crespo, R. (2007) · PubMed · Cited by 32 · see citing works
COPEP: A Development Process Model for a Community Platform for Cancer Patients
Arnold, Y., Leimeister, J. M., & Krcmar, H. (2003) · Alexandria (UniSG) (University of St.Gallen) · Cited by 31 · see citing works
Mendizabal, G. A., Nuño-Solínis, R., & González, I. Z. (2013) · BMC Family Practice · doi:10.1186/1471-2296-14-168 · Cited by 27 · see citing works
An evaluative methodology for virtual communities using web analytics
Phippen, A. (2004) · Campus-Wide Information Systems · doi:10.1108/10650740410567518 · Cited by 26 · see citing works
A Theoretical Framework for a Virtual Diabetes Self-Management Community Intervention
Vorderstrasse, A. A., Shaw, R. J., Blascovich, J., & Johnson, C. (2014) · Western Journal of Nursing Research · doi:10.1177/0193945913518993 · Cited by 23 · see citing works
Collaboration platforms for virtual student communities
Schubert, P. L., & Koch, M. R. (2003) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2003.1174571 · Cited by 19 · see citing works
Guide to Monitoring and Evaluating Health Information Products and Services
Sullivan, T. M., Strachan, M., & Timmons, B. K. (2007) · Cited by 19 · see citing works
Comprehensive Health Enhancement Support System (CHESS) for People with HIV Infection
Temesgen, Z., Knappe-Langworthy, J. E., Marie, M. M. S., Smith, B. A., & Dierkhising, R. (2006) · AIDS and Behavior · doi:10.1007/s10461-005-9026-x · Cited by 16 · see citing works
Kaji, N., Ragab, K., Ono, T., & Mori, K. (2003) · International Workshop on Autonomous Decentralized System (IWADS) · doi:10.1109/iwads.2002.1194643 · Cited by 13 · see citing works
Terrizzi, S., Sherer, S. A., Meyerhoefer, C. D., Sheinberg, M., & Levick, D. (2012) · Journal of the Association for Information Systems · Cited by 12 · see citing works
A Health Virtual Community for Patients with Chronic Kidney Disease
Morr, C. E., Cole, C., & Perl, J. M. (2014) · Procedia Computer Science · doi:10.1016/j.procs.2014.08.049 · Cited by 12 · see citing works
Towards m-communities: the case of COSMOS healthcare
Leimeister, J. M., Daum, M., & Krcmar, H. (2003) · Proceedings of the ... Annual Hawaii International Conference on System Sciences/Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.1109/hicss.2003.1174572 · Cited by 12 · see citing works
A Virtual Knowledge Community for Human Rights Monitoring for People with Disabilities.
Morr, C. E., Subercaze, J., Maret, P., & Rioux, M. H. (2008) · Cited by 11 · see citing works
The positive outcomes of a sense of virtual community
Ellonen, H., Kosonen, M., Tarkiainen, A., & Tonteri, L. (2013) · International Journal of Web Based Communities · doi:10.1504/ijwbc.2013.057216 · Cited by 9 · see citing works
Practical Web Traffic Analysis
Fletcher, P., Poon, A., Pearce, B., & Comber, P. (2002) · Apress eBooks · doi:10.1007/978-1-4302-5366-2 · Cited by 9 · see citing works
Six years of lessons learned in monitoring and evaluating online discussion forums
Avila, M., Nallathambi, K., Richey, C., & Mwaikambo, L. (2011) · Knowledge Management & E-Learning An International Journal · doi:10.34105/j.kmel.2011.03.041 · Cited by 6 · see citing works
Zarinah, M. K., & Siti, S. S. (2009) · The University of Malaya Research Repository (University of Malaya) · Cited by 6 · see citing works
Requirements Elicitation With Focus Groups: Lessons Learnt
Farinha, C. S. E. S., & da Silva, M. M. (2013) · European Conference on Information Systems · Cited by 5 · see citing works
Cheng, E., & Arthur, D. (2002) · Journal of the Association for Information Systems · Cited by 5 · see citing works
Fostering innovation and collaboration.
Ansell, C. (2007) · PubMed · Cited by 5 · see citing works
Carter, W., & Fisher, S. S. (2004) · ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE) · doi:10.1145/1067343.1067399 · Cited by 4 · see citing works
Alfano, I., Carini, M., & Gabriele, L. (2012) · Annals of information systems · doi:10.1007/978-1-4614-3634-8_8 · Cited by 3 · see citing works
CoPEP: A Development Process Model for Community Platforms for Cancer Patients
Arnold, Y., Leimeister, J. M., & Krcmar, H. (2003) · Journal of the Association for Information Systems · Cited by 3 · see citing works
Vasconcellos-Silva, P. R., Carvalho, D., Lucena, C., & ANAMNESIS (2013) · Interactive Journal of Medical Research · doi:10.2196/ijmr.2384 · Cited by 2 · see citing works
Making it work 2: using a virtual community to focus on rural health issues
Godden, D., & Aaraas, I. J. (2006) · Rural and Remote Health · doi:10.22605/rrh540 · Cited by 2 · see citing works
Flagged first, then by citations
1 of the 87 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Effectiveness of ICT-based intimate partner violence interventions: a systematic review
El Morr, C., & Layal, M. (2020) · BMC Public Health · doi:10.1186/s12889-020-09408-8 · Cited by 90 · see citing works
Yogeswaran, V., & El Morr, C. (2021) · BMC Public Health · doi:10.1186/s12889-021-12341-z · Cited by 54 · see citing works
“It is important to note that frameworks to evaluate the successful implementation and adoption of online communities do exist [57].”
BackgroundAudrain‐Pontevia, A., Menvielle, L., & Ertz, M. (2019) · Journal of Medical Internet Research · doi:10.2196/14006 · Cited by 37 · see citing works
Christo, E. M., & Layal, M. (2019) · Studies in health technology and informatics · doi:10.3233/978-1-61499-951-5-103 · Cited by 34 · see citing works
El Morr, C., Maret, P., Muhlenbach, F., Dharmalingam, D., Tadesse, R., Creighton, A., Kundi, B., Buettgen, A., Mgwigwi, T., Dinca-Panaitescu, S., et al. (2021) · JMIR Formative Research · doi:10.2196/33335 · Cited by 12 · see citing works
AUX and UX Evaluation of User Tools in Social Networks
Sánchez-Adame, L. M., Mendoza, S., González-Beltrán, B. A., Rodríguez, J., & Meneses‐Viveros, A. (2018) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1109/wi.2018.0-101 · Cited by 5 · see citing works
Mental Health for Medical Students, what do we know today?
Yogeswaran, V., & Morr, C. E. (2022) · Procedia Computer Science · doi:10.1016/j.procs.2021.12.245 · Cited by 4 · see citing works
UX Evaluation Over Time: User Tools in Social Networks
Sánchez-Adame, L. M., Mendoza, S., González-Beltrán, B. A., Rodríguez, J., & Meneses‐Viveros, A. (2018) · International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE) · doi:10.1109/iceee.2018.8533950 · Cited by 2 · see citing works
Towards an AUX Evaluation Framework for User Tools in Virtual Communities
Sánchez-Adame, L. M., Mendoza, S., González-Beltrán, B. A., Meneses‐Viveros, A., & Rodríguez, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-99504-5_3
- Authors
- El Morr, C., & Eftychiou, L.
- Affiliations
- York University (CA)
- ORCID
- Eftychiou, Linda
- Venue
- The Digitization of Healthcare
- Publisher
- Palgrave Macmillan UK
- Published
- 2017
- Pages
- 99-118
- OpenAlex
- W2744742119
- Cited by
- 9
- References
- 87
- Citations by year
- 2022: 1 · 2021: 2 · 2020: 1 · 2019: 2 · 2018: 3
- FWCI
- 10.90
- Type
- book chapter
- Language
- en
- Open access
- closed
- Locations
- Palgrave Macmillan UK eBooks (publishedVersion)
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- evaluation framework, usability, health outcomes, information system evaluation, online health communities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
