Comparing the use of an online expert health network against common information sources to answer health questions
Open accessAbstract
BACKGROUND: Many workers have questions about occupational safety and health (OSH). It is unknown whether workers are able to find correct, evidence-based answers to OSH questions when they use common information sources, such as websites, or whether they would benefit from using an easily accessible, free-of-charge online network of OSH experts providing advice. OBJECTIVE: To assess the rate of correct, evidence-based answers to OSH questions in a group of workers who used an online network of OSH experts (intervention group) compared with a group of workers who used common information sources (control group). METHODS: In a quasi-experimental study, workers in the intervention and control groups were randomly offered 2 questions from a pool of 16 standardized OSH questions. Both questions were sent by mail to all participants, who had 3 weeks to answer them. The intervention group was instructed to use only the online network ArboAntwoord, a network of about 80 OSH experts, to solve the questions. The control group was instructed that they could use all information sources available to them. To assess answer correctness as the main study outcome, 16 standardized correct model answers were constructed with the help of reviewers who performed literature searches. Subsequently, the answers provided by all participants in the intervention (n = 94 answers) and control groups (n = 124 answers) were blinded and compared with the correct model answers on the degree of correctness. RESULTS: Of the 94 answers given by participants in the intervention group, 58 were correct (62%), compared with 24 of the 124 answers (19%) in the control group, who mainly used informational websites found via Google. The difference between the 2 groups was significant (rate difference = 43%, 95% confidence interval [CI] 30%-54%). Additional analysis showed that the rate of correct main conclusions of the answers was 85 of 94 answers (90%) in the intervention group and 75 of 124 answers (61%) in the control group (rate difference = 29%, 95% CI 19%-40%). Remarkably, we could not identify differences between workers who provided correct answers and workers who did not on how they experienced the credibility, completeness, and applicability of the information found (P > .05). CONCLUSIONS: Workers are often unable to find correct answers to OSH questions when using common information sources, generally informational websites. Because workers frequently misjudge the quality of the information they find, other strategies are required to assist workers in finding correct answers. Expert advice provided through an online expert network can be effective for this purpose. As many people experience difficulties in finding correct answers to their health questions, expert networks may be an attractive new source of information for health fields in general.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous research suggests that patients frequently misjudge the quality of online health information.
“Previous research suggests that patients frequently misjudge the quality of online health information, and many HCPs fear that social media sites spread misinformation and eclipse provider input into individuals’ health decisions.[20–21,33]”
From Perceived healthcare provider reactions to patient and caregiver use of online health communities (2014) · cites this paper as [33]
Many HCPs fear that social media sites spread misinformation and eclipse provider input into individuals' health decisions.
“Previous research suggests that patients frequently misjudge the quality of online health information, and many HCPs fear that social media sites spread misinformation and eclipse provider input into individuals’ health decisions.[20–21,33]”
From Perceived healthcare provider reactions to patient and caregiver use of online health communities (2014) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Evidence-Based Medicine: How to Practice and Teach EBM
Sackett, D. L. (2000) · Cited by 5,950 · see citing works
Eysenbach, G., Powell, J., Kuss, O., & Sa, E.-R. (2002) · JAMA · doi:10.1001/jama.287.20.2691 · Cited by 1,871 · see citing works
Research Methods in Physical Activity
Thomas, J. R., & Nelson, J. K. (1990) · Cited by 1,562 · see citing works
Evidence-based medicine : how to practice and teach EBM
Straus, S. E. (2005) · Cited by 1,378 · see citing works
Research methods in physical activity
Maughan, R. (2016) · Journal of Sports Sciences · doi:10.1080/02640414.2016.1169035 · Cited by 1,256 · see citing works
Predictors of answer quality in online Q&A sites
Harper, F. M., Raban, D. R., Rafaeli, S., & Konstan, J. A. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357191 · Cited by 413 · see citing works
Googling for a diagnosis—use of Google as a diagnostic aid: internet based study
Tang, H., & Ng, J. (2006) · BMJ · doi:10.1136/bmj.39003.640567.ae · Cited by 386 · see citing works
Adolescents Searching for Health Information on the Internet: An Observational Study
Hansen, D. L., Derry, H. A., Resnick, P., & Richardson, C. R. (2003) · Journal of Medical Internet Research · doi:10.2196/jmir.5.4.e25 · Cited by 257 · see citing works
Emotion rating from short blog texts
Gill, A. J., Gergle, D., French, R. M., & Oberlander, J. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357229 · Cited by 232 · see citing works
Peterson, G., Aslani, P., & Williams, K. A. (2003) · Journal of Medical Internet Research · doi:10.2196/jmir.5.4.e33 · Cited by 218 · see citing works
Helping patients access high quality health information
Shepperd, S., Charnock, D., & Gann, B. (1999) · BMJ · doi:10.1136/bmj.319.7212.764 · Cited by 203 · see citing works
The Impact of Inaccurate Internet Health Information in a Secondary School Learning Environment
Kortum, P. T., Edwards, C. C., & Richards-Kortum, R. R. (2008) · Journal of Medical Internet Research · doi:10.2196/jmir.986 · Cited by 114 · see citing works
Kingsley, K., Galbraith, G. M. P., Herring, M., Stowers, E. D., Stewart, T., & Kingsley, K. V. (2011) · BMC Medical Education · doi:10.1186/1472-6920-11-17 · Cited by 97 · see citing works
Inleiding in evidence-based medicine : klinisch handelen gebaseerd op bewijsmateriaal
Offringa, M., Assendelft, W. J., & Scholten, R. J. P. M. (2008) · Data Archiving and Networked Services (DANS) · Cited by 88 · see citing works
Wikipedia and osteosarcoma: a trustworthy patients' information?
Leithner, A., Maurer-Ertl, W., Glehr, M., Friesenbichler, J., Leithner, K., & Windhager, R. H. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.004507 · Cited by 85 · see citing works
Wikipedia as an evidence source for nursing and healthcare students
Haigh, C. A. (2010) · Nurse Education Today · doi:10.1016/j.nedt.2010.05.004 · Cited by 79 · see citing works
Information Needs and Visitors' Experience of an Internet Expert Forum on Infertility
Himmel, W., Meyer, J., Kochen, M. M., & Michelmann, H. W. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.2.e20 · Cited by 71 · see citing works
Speed, accuracy, and confidence in Google, Ovid, PubMed, and UpToDate: results of a randomised trial
Thiele, R. H., Poiro, N. C., Scalzo, D. C., & Nemergut, E. C. (2010) · Postgraduate Medical Journal · doi:10.1136/pgmj.2010.098053 · Cited by 64 · see citing works
Factors predictive of successful learning in postgraduate medical education
Smits, P. B. A., Verbeek, J. H. A. M., Nauta, M. C. E., ten Cate, O., Metz, J. C. M., & van Dijk, F. J. H. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01846.x · Cited by 62 · see citing works
Schaafsma, F. G., Verbeek, J. H. A. M., Hulshof, C. T. J., & van Dijk, F. J. H. (2005) · BMC Health Services Research · doi:10.1186/1472-6963-5-59 · Cited by 50 · see citing works
Iske, P. L., & Boersma, W. (2005) · Journal of Knowledge Management · doi:10.1108/13673270510583018 · Cited by 49 · see citing works
Burg, M. A., Zebrack, B., Walsh, K., Maramaldi, P., Lim, J.-W., Smolinski, K. M., & Lawson, K. (2010) · Social Work in Health Care · doi:10.1080/00981380903018470 · Cited by 47 · see citing works
Gender and achievement in clinical medical students: a path analysis
de Saintonge, D. M. C., & Dunn, D. M. (2001) · Medical Education · doi:10.1046/j.1365-2923.2001.01043.x · Cited by 44 · see citing works
Ilić, D. (2003) · Human Reproduction · doi:10.1093/humrep/deg154 · Cited by 42 · see citing works
Health information literacy meets evidence-based practice
Schardt, C. M. (2011) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.99.1.001 · Cited by 39 · see citing works
A Knowledge Infrastructure for Occupational Safety and Health
van Dijk, F. J. H., Verbeek, J. H. A. M., Hoving, J. L., & Hulshof, C. T. J. (2010) · Journal of Occupational and Environmental Medicine · doi:10.1097/jom.0b013e318202f2c5 · Cited by 37 · see citing works
Enhancing evidence-based advice of occupational health physicians
Schaafsma, F. G., Hugenholtz, N. I. R., de Boer, A. G. E. M., Smits, P. B. A., Hulshof, C. T. J., & van Dijk, F. J. H. (2007) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.1156 · Cited by 30 · see citing works
Hugenholtz, N. I. R., Schaafsma, F. G., Nieuwenhuijsen, K., & van Dijk, F. J. H. (2008) · International Archives of Occupational and Environmental Health · doi:10.1007/s00420-008-0315-3 · Cited by 29 · see citing works
Knowledge Infrastructure Needed for Occupational Healtha
Hugenholtz, N. I. R., Schreinemakers, J. F., A-TJAK, M. A., & van Dijk, F. J. H. (2007) · Industrial Health · doi:10.2486/indhealth.45.13 · Cited by 25 · see citing works
Hugenholtz, N. I. R., Nieuwenhuijsen, K., Sluiter, J. K., & van Dijk, F. J. H. (2009) · BMC Health Services Research · doi:10.1186/1472-6963-9-18 · Cited by 23 · see citing works
Information about infantile hemangiomas on the Internet: How accurate is it?
Minzer-Conzetti, K., Garzón, M. C., Haggstrom, A. N., Horii, K. A., Mancini, A. J., Morel, K. D., Newell, B. D., Nopper, A. J., & Frieden, I. J. (2007) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2007.06.038 · Cited by 19 · see citing works
Rhebergen, M. D. F., Hulshof, C. T. J., Lenderink, A. F., & van Dijk, F. J. H. (2010) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-10-63 · Cited by 18 · see citing works
Occupational health needs in small industry in New Zealand: preferred sources of information
Dryson, E. W. (1993) · Occupational Medicine · doi:10.1093/occmed/43.4.176 · Cited by 17 · see citing works
Whelan, A. M., Jurgens, T. M., Bowles, S. K., & Doyle, H. (2009) · Annals of Pharmacotherapy · doi:10.1345/aph.1l688 · Cited by 16 · see citing works
Literacy, consumer informatics, and health care outcomes: Interrelations and implications.
Anton, B. B., & Nelson, R. (2006) · PubMed · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Moorhead, S. A., Hazlett, D. E., Harrison, L., Carroll, J. K., Irwin, A., & Hoving, C. (2013) · Journal of Medical Internet Research · doi:10.2196/jmir.1933 · Cited by 2,517 · see citing works
Perceived healthcare provider reactions to patient and caregiver use of online health communities
Rupert, D. J., Moultrie, R. R., Read, J. G., Amoozegar, J. B., Bornkessel, A. S., O’Donoghue, A. C., & Sullivan, H. W. (2014) · Patient Education and Counseling · doi:10.1016/j.pec.2014.05.015 · Cited by 149 · see citing works
A Scoping Review of Social Media Use in Social Work Practice
Chan, C. (2015) · Journal of Evidence-Informed Social Work · doi:10.1080/23761407.2015.1052908 · Cited by 82 · see citing works
Alhuwail, D., AlMeraj, Z., & Boujarwah, F. A. (2018) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-018-0660-4 · Cited by 47 · see citing works
Moick, M., & Terlutter, R. (2012) · Medicine 2.0 · doi:10.2196/med20.1996 · Cited by 37 · see citing works
van Dijk, F. J. H., Bubaš, M., & Smits, P. B. A. (2015) · Annals of Global Health · doi:10.1016/j.aogh.2015.08.023 · Cited by 36 · see citing works
Bientzle, M., Griewatz, J., Kimmerle, J., Küppers, J., Creß, U., & Lammerding‐Koeppel, M. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4597 · Cited by 34 · see citing works
Bornkessel, A. S., Furberg, R., & Lefebvre, R. C. (2014) · Current Cardiology Reports · doi:10.1007/s11886-014-0504-5 · Cited by 34 · see citing works
Reininger, B. M., Mecca, L. P., Stine, K., Schultz, K. C., Ling, L., & Halpern, D. V. (2013) · JMIR Research Protocols · doi:10.2196/resprot.2573 · Cited by 20 · see citing works
Ceballos, D. M., Herrick, R. F., Carreón, T., Nguyen, V. T., Chu, M. T., Sadowski, J. P., Blumenthal, H., & Morata, T. C. (2021) · INQUIRY The Journal of Health Care Organization Provision and Financing · doi:10.1177/00469580211035735 · Cited by 15 · see citing works
Can Workers Answer Their Questions about Occupational Safety and Health: Challenges and Solutions
Rhebergen, M. D. F., van Dijk, F. J. H., & Hulshof, C. T. J. (2012) · Industrial Health · doi:10.2486/indhealth.ms1344 · Cited by 13 · see citing works
Analysis of Free Online Physician Advice Services
Cohen, R., Elhadad, M., & Birk, O. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059963 · Cited by 9 · see citing works
Hazard information needs and information seeking in French workers
Rollin, L., Ladner, J., Gislard, A., Monfrin, F., Larchevesque, J. Y., Deslandes, P., & Gehanno, J.-F. (2013) · Occupational Medicine · doi:10.1093/occmed/kqt091 · Cited by 8 · see citing works
Walter, B., Schmid, R., & von Delius, S. (2017) · Zeitschrift für Gastroenterologie - German Journal of Gastroenterology · doi:10.1055/s-0043-106860 · Cited by 4 · see citing works
An examination of the content of diabetes websites targeting Arabic speakers
Alhadreti, O. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2023.105242 · Cited by 4 · see citing works
FORMATION OF ENTERPRISE EMPLOYEES’ RISK-ORIENTED THINKING OF SAFE WORK
Чеберячко, С. І., Hilpert, V., Cheberiachko, Y., Shaikhlislamova, I., & Borovytskyi, O. (2021) · Labour protection problems in Ukraine · doi:10.36804/nndipbop.37-1.2021.9-15 · Cited by 2 · see citing works
van Dijk, F. J. H. (2015) · Occupational and Environmental Medicine · doi:10.1136/oemed-2014-102545 · Cited by 2 · see citing works
(2) De oogst van het programma: kennis, instrumenten en interventies
van Dijk, F. J. H., & Schaufeli, W. B. (2014) · Tijdschrift voor bedrijfs- en verzekeringsgeneeskunde · doi:10.1007/s12498-014-0182-7 · Cited by 1 · see citing works
Knowledge Transfer and Exchange in Work and Health Research
Eerd, D. V. (2016) · UWSpace (University of Waterloo)
Rezaeiaghdam, A. (2022) · Queensland University of Technology · doi:10.5204/thesis.eprints.231220
From Compliance to Comprehension: A Work Health and Safety Literacy Framework for Safer WorkplacesPreprint
Ong, B., & Neto, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6505415
- Authors
- Rhebergen, M. D., Lenderink, A. F., van Dijk, F. J., & Hulshof, C. T.
- Affiliations
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Rhebergen, Martijn DF
- ORCID
- Hulshof, Carel TJ
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2012-02-02
- Volume
- 14
- Issue
- 1
- Pages
- e9
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC3374530
- OpenAlex
- W2055142640
- Cited by
- 21
- References
- 36
- Citations by year
- 2026: 1 · 2023: 1 · 2022: 1 · 2021: 2 · 2018: 1 · 2017: 1 · 2016: 1 · 2015: 4 · 2014: 3 · 2013: 4 · 2012: 2
- FWCI
- 7.38
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Instituut Gak
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Literacy and Information Accessibility
- Health Sciences › Health Professions › Occupational Therapy › Occupational Health and Safety in Workplaces
- Physical Sciences › Computer Science › Information Systems › Expert finding and Q&A systems
- Keywords
- occupational health and safety, health information seeking, information quality, information credibility, information websites, Google search, occupational health advice, health literacy, information overload
- MeSH terms
- Information Services, Online Systems, Occupational Health, Access to Information
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
