Evaluation of generic medical information accessed via mobile phones at the point of care in resource-limited settings
Open accessAbstract
OBJECTIVE: Many mobile phone resources have been developed to increase access to health education in the developing world, yet few studies have compared these resources or quantified their performance in a resource-limited setting. This study aims to compare the performance of resident physicians in answering clinical scenarios using PubMed abstracts accessed via the PubMed for Handhelds (PubMed4Hh) website versus medical/drug reference applications (Medical Apps) accessed via software on the mobile phone. METHODS: A two-arm comparative study with crossover design was conducted. Subjects, who were resident physicians at the University of Botswana, completed eight scenarios, each with multi-part questions. The primary outcome was a grade for each question. The primary independent variable was the intervention arm and other independent variables included residency and question. RESULTS: Within each question type there were significant differences in 'percentage correct' between Medical Apps and PubMed4Hh for three of the six types of questions: drug-related, diagnosis/definitions, and treatment/management. Within each of these question types, Medical Apps had a higher percentage of fully correct responses than PubMed4Hh (63% vs 13%, 33% vs 12%, and 41% vs 13%, respectively). PubMed4Hh performed better for epidemiologic questions. CONCLUSIONS: While mobile access to primary literature remains important and serves an information niche, mobile applications with condensed content may be more appropriate for point-of-care information needs. Further research is required to examine the specific information needs of clinicians in resource-limited settings and to evaluate the appropriateness of current resources in bridging location- and context-specific information gaps.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Almost all included studies highlighted that mHealth interventions facilitate access to previously unavailable services.
“In fact, almost all included studies highlighted this characteristic feature of mHealth interventions [20, 36–51].”
From A realist review of mobile phone-based health interventions for non-communicable disease management in sub-Saharan Africa (2017) · cites this paper as [44]
Results suggested that the use of medical apps led to a higher percentage of fully correct responses compared to PubMed (range 33-63% vs. 12-13%: Apps vs. PubMed).
“Results suggested that the use of medical apps led to a higher percentage of fully correct responses compared to PubMed (range 33–63% vs. 12–13%: Apps vs. PubMed).”
From Assessing the impact of mHealth interventions in low- and middle-income countries – what has been shown to work? (2014) · cites this paper as [15]
In Botswana, short-term learning outcome analysis has shown benefits of access to medical apps with condensed medical information, but does not analyse long-term benefits to knowledge retention.
“Short-term learning outcome analysis has shown the benefits of access to medical apps with condensed medical information in Botswana, but does not analyse the long-term benefits to knowledge retention (15).”
From Assessing the impact of mHealth interventions in low- and middle-income countries – what has been shown to work? (2014) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The International Telecommunication Union
Jones, R. W. (2002) · Digest - IEEE Antennas and Propagation Society. International Symposium · doi:10.1109/aps.1997.631791 · Cited by 1,910 · see citing works
Health professions education: a bridge to quality
Greiner, A. C., Knebel, E., Institute of Medicine (U.S.), & Institute of Medicine (U.S.) (2004) · Choice Reviews Online · doi:10.5860/choice.41-3474 · Cited by 1,783 · see citing works
What clinical information do doctors need?
Smith, R. W. (1996) · BMJ · doi:10.1136/bmj.313.7064.1062 · Cited by 593 · see citing works
Patrick, K., Griswold, W. G., Raab, F., & Intille, S. S. (2008) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2008.05.001 · Cited by 479 · see citing works
(2006) · Globalization and Health · doi:10.1186/1744-8603-2-9 · Cited by 467 · see citing works
Zurovac, D., Sudoi, R. K., Akhwale, W. S., Ndiritu, M., Hamer, D. H., Rowe, A. K., & Snow, R. W. (2011) · The Lancet · doi:10.1016/s0140-6736(11)60783-6 · Cited by 403 · see citing works
Medical apps for smartphones: lack of evidence undermines quality and safety
Buijink, A. W. G., Visser, B. J., & Marshall, L. (2013) · Evidence Based Medicine · doi:10.1136/eb-2012-100885 · Cited by 303 · see citing works
How do primary care physicians seek answers to clinical questions? A literature review.
Coumou, H. C. H., & Meijman, F. J. (2006) · PubMed · Cited by 255 · see citing works
DeRenzi, B., Lesh, N. B., Parikh, T., Sims, C., Maokla, W., Chemba, M., Hamisi, Y., Schellenberg, D. M., Mitchell, M., & Borriello, G. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357174 · Cited by 166 · see citing works
Use of handheld computers in medical education
Kho, A., Henderson, L. E., Dressler, D. D., & Kripalani, S. B. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00444.x · Cited by 157 · see citing works
Clinician Use of a Palmtop Drug Reference Guide
Rothschild, J. M. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1001 · Cited by 154 · see citing works
Pakenham-Walsh, N. M., & Bukachi, F. (2009) · Human Resources for Health · doi:10.1186/1478-4491-7-30 · Cited by 144 · see citing works
Leung, G. M., Johnston, J. M., Tin, K. Y. K., Wong, I. O. L., Ho, L.-M., Lam, W. W. T., & Lam, T.-H. (2003) · BMJ · doi:10.1136/bmj.327.7423.1090 · Cited by 130 · see citing works
Family physicians' use of medical abstracts to guide decision making: style or substance?
Barry, H. C., Ebell, M. H., Shaughnessy, A. F., Slawson, D. C., & Nietzke, F. (2001) · PubMed · Cited by 113 · see citing works
Internet access in Africa: empirical evidence from Kenya and Nigeria
Oyelaran‐Oyeyinka, B., & Adeya, C. N. (2003) · Telematics and Informatics · doi:10.1016/s0736-5853(03)00023-6 · Cited by 111 · see citing works
Pediatricians’ Use of and Attitudes About Personal Digital Assistants
Carroll, A. E., & Christakis, D. (2004) · PEDIATRICS · doi:10.1542/peds.113.2.238 · Cited by 104 · see citing works
Tonks, A., & Smith, R. (1996) · BMJ · doi:10.1136/bmj.313.7055.438 · Cited by 95 · see citing works
Smith, R. W. (2003) · BMJ · doi:10.1136/bmj.326.7383.238 · Cited by 87 · see citing works
Lancaster, F. W., & Warner, A. J. (1993) · Medical Entomology and Zoology · Cited by 84 · see citing works
León, S. A., Fontelo, P. A., Green, L., Ackerman, M. J., & Liu, F. (2007) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-7-5 · Cited by 70 · see citing works
Self-citationA comparative case study of two models of a clinical informaticist service
Greenhalgh, T. (2002) · BMJ · doi:10.1136/bmj.324.7336.524 · Cited by 68 · see citing works
Mobile learning for HIV/AIDS healthcare worker training in resource-limited settings
Zolfo, M., Iglesias, D., Kiyan, C., Echevarria, J., Fucay, L., Llacsahuanga, E., de Waard, I., Suàrez, V., Llaque, W., & Lynen, L. (2010) · AIDS Research and Therapy · doi:10.1186/1742-6405-7-35 · Cited by 67 · see citing works
Health information for the developing world
Kale, R. (1994) · BMJ · doi:10.1136/bmj.309.6959.939 · Cited by 62 · see citing works
Johnston, J. M., Leung, G. M., Tin, K. Y. K., Ho, L., Lam, W., & Fielding, R. (2004) · Medical Education · doi:10.1111/j.1365-2929.2004.01842.x · Cited by 55 · see citing works
Physicians' use of the personal digital assistant (PDA) in clinical decision making.
Dee, C. R., Teolis, M. G., & Todd, A. D. (2005) · PubMed · Cited by 47 · see citing works
Assessing the global accessibility of the Internet
Ngini, C. U., Furnell, S., & Ghita, B. (2002) · Internet Research · doi:10.1108/10662240210438399 · Cited by 42 · see citing works
Recent Advances in Mobile Technology Benefit Global Health, Research, and Care
Hampton, T. (2012) · JAMA · doi:10.1001/jama.2012.4465 · Cited by 40 · see citing works
Global Information and Africa: the telecommunications infrastructure for cyberspace
Afullo, T. J. O. (2000) · Library Management · doi:10.1108/01435120010317507 · Cited by 36 · see citing works
Oliver, K. B., Dalrymple, P. W., Lehmann, H. P., McClellan, D. A., Robinson, K. A., & Twose, C. (2008) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.96.1.50 · Cited by 20 · see citing works
Lawn, J. E., Blencowe, H. K., Pattinson, R., Stanton, C., & Cousens, S. N. (2011) · doi:10.1002/9781444398038.ch2 · Cited by 18 · see citing works
Consensus abstracts for evidence-based medicine
Fontelo, P. A. (2011) · BMJ Evidence-Based Medicine · doi:10.1136/ebm20003 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
1 of the 32 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- USAID. International m4Ed4Dev Symposium. http://usaid.stanford.edu/node/36 (accessed Jan 2012).
Cited by
Sorted by citations
Hall, C. S., Fottrell, E., Wilkinson, S., & Byass, P. (2014) · Global Health Action · doi:10.3402/gha.v7.25606 · Cited by 384 · see citing works
Hussain, M., Al-Haiqi, A., Zaidan, A. A., Zaidan, B. B., Kiah, M. L. M., Anuar, N. B., & Abdulnabi, M. S. H. (2015) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.08.015 · Cited by 130 · see citing works
Clinical information seeking behavior of physicians: A systematic review
Daei, A., Soleymani, M. R., Ashrafi‐rizi, H., Zargham‐Boroujeni, A., & Kelishadi, R. (2020) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2020.104144 · Cited by 119 · see citing works
How to identify, assess and utilise mobile medical applications in clinical practice
Aungst, T. D., Clauson, K. A., Misra, S., Lewis, T., & Husain, I. (2014) · International Journal of Clinical Practice · doi:10.1111/ijcp.12375 · Cited by 108 · see citing works
Opoku, D., Stephani, V., & Quentin, W. (2017) · BMC Medicine · doi:10.1186/s12916-017-0782-z · Cited by 97 · see citing works
Karageorgos, G. M., Andreadis, I., Psychas, K., Mourkousis, G., Kiourti, A., Lazzi, G., & Nikita, K. S. (2018) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2018.2868896 · Cited by 43 · see citing works
The need for a telemedicine strategy for Botswana? A scoping review and situational assessment
Ncube, B., Mars, M., & Scott, R. E. (2020) · BMC Health Services Research · doi:10.1186/s12913-020-05653-0 · Cited by 30 · see citing works
Use of Mobile Devices to Access Resources Among Health Professions Students: A Systematic Review
Mi, M., Wu, W. G., Qiu, M., Zhang, Y., Wu, L., & Li, J. (2016) · Medical Reference Services Quarterly · doi:10.1080/02763869.2016.1117290 · Cited by 24 · see citing works
McHenry, M. S., Fischer, L. J., Chun, Y., & Vreeman, R. (2017) · Global Health Promotion · doi:10.1177/1757975917715035 · Cited by 24 · see citing works
Johnson-Barlow, E., Emani, V. K., & Ren, J. (2016) · JMIR mhealth and uhealth · doi:10.2196/mhealth.6189 · Cited by 19 · see citing works
Kebaetse, M. B., Nkomazana, O., & Haverkamp, C. (2014) · The Electronic Journal of e-Learning · Cited by 18 · see citing works
The case for mindfulness interventions for traumatic stress in high violence, low resource settings
Pillay, K., & Eagle, G. (2019) · Current Psychology · doi:10.1007/s12144-019-00177-1 · Cited by 1 · see citing works
A Pictorial Schema for a Comprehensive User-oriented Identification of Medical Apps
Marceglia, S. R. F., Pinciroli, F., & Bonacina, S. (2014) · Methods of Information in Medicine · doi:10.3414/me13-01-0093 · Cited by 15 · see citing works
Bock, A., Elvers, D., Peters, F., Kramer, C., Kniha, K., Hölzle, F., Spreckelsen, C., & Modabber, A. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02889-0 · Cited by 12 · see citing works
Information needs of Botswana health care workers and perceptions of wikipedia
Park, E., Masupe, T., Joseph, J., Foster, A. H., Chavez, A., Jammalamadugu, S. B., Marek, A. J., Arumala, R., Ketshogileng, D., Littman–Quinn, R., et al. (2016) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2016.07.013 · Cited by 12 · see citing works
Self-citationShaarani, I., Berjaoui, H., Daher, A., Khalil, M., Rifai, A. E. R. A., Saati, R., & Antoun, J. (2019) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2019.07.015 · Cited by 10 · see citing works
Informatics and Technology in Resident Education
Niehaus, W. (2017) · PM&R · doi:10.1016/j.pmrj.2017.02.010 · Cited by 10 · see citing works
Haluza, D., & Hofer, F. (2019) · Health Informatics Journal · doi:10.1177/1460458219888420 · Cited by 11 · see citing works
Chreiman, K. M., Prakash, P. S., Martin, N. D., Kim, P. K., Mehta, S. D., McGinnis, K., Gallagher, J. J., & Reilly, P. M. (2017) · Trauma Surgery & Acute Care Open · doi:10.1136/tsaco-2017-000085 · Cited by 9 · see citing works
What is the impact of Apps in medical education? A study of CAPSULE, a case-based learning App
Sadler, J., Wright, J. E., Vincent, T., Kurka, T., & Howlett, D. C. (2020) · BMJ Simulation & Technology Enhanced Learning · doi:10.1136/bmjstel-2020-000593 · Cited by 8 · see citing works
Mohammadzadeh, N., & Safdari, R. (2016) · InTech eBooks · doi:10.5772/64704 · Cited by 7 · see citing works
Koh, E., Maranga, A., Yane, T., Ndlovu, K., Jereni, B. H., Nwako-Mohamadi, M. K., Kovarik, C., Forrestel, A. K., & Williams, V. L. (2022) · JMIR Dermatology · doi:10.2196/35254 · Cited by 8 · see citing works
Self-citationRecommendations for Developing a Telemedicine Strategy for Botswana: A Meta-Synthesis
Ncube, B., Mars, M., & Scott, R. E. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20186718 · Cited by 5 · see citing works
Senya, K., Ibrahim, A., Lindong, I., & Addo-Lartey, A. A. (2016) · Global Social Welfare · doi:10.1007/s40609-016-0078-9 · Cited by 6 · see citing works
The use of a medical application improves the diagnosis of acute kidney injury: A pre-post study
Gaspar, A., Iturricha-Cáceres, M. F., Macedo, E. M., Mehta, R. L., & Granado, R. C. (2022) · Frontiers in Medicine · doi:10.3389/fmed.2022.817387 · Cited by 5 · see citing works
Infobutton usage in Patient Portal MyHealth.
Long, J., Hulse, N. C., & Tao, C. (2015) · Europe PMC (PubMed Central) · Cited by 5 · see citing works
Perception of Patients on Usage of Smartphones by Health Care Professionals during Clinic Hours
Bolarinde, S. O. (2021) · INTERNATIONAL JOURNAL OF MULTIDISCIPLINARY RESEARCH AND ANALYSIS · doi:10.47191/ijmra/v4-i6-02 · Cited by 2 · see citing works
Building capacity in a health sciences library to support global health projects
Lackey, M. J., Swogger, S. E., & McGraw, K. A. (2014) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.102.2.006 · Cited by 2 · see citing works
Ahmed, S. (2017) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 2 · see citing works
Anjusha, I. B., Nayak, V. K. R., Nalini, Y. C., & Basavarajegowda, A. (2024) · Apollo Medicine · doi:10.1177/09760016241277640 · Cited by 1 · see citing works
Hulse, N. C., Long, J., Xu, X., & Tao, C. (2014) · PubMed Central · Cited by 1 · see citing works
An mHealth app counseling patients and general practitioners about Multiple Myeloma
Lanzola, G., Boninsegna, R., Losiouk, E., Zini, E. M., Quaglini, S., Ferretti, V. V., & Corso, A. (2017) · IEEE International Forum on Research and Technology for Society and Industry (RTSI) · doi:10.1109/rtsi.2017.8065889 · Cited by 1 · see citing works
Sources of drug information in neonatal guidelines of low- and middle-income countries†‡
Japs, K., Dennehy, C. E., Stoffella, S., Faldasz, J. D., & Bress, J. (2019) · International Journal of Pharmacy Practice · doi:10.1111/ijpp.12502 · Cited by 1 · see citing works
Al‐Worafi, Y. M. (2022) · CRC Press eBooks · doi:10.1201/9781003230458-20 · Cited by 1 · see citing works
Modelling of physicians’ clinical information-seeking behaviour in Iran: a grounded theory study
Daei, A., Soleymani, M. R., Zargham‐Boroujeni, A., Kelishadi, R., & Ashrafi‐rizi, H. (2024) · BMJ Open · doi:10.1136/bmjopen-2023-080602 · Cited by 1 · see citing works
Evaluation of WhatsApp as a Platform for Teledermatology in Botswana (Preprint)Preprint
Sawka, E., Maranga, A., Yane, T., Ndlovu, K., Jereni, B. H., Nwako-Mohamadi, M. K., Kovarik, C., Forrestel, A. K., & Williams, V. L. (2021) · doi:10.2196/preprints.35254
Self-citationBock, A., Elvers, D., Peters, F., Kramer, C., Kniha, K., Hölzle, F., Spreckelsen, C., & Modabber, A. (2021) · Figshare · doi:10.6084/m9.figshare.16557294
- Authors
- Goldbach, H., Chang, A. Y., Kyer, A., Ketshogileng, D., Taylor, L., Chandra, A., Dacso, M., Kung, S.-J., Rijken, T., Fontelo, P., Littman-Quinn, R., Seymour, A. K., & Kovarik, C. L.
- Affiliations
- University of Pennsylvania (US)
- University of Botswana (BW)
- The University of Texas Medical Branch at Galveston (US)
- Children's Hospital of Philadelphia (US)
- United States National Library of Medicine (US)
- Corresponding author
- Kovarik, Carrie L
- ORCID
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2014-01
- Volume
- 21
- Issue
- 1
- Pages
- 37-42
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3912712
- OpenAlex
- W2111070168
- Cited by
- 37
- References
- 32
- Citations by year
- 2024: 2 · 2023: 1 · 2022: 3 · 2021: 4 · 2020: 3 · 2019: 4 · 2018: 1 · 2017: 6 · 2016: 5 · 2015: 2 · 2014: 6
- FWCI
- 7.85
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- University of Pennsylvania
- Fondation Orange
- National Institutes of Health
- National Institute of Allergy and Infectious Diseases (P30AI045008)
- U.S. National Library of Medicine
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Information Systems › Mobile Learning in Education
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- mobile health, point-of-care information, mobile health apps, medical residents, clinical decision-making, resource-limited settings, health education
- MeSH terms
- Botswana, Health Resources, Humans, Internship and Residency, Software, Point-of-Care Systems, Computers, Handheld, PubMed, Cell Phone
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
