Point-of-care mobile application to guide health care professionals in conducting substance use screening and intervention: A mixed-methods user experience study
Open accessAbstract
Background Well-documented barriers have limited the widespread, sustained adoption of screening and intervention for substance use problems in health care settings. mHealth applications may address provider-related barriers; however, there is limited research on development and user experience of such applications. Objective This user experience study examines a provider-focused point-of-care app for substance use screening and intervention in health care settings. Method This mixed-methods study included think-aloud tasks, task success ratings, semistructured interviews, and usability questionnaires (e.g., System Usability Scale [SUS]) to examine user experience among 12 health coaches who provide substance use services in emergency department and primary care settings. Results The average rate of successful task completion was 94% and the mean SUS score was 76 out of 100. Qualitative data suggested the app enhanced participants' capability to complete tasks efficiently and effectively. Participants reported being satisfied with the app's features, content, screen layout, and navigation and felt it was easy to learn and could benefit patient interactions. Despite overwhelmingly positive user experience reports, there were some concerns that the app could negatively affect patient interactions due to reductions in eye contact and ability to build rapport. Conclusion Using the “Fit between Individuals, Task, and Technology” framework to guide interpretation, overall results indicate acceptable user experience and usability for this provider-focused point-of-care mobile app for substance use screening and intervention as well as favorable potential for adoption by health care practitioners. Such mobile health technologies may help to address well-known challenges related to implementing substance use services in health care settings.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Only a few technology products for substance use care with provider-facing aspects have undergone usability testing.
“Despite their promise, only a few technology products for substance use care with provider-facing aspects have undergone usability testing (Levesque et al., 2018; O’Grady et al., 2019; Satre et al., 2017).”
From Mobile technology and task shifting to improve access to alcohol treatment services in Mozambique (2022) · cites this paper as O’Grady et al., 2019
Providers in Mozambique shared concerns about maintaining rapport and about the importance of introducing the app to patients that were similar to those in the original mSBIRT study conducted in the US.
“While we found that providers in Mozambique shared similar concerns about maintaining rapport and the importance of introducing the app to patients as in the original mSBIRT study conducted in the US (O’Grady et al., 2019), they often mentioned patients’ satisfaction with the tablet and app.”
From Mobile technology and task shifting to improve access to alcohol treatment services in Mozambique (2022) · cites this paper as O’Grady et al., 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Three Approaches to Qualitative Content Analysis
Hsieh, H.-F., & Shannon, S. E. (2005) · Qualitative Health Research · doi:10.1177/1049732305276687 · Cited by 44,601 · see citing works
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
Creswell, J. W. (1994) · Cited by 35,227 · see citing works
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
ภู่พันธ์, ศ. (2018) · วารสารวิธีวิทยาการวิจัย (Journal of Research Methodology: JRM) · Cited by 12,777 · see citing works
Code Saturation Versus Meaning Saturation: How Many Interviews Are Enough?
Hennink, M. M., Kaiser, B. N., & Marconi, V. C. (2017) · Qualitative Health Research · doi:10.1177/1049732316665344 · Cited by 4,597 · see citing works
IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use
Lewis, J. R. (1995) · International Journal of Human-Computer Interaction · doi:10.1080/10447319509526110 · Cited by 2,429 · see citing works
A hands-on guide to doing content analysis
Erlingsson, C. L., & Brysiewicz, P. (2017) · African Journal of Emergency Medicine · doi:10.1016/j.afjem.2017.08.001 · Cited by 2,377 · see citing works
A mathematical model of the finding of usability problems
Nielsen, J., & Landauer, T. K. (1993) · Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '93 · doi:10.1145/169059.169166 · Cited by 1,961 · see citing works
Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests
Rubin, J. N. (1994) · Cited by 1,750 · see citing works
(2013) · Journal of Usability Studies archive · Cited by 1,357 · see citing works
The Factor Structure of the System Usability Scale
Lewis, J. R., & Sauro, J. (2009) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-02806-9_12 · Cited by 1,278 · see citing works
Mobile Health Technology Evaluation
Kumar, S., Nilsen, W. J., Abernethy, A., Atienza, A., Patrick, K., Pavel, M., Riley, W. T., Shar, A., Spring, B., Spruijt-Metz, D., et al. (2013) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2013.03.017 · Cited by 900 · see citing works
Mobile devices and apps for health care professionals: uses and benefits.
Ventola, C. L. (2014) · PubMed Central · doi:10.57945/manara.23966214 · Cited by 845 · see citing works
The impoverished life-world of outsiders
Chatman, E. A. (1996) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199603)47:3 · Cited by 688 · see citing works
Scaling Up mHealth: Where Is the Evidence?
Tomlinson, M., Rotheram-Borus, M. J., Swartz, L., & Tsai, A. C. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001382 · Cited by 673 · see citing works
The think aloud method: a guide to user interface design
JASPERS, M., STEEN, T., BOS, C., & GEENEN, M. (2004) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2004.08.003 · Cited by 544 · see citing works
Estimating the number of subjects needed for a thinking aloud test
Nielsen, J. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1065 · Cited by 476 · see citing works
Ammenwerth, E., Iller, C., & Mahler, C. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-3 · Cited by 459 · see citing works
Johnson, M., Jackson, R., Guillaume, L., Meier, P., & Goyder, E. (2011) · Journal of Public Health · doi:10.1093/pubmed/fdq095 · Cited by 374 · see citing works
Measuring Perceived Usability: The CSUQ, SUS, and UMUX
Lewis, J. R. (2018) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2017.1418805 · Cited by 302 · see citing works
(2008) · doi:10.1016/b978-0-12-373558-4.x0001-5 · Cited by 287 · see citing works
On the dimensionality of the System Usability Scale: a test of alternative measurement models
Borsci, S., Federici, S., & Lauriola, M. (2009) · Cognitive Processing · doi:10.1007/s10339-009-0268-9 · Cited by 275 · see citing works
(2001) · Ktisis at Cyprus University of Technology (Cyprus University of Technology) · Cited by 246 · see citing works
Expediting the Analysis of Qualitative Data in Evaluation
Neal, J. W., Neal, Z. P., VanDyke, E., & Kornbluh, M. (2014) · American Journal of Evaluation · doi:10.1177/1098214014536601 · Cited by 227 · see citing works
Measuring the Usability of Mobile Applications for Phones and Tablets
Kortum, P., & Sorber, M. (2015) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2015.1064658 · Cited by 222 · see citing works
Grier, R. A., Bangor, A. W., Kortum, P. T., & Peres, S. C. (2013) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931213571042 · Cited by 222 · see citing works
Brief alcohol intervention—where to from here? Challenges remain for research and practice
Nilsén, P. (2010) · Addiction · doi:10.1111/j.1360-0443.2009.02779.x · Cited by 166 · see citing works
Rahm, A. K., Boggs, J. M., Martin, C., Price, D. W., Beck, A., Backer, T. E., & Dearing, J. W. (2015) · Substance Abuse · doi:10.1080/08897077.2014.951140 · Cited by 150 · see citing works
Vendetti, J., Gmyrek, A., Damon, D., Singh, M., McRee, B., & Del Boca, F. (2017) · Addiction · doi:10.1111/add.13652 · Cited by 135 · see citing works
Extending Quality in Use to Provide a Framework for Usability Measurement
Bevan, N. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-02806-9_2 · Cited by 118 · see citing works
Alcohol Interventions Among Underage Drinkers in the ED: A Randomized Controlled Trial
Cunningham, R. M., Chermack, S. T., Ehrlich, P. F., Carter, P. M., Booth, B. M., Blow, F. C., Barry, K. L., & Walton, M. A. (2015) · PEDIATRICS · doi:10.1542/peds.2015-1260 · Cited by 87 · see citing works
Álvarez-Bueno, C., Rodríguez-Martín, B., García-Ortiz, L., Gómez-Marcos, M. Á., & Martínez-Vizcaíno, V. (2015) · Preventive Medicine · doi:10.1016/j.ypmed.2014.12.010 · Cited by 78 · see citing works
Technology-based Interventions for Preventing and Treating Substance Use Among Youth
Marsch, L. A., & Borodovsky, J. T. (2016) · Child and Adolescent Psychiatric Clinics of North America · doi:10.1016/j.chc.2016.06.005 · Cited by 75 · see citing works
Sheehan, B., Lee, Y., Rodríguez, M. C., Tiase, V. L., & Schnall, R. (2012) · Applied Clinical Informatics · doi:10.4338/aci-2012-06-ra-0021 · Cited by 73 · see citing works
Blow, F. C., Walton, M. A., Bohnert, A. M., Ignacio, R. V., Chermack, S. T., Cunningham, R. M., Booth, B. M., Ilgen, M. A., & Barry, K. L. (2017) · Addiction · doi:10.1111/add.13773 · Cited by 72 · see citing works
Glass, J. E., Andréasson, S., Bradley, K. A., Finn, S. W., Williams, E. C., Bakshi, A.-S., Gual, A., Heather, N., Sainz, M. T., Benegal, V., et al. (2017) · Addiction Science & Clinical Practice · doi:10.1186/s13722-017-0079-8 · Cited by 65 · see citing works
QUIM: a framework for quantifying usability metrics in software quality models
Seffah, A., Kececi, N., & Donyaee, M. (2002) · doi:10.1109/apaqs.2001.990036 · Cited by 49 · see citing works
User Experience Evaluation of a Smoking Cessation App in People With Serious Mental Illness
Vilardaga, R., Rizo, J. L., Kientz, J. A., McDonell, M. G., Ries, R. K., & Sobel, K. (2015) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntv256 · Cited by 45 · see citing works
Harris, S. K. (2014) · Alcohol research · doi:10.35946/arcr.v36.1.07 · Cited by 35 · see citing works
McKnight-Eily, L. R., Okoro, C. A., Mejía, R., Denny, C. H., Higgins-Biddle, J. C., Hungerford, D., Kanny, D., & Sniezek, J. E. (2017) · MMWR Morbidity and Mortality Weekly Report · doi:10.15585/mmwr.mm6612a1 · Cited by 32 · see citing works
Smartphone Application for Unhealthy Alcohol Use: A Pilot Study
Bertholet, N., Daeppen, J., McNeely, J., Kushnir, V., & Cunningham, J. A. (2017) · Substance Abuse · doi:10.1080/08897077.2017.1281860 · Cited by 25 · see citing works
Bonar, E. E., Walton, M. A., Cunningham, R. M., Chermack, S. T., Bohnert, A. M., Barry, K. L., Booth, B. M., & Blow, F. C. (2013) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2013.08.005 · Cited by 24 · see citing works
Naeem, F., Gire, N., Xiang, S., Yang, M., Syed, Y., Shokraneh, F., Adams, C., & Farooq, S. (2016) · World Journal of Psychiatry · doi:10.5498/wjp.v6.i2.187 · Cited by 18 · see citing works
Evaluation of a prototype health information system using the FITT framework
Honekamp, W., & Ostermann, H. (2011) · Journal of Innovation in Health Informatics · doi:10.14236/jhi.v19i1.793 · Cited by 14 · see citing works
Exploring physical therapists’ perceptions of mobile application usage utilizing the FITT framework
Noblin, A. M., Shettian, M., Cortelyou-Ward, K. H., & Dugre, J. S. (2016) · Informatics for Health and Social Care · doi:10.1080/17538157.2016.1178118 · Cited by 13 · see citing works
Satre, D. D., Ly, K., Wamsley, M., Curtis, A., & Satterfield, J. (2017) · JMIR Research Protocols · doi:10.2196/resprot.7070 · Cited by 13 · see citing works
(2016) · American Journal of Medical Research · doi:10.22381/ajmr3120163 · Cited by 13 · see citing works
Usability Testing of a National Substance Use Screening Tool Embedded in Electronic Health Records
Press, A., DeStio, C., McCullagh, L., Kapoor, S., Morley, J., SBIRT NY-II Team, & Conigliaro, J. C. (2016) · JMIR Human Factors · doi:10.2196/humanfactors.5820 · Cited by 12 · see citing works
Self-citationLevesque, D., Umanzor, C., & de Aguiar, E. (2018) · JMIR Medical Informatics · doi:10.2196/medinform.7355 · Cited by 7 · see citing works
Neilson, C. J., & Wilson, V. (2011) · Journal of the Canadian Health Libraries Association / Journal de l Association de bilbiothèques de la santé du Canada · doi:10.5596/c11-024 · Cited by 6 · see citing works
Alcohol Interventions Among Underage Drinkers in the ED: A Randomized Controlled Trial
(2015) · PEDIATRICS · doi:10.1542/peds.2015-1260d · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Sauro J, Zarolia P. SUPR-Qm: a questionnaire to measure the mobile app user experience. J Usability Stud 2017;13(01):17-37
Cited by
Sorted by citations
Mobile technology and task shifting to improve access to alcohol treatment services in Mozambique
O'Grady, M. A., Mootz, J., Suleman, A., Sweetland, A., Teodoro, E., Anube, A., Feliciano, P., Bezuidenhout, C., Dos Santos, P. F., Fumo, W., et al. (2022) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2021.108549 · Cited by 24 · see citing works
Self-citation
- Authors
- O'Grady, M., Kapoor, S., Gilmer, E., Neighbors, C., Conigliaro, J., Kwon, N., & Morgenstern, J.
- Affiliations
- Health Services Research, Center on Addiction, New York, New York, United States
- Northwell Health (US)
- Venue
- ACI Open
- Publisher
- Georg Thieme Verlag KG
- Published
- 2019-01
- Volume
- 03
- Issue
- 01
- Pages
- e1-e12
- ISSN
- 2566-9346 (ISSN-L)
- OpenAlex
- W2922911935
- Cited by
- 6
- References
- 50
- Citations by year
- 2024: 2 · 2022: 1 · 2021: 3
- FWCI
- 1.37
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- ACI Open (publishedVersion, cc-by)
- Funders
- Substance Abuse and Mental Health Services Administration (5U79TI025102)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Keywords
- substance use screening, substance use interventions, mobile health, health coaches, user experience, usability, emergency department, primary care, provider barriers, patient rapport, technology adoption
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
