Evaluating the usefulness of mobile services based on captured usage data from longitudinal field trials
Open accessAbstract
It is widely discussed whether the usefulness of mobile systems and services should be evaluated in the field or in the lab. The aim of this paper is to make a case for conducting longitudinal field trials when evaluating the usefulness of mobile services, and to partly base such evaluations on quantitative usage data automatically captured on the device. A framework for automatic capture and analysis of such data is introduced, and it is shown how it has been used to evaluate a mobile service for diabetes management. Results and experiences based on a three month field trial with a single diabetic user is presented and it is demonstrated how these results can be used to make inferences about the use of the service with regard to functionality and usage patterns.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Involving patient-users not only in the specification-design phase but also in the trial phase of a working prototype in a real-life setting for enough time is important to clarify how the chosen design works in relation to expectations and how the design can be improved.
“This study confirmed the importance of involving “patient-users”, not only in the specification-design phase but also in the trial phase of a working prototype in a real-life setting for enough time to clarify how the chosen design works in relation to expectations and how the design can be improved [40].”
From Long-Term Engagement With a Mobile Self-Management System for People With Type 2 Diabetes (2013) · cites this paper as [40]
Cites
(1993) · Cited by 9,879 · see citing works
Reality mining: sensing complex social systems
Eagle, N., & (Sandy) Pentland, A. (2006) · Personal and Ubiquitous Computing · doi:10.1007/s00779-005-0046-3 · Cited by 2,772 · see citing works
NIELSEN, J. (1993) · Usability Engineering · doi:10.1016/b978-0-08-052029-2.50009-7 · Cited by 934 · see citing works
The state of the art in automating usability evaluation of user interfaces
Ivory, M. Y., & Hearst, M. A. (2001) · ACM Computing Surveys · doi:10.1145/503112.503114 · Cited by 884 · see citing works
Froehlich, J. E., Chen, M. Y., Consolvo, S., Harrison, B. L., & Landay, J. A. (2007) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/1247660.1247670 · Cited by 488 · see citing works
ContextPhone: A Prototyping Platform for Context-Aware Mobile Applications
Raento, M., Oulasvirta, A., Petit, R., & Toivonen, H. T. T. (2005) · IEEE Pervasive Computing · doi:10.1109/mprv.2005.29 · Cited by 483 · see citing works
Extracting usability information from user interface events
Hilbert, D. M., & Redmiles, D. (2000) · ACM Computing Surveys · doi:10.1145/371578.371593 · Cited by 478 · see citing works
A Review of Mobile HCI Research Methods
Kjeldskov, J., & Graham, C. (2003) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-45233-1_23 · Cited by 375 · see citing works
Kjeldskov, J., Skov, M. B., Als, B. S., & Høegh, R. T. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-28637-0_6 · Cited by 287 · see citing works
An Approach to Large-Scale Collection of Application Usage Data Over the Internet
Hilbert, D. M., & Redmiles, D. F. (1997) · eScholarship (California Digital Library) · Cited by 75 · see citing works
Hejlesen, O. K., Andreassen, S., Hovorka, R., & Cavan, D. A. (1997) · Computer Methods and Programs in Biomedicine · doi:10.1016/s0169-2607(97)00033-3 · Cited by 40 · see citing works
DiasNet--an Internet tool for communication and education in diabetes.
Hejlesen, O. K., Plougmann, S., & Cavan, D. A. (2000) · PubMed · Cited by 16 · see citing works
Diasnet Mobile - A Personalized Mobile Diabetes Management and Advisory Service
Jensen, K. L., Pedersen, C. F., & Larsen, L. B. (2007) · VBN Forskningsportal (Aalborg Universitet) · Cited by 11 · see citing works
Self-citationTowards Useful and Usable Services in Personal Networks
Jensen, K. L., Pedersen, C. F., & Larsen, L. B. (2006) · doi:10.1109/mobiqw.2006.361747 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Mobile health applications to assist patients with diabetes: lessons learned and design implications
Årsand, E., Frøisland, D. H., Skrøvseth, S. O., Chomutare, T., Tatara, N., Hartvigsen, G., & Tufano, J. T. (2012) · Journal of Diabetes Science and Technology · doi:10.1177/193229681200600525 · Cited by 214 · see citing works
Mobile Phone-Based Self-Management Tools for Type 2 Diabetes: The Few Touch Application
Arsand, E., Tatara, N., ostengen, G., & Hartvigsen, G. (2010) · Journal of Diabetes Science and Technology · doi:10.1177/193229681000400213 · Cited by 188 · see citing works
Bonsignore, E., Quinn, A. J., Druin, A., & Bederson, B. B. (2013) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2491500.2491506 · Cited by 76 · see citing works
Long-Term Engagement With a Mobile Self-Management System for People With Type 2 Diabetes
Tatara, N., Årsand, E., Skrøvseth, S. O., & Hartvigsen, G. (2013) · JMIR mHealth and uHealth · doi:10.2196/mhealth.2432 · Cited by 62 · see citing works
Automated Usability Evaluation of Virtual Reality Applications
Harms, P. (2019) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3301423 · Cited by 60 · see citing works
Adu, M. D., Malabu, U. H., Callander, E. J., Malau‐Aduli, A. E. O., & Malau‐Aduli, B. S. (2018) · JMIR mhealth and uhealth · doi:10.2196/10115 · Cited by 60 · see citing works
Mobile apps use for wellness and fitness and university students’ subjective wellbeing
Aboelmaged, M. G. S., Ali, I., & Hashem, G. (2021) · Information Development · doi:10.1177/02666669211020498 · Cited by 29 · see citing works
Wireless and Mobile Technologies Improving Diabetes Self-Management
Årsand, E., Tatara, N., & Hartvigsen, G. (2011) · Handbook of Research on Mobility and Computing · doi:10.4018/978-1-60960-042-6.ch009 · Cited by 17 · see citing works
Schneider, C., Trukeschitz, B., & Rieser, H. (2019) · Applied Sciences · doi:10.3390/app10010038 · Cited by 14 · see citing works
Liu, N., Liu, Y., & Wang, X. (2010) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/1851600.1851650 · Cited by 11 · see citing works
Remote and Autonomous Studies of Mobile and Ubiquitous Applications in Real Contexts
Jensen, K. L. (2011) · International Journal of Mobile Human Computer Interaction · doi:10.4018/jmhci.2011040101 · Cited by 6 · see citing works
Self-citationAutomated Field Usability Evaluation Using Generated Task Trees
Harms, P. (2016) · doi:10.53846/goediss-5453 · Cited by 6 · see citing works
Evaluation des systèmes mobiles et ubiquitaires: proposition de méthodologie et retours d'expérience
Jambon, F., Mandran, N., Meillon, B., & Perrot, C. (2014) · Journal d Interaction Personne-Système · doi:10.46298/jips.61 · Cited by 5 · see citing works
Remote Usage Data Collection and Analysis for Mobile Accessibility Applications
Ahmetovic, D., Bernareggi, C., Ducci, M., Gerino, A., & Mascetti, S. (2021) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops51409.2021.9430968 · Cited by 4 · see citing works
The Challenge of Evaluating the Mobile and Ubiquitous User Experience
Jensen, K. L., & Larsen, L. B. (2008) · VBN Forskningsportal (Aalborg Universitet) · Cited by 4 · see citing works
Self-citationKronbauer, A. H., & Santos, C. A. S. (2013) · Brazilian Symposium on Multimedia and the Web · doi:10.1145/2526188.2526212 · Cited by 2 · see citing works
Impact of Contextual Factors on Smartphone Applications Use
Kronbauer, A. H., & Santos, C. A. S. (2014) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-319-11569-6_52 · Cited by 2 · see citing works
User experience evaluation of Post-Traumatic Mobile Service
Liu, N., Liu, Y., Liu, L. L., & Wang, X. (2009) · International Conference on Mobile Technology, Applications, and Systems · doi:10.1145/1710035.1710088 · Cited by 2 · see citing works
The analysis of contextual factors on the use of smartphones applications
Kronbauer, A. H., & Santos, C. A. S. (2015) · ICST Transactions on Ubiquitous Environments · doi:10.4108/ue.1.4.e2 · Cited by 2 · see citing works
Sherlock: uma plataforma para avaliar a experiência dos usuários com dispositivos móveis
Kronbauer, A. H., Machado, D., Omar, G., & Santos, C. A. S. (2014) · Cited by 1 · see citing works
Experimental Setups for User Evaluation of Mobile Devices and Ubiquitous Systems
Jambon, F. (2011) · IGI Global eBooks · doi:10.4018/978-1-60960-499-8.ch009
Evaluating Mobile, Ubiquitous and Context Sensitive Services in the Field
Jensen, K. L. (2007)
Self-citation
- Authors
- Jensen, K. L., & Larsen, L. B.
- Affiliations
- Aalborg University (DK)
- Venue
- Proceedings of the 4th international conference on mobile technology, applications, and systems and the 1st international symposium on Computer human interaction in mobile technology
- Publisher
- ACM
- Published
- 2007-09-10
- Pages
- 675-682
- OpenAlex
- W2046662042
- Cited by
- 22
- References
- 14
- Citations by year
- 2021: 2 · 2019: 2 · 2018: 1 · 2016: 1 · 2015: 1 · 2014: 3 · 2013: 3 · 2012: 1
- FWCI
- 23.66
- Type
- conference paper
- Language
- en
- Open access
- green
- vbn.aau.dk
- Locations
- International Conference on Mobile Technology, Applications, and Systems (publishedVersion)
- VBN Forskningsportal (Aalborg Universitet) (submittedVersion, other-oa)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- automated data collection, usability evaluation, diabetes management, mobile health, usage patterns
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
