A Study on Automatic Classification of Users’ Desktop Interactions
Abstract
Knowledge workers frequently change activities, either by choice or through interruptions. With an increasing number of activities and activity switches, it is becoming more and more difficult for knowledge workers to keep track of their desktop activities. This article presents our efforts to achieve activity awareness through automatic classification of user's everyday desktop activities. For getting a deeper understanding, we investigate performance of various classifiers with respect to discriminative power of time-, interaction-, and content-based feature sets for different work scenarios and users. Specifically, by viewing an activity as a sequence of desktop interactions we present (1) a methodology for translating a user's desktop interactions into activities, (2) evaluation of the discriminative power of different activity features and feature types, and (3) analysis of supervised classification models for classifying desktop activity under two different scenarios, i.e., an activity-centric scenario and a user-centric scenario. The experiments are carried out on a real-world dataset, and the results show satisfactory accuracy using relatively few and simple types of features.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The user interface used to collect ground truth for task switches and types resembles the one by Mirza et al.
“The user interface we used to collect the ground truth for task switches and types resembles the one by Mirza et al. [15], [25], [33].”
From Detecting Developers' Task Switches and Types (2022) · cites this paper as [33]
Mirza et al. found that they can predict the activity category for each 5 minute window with high accuracy (81 percent) and that interaction-based features work best.
“In a 6 hour field study with five participants and a controlled lab study, they found that they can predict the activity category for each of these 5 minute windows with high accuracy (81 percent) and that interaction-based features work best.”
From Detecting Developers' Task Switches and Types (2022) · cites this paper as [33]
Only very little prior work exists on automatically detecting, characterizing and summarizing developers' tasks.
“We have been able to find only very little work on automatically detecting, characterizing and summarizing (developers) tasks yet [22], [32], [33].”
From Detecting Developers' Task Switches and Types (2022) · cites this paper as [33]
Cites
Proceedings of the 25th international conference on Machine learning
Cohen, W. W., McCallum, A., & Roweis, S. T. (2008) · Cited by 5,691 · see citing works
Witten, I. H., & Frank, E. (2002) · ACM SIGMOD Record · doi:10.1145/507338.507355 · Cited by 5,163 · see citing works
Proceedings of the 20th conference on Uncertainty in artificial intelligence
Meek, C., Chickering, M., & Halpern, J. Y. (2004) · Cited by 1,548 · see citing works
Witten, I. H., Paynter, G. W., Frank, E., Gutwin, C. A., & Nevill-Manning, C. G. (1999) · doi:10.1145/313238.313437 · Cited by 860 · see citing works
A diary study of task switching and interruptions
Czerwinski, M., Horvitz, E., & Wilhite, S. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985715 · Cited by 830 · see citing works
The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software UsersPreprint
Horvitz, E., Breese, J. S., Heckerman, D. E., Hovel, D., & Rommelse, K. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.7385 · Cited by 655 · see citing works
No task left behind?: examining the nature of fragmented work
Mark, G., Gonzalez, V. M., & Harris, J. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1055017 · Cited by 503 · see citing works
An adaptation of Relief for attribute estimation in regression
Robnik‐Šikonja, M., & Kononenko, I. (1997) · Cited by 478 · see citing works
Fawcett, T. E., & Provost, F. J. (1999) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/312129.312195 · Cited by 422 · see citing works
Bellotti, V. M. E., Ducheneaut, N., Howard, M., & Smith, I. (2003) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/642611.642672 · Cited by 370 · see citing works
Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence
Boutilier, C. E., & Goldszmidt, M. (2000) · Cited by 350 · see citing works
Attention-Sensitive AlertingPreprint
Horvitz, E., Jacobs, A., & Hovel, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6707 · Cited by 317 · see citing works
Learning and reasoning about interruption
Horvitz, E., & Apacible, J. (2003) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/958432.958440 · Cited by 275 · see citing works
TaskTracer: a desktop environment to support multi-tasking knowledge workers
Dragunov, A. N., Dietterich, T. G., Johnsrude, K., McLaughlin, M., Li, L., & Herlocker, J. L. (2005) · Proceedings of the 10th international conference on Intelligent user interfaces · doi:10.1145/1040830.1040855 · Cited by 240 · see citing works
Journal of Digital Information Management
(2022) · Journal of Digital Information Management · doi:10.6025/jdim · Cited by 207 · see citing works
Programming by Demonstration Using Version Space Algebra
Lau, T. A., Wolfman, S. A., Domingos, P., & Weld, D. S. (2003) · Machine Learning · doi:10.1023/a:1025671410623 · Cited by 179 · see citing works
Automatically classifying emails into activities
Dredze, M. H., Lau, T. A., & Kushmerick, N. (2006) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1111449.1111471 · Cited by 111 · see citing works
A hybrid learning system for recognizing user tasks from desktop activities and email messages
Shen, J., Li, L., Dietterich, T. G., & Herlocker, J. L. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111473 · Cited by 109 · see citing works
Automation as Caregiver: A Survey of Issues and Technologies
Haigh, K. Z., & Yanco, H. A. (2003) · Cited by 97 · see citing works
SWISH: semantic analysis of window titles and switching history
Oliver, N., Smith, G., Thakkar, C., & Surendran, A. C. (2006) · Proceedings of the 11th international conference on Intelligent user interfaces · doi:10.1145/1111449.1111492 · Cited by 89 · see citing works
Proceedings of the 15th international conference on Intelligent user interfaces
Rich, C., Yang, Q., Cavazza, M., et al. (2010) · doi:10.1145/1719970 · Cited by 79 · see citing works
Lau, T. A., Bergman, L. D., Castelli, V., & Oblinger, D. (2004) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/964442.964464 · Cited by 51 · see citing works
Brdiczka, O., Su, N. M., & Begole, B. (2010) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1719970.1720011 · Cited by 39 · see citing works
Jacek Gwizdka, A., & Chignell, M. (2004) · Interacting with Computers · doi:10.1016/j.intcom.2004.04.008 · Cited by 34 · see citing works
Discovering frequent work procedures from resource connections
Shen, J., Fitzhenry, E., & Dietterich, T. G. (2009) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1502650.1502690 · Cited by 31 · see citing works
Enhancing Just-in-Time E-Learning Through Machine Learning on Desktop Context Sensors
Lokaiczyk, R., Faatz, A., Beckhaus, A., et al. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74255-5_25 · Cited by 30 · see citing works
Evaluating Personal Information Management Using an Activity Logs Enriched Desktop Dataset
Chernov, S., Demartini, G., Herder, E., et al. (2008) · Cited by 25 · see citing works
Oliver, N., Czerwinski, M. P., Smith, G., et al. (2008) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/1378773.1378836 · Cited by 13 · see citing works
Revealed Processes in Knowledge Management
Fenstermacher, K. D. (2005) · Lecture notes in computer science · doi:10.1007/11590019_51 · Cited by 11 · see citing works
Montabert, C., McCrickard, D. S., Winchester, W. W., et al. (2009) · Interacting with Computers · doi:10.1016/j.intcom.2009.06.003 · Cited by 11 · see citing works
Personalization in Knowledge Management
Tochtermann, K. (2003) · Lecture notes in computer science · doi:10.1007/3-540-44872-1_5 · Cited by 9 · see citing works
Moneypenny: lessons from the messy desk
Williamson, A. (1998) · Interacting with Computers · doi:10.1016/s0953-5438(97)00008-8 · Cited by 9 · see citing works
Machine Learning based Work Task Classification
Granitzer, M., Rath, A., Kröll, M., Seifert, C., Ipsmiller, D., Devaurs, D., Weber, N., & Lindstaedt, S. (2009) · Open Archive Toulouse Archive Ouverte (University of Toulouse) · Cited by 8 · see citing works
Task-stage knowledge support: coupling user information needs with stage identification
Wu, I., Liu, D., & Chen, W.-H. (2005) · doi:10.1109/iri-05.2005.1506443 · Cited by 8 · see citing works
Switch detector: an activity spotting system for desktop
Mirza, H. T., Chen, L., Chen, G., Hussain, I., & He, X. (2011) · Proceedings of the 20th ACM international conference on Information and knowledge management · doi:10.1145/2063576.2063947 · Cited by 5 · see citing works
Self-citationIdentifying Routine and Telltale Activity Patterns in Knowledge Work
Brdiczka, O., & Bellotti, V. M. E. (2011) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2011.12 · Cited by 4 · see citing works
Mining and Supporting Task-Stage Knowledge: A Hierarchical Clustering Technique
Liu, D., Wu, I., & Chen, W.-H. (2006) · Lecture notes in computer science · doi:10.1007/11944935_16 · Cited by 3 · see citing works
BUILDING USER TASK SPACE BY MINING TEMPORALLY PROXIMATE DESKTOP ACTIONS
Mirza, H. T., Chen, L., Majid, A., & Chen, G. (2011) · Cybernetics and Systems · doi:10.1080/01969722.2011.634675 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Detecting Developers' Task Switches and Types
Meyer, A. N., Satterfield, C., Zuger, M., Kevic, K., Murphy, G. C., Zimmermann, T., & Fritz, T. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2984086 · Cited by 14 · see citing works
“…category (13) : the percentage of the total duration of the task segment that was spent in each of the 13 application categories [14], [32], [33] NA 34.3% NA Switching Frequency Features 16.6% 13.7% 19.0% Difference in the window switches frequency (1) : difference of the number of switches…”
BackgroundMethodResultInfluential
- Authors
- Mirza, H. T., Chen, L., Hussain, I., Majid, A., & Chen, G.
- Affiliations
- COMSATS University Islamabad, Lahore Campus (PK)
- Zhejiang University (CN)
- COMSATS University Islamabad (PK)
- University of Azad Jammu and Kashmir (PK)
- Corresponding author
- Chen, Ling
- Venue
- Cybernetics and Systems
- Publisher
- Informa UK Limited
- Published
- 2015-05-06
- Volume
- 46
- Issue
- 5
- Pages
- 320-341
- ISSN
- 0196-9722 (ISSN-L)
- 1087-6553
- OpenAlex
- W2228653641
- Cited by
- 8
- References
- 38
- Citations by year
- 2026: 1 · 2024: 2 · 2023: 1 · 2021: 1 · 2020: 3
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Cybernetics & Systems (publishedVersion)
- Funders
- This work was funded by the Ministry of Industry and Information Technology of China (No. 2010ZX01042-002-003-001), China Knowledge Centre for Engineering Sciences and Technology (No. CKCEST-2014-1-5), the National Natural Science Foundation of China (Nos. 60703040 and 61332017), the Science and Technology Department of Zhejiang Province (Nos. 2007C13019, 2011C13042, and 2013C01046).
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Social Sciences › Decision Sciences › Information Systems and Management › Personal Information Management and User Behavior
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- activity awareness, knowledge workers, user activity recognition, interaction features, content-based features, time-based features, supervised classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
