CoChat

Automatically identifying targets users interact with during real world tasks

Hurst, Amy, Hudson, Scott E., Mankoff, Jennifer · 2010 · Proceedings of the 15th international conference on Intelligent user interfaces

The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.

Abstract

Information about the location and size of the targets that users interact with in real world settings can enable new innovations in human performance assessment and soft-ware usability analysis. Accessibility APIs provide some information about the size and location of targets. How-ever this information is incomplete because it does not sup-port all targets found in modern interfaces and the reported sizes can be inaccurate. These accessibility APIs access the size and location of targets through low-level hooks to the operating system or an application. We have developed an alternative solution for target identification that leverages visual affordances in the interface, and the visual cues produced as users interact with targets. We have used our novel target identification technique in a hybrid solution that combines machine learning, computer vision, and accessibility API data to find the size and location of targets users select with 89% accuracy. Our hybrid approach is superior to the performance of the accessibility API alone: in our dataset of 1355 targets covering 8 popular applications, only 74% of the targets were correctly identified by the API alone.

Overview

Integrity signals

Registry

Crossref · 10.1145/1719970.1719973

Cites

23 of 23 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 23 references

Research works printed but not linked: 5

  1. Microsoft Office Outlook 2003 Step By Step , Microsoft Press , 2004. Microsoft Office Outlook 2003 Step By Step, Microsoft Press, 2004.
  2. Microsoft Office Powerpoint 2003 Step By Step , Microsoft Press , 2004. Microsoft Office Powerpoint 2003 Step By Step, Microsoft Press, 2004.
  3. Microsoft Office Word 2003 Step By Step , Microsoft Press , 2004. Microsoft Office Word 2003 Step By Step, Microsoft Press, 2004.
  4. Mitchell , T. , ( 1997 ), Machine Learning , McGraw-Hill . Mitchell, T., (1997), Machine Learning, McGraw-Hill.
  5. Windows XP Step By Step , 2 nd edition, Microsoft Press , 2005 . Windows XP Step By Step, 2nd edition, Microsoft Press, 2005.

Cited by

See all 50

Metadata

Authors
Hurst, A., Hudson, S. E., & Mankoff, J.
Affiliations
Carnegie Mellon University (US)
Venue
Proceedings of the 15th international conference on Intelligent user interfaces
Publisher
ACM
Published
2010-02-07
Pages
11-20
OpenAlex
W2078091018
Cited by
50
References
23
Citations by year
2022: 1 · 2021: 1 · 2020: 3 · 2019: 6 · 2018: 1 · 2017: 3 · 2016: 3 · 2015: 8 · 2014: 4 · 2013: 2 · 2012: 6
FWCI
4.89
Type
conference paper
Language
en
Open access
closed
Locations
ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
Funders
National Science Foundation
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
  • Physical Sciences › Computer Science › Human-Computer Interaction › Interactive and Immersive Displays
Keywords
target identification, visual affordances, human performance assessment, usability analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Automatically identifying targets users interact with during real world tasks. https://app.cochat.ai/papers/10.1145/1719970.1719973
The paper
Hurst, A., Hudson, S. E., & Mankoff, J. (2010). Automatically identifying targets users interact with during real world tasks. Proceedings of the 15th international conference on Intelligent user interfaces. https://doi.org/10.1145/1719970.1719973

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.