Synergistic use of direct manipulation and natural language
Open accessThe DOI matches its Crossref record.
Abstract
This paper shows how the integration of natural language with direct manipulation produces a multimodal interface that overcomes limitations of these techniques when used separately. Natural language helps direct manipulation in being able to specify objects and actions by description, while direct manipulation enables users to learn which objects and actions are available in the system. Furthermore, graphical rendering and manipulation of context provides a partial solution to difficult problems of natural language anaphora.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Billinghurst, M., Clark, A., & Lee, G. (2015) · now publishers, Inc. eBooks · doi:10.1561/9781601989215 · Cited by 529
Oviatt, S. (2007) · Human Factors and Ergonomics · doi:10.1201/9781410615862.ch21 · Cited by 485
QuickSet: multimodal interaction for distributed applications
Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., & Clow, J. (1997) · Proceedings of the fifth ACM international conference on Multimedia - MULTIMEDIA '97 · doi:10.1145/266180.266328 · Cited by 466
Self-citationOviatt, S., Cohen, P., Wu, L., Duncan, L., Suhm, B., Bers, J., Holzman, T., Winograd, T., Landay, J., Larson, J., et al. (2000) · Human–Computer Interaction · doi:10.1207/s15327051hci1504_1 · Cited by 375
Self-citationIntegration and synchronization of input modes during multimodal human-computer interaction
Oviatt, S., DeAngeli, A., & Kuhn, K. (1997) · Proceedings of the ACM SIGCHI Conference on Human factors in computing systems · doi:10.1145/258549.258821 · Cited by 360
A survey of design issues in spatial input
Hinckley, K., Pausch, R., Goble, J. C., & Kassell, N. F. (1994) · Proceedings of the 7th annual ACM symposium on User interface software and technology - UIST '94 · doi:10.1145/192426.192501 · Cited by 360
Mutual disambiguation of recognition errors in a multimodel architecture
Oviatt, S. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303163 · Cited by 315
Multimodal human discourse: gesture and speech
Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.-F., Kirbas, C., McCullough, K. E., & Ansari, R. (2002) · ACM Transactions on Computer-Human Interaction · doi:10.1145/568513.568514 · Cited by 291
Multimodal integration-a statistical view
Wu, L., Oviatt, S. L., & Cohen, P. R. (1999) · IEEE Transactions on Multimedia · doi:10.1109/6046.807953 · Cited by 190
Self-citationThe role of natural language in a multimodal interface
Cohen, P. R. (1992) · Proceedings of the 5th annual ACM symposium on User interface software and technology · doi:10.1145/142621.142641 · Cited by 168
Self-citationOviatt, S. L., Cohen, P. R., & Wang, M. (1994) · Speech Communication · doi:10.1016/0167-6393(94)90079-5 · Cited by 76
Self-citationMultimodal interaction, interfaces, and analytics
Oviatt, S. (2022) · Handbook of Human Computer Interaction · doi:10.1007/978-3-319-27648-9_22-1 · Cited by 12
Metadata
- Authors
- Cohen, P. R., Dalrymple, M., Moran, D. B., Pereira, F. C., & Sullivan, J. W.
- Venue
- Proceedings of the SIGCHI conference on Human factors in computing systems Wings for the mind - CHI '89
- Publisher
- ACM Press
- Published
- 1989
- Pages
- 227-233
- OpenAlex
- W2023685874
- Cited by
- 142
- References
- 0
- FWCI
- 4.02
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.