The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Soylent: a word processor with a crowd inside
Open accessAbstract
This paper introduces architectural and interaction patterns for integrating crowdsourced human contributions directly into user interfaces. We focus on writing and editing, complex endeavors that span many levels of conceptual and pragmatic activity. Authoring tools offer help with pragmatics, but for higher-level help, writers commonly turn to other people. We thus present Soylent, a word processing interface that enables writers to call on Mechanical Turk workers to shorten, proofread, and otherwise edit parts of their documents on demand. To improve worker quality, we introduce the Find-Fix-Verify crowd programming pattern, which splits tasks into a series of generation and review stages. Evaluation studies demonstrate the feasibility of crowdsourced editing and investigate questions of reliability, cost, wait time, and work time for edits.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI reduces creative workloads by providing valuable suggestions and refining content for professional use, thereby helping writers overcome creative blocks.
“Prior studies have shown that AI reduces creative workloads by providing valuable suggestions [10, 33] and refining content for professional use [26, 43, 83], thereby helping writers overcome creative blocks.”
From Understanding Screenwriters' Practices, Attitudes, and Future Expectations in Human-AI Co-Creation (2025) · cites this paper as [10]
Citations
Cites
Awareness and coordination in shared workspaces
Dourish, P., & Bellotti, V. (1992) · Proceedings of the 1992 ACM conference on Computer-supported cooperative work - CSCW '92 · doi:10.1145/143457.143468 · Cited by 2,550 · cited 1×
“Writing is no exception [7]: we commonly recruit friends and colleagues to help us shape and polish our writing.”
BackgroundVizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601
Self-citation
Flagged first, then by citations
Research works printed but not linked: 24
- 10.1145/1753326.1753470 · DOI
- 10.1145/1622176.1622195 · DOI
- 10.3115/1220175.1220223 · DOI
- 10.5555/1599081.1599099 · DOI
- }} Cypher , A. Watch What I Do . MIT Press , Cambridge, MA , 1993 . }}Cypher, A. Watch What I Do. MIT Press, Cambridge, MA, 1993.
- 10.1145/1460563.1460641 · DOI
- 10.1145/1753326.1753478 · DOI
- 10.1145/1753326.1753357 · DOI
- 10.1145/1357054.1357127 · DOI
- 10.1016/S0004-3702(02)00222-9 · DOI
- 10.1145/1518701.1518927 · DOI
- 10.1145/146370.146380 · DOI
- }} Lieberman , H. Your Wish is My Command . Morgan Kaufmann , San Francisco , 2001 . }}Lieberman, H. Your Wish is My Command. Morgan Kaufmann, San Francisco, 2001.
- 10.1145/1866029.1866040 · DOI
- 10.1145/1240624.1240767 · DOI
- 10.5555/517637 · DOI
- 10.1145/1600150.1600175 · DOI
- }} Miller , R. and Myers , B . Interactive simultaneous editing of multiple text regions . USENIX '01 , ( 2001 ). }}Miller, R. and Myers, B. Interactive simultaneous editing of multiple text regions. USENIX '01, (2001).
- 10.1145/1753846.1753873 · DOI · Paper page
- 10.5555/1758156.1758163 · DOI
- 10.1145/1357054.1357169 · DOI
- 10.5555/1613715.1613751 · DOI
- 10.1109/CVPRW.2008.4562953 · DOI
- 10.1145/985692.985733 · DOI
Cited by
Sorted by citations
Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441923 · Cited by 1,034
Self-citationA Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497
Shepherding the crowd yields better work
Dow, S., Kulkarni, A., Klemmer, S., & Hartmann, B. (2012) · Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work · doi:10.1145/2145204.2145355 · Cited by 334
Self-citationAll That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.565 · Cited by 290
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text
Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021) · arXiv · doi:10.48550/arxiv.2107.00061 · Cited by 290
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257
Morris, R. R., Schueller, S. M., & Picard, R. W. (2015) · Journal of Medical Internet Research · doi:10.2196/jmir.4167 · Cited by 206
TaleBrush: Sketching Stories with Generative Pretrained Language Models
Chung, J. J. Y., Kim, W., Yoo, K. M., Lee, H., Adar, E., & Chang, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501819 · Cited by 206
Metaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146
Art or Artifice? Large Language Models and the False Promise of Creativity
Chakrabarty, T., Laban, P., Agarwal, D., Muresan, S., & Wu, C.-S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642731 · Cited by 135
Writer-Defined AI Personas for On-Demand Feedback Generation
Benharrak, K., Zindulka, T., Lehmann, F., Heuer, H., & Buschek, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642406 · Cited by 63
Cicero: Multi-Turn, Contextual Argumentation for Accurate Crowdsourcing
Chen, Q., Bragg, J., Chilton, L. B., & Weld, D. S. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300761 · Cited by 41
Evaluating the Quality of Learning Resources: A Learnersourcing Approach
Abdi, S., Khosravi, H., Sadiq, S., & Demartini, G. (2021) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2021.3058644 · Cited by 40
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Laban, P., Vig, J., Hearst, M., Xiong, C., & Wu, C.-S. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676419 · Cited by 31
The Invisible Labor of Access in Academic Writing Practices: A Case Analysis with Dyslexic Adults
Wang, E. Q., & Piper, A. M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512967 · Cited by 33
Self-Adapting Reliability in Distributed Software Systems
Brun, Y., Bang, J. Y., Edwards, G., & Medvidovic, N. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2015.2412134 · Cited by 16
Metadata
- Authors
- Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., & Panovich, K.
- Affiliations
- Massachusetts Institute of Technology (US)
- University of California, Berkeley (US)
- University of Michigan (US)
- Venue
- Proceedings of the 23nd annual ACM symposium on User interface software and technology
- Publisher
- ACM
- Published
- 2010-10-03
- Pages
- 313-322
- OpenAlex
- W2127008633
- Cited by
- 686
- References
- 2
- Citations by year
- 2026: 2 · 2025: 13 · 2024: 22 · 2023: 13 · 2022: 25 · 2021: 31 · 2020: 27 · 2019: 32 · 2018: 47 · 2017: 58 · 2016: 59 · 2015: 76 · 2014: 76 · 2013: 86 · 2012: 65
- FWCI
- 58.71
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- Funders
- National Science Foundation (0712793)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- word processing, Mechanical Turk, authoring tools, task decomposition, worker reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
