"I Hope This Is Helpful": Understanding Crowdworkers' Challenges and Motivations for an Image Description Task
Abstract
AI image captioning challenges encourage broad participation in designing algorithms that automatically create captions for a variety of images and users. To create large datasets necessary for these challenges, researchers typically employ a shared crowdsourcing task design for image captioning. This paper discusses findings from our thematic analysis of 1,064 comments left by Amazon Mechanical Turk workers using this task design to create captions for images taken by people who are blind. Workers discussed difficulties in understanding how to complete this task, provided suggestions of how to improve the task, gave explanations or clarifications about their work, and described why they found this particular task rewarding or interesting. Our analysis provides insights both into this particular genre of task as well as broader considerations for how to employ crowdsourcing to generate large datasets for developing AI algorithms.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Annotation by crowdworkers without disability-specific expertise leads to inconsistent or inaccurate labels in accessibility datasets.
“leading to inconsistent or inaccurate labels.”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [90]
Sighted crowd workers often apply inconsistent interpretive standards shaped by their own assumptions and biases.
“sighted crowd workers often apply inconsistent interpretive standards shaped by their own assumptions and biases [90].”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [90]
Accessibility datasets are often annotated by crowdworkers without disability-specific expertise.
“accessibility datasets are often annotated by crowdworkers without disability-specific expertise, leading to inconsistent or inaccurate labels.”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [90]
Sighted people often do not understand what information blind people want in visual descriptions.
“Sighted people often do not understand what information blind people want in visual descriptions [31, 90].”
From Disability-First AI Dataset Annotation: Co-designing Stuttered Speech Annotation Guidelines with People Who Stutter (2026) · cites this paper as [90]
Relying on crowdworkers is a common approach to annotation, but they may lack insight into disabled people's information needs and may apply varying standards of visual interpretation in BLV-focused datasets.
“Relying on crowdworkers is a common approach to annotation, but they may lack insight into disabled people's information needs and may apply varying standards of visual interpretation in BLV-focused datasets [113].”
From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [113]
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414 · see citing works
Proceedings of IEEE International Conference on Computer Vision
(1995) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.1995.466933 · Cited by 13,117 · see citing works
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107 · see citing works
CIDEr: Consensus-based image description evaluation
Vedantam, R., Zitnick, C. L., & Parikh, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299087 · Cited by 4,845 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Young, P., Lai, A., Hodosh, M., & Hockenmaier, J. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00166 · Cited by 2,439 · see citing works
Crowdsourcing user studies with Mechanical Turk
Kittur, A., Ed, H., & Suh, B. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357127 · Cited by 1,941 · see citing works
Towards an integrated crowdsourcing definition
Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. (2012) · Journal of Information Science · doi:10.1177/0165551512437638 · Cited by 1,859 · see citing works
Routledge International Handbook of Participatory Design
(2012) · doi:10.4324/9780203108543 · Cited by 1,606 · see citing works
International Journal of Information Management
Hills, P. (2005) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2005.10.001 · Cited by 1,197 · see citing works
BabyTalk: Understanding and Generating Simple Image Descriptions
Kulkarni, G. A., Premraj, V., Ordóñez, V., Dhar, S., Li, S., Choi, Y., Berg, A. C., & Berg, T. L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.162 · Cited by 839 · see citing works
VizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601 · see citing works
Task Design, Motivation, and Participation in Crowdsourcing Contests
Zheng, H., Li, D., & Hou, W. (2011) · International Journal of Electronic Commerce · doi:10.2753/jec1086-4415150402 · Cited by 537 · see citing works
Demographics and Dynamics of Mechanical Turk Workers
Difallah, D. E., Filatova, E., & Ipeirotis, P. G. (2018) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3159652.3159661 · Cited by 518 · see citing works
Breaking monotony with meaning: Motivation in crowdsourcing markets
Chandler, D. R., & Kapelner, A. D. (2013) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2013.03.003 · Cited by 400 · see citing works
StyleNet: Generating Attractive Visual Captions with Styles
Gan, C., Gan, Z., He, X., Gao, J., & Deng, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.108 · Cited by 287 · see citing works
Martin, D., Hanrahan, B. V., O’Neill, J., & Gupta, N. (2014) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2531602.2531663 · Cited by 279 · see citing works
Salehi, N., Irani, L. C., Bernstein, M. S., Alkhatib, A., Ogbe, E., Milland, K., & Clickhappier (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702508 · Cited by 274 · see citing works
The Crowd is a Collaborative Network
Gray, M. L., Suri, S., Ali, S. S., & Kulkarni, D. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2819942 · Cited by 239 · see citing works
Chang, J. C., Amershi, S., & Kamar, E. S. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026044 · Cited by 217 · see citing works
Brabham, D. C. (2012) · Journal of Applied Communication Research · doi:10.1080/00909882.2012.693940 · Cited by 200 · see citing works
What Are You Talking About? Text-to-Image Coreference
Kong, C., Lin, D., Bansal, M., Urtasun, R., & Fidler, S. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.455 · Cited by 191 · see citing works
How Blind People Interact with Visual Content on Social Networking Services
Voykinska, V., Azenkot, S., Wu, S., & Leshed, G. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2820013 · Cited by 180 · see citing works
Encyclopedia of Human‐Computer Interaction
(2007) · The Electronic Library · doi:10.1108/02640470710729173 · Cited by 169 · see citing works
Rich Representations of Visual Content for Screen Reader Users
Morris, M. R., Johnson, J., Bennett, C. L., & Cutrell, E. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173633 · Cited by 159 · see citing works
Improving accessibility of the web with a computer game
von Ahn, L., Ginosar, S., Kedia, M., Liu, R., & Blum, M. (2006) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124785 · Cited by 156 · see citing works
Learning the Visual Interpretation of Sentences
Zitnick, C. L., Parikh, D., & Vanderwende, L. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.211 · Cited by 156 · see citing works
Deep multimodal semantic embeddings for speech and images
Harwath, D. F., & Glass, J. R. (2015) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2015.7404800 · Cited by 150 · see citing works
VizWiz::LocateIt - enabling blind people to locate objects in their environment
Bigham, J. P., Jayant, C., Miller, A. E., White, B., & YEH, T. P. (2010) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2010.5543821 · Cited by 137 · see citing works
McInnis, B. J., Cosley, D., Nam, C., & Leshed, G. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858539 · Cited by 119 · see citing works
Framing image description as a ranking task
HodoshMicah, YoungPeter, & HockenmaierJulia (2013) · doi:10.5555/2566972.2566993 · Cited by 98 · see citing works
Crowdsourcing subjective fashion advice using VizWiz
Burton, M. A., Brady, E. L., Brewer, R. N., Neylan, C., Bigham, J. P., & Hurst, A. K. (2012) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2384916.2384941 · Cited by 95 · see citing works
How Teens with Visual Impairments Take, Edit, and Share Photos on Social Media
Bennett, C. L., E, J. L., Mott, M. E., Cutrell, E. B., & Morris, M. R. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173650 · Cited by 94 · see citing works
Turking in a Global Labour Market
Martin, D., O’Neill, J., Gupta, N., & Hanrahan, B. V. (2016) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-015-9241-6 · Cited by 87 · see citing works
Gupta, N., Martin, D., Hanrahan, B. V., & O’Neill, J. (2014) · International ACM SIGGROUP Conference on Supporting Group Work (GROUP) · doi:10.1145/2660398.2660403 · Cited by 82 · see citing works
Michie, L., Balaam, M., McCarthy, J., Osadchiy, T., & Morrissey, K. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173931 · Cited by 81 · see citing works
Kairam, S., & Heer, J. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2820016 · Cited by 72 · see citing works
"Could You Define That in Bot Terms"?: Requesting, Creating and Using Bots on Reddit
Long, K. S., Vines, J., Sutton, S., Brooker, P., Feltwell, T., Kirman, B., Barnett, J., & Lawson, S. (2017) · International Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025830 · Cited by 72 · see citing works
Guiding Novice Web Workers in Making Image Descriptions Using Templates
Morash, V. S., Siu, Y.-T., Miele, J. A., Hasty, L., & Landau, S. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2764916 · Cited by 69 · see citing works
Zhao, Y., Wu, S., Reynolds, L., & Azenkot, S. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134756 · Cited by 68 · see citing works
Crowdsourcing as a Tool for Research
Law, E., Gajos, K. Z., Wiggins, A., Gray, M. L., & Williams, A. C. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998197 · Cited by 68 · see citing works
Fiesler, C. L., & Hallinan, B. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173627 · Cited by 67 · see citing works
BrowseWithMe: An Online Clothes Shopping Assistant for People with Visual Impairments
Stangl, A. J., Kothari, E., Jain, S. D., Yeh, T., Grauman, K., & Gurari, D. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236337 · Cited by 66 · see citing works
Self-citationUnderstanding Volunteer AT Fabricators
Parry-Hill, J. L., Shih, P. C., Mankoff, J. C., & Ashbrook, D. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3026045 · Cited by 66 · see citing works
Assessing Image Quality Issues for Real-World Problems
Chiu, T.-Y., Zhao, Y., & Gurari, D. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00370 · Cited by 63 · see citing works
Self-citationWhy Does a Visual Question Have Different Answers?
Bhattacharya, N., Li, Q., & Gurari, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00437 · Cited by 59 · see citing works
Self-citationChiang, C., Kasunic, A., & Savage, S. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274306 · Cited by 57 · see citing works
"Our Privacy Needs to be Protected at All Costs"
Xia, H., Wang, Y., Huang, Y., & Shah, A. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134748 · Cited by 52 · see citing works
Chang, J. C., Kittur, A., & Hahn, N. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858411 · Cited by 46 · see citing works
CrowdVerge: Predicting If People Will Agree on the Answer to a Visual Question
Gurari, D., & Grauman, K. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025781 · Cited by 45 · see citing works
Self-citationWorker Demographics and Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Hanrahan, B. V., Bigham, J. P., & Callison-Burch, C. (2019) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312970 · Cited by 45 · see citing works
Sudhahar, S., Lansdall-Welfare, T., Flaounas, I., & Cristianini, N. (2012) · Cited by 39 · see citing works
Déjà Image-Captions: A Corpus of Expressive Descriptions in Repetition
Chen, J., Kuznetsova, P., Warren, D. S., & Choi, Y. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1053 · Cited by 32 · see citing works
The Application and Its Consequences for Non-Standard Knowledge Work
Nouwens, M., & Klokmose, C. N. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173973 · Cited by 27 · see citing works
Motivation of Workers on Microtask Crowdsourcing Platforms
Naderi, B. (2018) · T-labs series in telecommunication services · doi:10.1007/978-3-319-72700-4 · Cited by 27 · see citing works
Diversity & Inclusion and innovation: a virtuous cycle
Steele, R., & Derven, M. (2015) · Industrial and Commercial Training · doi:10.1108/ict-09-2014-0063 · Cited by 23 · see citing works
Investigating the Amazon Mechanical Turk Market Through Tool Design
Hanrahan, B. V., MARTIN, D. J., Willamowski, J. K., & Carroll, J. M. (2018) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-018-9312-6 · Cited by 17 · see citing works
Enabling the Disagreement among Crowds: A Collaborative Crowdsourcing Framework
Wang, M., Sun, Y., Yang, J., & He, L. (2018) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd.2018.8465368 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Stangl, A., Verma, N., Fleischmann, K. R., Morris, M. R., & Gurari, D. (2021) · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3471233 · Cited by 81 · see citing works
Self-citationImage Captioning as an Assistive Technology: Lessons Learned from VizWiz 2020 Challenge
Dognin, P., Melnyk, I., Mroueh, Y., Padhi, I., Rigotti, M., Ross, J., Schiff, Y., Young, R. A., & Belgodere, B. M. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13113 · Cited by 54 · see citing works
Privacy Concerns for Visual Assistance Technologies
Stangl, A., Shiroma, K., Davis, N., Xie, B., Fleischmann, K. R., Findlater, L., & Gurari, D. (2022) · ACM Transactions on Accessible Computing · doi:10.1145/3517384 · Cited by 35 · see citing works
Self-citationAngerbauer, K., Rodrigues, N., Cutura, R., Öney, S., Pathmanathan, N., Morariu, C., Weiskopf, D., & Sedlmair, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502133 · Cited by 32 · see citing works
Jiang, L., Jung, C., Phutane, M., Stangl, A. J., & Azenkot, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642238 · Cited by 26 · see citing works
Ga11y: An Automated GIF Annotation System for Visually Impaired Users
Zhang, M. R., Zhong, M., & Wobbrock, J. O. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502092 · Cited by 25 · see citing works
Jung, J. Y., Steinberger, T., Kim, J., & Ackerman, M. S. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533522 · Cited by 23 · see citing works
Stangl, A., Sadjo, E., Emami-Naeini, P., Wang, Y., Gurari, D., & Findlater, L. (2023) · 20th International Web for All Conference · doi:10.1145/3587281.3587296 · Cited by 15 · see citing works
Self-citationSuggestions for Online User Studies
Salminen, J. O., Jung, S., & Jansen, B. J. J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-90238-4_11 · Cited by 13 · see citing works
Edwards, E. J., Polster, K. L., Tuason, I., Blank, E., Gilbert, M., & Branham, S. M. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3471222 · Cited by 12 · see citing works
The State of Pilot Study Reporting in Crowdsourcing: A Reflection on Best Practices and Guidelines
Oppenlaender, J., Abbas, T., & Gadiraju, U. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641023 · Cited by 11 · see citing works
“Malicious” Pictorials: How Alt Text Matters to Screen Reader Users' Experience of Image-Dense Media
Yin, A., Sogani, R., Oewel, B., Phan, K., Park, J. S. Y., Yeo, M. A., Yazzolino, L. A., Arcos, K., Abdolrahmani, A., Blank, E., et al. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660747 · Cited by 9 · see citing works
Quality Control in Crowdsourcing based on Fine-Grained Behavioral Features
Pei, W., Yang, Z., Chen, M., & Yue, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479586 · Cited by 8 · see citing works
Landscape Analysis of Commercial Visual Assistance Technologies
Sadjo, E., Findlater, L., & Stangl, A. J. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3476521 · Cited by 8 · see citing works
Crowdsourcing Thumbnail Captions: Data Collection and Validation
Aguirre, C., Cao, S., Mahmood, A., & Huang, C.-M. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3589346 · Cited by 6 · see citing works
AccessShare: Co-designing Data Access and Sharing with Blind People
Kamikubo, R., Zeraati, F. Z., Lee, K., & Kacorri, H. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675612 · Cited by 6 · see citing works
Salameh, R., Lanir, J., & Szpiro, S. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2531268 · Cited by 6 · see citing works
Tang, X., Li, J., & Wu, S. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790405 · Cited by 4 · see citing works
Case Law Grounding: Using Precedents to Align Decision-Making for Humans and AI
Chen, Q. Z., & Zhang, A. X. (2025) · ACM Collective Intelligence Conference (CI) · doi:10.1145/3715928.3737487 · Cited by 4 · see citing works
Angerbauer, K., & Sedlmair, M. (2022) · IEEE Evaluation and Beyond - Methodological Approaches for Visualization (BELIV) · doi:10.1109/beliv57783.2022.00007 · Cited by 4 · see citing works
Garg, K., Tang, X., Heo, J., Morgan, D. R., Gergle, D., Sudderth, E. B., & Piper, A. M. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791309 · Cited by 3 · see citing works
Fernandes, D. L., Ribeiro, M. H. F., Silva, M. M., & Cerqueira, F. R. (2024) · Disability and Rehabilitation Assistive Technology · doi:10.1080/17483107.2024.2437567 · Cited by 3 · see citing works
Cripping AI: Reimagining AI Through Lived Disability Experiences
Tang, X., Lin, T.-A., Li, J., & Wu, S. (2026) · Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3805689.3806744 · Cited by 2 · see citing works
PRACTICES OF BENCHMARKING, VULNERABILITY IN THE COMPUTER VISION PIPELINE
maleve, N. (2023) · Photographies · doi:10.1080/17540763.2023.2189159 · Cited by 2 · see citing works
Qiu, S., Gadiraju, U., & Zheng, X. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729201 · Cited by 1 · see citing works
CoSight: Exploring Viewer Contributions to Online Video Accessibility Through Descriptive Commenting
Wang, R., Liu, X., Wang, B., Zhang, W., Liao, Z., Li, Z., Pavel, A., & Chen, X. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747747 · Cited by 1 · see citing works
Towards Adaptable and Interactive Image Captioning with Data Augmentation and Episodic Memory
Anagnostopoulou, A., Hartmann, M., & Sonntag, D. (2023) · Workshop on Simple and Efficient Natural Language Processing (SustaiNLP) · doi:10.18653/v1/2023.sustainlp-1.19 · Cited by 1 · see citing works
The Consensus Trap: Dissecting Subjectivity and the “Ground Truth” Illusion in Data Annotation
Munir, S., Mah, B., Kalsi, K., Kapania, S., Posada, J., Law, E., Wang, D., & Ahmed, S. I. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806510
Generating clothing graphic captions for visually impaired users
Alluqmani, A., Harvey, M., & Paramita, M. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21774-w
Adapting Socio-Technical Congruence to Human-Agent Collectives
Oppenlaender, J., Visuri, A., & Hosio, S. J. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838753
- Authors
- Simons, R. N., Gurari, D., & Fleischmann, K. R.
- Affiliations
- Texas Woman's University (US)
- The University of Texas at Austin (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2020-10-14
- Volume
- 4
- Issue
- CSCW2
- Pages
- 1-26
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3094043653
- Cited by
- 27
- References
- 60
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 5 · 2023: 4 · 2022: 6 · 2021: 5
- FWCI
- 3.42
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- image captioning, crowdsourcing, Amazon Mechanical Turk, blind users, task design, dataset construction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
