Shepherding the crowd yields better work

Dow, Steven, Kulkarni, Anand, Klemmer, Scott, Hartmann, Björn·2012·Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work
DOI 10.1145/2145204.2145355

Cited by (334)

Items 201–300 of 334

  1. Bridge the Gap!
    Obinna Anya · 2015 · ACM Conference on Computer Supported Cooperative Work (CSCW)
  2. Predicting Crowd Work Quality under Monetary Interventions
    Ming Yin, Yiling Chen · 2016 · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
  3. SeCrowd: Efficient secure interactive crowdsourcing via permission-based signatures
    Jing Chen, Jiong Chen, Kun He, Ruiying Du · 2020 · Future Generation Computer Systems
  4. Predicting budget for Crowdsourced and Freelance Software development Projects
    K. Abhinav, Alpana Dubey · 2017 · International Symposium on Electronic Commerce
  5. You Know What I Meme: Enhancing People's Understanding and Awareness of Hateful Memes Using Crowdsourced Explanations
    Nanyi Bi, Yi‐Ching Huang, Chao-Chun Han, Jane Yung-jen Hsu · 2023 · Proceedings of the ACM on Human-Computer Interaction
  6. The role of rating feedback and its implications for solver submission behavior in crowdsourcing contests
    Mengmeng Wang, Xiaocong Cui · 2023 · Information & Management
  7. Hidden Indicators of Collective Intelligence in Crowdfunding
    Emőke-Ágnes Horvát, Henry Kudzanai Dambanemuya, Jayaram Suryanarayana Uparna, Brian Uzzi · 2023 · International World Wide Web Conference (WWW)
  8. Collective Creativity through a Micro-Tasks Crowdsourcing Approach
    Victor Girotto · 2016 · ACM Conference on Computer Supported Cooperative Work (CSCW)
  9. Multi-Winner Contests for Strategic Diffusion in Social Networks
    Wen Shen, Yang Feng, Cristina V. Lopes · 2019 · Proceedings of the AAAI Conference on Artificial Intelligence
  10. In Search of Ambiguity: A Three-Stage Workflow Design to Clarify Annotation Guidelines for Crowd Workers
    Vivek Krishna Pradhan, Mike Schaekermann, Matthew Lease · 2022 · Frontiers in Artificial Intelligence
  11. Managing Conflict in Online Debate Communities: Foregrounding Moderators' Beliefs and Values on Kialo
    Jordan Beck, Bikalpa Neupane, John M. Carroll · 2018
  12. OSINT Clinic: Co-designing AI-Augmented Collaborative OSINT Investigations for Vulnerability Assessment
    Anirban Mukhopadhyay, Kurt Luther · 2025 · CHI Conference on Human Factors in Computing Systems
  13. Crafting Policy Discussion Prompts as a Task for Newcomers
    Brian James McInnis, Gilly Leshed, Dan Cosley · 2018 · Proceedings of the ACM on Human-Computer Interaction
  14. Crowd-Worker Skill Improvement with AI Co-Learners
    Takumi Nakayama, Masaki Matsubara, Atsuyuki Morishima · 2021 · International Conference on Human-Agent Interaction (HAI)
  15. A longitudinal study of hacker behaviour
    Thomas Walshe, Andrew C. Simpson · 2022 · ACM Symposium on Applied Computing
  16. Exploring the Challenges and Opportunities in Developing Systems to Improve Alcohol Use Disorder through Chatbot Technology
    Xiang-Zhi Qiu, Chien Wen Yuan, Nanyi Bi, Ming‐Chyi Huang, Chuang-Wen You · 2023 · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA)
  17. Meet your match rating
    Anurag Sarkar, Seth Cooper · 2018 · International Conference on the Foundations of Digital Games (FDG)
  18. CrowdTrace: Visualizing Provenance in Distributed Sensemaking
    Tianyi Li, Yasmine Belghith, Chris L. North, Kurt Luther · 2020 · Visualization
  19. Collaborative Solving in a Human Computing Game Using a Market, Skills and Challenges
    Olivier Tremblay-Savard, Alexander Butyaev, Jérôme Waldispühl · 2016 · Annual Symposium on Computer-Human Interaction in Play
  20. Comparing infrared and webcam eye tracking in the Visual World Paradigm
    Myrte Vos, Serge Minor, Gillian Catriona Ramchand · 2022
  21. Belief Miner: A Methodology for Discovering Causal Beliefs and Causal Illusions from General Populations
    Shahreen Salim, Md Naimul Hoque, Klaus D. Mueller · 2024 · Proceedings of the ACM on Human-Computer Interaction
  22. Revolutionizing machine learning: Blockchain-based crowdsourcing for transparent and fair labeled datasets supply
    Haitao Xu, Zheng He, Dapeng Lan · 2024 · Future Generation Computer Systems
  23. Rethinking Quality Assurance for Crowdsourced Multi-ROI Image Segmentation
    Xiaolu Lu, David Ratcliffe, Tsu-Ting Kao, Aristarkh Tikhonov, Lester Litchfield, Craig J. Rodger, Kaier Wang · 2023 · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
  24. Motivational feedback in crowdsourcing: a case study in speech transcription
    Giuseppe Riccardi, A. Ghosh, Shammur Absar Chowdhury, Ali Orkan Bayer · 2013 · Interspeech (USB)
  25. Agreement-based credibility assessment and task replication in human computation systems
    Lesandro Ponciano, Francisco Brasileiro · 2018 · Future Generation Computer Systems
  26. A Learning Effect by Presenting Machine Prediction as a Reference Answer in Self-correction
    Masaki Matsubara, Masaki Kobayashi, Atsuyuki Morishima · 2018 · IEEE International Conference on Big Data (BigData)
  27. PreSight: Enabling Real-Time Detection of Accessibility Problems on Sidewalks
    Li Zheng, Mahbubur Rahman, Ryan Robucci, Nilanjan Banerjee · 2017 · IEEE International Conference on Sensing, Communication, and Networking (SECON)
  28. Supporting ESL Writing by Prompting Crowdsourced Structural Feedback
    Yi‐Ching Huang, Jiunn-Chia Huang, Hao‐Chuan Wang, Jane Yung-jen Hsu · 2017 · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
  29. Dynamic Weighted Majority Approach for Detecting Malicious Crowd Workers
    Meisam Nazariani, Ahmad Abdollahzadeh Barforoush · 2019 · Canadian Journal of Electrical and Computer Engineering
  30. Is Quality Control Pointless?
    Markus Krause, Farhad Mohammad Afzali, Simon J. Caton, Margeret Hall · 2019 · Hawaii International Conference on System Sciences (HICSS)
  31. Mooqita
    Markus Krause, Doris Schiöberg, Jan David Smeddinck · 2018 · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA)
  32. Crowd-supervised training of spoken language systems
    Stephanie Seneff, Ian C. Mcgraw · 2012
  33. Methods and models for the quantitative analysis of crowd brainstorming
    Filip Krynicki · 2014 · UWSpace (University of Waterloo)
  34. Prototype Tasks: Improving Crowdsourcing Results through Rapid, Iterative Task Design
    Snehalkumar S. Gaikwad, Nalin Chhibber, Vibhor Sehgal, Alipta Ballav, Catherine A. Mullings, Nasser, Ahmed, Angela Richmond-Fuller, Aaron Gilbee, Dilrukshi Gamage, Mark E. Whiting et al. · 2017 · arXiv (Cornell University)
  35. Crowd Work on a CV? Understanding How AMT Fits into Turkers' Career Goals and Professional Profiles
    Anna Kasunic, Chiang, Chun-Wei, Geoff F. Kaufman, Saiph Savage · 2019 · arXiv (Cornell University)
  36. On the State of Reporting in Crowdsourcing Experiments and a Checklist to Aid Current Practices
    Ramírez, Jorge, Sayin, Burcu, Baez, Marcos, Casati, Fabio, Cernuzzi, Luca, Benatallah, Boualem, Demartini, Gianluca · 2021 · Proceedings of the ACM on Human-Computer Interaction
  37. Hybrid Machine-Crowd Interaction for Handling Complexity: Steps Toward a Scaffolding Design Framework
    António Correia, Shoaib Jameel, Hugo Paredes, Benjamim Fonseca, Daniel M. Schneider · 2019 · Human-computer interaction series
  38. A Systematic Literature Survey of Crowdsourcing: Current Status and Future Perspectives
    Himanshu Suyal, Avtar Singh · 2025 · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery
  39. Human computer interaction paradigm for business process task crowdsourcing
    Chithralekha Balamurugan, Shourya Roy · 2013 · Asia-Pacific Computer and Human Interaction
  40. Real-time trip planning with the crowd
    Joseph Rafidi · 2013 · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA)
  41. Automatically Providing Action Plans Helps People Complete Tasks
    Nicolas Kokkalis, Johannes Huebner, Steven Diamond, Dominic Becker, Michael Chang, Moontae Lee, Florian Schulze, Thomas Koehn, Scott Klemmer · 2012
  42. Planet Four: Craters—Optimizing task workflow to improve volunteer engagement and crater counting performance
    James Sprinks, Robert J. Houghton, S. P. Bamford, Jeremy G. Morley · 2019 · Meteoritics and Planetary Science
  43. FourEyes
    Jean Young Song, Raymond Fok, Juho Kim, Walter S. Lasecki · 2019 · ACM Transactions on Interactive Intelligent Systems
  44. On the Impact of Predicate Complexity in Crowdsourced Classification Tasks
    Jorge Ramírez, Marcos Báez, Fabio Casati, Luca Cernuzzi, Boualem Benatallah, Таран Екатерина Александровна, Вероника Анатольевна Маланина · 2021 · ACM International Conference on Web Search and Data Mining (WSDM)
  45. Paid Crowdsourcing as Concept and Content Generator to Enhance Museum Experiences
    L. van der Lans, Evy Ligia Ansems, Vassilis-Javed Khan · 2018 · Springer series on cultural computing
  46. It's about time: Online Macrotask Sequencing in Expert Crowdsourcing
    Heinz Schmitz, Ioanna Lykourentzou · 2016 · arXiv (Cornell University)
  47. Supporting Crowd Workers in Ideation Tasks Through Information Gathering and Reflective Activity
    Lingyun Sun, Weiyue Gao, Wei Xiang · 2022 · International Journal of Human-Computer Interaction
  48. Co-Designing Collaborative Generative AI Tools for Freelancers
    Kashif Imteyaz, Michael J. Muller, Claudia Flores-Saviaga, Saiph Savage · 2026 · CHI Conference on Human Factors in Computing Systems
  49. Quality management of user-generated content in participatory journalism
    Satu Jumisko Pyykko, Heli Väätäjä, Markus S. Jaakola · 2014 · Australian Conference on Computer-Human Interaction (OzCHI)
  50. Dynamic Labeling: A Control System for Labeling Styles in Image Annotation Tasks
    Chia-Ming Chang, Yi He, Xusheng Du, Xi Yang, Haoran Xie · 2024 · Lecture notes in computer science
  51. Knowledge based quality analysis of crowdsourced software development platforms
    Asad Habib, Shahid Hussain, Arif Ali Khan, Muhammad Khalid Sohail, Manzoor Ilahi, Muhammad Rafiq Mufti, Muhammad Imran Faisal · 2018 · Computational and Mathematical Organization Theory
  52. TO ESTABLISH CROWDSOURCING AS AN ORGANIZATIONAL BUSINESS PROCESS: AN EXPLORATORY STUDY
    David Johnstone · 2013
  53. Development of a general crowdsourcing maturity model
    Carlos Mario Durango Yepes, Víctor Daniel Gil Vera · 2016 · Cuadernos de Administración
  54. Read-Agree-Predict: A Crowdsourced Approach to Discovering Relevant Primary Sources for Historians
    Nai-Ching Wang, David L. Hicks, Paul Quigley, Kurt Luther · 2019 · Human Computation
  55. WearWrite: Orchestrating the Crowd to Complete Complex Tasks from Wearables (We Wrote This Paper on a Watch)
    Michael Nebeling, Anhong Guo, Kyle I. Murray, Annika Tostengard, Angelos Giannopoulos, Martin Mihajlov, Steven P. Dow, Jaime B. Teevan, Jeffrey P. Bigham · 2015 · arXiv (Cornell University)
  56. Learning on the Go: Understanding How Gig Economy Workers Learn with Recommendation Algorithms
    Shunan Jiang, W. Sinchaisri · 2025 · Proc. ACM Hum. Comput. Interact.
  57. Crowd-Enhanced Cloud Services: Issues and Directions
    Saeed Arbabı, Mohammad Allah Bakhsh, Mohsen Sharifi · 2015 · International Journal of Computer Applications
  58. Teachable Conversational Agents for Crowdwork: Effects on Performance and Trust
    Nalin Chhibber, Joslin Goh, Edith Law · 2022 · Proceedings of the ACM on Human-Computer Interaction
  59. Understanding how early-stage researchers leverage socio-technical affordances for distributed research support
    Yuchao Jiang, Boualem Benatallah, Marcos Báez · 2023 · Information and Software Technology
  60. Characterizing Time Spent in Video Object Tracking Annotation Tasks: A Study of Task Complexity in Vehicle Tracking
    Amy Rechkemmer, Alex C. Williams, Matthew Lease, Li Erran Li · 2023 · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
  61. Crowd Learning
    David Elijah Kalisz · 2015 · Advances in logistics, operations, and management science book series
  62. Accurate inference of crowdsourcing properties when using efficient allocation strategies
    Abigail Hotaling, James P. Bagrow · 2022 · Scientific Reports
  63. Frenzy: A Platform for Friendsourcing
    Lydia B. Chilton, Felicia Cordeiro, Daniel S. Weld, James A. Landay · 2013 · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
  64. Predicting the quality of contests on crowdsourcing-based software development platforms
    Mohammad Imran Faisal · 2017 · ACM Symposium on Applied Computing
  65. A Game-Theoretic Approach to Quality Improvement in Crowdsourcing Tasks
    Mohammad Allahbakhsh, Haleh Amintoosi, Salil S. Kanhere · 2018 · Lecture notes in business information processing
  66. An ML-Powered Human Behavior Management System
    Sihem Amer-Yahia, Reynold C. K. Cheng, Mohamed Bouadi, Abdelouahab Chibah, Mohammadreza Esfandiari, Jiangping Zhou, Nan Zhang, Eric H. Y. Lau, Yuguo Li, Xiaolin Han et al. · 2020 · HAL
  67. Modeling evolutionary responses in crowdsourcing MCQ using belief function theory
    Constance Thierry, Arnaud Martin B, Yolande Le Gall, Jean-Cristophe Dubois · 2023 · Procedia Computer Science
  68. On Using Hybrid Pedagogy as Guideline for Improving Assessment Design.
    Christian Köppe, Rody Middelkoop · 2019
  69. Big Data Analytics Using Cloud and Crowd
    Mohammad Allahbakhsh, Saeed Arbabı, Hamid Reza Motahari‐Nezhad, Boualem Benatallah · 2016 · arXiv (Cornell University)
  70. The Crowdrouter Framework: Addressing Issues of Software Design in Support of Crowdwork
    Mario Barrenechea · 2016 · CU Scholar (University of Colorado Boulder)
  71. CrowdTone: Crowd-powered tone feedback and improvement system for emails
    Rajan Vaish, Andrés Monroy‐Hernández · 2017 · arXiv (Cornell University)
  72. WeAudit: Scaffolding User Auditors and AI Practitioners in Auditing Generative AI
    Wesley Hanwen Deng, Claire Wang, Howard Ziyu Han, Jason Hong, Kenneth Holstein, Motahhare Eslami · 2025 · Qeios
  73. Context-aware worker selection for efficient quality control in crowdsourcing
    Tarek M. Awwad · 2018 · theses.fr (ABES)
  74. How well did I do? The effect of feedback on affective commitment in the context of microwork
    Anna-Maria Staiger, Johannes Schmidt, Dietrich von der Oelsnitz · 2022 · Hawaii International Conference on System Sciences (HICSS)
  75. Empirical Study on Effects of Self-Correction in Crowdsourced Microtasks
    Masaki Kobayashi, Hiromi Morita, Masaki Matsubara, Nobuyuki Shimizu, Atsuyuki Morishima · 2021 · Human Computation
  76. A Crowdsourcing Task Allocation Mechanism for Hybrid Worker Context Based on Skill Level Updating
    Jiuchuan Jiang, Jinpeng Wei · 2024 · Communications in computer and information science
  77. A System Design Perspective for Business Growth in a Crowdsourced Data Labeling Practice
    Vahid Hajipour, Sajjad Jalali, Francisco J. Santos‐Arteaga, Samira Vazifeh-Noshafagh, Debora Di Caprio · 2024 · Algorithms
  78. Living with Bats: Designing an Interactive Space for Human-Wildlife Cohabitation
    Joseph Samuel Johnson, Jacob Koch, Rob Grace, Feifei Pang, Roshanak Basty, Mustapha Awinsongya Yakubu, Emmanuel Kojo Gyamfi, Abigail Mesrenyame Dogbe, Adriana Navarro Sainz, Kemi Akanbi et al. · 2025 · ACM Journal on Computing and Sustainable Societies
  79. IdeaGens: A Social Ideation System for Guided Crowd Brainstorming
    Joel Chan, Steven Dang, Peter Kremer, Lucy Guo, Steven P. Dow · 2014 · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing
  80. What the Crowd Yields: Considerations when Crowdsourcing
    Jeff Scagnelli · 2013 · Survey Practice
  81. Large lectures are hard: an exploration in meaningful social visualizations for large group communication
    Andrew Harris · 2012 · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
  82. Components and Functions of Crowdsourcing Systems
    Lars Hetmank · 2013
  83. MobileWorks: Designing for Quality in a Managed Crowdsourcing Architecture (Extended Abstract).
    Anand P. Kulkarni, David Rolnitzky, Philipp Gutheim, Prayag Narula, Tapan S. Parikh, Björn Hartmann · 2012 · National Conference on Artificial Intelligence
  84. Outsider Perspectives
    Rhema Linder, Shamsi T. Iqbal, Jaime B. Teevan · 2018 · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA)
  85. RIPT
    Tiffany Wun, Claire Mikalauskas, Kevin Ta, Joshua Horacsek, Lora Oehlberg · 2018 · ACM Conference Companion Publication on Designing Interactive Systems (DIS Companion)
  86. Crowdsourcing Peer Review: As We May Do
    Michael Soprano, Stefano Mizzaro · 2019 · Communications in computer and information science
  87. Fostering Feedback Seeking Behavior in Novice Designers
    Sneha R. Krishna Kumaran · 2019 · Creativity and Cognition
  88. Software Crowdsourcing Design: An Experiment on the Relationship Between Task Design and Crowdsourcing Performance
    Turki Alelyani, Paul T. Grogan, Yla Tausczik, Ye Yang · 2020 · Lecture notes in computer science
  89. AsPeer: Method to Self and Peer Assessment in Large Online Design Classes
    Hariesh K. Sankaran, Raju Shanmugam, Bhavesh S. Shah, Sultan Ahmad, Hikmat A. M. Abdeljaber, Jabeen Nazeer · 2023 · International Journal of Emerging Technologies in Learning (iJET)
  90. Choosing Peers, Improve the Quality of Peer Assessment
    Jinshuang Li, Xiaohan Fu, Qiuwei Yang · 2016 · International Conference on Machinery, Materials and Information Technology Applications (ICMMITA)
  91. FOSSEE Fellowship 2019: Results of Crowdsourcing and Performance Based Selection
    Kannan M. Moudgalya, Usha Viswanathan, Vineeta Ghavri · 2019 · IEEE International Conference on Technology for Education (T4E)
  92. A Microtask Approach to Identifying Incomprehension for Facilitating Peer Learning
    Izumi Hinako, Masaki Matsubara, Chiemi Watanabe, Atsuyuki Morishima · 2019 · IEEE International Conference on Big Data (BigData)
  93. Summarizing the Crowdsourced Testing
    Hong Liang, Tieke He · 2019 · IEEE International Conference on Computer and Communications (ICCC)
  94. Sifter: A Hybrid Workflow for Theme-based Video Curation at Scale
    Yan Chen, Andrés Monroy‐Hernández, Ian Wehrman, Steve Oney, Walter S. Lasecki, Rajan Vaish · 2020 · ACM International Conference on Interactive Media Experiences
  95. Bashon: A Hybrid Crowd-Machine Workflow for Shell Command Synthesis
    Yan Chen, Jaylin Herskovitz, Walter S. Lasecki, Steve Oney · 2020 · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)
  96. Meta-Gig: Empowering anyone to create crowd marketplaces
    Carlos Toxtli, Saiph Savage · 2020 · Avances en Interacción Humano-Computadora
  97. Crowdsourcing
    Pär J. Ågerfalk, Brian F. Fitzgerald, Klaas-Jan Stol · 2015 · SpringerBriefs in computer science
  98. Crowd Learning
    David Elijah Kalisz · 2019 · IGI Global eBooks
  99. Spam elimination and bias correction : ensuring label quality in crowdsourced tasks.
    Lingyu Lyu · 2018 · ThinkIR: The University of Louisville's Institutional Repository (University of Louisville)
  100. Use and Effects of LLMs in Peer Review: A Randomized Experiment and Survey at ICML 2026
    Kim, Sunnie S. Y., Deng, Wesley Hanwen, Vaughan, Jennifer Wortman, Su, Buxin, Su, Weijie, Agarwal, Alekh, Li, Sharon, Jaggi, Martin, Goldstein, Daniel G., Shah, Nihar B. et al. · 2026 · arXiv