CoChat

Huddler: Convening Stable and Familiar Crowd Teams Despite Unpredictable Availability

Salehi, Niloufar, McCabe, Andrew, Valentine, Melissa, Bernstein, Michael · 2017 · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing
Cited by 63

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Abstract

Distributed, parallel crowd workers can accomplish simple tasks through workflows, but teams of collaborating crowd workers are necessary for complex goals. Unfortunately, a fundamental condition for effective teams -- familiarity with other members -- stands in contrast to crowd work's flexible, on-demand nature. We enable effective crowd teams with Huddler, a system for workers to assemble familiar teams even under unpredictable availability and strict time constraints. Huddler utilizes a dynamic programming algorithm to optimize for highly familiar teammates when individual availability is unknown. We first present a field experiment that demonstrates the value of familiarity for crowd teams: familiar crowd teams doubled the performance of ad-hoc (unfamiliar) teams on a collaborative task. We then report a two-week field deployment wherein Huddler enabled crowd workers to convene highly familiar teams in 18 minutes on average. This research advances the goal of supporting long-term, team-based collaborations without sacrificing the flexibility of crowd work.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2998181.2998300

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 44

  1. 10.1111/j.1083-6101.2010.01529.x · DOI · Paper page
  2. Linda Argote Paul Ingram John M. Levine and Richard L. Moreland. "Knowledge transfer in organizations: Learning from the experience of others." Organizational behavior and human decision processes 82 no. 1 (2000): 1--8. · DOI · Paper page
  3. Linda Argote. "Input uncertainty and organizational coordination in hospital emergency units." Administrative science quarterly (1982): 420--434.
  4. Jeffrey P. Bigham, Michael S. Bernstein, and Eytan Adar. "Human-Computer Interaction and Collective Intelligence." Handbook of Collective Intelligence (2015): 57.
  5. 10.1287/orsc.3.1.20 · DOI · Paper page
  6. Tom Finholt Lee Sproull and Sara Kiesler. "Communication and performance in ad hoc task groups." Intellectual teamwork: Social and technological foundations of cooperative work (1990): 291--325. · doi:10.5555/117848.117859
  7. 10.1080/13691180110097030 · DOI · Paper page
  8. Scott B. Droege, and Jenny M. Hoobler. "Employee turnover and tacit knowledge diffusion: A network perspective." Journal of Managerial Issues(2003): 5064.
  9. Amy C. Edmondson Teaming: How organizations learn, innovate, and compete in the knowledge economy. John Wiley & Sons, 2012.
  10. 10.2753/MIS0742-1222240104 · DOI · Paper page
  11. 10.1287/mnsc.31.9.1055 · doi:10.1287/mnsc.31.9.1055
  12. 10.1145/2818048.2819942 · DOI · Paper page
  13. Deborah H. Gruenfeld Elizabeth A. Mannix Katherine Y. Williams and Margaret A. Neale. "Group composition and decision making: How member familiarity and information distribution affect process and performance." Organizational behavior and human decision processes 67 no. 1 (1996): 1--15. · DOI · Paper page
  14. 10.1108/09696470910939215 · DOI
  15. 10.14778/2824032.2824062 · DOI
  16. J. Richard. Hackman, Leading teams: Setting the stage for great performances. Harvard Business Press, 2002.
  17. 10.1111/j.1744-6570.2003.tb00753.x · DOI · Paper page
  18. Mark Hefke, and Ljiljana Stojanovic. "An ontologybased approach for competence bundling and composition of ad-hoc teams in an organization." Proc. I-KNOW'04 (2004): 126--134.
  19. 10.5555/2007322.2007325 · doi:10.5555/2007322.2007325
  20. 10.1287/mnsc.1080.0921 · DOI · Paper page
  21. 10.1145/1240624.1240698 · DOI · Paper page
  22. 10.1145/2047196.2047202 · DOI · Paper page
  23. 10.1145/1869086.1869096 · DOI
  24. David A. Kravitz and Barbara Martin. "Ringelmann rediscovered: The original article." (1986): 936. · doi:10.1037//0022-3514.50.5.936 · Paper page
  25. Karim Lakhani, David A. Garvin, and Eric Lonstein. "Topcoder (a): Developing software through crowdsourcing." Harvard Business School General Management Unit Case 610-032 (2010).
  26. 10.1145/2702123.2702565 · DOI · Paper page
  27. 10.1177/0146167295214009 · DOI · Paper page
  28. 10.1145/1600150.1600159 · DOI · Paper page
  29. 10.1145/2851581.2892421 · DOI
  30. Ioanna Lykourentzou, Angeliki Antoniou, Yannick Naudet, and Steven P. Dow. "Personality Matters: Balancing for Personality Types Leads to Better Outcomes for Crowd Teams." (2016).
  31. 10.1371/journal.pone.0153048 · DOI
  32. 10.1145/2531602.2531663 · DOI · Paper page
  33. 10.1207/S15327957PSPR0401_8 · DOI · Paper page
  34. Mitzi M. Montoya-Weiss, Anne P. Massey, and Michael Song. "Getting it together: Temporal coordination and conflict management in global virtual teams." Academy of management Journal 44, no. 6 (2001): 1251--1262. · DOI · Paper page
  35. M. Mortensen 2014 "Constructing the Team: The Antecedents and Effects of Membership Model Divergence." Organization Science 25: 909--931. 10.1287/orsc.2013.0881 · DOI
  36. 10.1089/109493104323024410 · doi:10.1089/109493104323024410
  37. 10.1287/mnsc.1050.0366 · DOI · Paper page
  38. 10.1145/2642918.2647409 · DOI · Paper page
  39. 10.5465/19416520.2013.766405 · DOI
  40. 10.1145/1978942.1979102 · DOI · Paper page
  41. Anita Williams Woolley Christopher F. Chabris Alex Pentland Nada Hashmi and Thomas W. Malone. "Evidence for a collective intelligence factor in the performance of human groups." science 330 no. 6004 (2010): 686--688. · DOI · Paper page
  42. 10.1126/science.1205869 · DOI · Paper page
  43. 10.1145/2702123.2702508 · DOI · Paper page
  44. 10.1145/2531602.2531718 · DOI

Cited by

See all 63

Metadata

Authors
Salehi, N., McCabe, A., Valentine, M., & Bernstein, M.
Venue
Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing
Publisher
ACM
Published
2017-02-25
Pages
1700-1713
OpenAlex
W2544952174
Cited by
63
References
9
FWCI
8.86
Type
conference paper
Language
en
Open access
closed
Funders
  • Office of Naval Research (N00014-16-1-2894)
  • National Science Foundation (IIS-1351131)
  • Stanford Graduate Fellowship

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.