The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Human computation: a survey and taxonomy of a growing field
Abstract
The rapid growth of human computation within research and industry has produced many novel ideas aimed at organizing web users to do great things. However, the growth is not adequately supported by a framework with which to understand each new system in the context of the old. We classify human computation systems to help identify parallels between different systems and reveal "holes" in the existing work as opportunities for new research. Since human computation is often confused with "crowdsourcing" and other terms, we explore the position of human computation with respect to these related topics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1999) · Cited by 12,854
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290 · cited 3×
“The term human computation was used as early as 1838 [69] in philosophy and psychology literature, as well as more recently in the context of computer science theory [62].”
BackgroundMethodComputing Machinery and Intelligence (1950)
Turing, A. (2004) · The Essential Turing · doi:10.1093/oso/9780198250791.003.0017 · Cited by 6,118
Labeling images with a computer game
von Ahn, L., & Dabbish, L. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985733 · Cited by 2,204
CYC: a large-scale investment in knowledge infrastructure
Lenat, D. B. (1995) · Communications of the ACM · doi:10.1145/219717.219745 · Cited by 1,946
Licklider, J. C. R. (1960) · IRE Transactions on Human Factors in Electronics · doi:10.1109/thfe2.1960.4503259 · Cited by 1,938
(2008) · Cited by 1,657
Collective intelligence: mankind's emerging world in cyberspace
(1998) · Choice Reviews Online · doi:10.5860/choice.35-3911 · Cited by 1,526
Predicting protein structures with a multiplayer online game
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010) · Nature · doi:10.1038/nature09304 · Cited by 1,368 · cited 3×
“The game uses Rosetta energy a property of protein structures, to identify the most useful results [14].”
BackgroundInfluentialGet another label? improving data quality and data mining using multiple, noisy labelers
Sheng, V. S., Provost, F., & Ipeirotis, P. G. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401965 · Cited by 1,165
Designing games with a purpose
von Ahn, L., & Dabbish, L. (2008) · Communications of the ACM · doi:10.1145/1378704.1378719 · Cited by 1,155
reCAPTCHA: Human-Based Character Recognition via Web Security Measures
von Ahn, L., Maurer, B., McMillen, C., Abraham, D., & Blum, M. (2008) · Science · doi:10.1126/science.1160379 · Cited by 1,115
Quality management on Amazon Mechanical Turk
Ipeirotis, P. G., Provost, F., & Wang, J. (2010) · Proceedings of the ACM SIGKDD Workshop on Human Computation · doi:10.1145/1837885.1837906 · Cited by 973
von Ahn, L. (2006) · Computer · doi:10.1109/mc.2006.196 · Cited by 955
Wolfers, J., & Zitzewitz, E. (2004) · Journal of Economic Perspectives · doi:10.1257/0895330041371321 · Cited by 889 · cited 2×
“Several online polling web sites and prediction markets harness this concept to not only determine a group opinion, but to try to predict future events [72] (e.”
BackgroundAnalyzing the Amazon Mechanical Turk marketplace
Ipeirotis, P. G. (2010) · XRDS: Crossroads, The ACM Magazine for Students · doi:10.1145/1869086.1869094 · Cited by 879
Knowledge discovery and data mining: towards a unifying framework
Fayyad, U. M., Piatetsky-Shapiro, G., & Smyth, P. (1996) · Cited by 846
Soylent: a word processor with a crowd inside
Bernstein, M. S., Little, G., Miller, R. C., Hartmann, B., Ackerman, M. S., Karger, D. R., Crowell, D., & Panovich, K. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866078 · Cited by 686
VizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601
He says, she says: conflict and coordination in Wikipedia
Kittur, A., Suh, B., Pendleton, B. A., & Chi, E. H. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240698 · Cited by 574
VizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Tatarowicz, A., White, B., White, S., & Yeh, T. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806020 · Cited by 536
Open Mind Common Sense: Knowledge Acquisition from the General Public
Singh, P., Lin, T., Mueller, E. T., Lim, G., Perkins, T., & Li Zhu, W. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-36124-3_77 · Cited by 477
Financial incentives and the "performance of crowds"
Mason, W., & Watts, D. J. (2010) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1809400.1809422 · Cited by 463
Parameswaran, M., & Whinston, A. B. (2007) · Communications of the Association for Information Systems · doi:10.17705/1cais.01937 · Cited by 385
Designing Emergent Behaviors: From Local Interactions to Collective Intelligence
Mataric, M. J. (1993) · From Animals to Animats 2 · doi:10.7551/mitpress/3116.003.0059 · Cited by 312
Verbosity: a game for collecting common-sense facts
von Ahn, L., Kedia, M., & Blum, M. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124784 · Cited by 309
Creating Speech and Language Data With Amazon’s Mechanical Turk
Callison-Burch, C., & Dredze, M. H. (2010) · Cited by 294
Law, E., & von Ahn, L. (2009) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1518701.1518881 · Cited by 248
Picbreeder: evolving pictures collaboratively online
Secretan, J., Beato, N., D Ambrosio, D. B., Rodriguez, A., Campbell, A., & Stanley, K. O. (2008) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357328 · Cited by 192
A crowdsourceable QoE evaluation framework for multimedia content
Chen, K.-T., Wu, C.-C., Chang, Y.-C., & Lei, C.-L. (2009) · Proceedings of the 17th ACM international conference on Multimedia · doi:10.1145/1631272.1631339 · Cited by 192
von Ahn, L. (2008) · 2008 IEEE 24th International Conference on Data Engineering · doi:10.1109/icde.2008.4497403 · Cited by 172
Evaluation Of Algorithms Using Games: The Case Of Music Tagging.
Law, E., West, K., Mandel, M. I., Bay, M., & Downie, J. S. (2009) · Zenodo · doi:10.5281/zenodo.1417646 · Cited by 168
Exploring iterative and parallel human computation processes
Little, G., Chilton, L. B., Goldman, M., & Miller, R. C. (2010) · Proceedings of the ACM SIGKDD Workshop on Human Computation · doi:10.1145/1837885.1837907 · Cited by 166
Kosorukoff, A. L. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2001.972056 · Cited by 141
At what cost pervasive? A social computing view of mobile computing systems
Dryer, D. C., Eisbach, C., & Ark, W. (1999) · IBM Systems Journal · doi:10.1147/sj.384.0652 · Cited by 127
A Survey of Human Computation Systems
Yuen, M.-C., Chen, L., & King, I. (2009) · doi:10.1109/cse.2009.395 · Cited by 108
Chklovski, T. A. (2003) · International Conference on Knowledge Capture · doi:10.1145/945645.945650 · Cited by 81
PayCash: a secure efficient internet payment system
Peha, J. M., & Khamitov, I. M. (2004) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2004.06.001 · Cited by 55
Horvitz, E., & Paek, T. (2007) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-006-9026-1 · Cited by 26
KQQKQQ and the Kasparov-World Game
Nalimov, E. V., Wirth, C., & Haworth, G. M. (1999) · ICGA Journal · doi:10.3233/icg-1999-22402 · Cited by 22
Mathematical Modeling of Social Games
Chan, K. T., King, I., & Yuen, M.-C. (2009) · doi:10.1109/cse.2009.166 · Cited by 22
Towards Managing Knowledge Collection from Volunteer Contributors
Chklovski, T., & Gil, Y. (2005) · Cited by 21
Gomme, P. (2003) · RePEc: Research Papers in Economics · Cited by 20
A comprehensive human computation framework
Yang, Y., Zhu, B., Guo, R., Yang, L., Li, S., & Yu, N. (2008) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1459359.1459423 · Cited by 20
A gesture based game for image tagging
Gonçalves, D., Jesus, R. M., & Correia, N. M. R. (2008) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1358628.1358745 · Cited by 12
(2000) · CrossRef Listing of Deleted DOIs · doi:10.1162/153244302760185243
Cited by
Kittur, A., Nickerson, J. V., Bernstein, M., Gerber, E., Shaw, A., Zimmerman, J., Lease, M., & Horton, J. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441923 · Cited by 1,034
A Survey of Research on Cloud Robotics and Automation
Kehoe, B., Patil, S., Abbeel, P., & Goldberg, K. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2376492 · Cited by 833
Snorkel: rapid training data creation with weak supervision
Ratner, A., Bach, S. H., Ehrenberg, H., Fries, J., Wu, S., & Ré, C. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3157794.3157797 · Cited by 777
Mobile Crowd Sensing and Computing: The Review of an Emerging Human-Powered Sensing Paradigm
Guo, B., Wang, Z., Yu, Z., Wang, Y., Yen, N. Y., Huang, R., & Zhou, X. (2015) · ACM Computing Surveys · doi:10.1145/2794400 · Cited by 719
A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk
Hara, K., Adams, A., Milland, K., Savage, S., Callison-Burch, C., & Bigham, J. P. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174023 · Cited by 497
Metadata
- Authors
- Quinn, A. J., & Bederson, B. B.
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2011-05-07
- Pages
- 1403-1412
- OpenAlex
- W2170989440
- Cited by
- 801
- References
- 46
- FWCI
- 85.53
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.