The DOI matches its Crossref record; 104 of the 104 references it cites were found in a registry or an index.
Crowdsourced Data Management: A Survey
Abstract
Any important data management and analytics tasks cannot be completely addressed by automated processes. These tasks, such as entity resolution, sentiment analysis, and image recognition can be enhanced through the use of human cognitive ability. Crowdsouring platforms are an effective way to harness the capabilities of people (i.e., the crowd) to apply human computation for such tasks. Thus, crowdsourced data management has become an area of increasing interest in research and industry. We identify three important problems in crowdsourced data management. (1) Quality Control: Workers may return noisy or incorrect results so effective techniques are required to achieve high quality; (2) Cost Control: The crowd is not free, and cost control aims to reduce the monetary cost; (3) Latency Control: The human workers can be slow, particularly compared to automated computing time scales, so latency-control techniques are required. There has been significant work addressing these three factors for designing crowdsourced tasks, developing crowdsourced data manipulation operators, and optimizing plans consisting of multiple operators. In this paper, we survey and synthesize a wide spectrum of existing studies on crowdsourced data management. Based on this analysis we then outline key factors that need to be considered to improve crowdsourced data management.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Mining association rules between sets of items in large databases
Agrawal, R., Imieliński, T., & Swami, A. (1993) · Proceedings of the 1993 ACM SIGMOD international conference on Management of data · doi:10.1145/170035.170072 · Cited by 14,907
(2009) · Cited by 6,153
A survey of approaches to automatic schema matching
Rahm, E., & Bernstein, P. A. (2001) · The VLDB Journal · doi:10.1007/s007780100057 · Cited by 3,352
Rank Analysis of Incomplete Block Designs: I. The Method of Paired Comparisons
Bradley, R. A., & Terry, M. E. (1952) · Biometrika · doi:10.2307/2334029 · Cited by 2,412
Borzsony, S., Kossmann, D., & Stocker, K. (2002) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2001.914855 · Cited by 2,266
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187
Automated Planning: Theory & Practice
Nau, D. S., Ghallab, M., & Traverso, P. (2004) · HAL · Cited by 2,129
The Weighted Majority Algorithm
Littlestone, N., & Warmuth, M. K. (1994) · Information and Computation · doi:10.1006/inco.1994.1009 · Cited by 1,975
Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm
Dawid, A. P., & Skene, A. M. (1979) · Applied Statistics · doi:10.2307/2346806 · Cited by 1,692
RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS: THE METHOD OF PAIRED COMPARISONS
BRADLEY, R. A., & TERRY, M. E. (1952) · Biometrika · doi:10.1093/biomet/39.3-4.324 · Cited by 1,434
(1978) · Medical Entomology and Zoology · Cited by 1,199
reCAPTCHA: Human-Based Character Recognition via Web Security Measures
von Ahn, L., Maurer, B., McMillen, C., Abraham, D., & Blum, M. (2008) · Science · doi:10.1126/science.1160379 · Cited by 1,115
Whose Vote Should Count More: Optimal Integration of Labels from Labelers of Unknown Expertise
Whitehill, J., Wu, T., Bergsma, J., Movellan, J. R., & Ruvolo, P. (2009) · Cited by 1,004
Quality management on Amazon Mechanical Turk
Ipeirotis, P. G., Provost, F., & Wang, J. (2010) · Proceedings of the ACM SIGKDD Workshop on Human Computation · doi:10.1145/1837885.1837906 · Cited by 973
Self-citationOptimal aggregation algorithms for middleware
Fagin, R., Lotem, A., & Naor, M. (2003) · Journal of Computer and System Sciences · doi:10.1016/s0022-0000(03)00026-6 · Cited by 942
Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach
Pearl, J. (2022) · Probabilistic and Causal Inference · doi:10.1145/3501714.3501727 · Cited by 727
BRADLEY, R. A. (1954) · Biometrika · doi:10.1093/biomet/41.3-4.502 · Cited by 716
Interactive deduplication using active learning
Sarawagi, S., & Bhamidipaty, A. (2002) · Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/775047.775087 · Cited by 692
TrueSkill™: A Bayesian Skill Rating System
Herbrich, R., Minka, T., & Graepel, T. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0076 · Cited by 639
CrowdDB: answering queries with crowdsourcing
Franklin, M. J., Kossmann, D., Kraska, T., Ramesh, S., & Xin, R. (2011) · Proceedings of the 2011 ACM SIGMOD International Conference on Management of data · doi:10.1145/1989323.1989331 · Cited by 623
Self-citationCrowdER: crowdsourcing entity resolution
Wang, J., Kraska, T., Franklin, M. J., & Feng, J. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2350229.2350263 · Cited by 507
Self-citationKazemi, L., & Shahabi, C. (2012) · ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (SIGSPATIAL GIS) · doi:10.1145/2424321.2424346 · Cited by 502
Iterative Learning for Reliable Crowdsourcing Systems
Karger, D. R., Oh, S., & Shah, D. (2011) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 468
Demartini, G., Difallah, D. E., & Cudré-Mauroux, P. (2012) · Proceedings of the 21st international conference on World Wide Web · doi:10.1145/2187836.2187900 · Cited by 448
Multicriterion Decision in Management: Principles and Practice
Pomerol, J., & Barba-Romero, S. (2011) · University of Maribor digital library (University of Maribor) · Cited by 378
A framework for protecting worker location privacy in spatial crowdsourcing
To, H., Ghinita, G., & Shahabi, C. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732951.2732966 · Cited by 361
Limits on the majority vote accuracy in classifier fusion
Kuncheva, L. I., Whitaker, C. J., Shipp, C. A., & Duin, R. P. W. (2003) · Pattern Analysis & Applications · doi:10.1007/s10044-002-0173-7 · Cited by 356
Statistical ranking and combinatorial Hodge theory
Jiang, X., Lim, L.-H., Yao, Y., & Ye, Y. (2011) · Mathematical Programming · doi:10.1007/s10107-010-0419-x · Cited by 335
Supervised learning from multiple experts: whom to trust when everyone lies a bit
Raykar, V. C., Yu, S., Zhao, L. H., Jerebko, A., Florin, C., Valadez, G. H., Bogoni, L., & Moy, L. (2009) · Proceedings of the 26th Annual International Conference on Machine Learning · doi:10.1145/1553374.1553488 · Cited by 333
Increased Diels-Alderase activity through backbone remodeling guided by Foldit players
Eiben, C. B., Siegel, J. B., Bale, J. B., Cooper, S., Khatib, F., Shen, B. W., Players, F., Stoddard, B. L., Popovic, Z., & Baker, D. (2012) · Nature Biotechnology · doi:10.1038/nbt.2109 · Cited by 315
Online crowdsourcing: Rating annotators and obtaining cost-effective labels
Welinder, P., & Perona, P. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543189 · Cited by 314
Computing with Noisy Information
Feige, U., Raghavan, P., Peleg, D., & Upfal, E. (1994) · SIAM Journal on Computing · doi:10.1137/s0097539791195877 · Cited by 304
Variational Inference for Crowdsourcing
Liu, Q., Peng, J., & Ihler, A. (2012) · Cited by 301
Multicriterion Decision in Management
Pomerol, J.-C., & Barba-Romero, S. (2000) · International Series in Operations Research & Management Science · doi:10.1007/978-1-4615-4459-3 · Cited by 294
Online Task Assignment in Crowdsourcing Markets
Ho, C.-J., & Vaughan, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8120 · Cited by 293
Pairwise ranking aggregation in a crowdsourced setting
Chen, X., Bennett, P. N., Collins-Thompson, K., & Horvitz, E. (2013) · Proceedings of the sixth ACM international conference on Web search and data mining · doi:10.1145/2433396.2433420 · Cited by 290
Yan, Y., Rosales, R., Fung, G., & Dy, J. (2011) · Cited by 289
Marcus, A., Wu, E., Karger, D., Madden, S., & Miller, R. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/2047485.2047487 · Cited by 278
Inferring Ground Truth from Subjective Labelling of Venus Images
Smyth, P., Fayyad, U. M., Burl, M. C., Perona, P., & Baldi, P. (1994) · CaltechAUTHORS (California Institute of Technology) · Cited by 268
Eliminating spammers and ranking annotators for crowdsourced labeling tasks
Raykar, V. C., Yu, S., & Taskar, B. (2012) · Cited by 266
CrowdSearch: exploiting crowds for accurate real-time image search on mobile phones
Yan, T., Kumar, V., & Ganesan, D. (2010) · Proceedings of the 8th international conference on Mobile systems, applications, and services · doi:10.1145/1814433.1814443 · Cited by 255
CDAS: a crowdsourcing data analytics system
Liu, X., Lu, M., Ooi, B. C., Shen, Y., Wu, S., & Zhang, M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2336664.2336676 · Cited by 251
Community-based bayesian aggregation models for crowdsourcing
Venanzi, M., Guiver, J., Kazai, G., Kohli, P., & Shokouhi, M. (2014) · Proceedings of the 23rd international conference on World wide web · doi:10.1145/2566486.2567989 · Cited by 242
Corleone: hands-off crowdsourcing for entity matching
Gokhale, C., Das, S., Doan, A., Naughton, J. F., Rampalli, N., Shavlik, J., & Zhu, X. (2014) · Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2588555.2588576 · Cited by 219
Pay-as-you-go user feedback for dataspace systems
Jeffery, S. R., Franklin, M. J., & Halevy, A. Y. (2008) · Proceedings of the 2008 ACM SIGMOD international conference on Management of data · doi:10.1145/1376616.1376701 · Cited by 215
Self-citationAdaptive Task Assignment for Crowdsourced Classification
Ho, C.-J., Jabbari, S., & Vaughan, J. W. (2013) · Cited by 208
CrowdScreen: algorithms for filtering data with humans
Parameswaran, A. G., Garcia-Molina, H., Park, H., Polyzotis, N., Ramesh, A., & Widom, J. (2012) · Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2213836.2213878 · Cited by 207
Leveraging transitive relations for crowdsourced joins
Wang, J., Li, G., Kraska, T., Franklin, M. J., & Feng, J. (2013) · Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2463676.2465280 · Cited by 205
Self-citationCrowdsourced Databases: Query Processing with People
Marcus, A., Wu, E., Karger, D. R., Madden, S. R., & Miller, R. C. (2011) · Cited by 191
QASCA: A Quality-Aware Task Assignment System for Crowdsourcing Applications
Zheng, Y., Wang, J., Li, G., Cheng, R., & Feng, J. (2015) · Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2723372.2749430 · Cited by 186
Self-citationiCrowd: An Adaptive Crowdsourcing Framework
Fan, J., Li, G., Ooi, B. C., Tan, K.-L., & Feng, J. (2015) · Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2723372.2750550 · Cited by 181
Self-citationSo who won?: dynamic max discovery with the crowd
Guo, S., Parameswaran, A., & Garcia-Molina, H. (2012) · Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data · doi:10.1145/2213836.2213880 · Cited by 178
Iterative Ranking from Pair-wise ComparisonsPreprint
Negahban, S. N., Oh, S., & Shah, D. (2012) · arXiv (Cornell University) · Cited by 176
Scaling up crowd-sourcing to very large datasets: a case for active learning
Mozafari, B., Sarkar, P., Franklin, M., Jordan, M., & Madden, S. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2735471.2735474 · Cited by 165
Self-citationCrowdsourcing algorithms for entity resolution
Vesdapunt, N., Bellare, K., & Dalvi, N. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732977.2732982 · Cited by 162
Whom to ask? jury selection for decision making tasks on micro-blog services
Cao, C. C., She, J., Tong, Y., & Chen, L. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2350229.2350264 · Cited by 148
Question selection for crowd entity resolution
Whang, S. E., Lofgren, P., & García-Molina, H. (2013) · Proceedings of the VLDB Endowment · doi:10.14778/2536336.2536337 · Cited by 146
Efficient Ranking from Pairwise Comparisons
Wauthier, F. L., Jordan, M. I., & Jojić, N. (2013) · Cited by 141
Crowdsourcing for Multiple-Choice Question Answering
Aydin, B. I., Yilmaz, Y. S., Li, Y., Li, Q., Gao, J., & Demirbaş, M. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i2.19016 · Cited by 136
Parameswaran, A., Park, H., García-Molina, H., Polyzotis, N., & Widom, J. (2012) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2396761.2398421 · Cited by 131
Using the crowd for top-k and group-by queries
Davidson, S. B., Khanna, S., Milo, T., & Roy, S. (2013) · International Conference on Database Theory · doi:10.1145/2448496.2448524 · Cited by 129
What's the right price? pricing tasks for finishing on time
Faridani, S., Hartmann, B., & Ipeirotis, P. G. (2011) · Cited by 126
A sample-and-clean framework for fast and accurate query processing on dirty data
Wang, J., Krishnan, S., Franklin, M. J. M., et al. (2014) · Proceedings - ACM-SIGMOD International Conference on Management of Data · doi:10.1145/2588555.2610505 · Cited by 123
Self-citationPOMDP-based control of workflows for crowdsourcing
Dai, P., Lin, C. H., Mausam, M., & Weld, D. S. (2013) · Artificial Intelligence · doi:10.1016/j.artint.2013.06.002 · Cited by 117
Crowdsourced enumeration queries
Trushkowsky, B., Kraska, T., Franklin, M. J. M., & Sarkar, P. (2013) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2013.6544865 · Cited by 117
Self-citationMax algorithms in crowdsourcing environments
Venetis, P., García-Molina, H., Huang, K., & Polyzotis, N. (2012) · International World Wide Web Conference (WWW) · doi:10.1145/2187836.2187969 · Cited by 109
Optimistic Knowledge Gradient Policy for Optimal Budget Allocation in Crowdsourcing
Chen, X., Lin, Q., & Zhou, D. (2013) · Cited by 103
Crowdsourced POI labelling: Location-aware result inference and Task Assignment
Hu, H., Zheng, Y., Bao, Z., Li, G., Feng, J., & Cheng, R. C. K. (2016) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2016.7498229 · Cited by 94
Self-citationMarcus, A., Karger, D. R., Madden, S. R., Miller, R. C., & Oh, S. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2535568.2448944 · Cited by 92
Asking the Right Questions in Crowd Data Sourcing
Boim, R., Greenshpan, O., Milo, T., Novgorodov, S., Polyzotis, N., & Tan, W.-C. (2012) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2012.122 · Cited by 84
Heterogeneous Stream Processing and Crowdsourcing for Urban Traffic Management
Artikis, A., Weidlich, M., Schnitzler, F., Boutsis, I., Liebig, T., Piatkowski, N., Bockermann, C., Morik, K. J., Kalogeraki, V., Mareček, J., et al. (2014) · doi:10.5441/002/edbt.2014.77 · Cited by 82
Amsterdamer, Y., Grossman, Y., Milo, T., & Senellart, P. P. (2013) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2463676.2465318 · Cited by 79
CrowdPlanner: A crowd-based route recommendation system
Su, H., Zheng, K., Huang, J. M., Jeung, H., Chen, L., & Zhou, X. (2014) · Proceedings - International Conference on Data Engineering · doi:10.1109/icde.2014.6816730 · Cited by 78
Evaluating the crowd with confidence
Joglekar, M., García-Molina, H., & Parameswaran, A. (2013) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487595 · Cited by 78
Argonaut: macrotask crowdsourcing for complex data processing
Haas, D., Ansel, J., Gu, L., & Marcus, A. (2015) · Proceedings of the VLDB Endowment · doi:10.14778/2824032.2824062 · Cited by 72
Kurve, A., Miller, D. J., & Kesidis, G. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2327026 · Cited by 70
A Multi-armed Bandit Approach to Online Spatial Task Assignment
Hassan, U. U., & Curry, E. (2014) · IEEE International Conference on Ubiquitous Intelligence and Computing, Autonomic and Trusted Computing, and Scalable Computing and Communications (UIC-ATC-ScalCom) · doi:10.1109/uic-atc-scalcom.2014.68 · Cited by 68
Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning
Yi, J., Jin, R., Jain, S., Yang, T., & Jain, A. K. (2012) · Cited by 67
Gao, Y., & Parameswaran, A. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2733085.2733101 · Cited by 66
Skyline queries in crowd-enabled databases
Lofi, C., Maarry, K. E., & Balke, W. (2013) · International Conference on Extending Database Technology (EDBT) · doi:10.1145/2452376.2452431 · Cited by 66
Adaptive Polling for Information Aggregation
Pfeiffer, T., Gao, X. A., Chen, Y., Mao, A., & Rand, D. G. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8099 · Cited by 63
Active Learning for Crowdsourcing Using Knowledge Transfer
Fang, M., Yin, J., & Tao, D. (2014) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v28i1.8993 · Cited by 61
CLAMShell: speeding up crowds for low-latency data labeling
Haas, D., Wang, J., Wu, E., & Franklin, M. J. M. (2015) · Proceedings of the VLDB Endowment · doi:10.14778/2856318.2856331 · Cited by 60
Self-citationOn Optimality of Jury Selection in Crowdsourcing
Zheng, Y., Cheng, R. C. K., Maniu, S., & Mo, L. (2015) · The HKU Scholars Hub (University of Hong Kong) · doi:10.5441/002/edbt.2015.18 · Cited by 59
Self-citationPark, H., & Widom, J. (2014) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/2588555.2610503 · Cited by 57
Top-k Selection based on Adaptive Sampling of Noisy Preferences
Busa‐Fekete, R., Szörényi, B., Cheng, W., Weng, P., Hüllermeier, E., Dasgupta, S., & McAllester, D. (2013) · Open access LMU (Ludwid Maxmilian's Universitat Munchen) · Cited by 51
Zhang, X., Li, G., & Feng, J. (2016) · Proceedings of the VLDB Endowment · doi:10.14778/2921558.2921559 · Cited by 50
Self-citationSarma, A. D., Parameswaran, A., García-Molina, H., & Halevy, A. Y. (2014) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2014.6816715 · Cited by 49
Crowdsourcing Entity Resolution: When is A=B?
Gruenheid, A., Kossmann, D., Ramesh, S., & Widmer, F. (2012) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-a-009761323 · Cited by 49
Active learning from crowds with unsure option
Zhong, J., Tang, K., & Zhou, Z. (2015) · Cited by 45
K-Anonymity for Crowdsourcing Database
Wu, S., Wang, X., Wang, S., Zhang, Z., & Tung, A. K. H. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.93 · Cited by 44
Optimal crowd-powered rating and filtering algorithms
Parameswaran, A., Boyd, S., García-Molina, H., Gupta, A., Polyzotis, N., & Widom, J. (2014) · Proceedings of the VLDB Endowment · doi:10.14778/2732939.2732942 · Cited by 43
Comprehensive and reliable crowd assessment algorithms
Joglekar, M., García-Molina, H., & Parameswaran, A. (2015) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2015.7113284 · Cited by 41
Debiasing crowdsourced quantitative characteristics in local businesses and services
Ouyang, R. W., Kaplan, L., Martin, P. D., Toniolo, A., Srivastava, M. B., & Norman, T. J. (2015) · ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) · doi:10.1145/2737095.2737116 · Cited by 41
Learning to Top-K Search using Pairwise Comparisons
Eriksson, B. (2013) · Cited by 38
Crowd-Selection Query Processing in Crowdsourcing Databases: A Task-Driven Approach
Zhao, Z., Wei, F., Zhou, M. Q., Chen, W., & Ng, W. (2015) · Rare & Special e-Zone (The Hong Kong University of Science and Technology) · doi:10.5441/002/edbt.2015.35 · Cited by 32
A transfer learning based framework of crowd-selection on twitter
Zhao, Z. L., Yan, D., Ng, W., & Gao, S. (2013) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2487575.2487708 · Cited by 30
Crowdsourced Action-Model Acquisition for Planning
Zhuo, H. H. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9667 · Cited by 27
Herding the Crowd: Automated Planning for Crowdsourced Planning
Talamadupula, K., Subbarao Kambhampati, B., Hu, Y., Nguyen, T. A., & Zhuo, H. H. (2013) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v1i1.13128 · Cited by 18
Effective Result Inference for Context-Sensitive Tasks in Crowdsourcing
Fang, Y., Sun, H., Li, G., Zhang, R., & Huai, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-32025-0_3 · Cited by 16
Self-citationManaging General and Individual Knowledge in Crowd Mining Applications
Amsterdamer, Y., Davidson, S. B., Kukliansky, A., Milo, T., Novgorodov, S., & Somech, A. (2015) · Cited by 13
Cited by
The ApolloScape Open Dataset for Autonomous Driving and Its Application
Huang, X., Wang, P., Cheng, X., Zhou, D., Geng, Q., & Yang, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2926463 · Cited by 615
The ApolloScape Open Dataset for Autonomous Driving and its Application
Huang, X., Wang, P., Cheng, X., Zhou, D., Geng, Q., & Yang, R. (2018) · arXiv · doi:10.48550/arxiv.1803.06184 · Cited by 615
Truth inference in crowdsourcing: is the problem solved?
Zheng, Y., Li, G., Li, Y., Shan, C., & Cheng, R. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3055540.3055547 · Cited by 421
Self-citationA Survey of Traffic Prediction: from Spatio-Temporal Data to Intelligent Transportation
Yuan, H., & Li, G. (2021) · Data Science and Engineering · doi:10.1007/s41019-020-00151-z · Cited by 389
Self-citationKnowledge Graph Quality Management: a Comprehensive Survey
Xue, B., & Zou, L. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2022.3150080 · Cited by 108
Metadata
- Authors
- Li, G., Wang, J., Zheng, Y., & Franklin, M. J.
- Venue
- IEEE Transactions on Knowledge and Data Engineering
- Published
- 2016-09-01
- Volume
- 28
- Issue
- 9
- Pages
- 2296-2319
- ISSN
- 1041-4347 (ISSN-L)
- OpenAlex
- W2329105431
- Cited by
- 283
- References
- 104
- FWCI
- 62.56
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.