The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking
Abstract
We tackle the problem of entity linking for large collections of online pages; Our system, ZenCrowd, identifies entities from natural language text using state of the art techniques and automatically connects them to the Linked Open Data cloud. We show how one can take advantage of human intelligence to improve the quality of the links by dynamically generating micro-tasks on an online crowdsourcing platform. We develop a probabilistic framework to make sensible decisions about candidate links and to identify unreliable human workers. We evaluate ZenCrowd in a real deployment and show how a combination of both probabilistic reasoning and crowdsourcing techniques can significantly improve the quality of the links, while limiting the amount of work performed by the crowd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Binary Codes capable of currecting deletions, insertions, and reversals
Levenshtein, V. I. (1965) · Soviet physics. Doklady · Cited by 14,807
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 2×
“For a more in-depth coverage, we refer the interested reader to one of the many overviews on this domain, such as [23].”
BackgroundMethodMaximum Likelihood from Incomplete Data via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1976) · Cited by 4,517
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - ACL '03 · doi:10.3115/1075096.1075150 · Cited by 3,008
(1961) · Journal of the Optical Society of America · doi:10.1364/josa.51.000248 · Cited by 2,288
Advances in Record-Linkage Methodology as Applied to Matching the 1985 Census of Tampa, Florida
Jaro, M. A. (1989) · Journal of the American Statistical Association · doi:10.1080/01621459.1989.10478785 · Cited by 1,401
Open information extraction from the web
Banko, M., Cafarella, M., Soderland, S. G., Broadhead, M., & Etzioni, O. (2007) · International Joint Conference on Artificial Intelligence · Cited by 1,398
DBpedia spotlight: shedding light on the web of documents
Mendes, P. N., Jakob, M., García-Silva, A., & Bizer, C. (2011) · Proceedings of the 7th International Conference on Semantic Systems · doi:10.1145/2063518.2063519 · Cited by 1,180
The State of Record Linkage and Current Research Problems
Winkler, W. E. (1999) · Cited by 1,015
Wikify!: linking documents to encyclopedic knowledge
Mihalcea, R., & Csomai, A. (2007) · Proceedings of the sixteenth ACM conference on Conference on information and knowledge management · doi:10.1145/1321440.1321475 · Cited by 909
Reference reconciliation in complex information spaces
Dong, X., Halevy, A., & Madhavan, J. (2005) · Proceedings of the 2005 ACM SIGMOD international conference on Management of data · doi:10.1145/1066157.1066168 · Cited by 518
Boilerplate detection using shallow text features
Kohlschütter, C., Fankhauser, P., & Nejdl, W. (2010) · Proceedings of the third ACM international conference on Web search and data mining · doi:10.1145/1718487.1718542 · Cited by 476
Marcus, A., Wu, E., Karger, D., Madden, S., & Miller, R. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/2047485.2047487 · Cited by 278
Annotating named entities in Twitter data with crowdsourcing
Finin, T., Murnane, W., Karandikar, A., Keller, N., Martineau, J., & Dredze, M. H. (2010) · doi:10.13016/m21j97c2p · Cited by 276
Ad-hoc object retrieval in the web of data
Pound, J., Mika, P., & Zaragoza, H. (2010) · Proceedings of the 19th international conference on World wide web · doi:10.1145/1772690.1772769 · Cited by 211
Broad-coverage sense disambiguation and information extraction with a supersense sequence tagger
Ciaramita, M., & Altun, Y. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610158 · Cited by 204
Crowdsourcing for book search evaluation: impact of hit design on comparative system ranking
Kazai, G., Kamps, J., Koolen, M., & Milić-Frayling, N. (2011) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2009916.2009947 · Cited by 157
Design and Implementation of Relevance Assessments Using Crowdsourcing
Alonso, O., & Baeza-Yates, R. A. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-20161-5_16 · Cited by 152
Repeatable and reliable search system evaluation using crowdsourcing
Blanco, R., Halpin, H., Herzig, D. M., Mika, P., Pound, J., Thompson, H. S., & Duc, T. T. (2011) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2009916.2010039 · Cited by 111
Effective and Efficient Entity Search in RDF Data
Blanco, R., Mika, P., & Vigna, S. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-25073-6_6 · Cited by 109
Overview of the TREC 2007 Enterprise Track.
Bailey, P., de Vries, A. P., Craswell, N., & Soboroff, I. M. (2007) · Cited by 76
Entity Name System: The Back-Bone of an Open and Scalable Web of Data
Bouquet, P., Stoermer, H., Niederée, C., & Maña, A. (2008) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icsc.2008.37 · Cited by 74
On, B.-W., Koudas, N., Lee, D., & Srivastava, D. (2007) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde.2007.367895 · Cited by 68
Enhanced results for web search
Haas, K., Mika, P., Tarjan, P., & Blanco, R. (2011) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2009916.2010014 · Cited by 54
idMesh: graph-based disambiguation of linked data
Cudré-Mauroux, P., Haghani, P., Jost, M., Aberer, K., & de Meer, H. (2009) · International World Wide Web Conference (WWW) · doi:10.1145/1526709.1526789 · Cited by 49
Self-citationdipLODocus[RDF]—Short and Long-Tail RDF Analytics for Massive Webs of Data
Wylot, M., Pont, J., Wisniewski, M., & Cudré-Mauroux, P. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-25073-6_49 · Cited by 43
Self-citation
Cited by
Entity Linking with a Knowledge Base: Issues, Techniques, and Solutions
Shen, W., Wang, J., & Han, J. (2015) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2014.2327028 · Cited by 761
Truth inference in crowdsourcing: is the problem solved?
Zheng, Y., Li, G., Li, Y., Shan, C., & Cheng, R. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3055540.3055547 · Cited by 421
“The single-choice tasks are especially studied in [34, 16, 15, 53, 41, 61, 35, 30, 27, 46, 5].”
BackgroundMethodInfluentialAnalysis of named entity recognition and linking for tweets
Derczynski, L., Maynard, D., Rizzo, G., van Erp, M., Gorrell, G., Troncy, R., Petrak, J., & Bontcheva, K. (2015) · Information Processing & Management · doi:10.1016/j.ipm.2014.10.006 · Cited by 340
Spatial crowdsourcing: a survey
Tong, Y., Zhou, Z., Zeng, Y., Chen, L., & Shahabi, C. (2020) · The VLDB Journal · doi:10.1007/s00778-019-00568-7 · Cited by 305
Crowdsourced Data Management: A Survey
Li, G., Wang, J., Zheng, Y., & Franklin, M. J. (2016) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2016.2535242 · Cited by 283
Metadata
- Authors
- Demartini, G., Difallah, D. E., & Cudré-Mauroux, P.
- Venue
- Proceedings of the 21st international conference on World Wide Web
- Publisher
- ACM
- Published
- 2012-04-16
- Pages
- 469-478
- OpenAlex
- W2113878109
- Cited by
- 448
- References
- 27
- FWCI
- 27.08
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.