Unsupervised construction of large paraphrase corpora: exploiting massively parallel news sources
Open accessAbstract
We investigate unsupervised techniques for acquiring monolingual sentence-level paraphrases from a corpus of temporally and topically clustered news articles collected from thousands of web-based news sources. Two techniques are employed: (1) simple string edit distance, and (2) a heuristic strategy that pairs initial (presumably summary) sentences from different news stories in the same cluster. We evaluate both datasets using a word alignment algorithm and a metric borrowed from machine translation. Results show that edit distance data is cleaner and more easily-aligned than the heuristic data, with an overall alignment error rate (AER) of 11.58% on a similarly-extracted test set. On test data extracted by the heuristic strategy, however, performance of the two training sets is similar, with AERs of 13.2% and 14.7% respectively. Analysis of 100 pairs of sentences from each set reveals that the edit distance data lacks many of the complex lexical and syntactic alternations that characterize monolingual paraphrase. The summary sentences, while less readily alignable, retain more of the non-trivial alternations that are of greatest interest learning paraphrase relationships.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When they can, researchers use heuristics as well.
“Therefore, when they can, researchers use heuristics as well (Dolan et al., 2004; El Bachir Menai & Bagais, 2011; Ganitkevitch et al., 2013; HaCohen-Kerner & Tayeb, 2017).”
From Taxonomy of academic plagiarism methods (2021) · cites this paper as Dolan et al., 2004
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 10
- R. Barzilay and K. R. McKeown. 2001. Extracting Paraphrases from a parallel corpus. In Proceedings of the ACL/EACL.
- R. Barzilay and L. Lee. 2003. Learning to Paraphrase: an unsupervised approach using multiple-sequence alignment. In Proceedings of HLT/NAACL.
- P. Brown, S. A. Della Pietra, V.J. Della Pietra and R. L. Mercer. 1993. The Mathematics of Statistical Machine Translation. Computational Linguistics, 19(2): 263-311.
- D. Lin and P. Pantel. 2001. DIRT -Discovery of Inference Rules from Text. In Proceedings of ACM SIGKDD Conference on Knowledge Discovery and Data Mining.
- I. D. Melamed. 2001. Empirical Methods for Exploiting Parallel Texts. MIT Press.
- R. Mihalcea and T. Pedersen. 2003 An Evaluation Exercise for Word Alignment. In Proceedings of the Workshop on Building and Using Parallel Texts: Data Driven Machine Translation and Beyond. May 31, 2003. Edmonton, Canada.
- F. Och and H. Ney. 2000. Improved Statistical Alignment Models. In Proceedings of the 38th Annual Meeting of the ACL, Hong Kong, China.
- F. Och and H. Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics, 29(1):19-52.
- Y. Shinyama, S. Sekine and K. Sudo. 2002. Automatic Paraphrase Acquisition from News Articles. In Proceedings of NAACL-HLT.
- S. Vogel, H. Ney and C. Tillmann. 1996. HMM-Based Word Alignment in Statistical Translation. In Proceedings of the Annual Meeting of the ACL, Copenhagen, Denmark.
Cited by
Sorted by citations
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · see citing works
Learning to Generate Reviews and Discovering Sentiment
Radford, A., Jozefowicz, R., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1704.01444 · Cited by 330 · see citing works
“The package received was blank and has no barcode.”
BackgroundMethodSentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · arXiv · doi:10.48550/arxiv.1908.10084 · Cited by 86 · see citing works
Measuring text similarity based on structure and word embedding
Farouk, M. (2020) · Cognitive Systems Research · doi:10.1016/j.cogsys.2020.04.002 · Cited by 47 · see citing works
Taxonomy of academic plagiarism methods
Vrbanec, T., & Meštrović, A. (2021) · Zbornik Veleučilišta u Rijeci · doi:10.31784/zvr.9.1.17 · Cited by 4 · see citing works
- Authors
- Dolan, B., Quirk, C., & Brockett, C.
- Affiliations
- Microsoft (United States) (US)
- Venue
- Proceedings of the 20th international conference on Computational Linguistics - COLING '04
- Published
- 2004
- Pages
- 350-es
- ISSN
- 1525-2477 (ISSN-L)
- OpenAlex
- W1980776243
- Cited by
- 728
- References
- 0
- Citations by year
- 2026: 5 · 2025: 6 · 2024: 13 · 2023: 16 · 2022: 24 · 2021: 49 · 2020: 63 · 2019: 70 · 2018: 60 · 2017: 33 · 2016: 47 · 2015: 55 · 2014: 40 · 2013: 43 · 2012: 42
- FWCI
- 8.85
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Conference on Computational Linguistics (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- paraphrase corpus, edit distance, word alignment, news articles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
