Generating Sentences by Editing Prototypes
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
We propose a new generative language model for sentences that first samples a prototype sentence from the training corpus and then edits it into a new sentence. Compared to traditional language models that generate from scratch either left-to-right or by first sampling a latent sentence vector, our prototype-then-edit model improves perplexity on language modeling and generates higher quality outputs according to human evaluation. Furthermore, the model gives rise to a latent edit vector that captures interpretable semantics such as sentence similarity and sentence-level analogies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Non-parametric neural language models (NLMs) learn predictive distributions of text utilizing an external datastore, which allows them to learn through explicitly memorizing the training datapoints.”
From Efficient Nearest Neighbor Language Models (2021) · cites this paper as Guu et al., 2018
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prompt augmentation is closely related to retrieval-based methods that provide more textual context to the model to improve performance [37], a method that has also been shown to be effective in prompt-based learning [99].”
From Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing (2023) · cites this paper as [37]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 3×
“Interestingly, the top-10 performance of our model in Table 5 is nearly as good as the performance of GloVe vectors on the simpler lexical analogy task, despite the fact that the sentence prediction task is harder.”
MethodInfluentialEfficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568 · cited 2×
“Our use of lexical similarity neighborhoods is comparable to context windows used in word vector training (Mikolov et al., 2013a).”
BackgroundMethodAuto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 6×
“aining p(xjx0) as a conditional variational autoencoder (C-VAE) (it is conditioned on x0). We maximize the objective via SGD, approximating the gradient using the usual Monte Carlo estimate for VAEs (Kingma and Welling, 2014). Neural editor p edit(xjx0;z): We implement our neural editor as a left-to-right sequence-to-…”
MethodInfluentialSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 2×
“Neural language models (Bengio et al., 2003) based upon recurrent neural networks and sequence-to-sequence architectures (Sutskever et al., 2014) have been widely used due to their flexibility and performance across a wide-range of NLP tasks (Ju-rafsky and Martin, 2000; Kalchbrenner and Blun-som,…”
BackgroundMethodA Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585 · cited 1×
“More generally, results in manifold learning demonstrate that a weak metric such as lexical similarity can be used to extract semantic similarity through distributional statistics (Tenenbaum et al., 2000; Hashimoto et al., 2016).”
BackgroundOn Estimation of a Probability Density Function and Mode
Parzen, E. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704472 · Cited by 10,585 · cited 1×
“From a generative modeling perspective, editing randomly sampled training sentences closely resembles nonparametric kernel density estimation (Parzen, 1962) where one samples points from a training set, and adds noise to smooth the density.”
BackgroundGoogle's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016) · arXiv · doi:10.48550/arxiv.1609.08144 · Cited by 5,319 · cited 2×
“Our work is motivated by an emerging consensus that attention-based mechanisms (Bahdanau et al., 2015) can substantially improve performance on various sequence to sequence tasks by capturing more information from the input sequence (Wu et al., 2016; Vaswani et al., 2017).”
BackgroundMethodAn Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Linguistic Regularities in Continuous Space Word Representations
Mikolov, T., Yih, W.-T., & Zweig, G. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 2,912
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
Nallapati, R., Zhou, B., dos Santos, C., Gulcehre, C., & Xiang, B. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1028 · Cited by 2,361
Research works printed but not linked: 1
- Bengio Y., 2003, Journal of machine learning research, 3, 1137
Cited by
Items 101–200 of 293
Exemplars-Guided Empathetic Response Generation Controlled by the Elements of Human Communication
Majumder, N., Ghosal, D., Hazarika, D., Gelbukh, A. F., Mihalcea, R. F., & Poria, S. (2022) · IEEE Access · doi:10.1109/access.2022.3193159 · Cited by 37
Neural Language Generation: Formulation, Methods, and Evaluation
Gârbacea, C., & Mei, Q. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.15780 · Cited by 24
“tion does not reflect how humans write text (Guu et al., 2018) and is limiting content diversity (Mehri and Sigal, 2018).”
BackgroundDictionary-Guided Editing Networks for Paraphrase Generation
Huang, S., Wu, Y., Wei, F., & Luan, Z. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33016546 · Cited by 22
MetAdapt: Meta-learned task-adaptive architecture for few-shot classification
Doveh, S., Schwartz, E., Xue, C., Feris, R., Bronstein, A. M., Giryes, R., & Karlinsky, L. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.05.010 · Cited by 23
Emotional Text Generation Based on Cross-Domain Sentiment Transfer
Zhang, R., Wang, Z., Yin, K., & Huang, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2931036 · Cited by 21
“[13] investigated a prototypethen-edit approach which first samples a prototype sentence from the training corpus, then edit this sentence using neural network to change its style.”
MethodImportance-Aware Learning for Neural Headline Editing
Wu, Q., Li, L., Zhou, H., Zeng, Y., & Yu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6467 · Cited by 21
Heuristic Authorship Obfuscation
Bevendorff, J., Potthast, M., Hagen, M., & Stein, B. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1104 · Cited by 21
“Promising approaches already exist, such as neural editing and paraphrasing (Grangier and Auli, 2017; Guu et al., 2017).”
MethodQuickEdit: Editing Text and Translations by Crossing Words Out
Grangier, D., & Auli, M. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1025 · Cited by 20
“Our generation is more involved as our model rephrases the sentences, which includes introducing new words, re-ordering text, inflecting nouns and verbs, etc. Guu et al. (2017) considers generating text with latent edits.”
MethodE2E NLG Challenge Submission: Towards Controllable Generation of Diverse Natural Language
Elder, H., Gehrmann, S., O’Connor, A., & Liu, Q. (2018) · International Conference on Natural Language Generation · doi:10.18653/v1/w18-6556 · Cited by 22
“The approach of augmenting the source sequence takes inspiration from recent work in para-phrase generation (Guu et al., 2017) and generating structured queries from natural language (Zhong et al., 2017).”
BackgroundAn Enhanced Knowledge Injection Model for Commonsense Generation
Fan, Z., Gong, Y., Wei, Z., Wang, S., Huang, Y., Jiao, J., Huang, X., Duan, N., & Zhang, R. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.182 · Cited by 35
An Empirical Survey of Data Augmentation for Limited Data Learning in NLPPreprint
Chen, J., Tam, D., Raffel, C., Bansal, M., & Yang, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07499 · Cited by 20
“Generative models used in this setting have been based on conditional VAE (Bowman et al., 2016; Hu et al., 2017; Guu et al., 2017; Malandrakis et al., 2019), GAN (Iyyer et al., 2018; Xu et al., 2018) or pre-trained language models like GPT-2 (Anaby-Tavor et al., 2020; Kumar et al., 2020).”
MethodLatent Space Expanded Variational Autoencoder for Sentence Generation
Song, T., Sun, J., Chen, B., Peng, W., & Song, J. (2019) · IEEE Access · doi:10.1109/access.2019.2944630 · Cited by 21
“There are variants of the VAE that have demonstrated strong performance in, for example, modeling images [31], [32], documents [33], dialogues [34], and sentences [9], [35], [36], [37].”
BackgroundNasir, T., & Malik, M. K. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2023.103544 · Cited by 21
SeGMA: Semi-Supervised Gaussian Mixture Autoencoder
Śmieja, M., Wołczyk, M., Tabor, J., & Geiger, B. C. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3016221 · Cited by 25
Garbacea, C., Carton, S., Yan, S., & Mei, Q. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1409 · Cited by 18
Comparative Explanations of Recommendations
Yang, A., Wang, N., Cai, R., Deng, H., & Wang, H. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512031 · Cited by 24
Ding, Z., Tang, Y., Cheng, X., Li, H., & Shang, W. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3624740 · Cited by 31
“Meanwhile, recent studies [18, 27, 81, 85, 90] on text generation tasks (e.g., text summarization, sentence generation) show that incorporating the templates of the target sentences can produce promising results in generating better translations, as templates can provide a positive impact for…”
BackgroundImprobotics: Exploring the Imitation Game Using Machine Intelligence in Improvised Theatre
Mathewson, K., & Mirowski, P. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment · doi:10.1609/aiide.v14i1.13030 · Cited by 18
Learning to Decipher Hate Symbols
Qian, J., ElSherief, M., Belding, E., & Wang, W. Y. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1305 · Cited by 24
“For example, Zhao et al. (2017) apply CVAE to dialog generation, while Guu et al. (2018) use CVAE for sentence generation.”
MethodIso, H., Qiao, C., & Li, H. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.17 · Cited by 28
Faltings, F., Galley, M., Hintz, G., Brockett, C., Quirk, C., Gao, J., & Dolan, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.414 · Cited by 51
Diversifying Reply Suggestions Using a Matching-Conditional Variational Autoencoder
Deb, B., Bailey, P., & Shokouhi, M. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-2006 · Cited by 24
“Several researchers have used CVAEs (Sohn et al., 2015) for generating text (Miao et al., 2016; Guu et al., 2018; Bowman et al., 2016), modeling conversations (Park et al., 2018), diversifying responses in dialogues (Zhao et al., 2017; Shen et al., 2017) and improving translations (Schulz et al.,…”
BackgroundMeta-Learning for Few-Shot Named Entity Recognition
de Lichy, C., Glaude, H., & Campbell, W. M. (2021) · doi:10.18653/v1/2021.metanlp-1.6 · Cited by 28
“More advanced approaches rely on generative models (Gupta, 2019; Hou et al., 2018; Zhao et al., 2019; Guu et al., 2018; Yoo et al., 2018), paraphrasing (Cho et al.”
MethodDeep Learning for Dialogue Systems: Chit-Chat and Beyond
Yan, R., Li, J., & Zhou, Y. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000083 · Cited by 30
Neural Machine Translation with Monolingual Translation MemoryPreprint
Cai, D., Yan, W., Li, H., Lam, W. P., & Liu, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.11269 · Cited by 16
“Recently, retrieval-guided generation has gained increasing interest in a wide range of text generation tasks such as language modeling (Guu et al., 2018; Khandelwal et al., 2019; Guu et al., 2020), dialogue response generation (Weston et al.”
BackgroundVariational Knowledge Graph ReasoningPreprint
Chen, W., Xiong, W., Yan, X., & Wang, W. Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.06581 · Cited by 15
“Besides, we also try to directly maximize the marginal likelihood p ( r | e s , e d ) = L p ( L | e s , e d ) p ( r | L ) using only the prior and likelihood model following MML (Guu et al., 2017b), which enables us to understand the superiority of introducing an approximate posterior.”
BackgroundMethodInfluentialSkeleton-to-Response: Dialogue Generation Guided by Retrieval MemoryPreprint
Cai, D., Wang, Y., Bi, V., Tu, Z., Liu, X., Lam, W. P., & Shi, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.05296 · Cited by 15
“The idea of feeding IR results to generative models have also been explored in neural machine translation (Guu et al. 2018), where each retrieved pair is fed to a standard encoder-decoder model.”
BackgroundLearning Sparse Prototypes for Text GenerationPreprint
He, J.-X., Berg-Kirkpatrick, T., & Neubig, G. (2020) · Neural Information Processing Systems · doi:10.48550/arxiv.2006.16336 · Cited by 26
“Intuitively, these non-parametric LMs are attractive because they help remove some of the pressure on the parametric model to memorize the entirety of the language it must model.”
BackgroundMethodInfluentialDeep Learning for Text Style Transfer: A SurveyPreprint
Jin, D., Jin, Z., Hu, Z., Olga Vechtomova, A., & Mihalcea, R. F. (2020) · arXiv (Cornell University) · doi:10.3929/ethz-b-000543432 · Cited by 15
FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation
Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.-T., Koh, P. W., Iyyer, M., Zettlemoyer, L., & Hajishirzi, H. (2023) · arXiv · doi:10.48550/arxiv.2305.14251 · Cited by 14
Few-shot Controllable Style Transfer for Low-Resource Multilingual Settings
Krishna, K., Nathani, D., Garcia, X., Samanta, B., & Talukdar, P. P. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.514 · Cited by 34
“DIFFUR is related to neural editor models (Guu et al., 2018; He et al., 2020), where language models are decomposed into a probabilistic space of edit vectors over prototype sentences.”
BackgroundLearning to Summarize Chinese Radiology Findings With a Pre-Trained Encoder
Jiang, Z., Cai, X., Yang, L., Gao, D., Zhao, W., Han, J., Liu, J., Shen, D. G., & Liu, T. (2023) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2023.3280987 · Cited by 14
“First, previous abstractive summarization studies mainly focused on documents written in English, and abstractive summarization on Chinese clinical texts remains challenging due to limitations of the related corpus [7], [9].”
BackgroundEmotional editing constraint conversation content generation based on reinforcement learning
Sun, X. N., Li, J., Wei, X., Li, C., & Tao, J. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.10.007 · Cited by 34
Political Depolarization of News Articles Using Attribute-Aware Word Embeddings
Liu, R., Wang, L., Jia, C., & Vosoughi, S. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18069 · Cited by 15
Retrieve and Refine: Improved Sequence Generation Models For DialoguePreprint
Weston, J., Dinan, E., & Miller, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04776 · Cited by 70
“…(Junczys-Dowmunt and Grundkiewicz, 2017; Niehues et al., 2016; Novak et al., 2016; Xia et al., 2017; Grangier and Auli, 2017) or combining with retrieval (Gu et al., 2017; Cao et al., 2018), as well as for sentence correction or refinement without context (Guu et al., 2017; Schmaltz et al., 2017).”
BackgroundImproving Text Generation with Student-Forcing Optimal Transport
Li, J., Li, C., Wang, G., Fu, H., Lin, Y.-C., Chen, L., Zhang, Y., Tao, C., Zhang, R., Wang, W., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.735 · Cited by 22
vONTSS: vMF based semi-supervised neural topic modeling with optimal transport
Xu, W., Jiang, X., Rao, S. S. H., Iannacci, F., & Zhao, J. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.271 · Cited by 20
“The KL divergence with vMF(., 0) (Davidson et al., 2018) is vMF based VAE has better clusterability of data points especially in low dimensions (Guu et al., 2018).”
BackgroundSounds Wilde. Phonetically Extended Embeddings for Author-Stylized Poetry Generation
Tikhonov, A., & Yamshchikov, I. P. (2018) · doi:10.18653/v1/w18-5813 · Cited by 11
“In (Guu et al., 2017) it is shown that an existent human-written source used to control the saliency of the output can significantly improve the quality of the resulting texts.”
BackgroundExploring Methods for Generating Feedback Comments for Writing Learning
Hanawa, K., Nagata, R., & Inui, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.766 · Cited by 17
“Weston et al. (2018); Guu et al. (2018); Cao et al. (2018); Hossain et al. (2020) retrieve a similar instance using a superficial similarity of the entire input.”
Utterance Generation With Variational Auto-Encoder for Slot Filling in Spoken Language Understanding
Shin, Y., Yoo, K. M., & Lee, S.-G. (2019) · IEEE Signal Processing Letters · doi:10.1109/lsp.2019.2895284 · Cited by 14
“To make plausible text, we use temperature τ below one during generation [18] which is x̂t = softmax(f(xt)/τ) to suppress incoherent and ungrammatical sentences [19].”
MethodA Two-class Hyper-spherical Autoencoder for Supervised Anomaly Detection
Kawachi, Y., Koizumi, Y., Murata, S., & Harada, N. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8683790 · Cited by 13
“There are several DNNs that handle spherical space, such as spherical CNN [24], and those DNNs are applied to language modeling applications [25].”
BackgroundDecomposing Textual Information For Style Transfer
Yamshchikov, I. P., Shibaev, V., Nagaev, A., Jost, J., & Tikhonov, A. (2019) · Workshop on Neural Generation and Translation (WNGT) · doi:10.18653/v1/d19-5613 · Cited by 15
“For example, a retrieve-edit approach by (Guu et al., 2018) works for sentiment transfer.”
BackgroundDiscrete Latent Variable Representations for Low-Resource Text Classification
Jin, S., Wiseman, S., Stratos, K., & Livescu, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.437 · Cited by 15
So Different Yet So Alike! Constrained Unsupervised Text Style Transfer
Kashyap, A. R., Hazarika, D., Kan, M., Zimmermann, R., & Poria, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.32 · Cited by 19
SHARE: a System for Hierarchical Assistive Recipe Editing
Li, S. X., Li, Y., Ni, J., & McAuley, J. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.761 · Cited by 29
Shen, L., Guo, X., & Chen, M. (2020) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn48605.2020.9206888 · Cited by 17
Fast Cross-domain Data Augmentation through Neural Sentence EditingPreprint
Raille, G., Djambazovska, S., & Musat, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.10254 · Cited by 14
“Weshow that the Edit-transformer has a great speed advantage over the Neural Editor baseline - with nearly three times faster training and five times faster inference.”
BackgroundMethodInfluentialFrom theories on styles to their transfer in text: Bridging the gap with a hierarchical survey
Troiano, E., Velutharambath, A., & Klinger, R. (2022) · Natural Language Engineering · doi:10.1017/s1351324922000407 · Cited by 24
“Consistent with this is an observation of Guu et al. (2018), who proposed a generation system able to control for the attribute of a prototype text with a series of edits.”
BackgroundBetter Exploiting Latent Variables in Text Modeling
Kruengkrai, C. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1553 · Cited by 10
“Guu et al. (2018) first proposed using the von Mises– Fisher (vMF) distribution to model the VAE instead of using the Gaussian distribution.”
MethodChang, J., He, R., Wang, L., Zhao, X., Yang, T., & Wang, R. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1200 · Cited by 12
Targeted optimization of regulatory DNA sequences with neural editing architecturesPreprint
Gupta, A., & Kundaje, A. B. (2019) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/714402 · Cited by 11
Siamese Networks for Semantic Pattern Similarity
Benajiba, Y., Sun, J., Zhang, Y., Jiang, L., Weng, Z., & Biran, O. (2019) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icosc.2019.8665512 · Cited by 11
Fan, A., & Gardent, C. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.586 · Cited by 38
A Probabilistic Generative Model for Typographical Analysis of Early Modern Printing
Goyal, K., Dyer, C., Warren, C., G’Sell, M., & Berg-Kirkpatrick, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.266 · Cited by 10
ProtoTEx: Explaining Model Decisions with Prototype Tensors
Das, A., Gupta, C., Kovatchev, V., Lease, M., & Li, J. J. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.213 · Cited by 35
Xiang, X., Xu, G., Fu, X., Wei, Y., Jin, L., & Wang, L. (2018) · Information · doi:10.3390/info9090217 · Cited by 11
“[20] added latent structured information to the decoder and introduced an editing vector [21] to edit the generated summary, thereby enhancing the readability of the summary.”
BackgroundShaping the Narrative Arc: An Information-Theoretic Approach to Collaborative DialoguePreprint
Mathewson, K. W., Castro, P. S., Cherry, C., Foster, G. F., & Bellemare, M. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.11528 · Cited by 19
“Previous work has used a topic-transition generative adversarial network to enforce smoothness of transition of subsequent utterances (Liang et al., 2017).”
MethodGao, Y. Q., Qi, J., Ye, H., Sun, Y., Hu, X., Dong, Z., & Sun, Y. (2024) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2024.3502890 · Cited by 10
A Deep Generative Model for Reordering Adjacency Matrices
Kwon, O.-H., Kao, C., Chen, C., & Ma, K. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2022.3153838 · Cited by 12
“Deep generative models, such as variational autoencoders [21] and generative adversarial networks [22], have shown great success in various domains, such as image generation [23], [24], text generation [25], [26], and music generation [27], [28].”
BackgroundDynamic Demonstration Retrieval and Cognitive Understanding for Emotional Support Conversation
Xu, Z., Chen, D., Kuang, J., Yi, Z., Li, Y., & Shen, Y. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657695 · Cited by 25
“Retrieval-enhanced models address this by using reference utterances to guide response generation [12, 14, 40].”
COD3S: Diverse Generation with Discrete Semantic Signatures
Weir, N., Sedoc, J., & Durme, B. V. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.421 · Cited by 14
Data-to-text Generation by Splicing Together Nearest Neighbors
Wiseman, S., Bačkurs, A., & Stratos, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.352 · Cited by 15
“…with the source information on which these example texts condition — on the hope that these neighbors might make a generation task easier, or the system more interpretable or controllable (Song et al., 2016; Weston et al., 2018; Guu et al., 2018; Zhang et al., 2018; Peng et al., 2019, inter alia ).”
BackgroundZhao, J., & Cho, K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.09502 · Cited by 9
Diversifying Reply Suggestions using a Matching-Conditional Variational AutoencoderPreprint
Deb, B., Bailey, P., & Shokouhi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.10630 · Cited by 13
“Several researchers have used CVAEs (Sohn et al., 2015) for generating text (Miao et al., 2016; Guu et al., 2018; Bowman et al., 2016), modeling conversations (Park et al., 2018), diversifying responses in dialogues (Zhao et al., 2017; Shen et al., 2017) and improving translations (Schulz et al.,…”
BackgroundDiversifying Task-oriented Dialogue Response Generation with Prototype Guided ParaphrasingPreprint
Lippe, P., Ren, P., Haned, H., Voorn, B., & de Rijke, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03391 · Cited by 13
“ing and paraphrasing. The idea of prototype editing is to first sample a prototype sentence from the training corpus and then edit it into a new sentence, instead of generating a sentence from scratch [9]. The prototype sentences have different styles so that we expect to get diverse responses w.r.t. different prototyp…”
BackgroundMethodFaltings, F., Galley, M., Brantley, K., Peng, B., Cai, W., Zhang, Y., Gao, J., & Dolan, B. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.270 · Cited by 24
Nutri-bullets Hybrid: Consensual Multi-document Summarization
Shah, D., Yu, L., Lei, T., & Barzilay, R. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.411 · Cited by 14
“Prior work (Mueller et al., 2017; Fan et al., 2017; Guu et al., 2018) on text generation also control aspects of the produced text, such as style and length.”
BackgroundEmotional Conversation Generation Based on a Bayesian Deep Neural Network
Sun, X. N., Li, J., Wei, X., Li, C., & Tao, J. (2019) · ACM Transactions on Information Systems · doi:10.1145/3368960 · Cited by 11
“In recent years, data-driven neural network models have shown great potential in the field of text generation, which dramatically improves the quality of raw text (fluency, etc.) [8, 21], but these models generally ignore the emotional factors in the text.”
BackgroundVector-to-Sequence Models for Sentence Analogies
Wang, L., & Lepage, Y. (2020) · International Conference on Advanced Computer Science and Information Systems · doi:10.1109/icacsis51025.2020.9263191 · Cited by 18
“Guu et al. (2019) [25] trained a neural editor to obtain the edit vector encoding transformations between two sentences, and applied it on a sentence to generate a new sentence conditioned on this edit vector.”
Coupling Retrieval and Meta-Learning for Context-Dependent Semantic ParsingPreprint
Guo, D., Tang, D., Duan, N., Zhou, M., & Yin, J. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.07108 · Cited by 13
“Retrieval-augmented models have also been studied in text generation (Gu et al., 2017; Huang et al., 2018b; Guu et al., 2018; Wu et al., 2018).”
BackgroundSelf-Imitation Learning via Trajectory-Conditioned Policy for Hard-Exploration TasksPreprint
Guo, Y., Choi, J., Moczulski, M., Bengio, S., Norouzi, M., & Lee, H. (2019) · arXiv (Cornell University) · Cited by 12
Luo, Z., Fan, W., Amayri, M., & Bouguila, N. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3690624.3709230 · Cited by 18
“The vMF distribution has also gained popularity in deep learning for its efficacy in modeling directional data [14, 22, 61].”
Hyperspherical Variational Co-embedding for Attributed Networks
Fang, J., Liang, S., Meng, Z., & de Rijke, M. (2021) · ACM Transactions on Information Systems · doi:10.1145/3478284 · Cited by 7
A Cross-Sentence Latent Variable Model for Semi-Supervised Text Sequence Matching
Choi, J., Kim, T., & Lee, S. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1469 · Cited by 9
“, 2018), and using alternative distributions (Guu et al., 2018).”
MethodInfluentialTriple-to-Text Generation with an Anchor-to-Prototype Framework
Li, Z., Lin, Z., Ding, N., Zheng, H.-T., & Shen, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/523 · Cited by 9
“Additionally, the idea of utilizing retrieved prototypes has also been exploited in other generation task such as unconditional text generation [ Guu et al. , 2018 ] , dialogue response generation [ Wu et al. , 2019; Cai et al. , 2019a; Cai et al. , 2019b ] and abstractive summarization [ Cao et…”
MethodBevendorff, J., Wenzel, T., Potthast, M., Hagen, M., & Stein, B. (2020) · it - Information Technology · doi:10.1515/itit-2019-0046 · Cited by 8
“Promising approaches already exist, such as neural editing, paraphrasing, and targeted neural style transfer [16, 17, 55].”
BackgroundMethodToward Unsupervised Text Content Manipulation.Preprint
Wang, W., Hu, Z., Yang, Z., Shi, H., Xu, F. F., & Xing, E. P. (2019) · arXiv (Cornell University) · Cited by 7
Learning Autocomplete Systems as a Communication GamePreprint
Lee, M., Hashimoto, T., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06964 · Cited by 11
“We segment the reviews into sentences following the same procedure as Guu et al. [14] while treating capitalization as a separate, special token (e.g.”
Self-citationTemplate Guided Text Generation for Task-Oriented DialoguePreprint
Kale, M., & Rastogi, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.15006 · Cited by 26
Towards Minimal Supervision BERT-Based Grammar Error Correction (Student Abstract)
Li, Y., Anastasopoulos, A., & Black, A. W. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i10.7202 · Cited by 8
ALTER: Auxiliary Text Rewriting Tool for Natural Language Generation
Xu, Q., Xu, C., & Qu, L. (2019) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d19-3003 · Cited by 8
“Recently, there has been a series of work on conducting edit operations on text to advance automatic natural language generation (Guu et al., 2018; Li et al., 2018).”
BackgroundIncorporating bilingual translation templates into neural machine translation
Li, F., Liu, B., Yan, H., Xie, P., Li, J., & Zhang, Z. (2025) · Scientific Reports · doi:10.1038/s41598-025-86754-w · Cited by 10
Sequentially Controlled Text Generation
Spangher, A., Ming, Y., Hua, X., & Peng, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.509 · Cited by 26
Improving Aspect Sentiment Classification via Retrieving from Training Data
Ling, T., Chen, L., Liao, C., Huang, S., Yu, Z., & Liu, Y. (2023) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipaasc58517.2023.10317451 · Cited by 6
“As a result, a series of retrieval-based approaches have been applied with great success to many NLP tasks such as language modeling [13, 14], open-domain question answering [15] and machine translation [16].”
BackgroundReducing Sequence Length by Predicting Edit Spans with Large Language Models
Kaneko, M., & Okazaki, N. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.619 · Cited by 28
“Guu et al. (2018) proposed a language model that initially selects a prototype sentence from the training dataset and subsequently modifies it to create a new sentence.”
BackgroundMath Word Problem Generation via Disentangled Memory Retrieval
Qin, W., Wang, X., Hu, Z., Wang, L., Lan, Y., & Hong, R. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3639569 · Cited by 6
Look-up and Adapt: A One-shot Semantic Parser
Lu, Z., Arabshahi, F., Labutov, I., & Mitchell, T. M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1104 · Cited by 4
“(Guu et al., 2018) Gu et al. (2018).”
Stochastic graph recurrent neural network
Yan, T., Zhang, H., Li, Z., & Xia, Y. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.05.105 · Cited by 6
Supporting search engines with knowledge and context
Voskarides, N. (2021) · ACM SIGIR Forum · doi:10.1145/3527546.3527573 · Cited by 5
QuickEdit: Editing Text & Translations by Crossing Words OutPreprint
Grangier, D., & Auli, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.04805 · Cited by 14
“Guu et al. (2017) considers generating text with latent edition.”
Nutribullets Hybrid: Multi-document Health SummarizationPreprint
Shah, D., Yu, L., Lei, T., & Barzilay, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.03465 · Cited by 5
“Prior work (Mueller et al., 2017; Fan et al., 2017; Guu et al., 2018) on text generation also control aspects of the produced text, such as style and length.”
BackgroundCapturing Structural Locality in Non-parametric Language ModelsPreprint
Xu, F. F., He, J., Neubig, G., & Hellendoorn, V. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.02870 · Cited by 9
Intensive vision-guided network for radiology report generation
Zheng, F., Li, M., Wang, Y., Yu, W., Wang, R., Chen, Z., Xiao, N., & Lu, Y. (2024) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ad1995 · Cited by 6
“There are some other studies on long text generation (Hu et al. 2019), sentence generation (Guu et al. 2018), paragraph generation (Liang et al. 2017), visual description (Karpathy & Fei-Fei 2017), and text summarization (Rush et al. 2015).”
BackgroundComposing Like an Ancient Chinese Poet: Learn to Generate Rhythmic Chinese Poetry
He, M., Yan, C., Zhao, H., Liu, Q., Wu, L., Cui, Y. J., Zeng, G., & Liu, G. (2023) · Journal of Computer Science and Technology · doi:10.1007/s11390-023-1295-1 · Cited by 4
Counterfactual and Prototypical Explanations for Tabular Data via Interpretable Latent Space
Piaggesi, S., Bodria, F., Guidotti, R., Giannotti, F., & Pedreschi, D. (2024) · IEEE Access · doi:10.1109/access.2024.3496114 · Cited by 6
“Other uses of the latent space prototypes are for face identification [69], guided generation of sentences [70], and for other explanations generations like saliency maps [71].”
Kim, B., Seo, S., Han, S., Erdenee, E., & Chang, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.286 · Cited by 9
Learning to Selectively Learn for Weakly-supervised Paraphrase Generation
Ding, K., Li, D., Li, A. H., Fan, X., Guo, C., Liu, Y., & Liu, H. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.480 · Cited by 4
“…for solving the studied problem; and (iii) the state-of-the-art paraphrasing methods are predominantly built upon traditional Seq2Seq models, while the necessity of learning from scratch largely magnifies the learning difficulty when dealing with scarce or noisy training data (Guu et al., 2018).”
Background
Metadata
- Authors
- Guu, K., Hashimoto, T. B., Oren, Y., & Liang, P.
- Affiliations
- Stanford University (US)
- Corresponding author
- Guu, Kelvin
- Hashimoto, Tatsunori B.
- Oren, Yonatan
- Liang, Percy
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2018-12
- Volume
- 6
- Pages
- 437-450
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2963018920
- Cited by
- 300
- References
- 29
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 7 · 2023: 30 · 2022: 30 · 2021: 48 · 2020: 56 · 2019: 72 · 2018: 44
- FWCI
- 30.55
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Transactions of the Association for Computational Linguistics (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defense Advanced Research Projects Agency
- Army Research Office (W911NF-15-1-, W911NF, W911NF-15)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- sentence generation, generative language models, sentence similarity, perplexity, interpretability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.